Multifaceted fused-CNN based scoring of breast cancer whole-slide histopathology images

Robust ROI Detection in Whole Slide Images Guided by Pathologists' Viewing Patterns

Deep learning techniques offer improvements in computer-aided diagnosis systems. However, acquiring image domain annotations is challenging due to the knowledge and commitment required of expert pathologists. Pathologists often identify regions in whole slide images with diagnostic relevance rather than examining the entire slide, with a positive correlation between the time spent on these critical image regions and diagnostic accuracy. In this paper, a heatmap is generated to represent pathologists' viewing patterns during diagnosis and used to guide a deep learning architecture during training. The proposed system outperforms traditional approaches based on color and texture image characteristics, integrating pathologists' domain expertise to enhance region of interest detection without needing individual case annotations. Evaluating our best model, a U-Net model with a pre-trained ResNet-18 encoder, on a skin biopsy whole slide image dataset for melanoma diagnosis, shows its potential in detecting regions of interest, surpassing conventional methods with an increase of 20%, 11%, 22%, and 12% in precision, recall, F1-score, and Intersection over Union, respectively. In a clinical evaluation, three dermatopathologists agreed on the model's effectiveness in replicating pathologists' diagnostic viewing behavior and accurately identifying critical regions. Finally, our study demonstrates that incorporating heatmaps as supplementary signals can enhance the performance of computer-aided diagnosis systems. Without the availability of eye tracking data, identifying precise focus areas is challenging, but our approach shows promise in assisting pathologists in improving diagnostic accuracy and efficiency, streamlining annotation processes, and aiding the training of new pathologists.

AI-enabled routine H&E image based prognostic marker for early-stage luminal breast cancer

Breast cancer (BC) grade is a well-established subjective prognostic indicator of tumour aggressiveness. Tumour heterogeneity and subjective assessment result in high degree of variability among observers in BC grading. Here we propose an objective Haematoxylin & Eosin (H&E) image-based prognostic marker for early-stage luminal/Her2-negative BReAst CancEr that we term as the BRACE marker. The proposed BRACE marker is derived from AI based assessment of heterogeneity in BC at a detailed level using the power of deep learning. The prognostic ability of the marker is validated in two well-annotated cohorts (Cohort-A/Nottingham: n = 2122 and Cohort-B/Coventry: n = 311) on early-stage luminal/HER2-negative BC patients treated with endocrine therapy and with long-term follow-up. The BRACE marker is able to stratify patients for both distant metastasis free survival (p = 0.001, C-index: 0.73) and BC specific survival (p < 0.0001, C-index: 0.84) showing comparable prediction accuracy to Nottingham Prognostic Index and Magee scores, which are both derived from manual histopathological assessment, to identify luminal BC patients that may be likely to benefit from adjuvant chemotherapy.

A Comprehensive Review of Computer-Aided Models for Breast Cancer Diagnosis Using Histopathology Images

Breast cancer is the second most common cancer in women who are mainly middle-aged and older. The American Cancer Society reported that the average risk of developing breast cancer sometime in their life is about 13%, and this incident rate has increased by 0.5% per year in recent years. A biopsy is done when screening tests and imaging results show suspicious breast changes. Advancements in computer-aided system capabilities and performance have fueled research using histopathology images in cancer diagnosis. Advances in machine learning and deep neural networks have tremendously increased the number of studies developing computerized detection and classification models. The dataset-dependent nature and trial-and-error approach of the deep networks' performance produced varying results in the literature. This work comprehensively reviews the studies published between 2010 and 2022 regarding commonly used public-domain datasets and methodologies used in preprocessing, segmentation, feature engineering, machine-learning approaches, classifiers, and performance metrics.

Breast tumor localization and segmentation using machine learning techniques: Overview of datasets, findings, and methods

The Global Cancer Statistics 2020 reported breast cancer (BC) as the most common diagnosis of cancer type. Therefore, early detection of such type of cancer would reduce the risk of death from it. Breast imaging techniques are one of the most frequently used techniques to detect the position of cancerous cells or suspicious lesions. Computer-aided diagnosis (CAD) is a particular generation of computer systems that assist experts in detecting medical image abnormalities. In the last decades, CAD has applied deep learning (DL) and machine learning approaches to perform complex medical tasks in the computer vision area and improve the ability to make decisions for doctors and radiologists. The most popular and widely used technique of image processing in CAD systems is segmentation which consists of extracting the region of interest (ROI) through various techniques. This research provides a detailed description of the main categories of segmentation procedures which are classified into three classes: supervised, unsupervised, and DL. The main aim of this work is to provide an overview of each of these techniques and discuss their pros and cons. This will help researchers better understand these techniques and assist them in choosing the appropriate method for a given use case.

Semantic annotation for computational pathology: multidisciplinary experience and best practice recommendations

Recent advances in whole‐slide imaging (WSI) technology have led to the development of a myriad of computer vision and artificial intelligence‐based diagnostic, prognostic, and predictive algorithms. Computational Pathology (CPath) offers an integrated solution to utilise information embedded in pathology WSIs beyond what can be obtained through visual assessment. For automated analysis of WSIs and validation of machine learning (ML) models, annotations at the slide, tissue, and cellular levels are required. The annotation of important visual constructs in pathology images is an important component of CPath projects. Improper annotations can result in algorithms that are hard to interpret and can potentially produce inaccurate and inconsistent results. Despite the crucial role of annotations in CPath projects, there are no well‐defined guidelines or best practices on how annotations should be carried out. In this paper, we address this shortcoming by presenting the experience and best practices acquired during the execution of a large‐scale annotation exercise involving a multidisciplinary team of pathologists, ML experts, and researchers as part of the Pathology image data Lake for Analytics, Knowledge and Education (PathLAKE) consortium. We present a real‐world case study along with examples of different types of annotations, diagnostic algorithm, annotation data dictionary, and annotation constructs. The analyses reported in this work highlight best practice recommendations that can be used as annotation guidelines over the lifecycle of a CPath project.

COVID-19 detection in chest X-ray images using deep boosted hybrid learning

The new emerging COVID-19, declared a pandemic disease, has affected millions of human lives and caused a massive burden on healthcare centers. Therefore, a quick, accurate, and low-cost computer-based tool is required to timely detect and treat COVID-19 patients. In this work, two new deep learning frameworks: Deep Hybrid Learning (DHL) and Deep Boosted Hybrid Learning (DBHL), is proposed for effective COVID-19 detection in X-ray dataset. In the proposed DHL framework, the representation learning ability of the two developed COVID-RENet-1 & 2 models is exploited individually through a machine learning (ML) classifier. In COVID-RENet models, Region and Edge-based operations are carefully applied to learn region homogeneity and extract boundaries features. While in the case of the proposed DBHL framework, COVID-RENet-1 & 2 are fine-tuned using transfer learning on the chest X-rays. Furthermore, deep feature spaces are generated from the penultimate layers of the two models and then concatenated to get a single enriched boosted feature space. A conventional ML classifier exploits the enriched feature space to achieve better COVID-19 detection performance. The proposed COVID-19 detection frameworks are evaluated on radiologist's authenticated chest X-ray data, and their performance is compared with the well-established CNNs. It is observed through experiments that the proposed DBHL framework, which merges the two-deep CNN feature spaces, yields good performance (accuracy: 98.53%, sensitivity: 0.99, F-score: 0.98, and precision: 0.98). Furthermore, a web-based interface is developed, which takes only 5-10s to detect COVID-19 in each unseen chest X-ray image. This web-predictor is expected to help early diagnosis, save precious lives, and thus positively impact society.

Mitotic nuclei analysis in breast cancer histopathology images using deep ensemble classifier

Mitotic nuclei estimation in breast tumour samples has a prognostic significance in analysing tumour aggressiveness and grading system. The automated assessment of mitotic nuclei is challenging because of their high similarity with non-mitotic nuclei and heteromorphic appearance. In this work, we have proposed a new Deep Convolutional Neural Network (CNN) based Heterogeneous Ensemble technique "DHE-Mit-Classifier" for analysis of mitotic nuclei in breast histopathology images. The proposed technique in the first step detects candidate mitotic patches from the histopathological biopsy regions, whereas, in the second step, these patches are classified into mitotic and non-mitotic nuclei using the proposed DHE-Mit-Classifier. For the development of a heterogeneous ensemble, five different deep CNNs are designed and used as base-classifiers. These deep CNNs have varying architectural designs to capture the structural, textural, and morphological properties of the mitotic nuclei. The developed base-classifiers exploit different ideas, including (i) region homogeneity and feature invariance, (ii) asymmetric split-transform-merge, (iii) dilated convolution based multi-scale transformation, (iv) spatial and channel attention, and (v) residual learning. Multi-layer-perceptron is used as a meta-classifier to develop a robust and accurate classifier for providing the final decision. The performance of the proposed ensemble "DHE-Mit-Classifier" is evaluated against state-of-the-art CNNs. The performance evaluation on the test set suggests the superiority of the proposed ensemble with an F-score (0.77), recall (0.71), precision (0.83), and area under the precision-recall curve (0.80). The good generalisation of the proposed ensemble with a considerably high F-score and precision suggests its potential use in the development of an assistance tool for pathologists.

Boosting Breast Cancer Detection Using Convolutional Neural Network

Breast cancer forms in breast cells and is considered as a very common type of cancer in women. Breast cancer is also a very life-threatening disease of women after lung cancer. A convolutional neural network (CNN) method is proposed in this study to boost the automatic identification of breast cancer by analyzing hostile ductal carcinoma tissue zones in whole-slide images (WSIs). The paper investigates the proposed system that uses various convolutional neural network (CNN) architectures to automatically detect breast cancer, comparing the results with those from machine learning (ML) algorithms. All architectures were guided by a big dataset of about 275,000, 50 × 50-pixel RGB image patches. Validation tests were done for quantitative results using the performance measures for every methodology. The proposed system is found to be successful, achieving results with 87% accuracy, which could reduce human mistakes in the diagnosis process. Moreover, our proposed system achieves accuracy higher than the 78% accuracy of machine learning (ML) algorithms. The proposed system therefore improves accuracy by 9% above results from machine learning (ML) algorithms.

Deep convolutional neural network and emotional learning based breast cancer detection using digital mammography

Breast cancer is one of the deadly diseases among women. However, the chances of death are highly reduced if it gets diagnosed and treated at its early stage. Mammography is one of the reliable methods used by the radiologist to detect breast cancer at its initial stage. Therefore, an automatic and secure breast cancer detection system that accurately detects abnormalities not only increases the radiologist's diagnostic confidence but also provides more objective evidence. In this work, an automatic Diverse Features based Breast Cancer Detection (DFeBCD) system is proposed to classify a mammogram as normal or abnormal. Four sets of distinct feature types are used. Among them, features based on taxonomic indexes, statistical measures and local binary patterns are static. The proposed DFeBCD dynamically extracts the fourth set of features from mammogram images using a highway-network based deep convolution neural network (CNN). Two classifiers, Support Vector Machine (SVM) and Emotional Learning inspired Ensemble Classifier (ELiEC), are trained on these distinct features using a standard IRMA mammogram dataset. The reliability of the system performance is ensured by applying 5-folds cross-validation. Through experiments, we have observed that the performance of the DFeBCD system on dynamically generated features through highway network-based CNN is better than that of all the three individual sets of ad-hoc features. Furthermore, the hybridization of all four types of features improves the system's performance by nearly 2-3%. The performance of both the classifiers is comparable using the individual sets of ad-hoc features. However, the ELiEC classifier's performance is better than SVM using both hybrid and dynamic features.