Multispectral band selection and spatial characterization: Application to mitosis detection in breast cancer histopathology

A novel multilevel iterative training strategy for the ResNet50 based mitotic cell classifier

The number of mitotic cells is an important indicator of grading invasive breast cancer. It is very challenging for pathologists to identify and count mitotic cells in pathological sections with naked eyes under the microscope. Therefore, many computational models for the automatic identification of mitotic cells based on machine learning, especially deep learning, have been proposed. However, converging to the local optimal solution is one of the main problems in model training. In this paper, we proposed a novel multilevel iterative training strategy to address the problem. To evaluate the proposed training strategy, we constructed the mitotic cell classification model with ResNet50 and trained the model with different training strategies. The results showed that the models trained with the proposed training strategy performed better than those trained with the conventional strategy in the independent test set, illustrating the effectiveness of the new training strategy. Furthermore, after training with our proposed strategy, the ResNet50 model with Adam optimizer has achieved 89.26% F1 score on the public MITOSI14 dataset, which is higher than that of the state-of-the-art methods reported in the literature.

A novel dilated contextual attention module for breast cancer mitosis cell detection

Background and object: Mitotic count (MC) is a critical histological parameter for accurately assessing the degree of invasiveness in breast cancer, holding significant clinical value for cancer treatment and prognosis. However, accurately identifying mitotic cells poses a challenge due to their morphological and size diversity. Objective: We propose a novel end-to-end deep-learning method for identifying mitotic cells in breast cancer pathological images, with the aim of enhancing the performance of recognizing mitotic cells. Methods: We introduced the Dilated Cascading Network (DilCasNet) composed of detection and classification stages. To enhance the model's ability to capture distant feature dependencies in mitotic cells, we devised a novel Dilated Contextual Attention Module (DiCoA) that utilizes sparse global attention during the detection. For reclassifying mitotic cell areas localized in the detection stage, we integrate the EfficientNet-B7 and VGG16 pre-trained models (InPreMo) in the classification step. Results: Based on the canine mammary carcinoma (CMC) mitosis dataset, DilCasNet demonstrates superior overall performance compared to the benchmark model. The specific metrics of the model's performance are as follows: F1 score of 82.9%, Precision of 82.6%, and Recall of 83.2%. With the incorporation of the DiCoA attention module, the model exhibited an improvement of over 3.5% in the F1 during the detection stage. Conclusion: The DilCasNet achieved a favorable detection performance of mitotic cells in breast cancer and provides a solution for detecting mitotic cells in pathological images of other cancers.

Unsupervised band selection of medical hyperspectral images guided by data gravitation and weak correlation

BACKGROUND AND OBJECTIVE

Medical hyperspectral images (MHSIs) are used for a contact-free examination of patients without harmful radiation. However, high-dimensionality images contain large amounts of data that are sparsely distributed in a high-dimensional space, which leads to the "curse of dimensionality" (called Hughes' phenomenon) and increases the complexity and cost of data processing and storage. Hence, there is a need for spectral dimensionality reduction before the clinical application of MHSIs. Some dimensionality-reducing strategies have been proposed; however, they distort the data within MHSIs.

METHODS

To compress dimensionality without destroying the original data structure, we propose a method that involves data gravitation and weak correlation-based ranking (DGWCR) for removing bands of noise from MHSIs while clustering signal-containing bands. Band clustering is done by using the connection centre evolution (CCE) algorithm and selecting the most representative bands in each cluster based on the composite force. The bands within the clusters are ranked using the new entropy-containing matrix, and a global ranking of bands is obtained by applying an S-shaped strategy. The source code is available at https://www.github.com/zhangchenglong1116/DGWCR.

RESULTS

Upon feeding the reduced-dimensional images into various classifiers, the experimental results demonstrated that the small number of bands selected by the proposed DGWCR consistently achieved higher classification accuracy than the original data. Unlike other reference methods (e.g. the latest deep-learning-based strategies), DGWCR chooses the spectral bands with the least redundancy and greatest discrimination.

CONCLUSION

In this study, we present a method for efficient band selection for MHSIs that alleviates the "curse of dimensionality". Experiments were validated with three MHSIs in the human brain, and they outperformed several other band selection methods, demonstrating the clinical potential of DGWCR.

Artificial intelligence in breast cancer histopathology

This is a review on the use of artificial intelligence for digital breast pathology. A systematic search on PubMed was conducted, identifying 17,324 research papers related to breast cancer pathology. Following a semimanual screening, 664 papers were retrieved and pursued. The papers are grouped into six major tasks performed by pathologists-namely, molecular and hormonal analysis, grading, mitotic figure counting, ki-67 indexing, tumour-infiltrating lymphocyte assessment, and lymph node metastases identification. Under each task, open-source datasets for research to build artificial intelligence (AI) tools are also listed. Many AI tools showed promise and demonstrated feasibility in the automation of routine pathology investigations. We expect continued growth of AI in this field as new algorithms mature.

Artificial intelligence techniques for neuropathological diagnostics and research

Artificial intelligence (AI) research began in theoretical neurophysiology, and the resulting classical paper on the McCulloch-Pitts mathematical neuron was written in a psychiatry department almost 80 years ago. However, the application of AI in digital neuropathology is still in its infancy. Rapid progress is now being made, which prompted this article. Human brain diseases represent distinct system states that fall outside the normal spectrum. Many differ not only in functional but also in structural terms, and the morphology of abnormal nervous tissue forms the traditional basis of neuropathological disease classifications. However, only a few countries have the medical specialty of neuropathology, and, given the sheer number of newly developed histological tools that can be applied to the study of brain diseases, a tremendous shortage of qualified hands and eyes at the microscope is obvious. Similarly, in neuroanatomy, human observers no longer have the capacity to process the vast amounts of connectomics data. Therefore, it is reasonable to assume that advances in AI technology and, especially, whole-slide image (WSI) analysis will greatly aid neuropathological practice. In this paper, we discuss machine learning (ML) techniques that are important for understanding WSI analysis, such as traditional ML and deep learning, introduce a recently developed neuropathological AI termed PathoFusion, and present thoughts on some of the challenges that must be overcome before the full potential of AI in digital neuropathology can be realized.

MiNuGAN: Dual Segmentation of Mitoses and Nuclei Using Conditional GANs on Multi-center Breast H&E Images

Breast cancer is the second most commonly diagnosed type of cancer among women as of 2021. Grading of histopathological images is used to guide breast cancer treatment decisions and a critical component of this is a mitotic score, which is related to tumor aggressiveness. Manual mitosis counting is an extremely tedious manual task, but automated approaches can be used to overcome inefficiency and subjectivity. In this paper, we propose an automatic mitosis and nuclear segmentation method for a diverse set of H&E breast cancer pathology images. The method is based on a conditional generative adversarial network to segment both mitoses and nuclei at the same time. Architecture optimizations are investigated, including hyper parameters and the addition of a focal loss. The accuracy of the proposed method is investigated using images from multiple centers and scanners, including TUPAC16, ICPR14 and ICPR12 datasets. In TUPAC16, we use 618 carefully annotated images of size 256×256 scanned at 40×. TUPAC16 is used to train the model, and segmentation performance is measured on the test set for both nuclei and mitoses. Results on 200 held-out testing images from the TUPAC16 dataset were mean DSC = 0.784 and 0.721 for nuclear and mitosis, respectively. On 202 ICPR12 images, mitosis segmentation accuracy had a mean DSC = 0.782, indicating the model generalizes well to unseen datasets. For datasets that had mitosis centroid annotations, which included 200 TUPAC16, 202 ICPR12 and 524 ICPR14, a mean F1-score of 0.854 was found indicating high mitosis detection accuracy.

Automated knowledge-assisted mitosis cells detection framework in breast histopathology images

Based on the Nottingham Histopathology Grading (NHG) system, mitosis cells detection is one of the important criteria to determine the grade of breast carcinoma. Mitosis cells detection is a challenging task due to the heterogeneous microenvironment of breast histopathology images. Recognition of complex and inconsistent objects in the medical images could be achieved by incorporating domain knowledge in the field of interest. In this study, the strategies of the histopathologist and domain knowledge approach were used to guide the development of the image processing framework for automated mitosis cells detection in breast histopathology images. The detection framework starts with color normalization and hyperchromatic nucleus segmentation. Then, a knowledge-assisted false positive reduction method is proposed to eliminate the false positive (i.e., non-mitosis cells). This stage aims to minimize the percentage of false positive and thus increase the F1-score. Next, features extraction was performed. The mitosis candidates were classified using a Support Vector Machine (SVM) classifier. For evaluation purposes, the knowledge-assisted detection framework was tested using two datasets: a custom dataset and a publicly available dataset (i.e., MITOS dataset). The proposed knowledge-assisted false positive reduction method was found promising by eliminating at least 87.1% of false positive in both the dataset producing promising results in the F1-score. Experimental results demonstrate that the knowledge-assisted detection framework can achieve promising results in F1-score (custom dataset: 89.1%; MITOS dataset: 88.9%) and outperforms the recent works.

Machine Learning Methods for Histopathological Image Analysis: A Review

Histopathological images (HIs) are the gold standard for evaluating some types of tumors for cancer diagnosis. The analysis of such images is time and resource-consuming and very challenging even for experienced pathologists, resulting in inter-observer and intra-observer disagreements. One of the ways of accelerating such an analysis is to use computer-aided diagnosis (CAD) systems. This paper presents a review on machine learning methods for histopathological image analysis, including shallow and deep learning methods. We also cover the most common tasks in HI analysis, such as segmentation and feature extraction. Besides, we present a list of publicly available and private datasets that have been used in HI research.

Hyperspectral and multispectral imaging in digital and computational pathology: a systematic review [Invited]

Hyperspectral imaging (HSI) and multispectral imaging (MSI) technologies have the potential to transform the fields of digital and computational pathology. Traditional digitized histopathological slides are imaged with RGB imaging. Utilizing HSI/MSI, spectral information across wavelengths within and beyond the visual range can complement spatial information for the creation of computer-aided diagnostic tools for both stained and unstained histological specimens. In this systematic review, we summarize the methods and uses of HSI/MSI for staining and color correction, immunohistochemistry, autofluorescence, and histopathological diagnostic research. Studies include hematology, breast cancer, head and neck cancer, skin cancer, and diseases of central nervous, gastrointestinal, and genitourinary systems. The use of HSI/MSI suggest an improvement in the detection of diseases and clinical practice compared with traditional RGB analysis, and brings new opportunities in histological analysis of samples, such as digital staining or alleviating the inter-laboratory variability of digitized samples. Nevertheless, the number of studies in this field is currently limited, and more research is needed to confirm the advantages of this technology compared to conventional imagery.

Using spectral imaging for the analysis of abnormalities for colorectal cancer: When is it helpful?

The spectral imaging technique has been shown to provide more discriminative information than the RGB images and has been proposed for a range of problems. There are many studies demonstrating its potential for the analysis of histopathology images for abnormality detection but there have been discrepancies among previous studies as well. Many multispectral based methods have been proposed for histopathology images but the significance of the use of whole multispectral cube versus a subset of bands or a single band is still arguable. We performed comprehensive analysis using individual bands and different subsets of bands to determine the effectiveness of spectral information for determining the anomaly in colorectal images. Our multispectral colorectal dataset consists of four classes, each represented by infra-red spectrum bands in addition to the visual spectrum bands. We performed our analysis of spectral imaging by stratifying the abnormalities using both spatial and spectral information. For our experiments, we used a combination of texture descriptors with an ensemble classification approach that performed best on our dataset. We applied our method to another dataset and got comparable results with those obtained using the state-of-the-art method and convolutional neural network based method. Moreover, we explored the relationship of the number of bands with the problem complexity and found that higher number of bands is required for a complex task to achieve improved performance. Our results demonstrate a synergy between infra-red and visual spectrum by improving the classification accuracy (by 6%) on incorporating the infra-red representation. We also highlight the importance of how the dataset should be divided into training and testing set for evaluating the histopathology image-based approaches, which has not been considered in previous studies on multispectral histopathology images.

Efficient deep learning model for mitosis detection using breast histopathology images

Mitosis detection is one of the critical factors of cancer prognosis, carrying significant diagnostic information required for breast cancer grading. It provides vital clues to estimate the aggressiveness and the proliferation rate of the tumour. The manual mitosis quantification from whole slide images is a very labor-intensive and challenging task. The aim of this study is to propose a supervised model to detect mitosis signature from breast histopathology WSI images. The model has been designed using deep learning architecture with handcrafted features. We used handcrafted features issued from previous medical challenges MITOS @ ICPR 2012, AMIDA-13 and projects (MICO ANR TecSan) expertise. The deep learning architecture mainly consists of five convolution layers, four max-pooling layers, four rectified linear units (ReLU), and two fully connected layers. ReLU has been used after each convolution layer as an activation function. Dropout layer has been included after first fully connected layer to avoid overfitting. Handcrafted features mainly consist of morphological, textural and intensity features. The proposed architecture has shown to have an improved 92% precision, 88% recall and 90% F-score. Prospectively, the proposed model will be very beneficial in routine exam, providing pathologists with efficient and - as we will prove - effective second opinion for breast cancer grading from whole slide images. Last but not the least, this model could lead junior and senior pathologists, as medical researchers, to a superior understanding and evaluation of breast cancer stage and genesis.

Screening of the prognostic targets for breast cancer based co-expression modules analysis

The purpose of the present study was to screen the prognostic targets for breast cancer based on a co-expression modules analysis. The microarray dataset GSE73383 was downloaded from the Gene Expression Omnibus (GEO) database, including 15 breast cancer samples with good prognosis and 9 breast cancer samples with poor prognosis. The differentially expressed genes (DEGs) were identified with the limma package. The Database for Annotation, Visualization and Integrated Discovery was used to perform Gene Ontology (GO) and Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway enrichment analysis. Furthermore, the co-expression analysis of DEGs was conducted with weighted correlation analysis. The interaction associations were analyzed with the Human Protein Reference Database and BioGRID. The protein-protein interactions (PPI) network was constructed and visualized by Cytoscape software. A total of 491 DEGs were identified in breast cancer samples with poor prognosis compared with those with good prognosis, and they were enriched in 85 GO terms and 4 KEGG pathways. 368 DEGs were co-expressed with others, and they were clustered into 10 modules. Module 6 was the most relevant to the clinical features, and 21 genes and 273 interaction pairs were selected out. Abnormal expression levels of required for meiotic nuclear division 5 homolog A (RMND5A) and angiopoietin-like protein 1 (ANGPTL1) were associated with a poor prognosis. It was indicated that SWI/SNF related, matrix associated, actin dependent regulator of chromatin, subfamily D, member 1, SWI/SNF related, matrix associated, actin dependent regulator of chromatin, subfamily D, member 1, dihydropyrimidinase-like 2, RMND5A and ANGPTL1 were potential prognostic markers in breast cancer, and the cell cycle may be involved in the regulation of breast cancer.

A Multi-Classifier System for Automatic Mitosis Detection in Breast Histopathology Images Using Deep Belief Networks

Mitotic count is an important diagnostic factor in breast cancer grading and prognosis. Detection of mitosis in breast histopathology images is very challenging mainly due to diffused intensities along object boundary and shape variation in different stages of mitosis. This paper demonstrates an accurate technique for detecting the mitotic cells in Hematoxyline and Eosin stained images by step by step refinement of segmentation and classification stages. Krill Herd Algorithm-based localized active contour model precisely segments cell nuclei from background stroma. A deep belief network based multi-classifier system classifies the labeled cells into mitotic and nonmitotic groups. The proposed method has been evaluated on MITOS data set provided for MITOS-ATYPIA contest 2014 and also on clinical images obtained from Regional Cancer Centre (RCC), Thiruvananthapuram, which is a pioneer institute specifically for cancer diagnosis and research in India. The algorithm provides improved performance compared with other state-of-the-art techniques with average F-score of 84.29% for the MITOS data set and 75% for the clinical data set from RCC.

Nuclear spatial and spectral features based evolutionary method for meningioma subtypes classification in histopathology

Meningioma subtypes classification is a real-world multiclass problem from the realm of neuropathology. The major challenge in solving this problem is the inherent complexity due to high intra-class variability and low inter-class variation in tissue samples. The development of computational methods to assist pathologists in characterization of these tissue samples would have great diagnostic and prognostic value. In this article, we proposed an optimized evolutionary framework for the classification of benign meningioma into four subtypes. This framework investigates the imperative role of RGB color channels for discrimination of tumor subtypes and compute structural, statistical and spectral phenotypes. An evolutionary technique, Genetic Algorithm, in combination with Support Vector Machine is applied to tune classifier parameters and to select the best possible combination of extracted phenotypes that improved the classification accuracy (94.88%) on meningioma histology dataset, provided by the Institute of Neuropathology, Bielefeld. These statistics show that computational framework can robustly discriminate four subtypes of benign meningioma and may aid pathologists in the diagnosis and classification of these lesions.

A Comparative Performance Analysis of Multispectral and RGB Imaging on HER2 Status Evaluation for the Prediction of Breast Cancer Prognosis

Despite the extensive application of multispectral imaging (MSI) in biomedical multidisciplinary researches, there is a paucity of data available regarding the implication of MSI in tumor prognosis prediction. We compared the behaviors of multispectral (MS) and conventional red-green-blue (RGB) images on assessment of human epidermal growth factor receptor 2 (HER2) immunohistochemistry to explore their impact on outcome in patients with invasive breast cancer (BC). Tissue microarrays containing 240 BC patients were introduced to compare the performance of MS and RGB imaging methods on the quantitative assessment of HER2 status and the prognostic value of 5-year disease-free survival (5-DFS). Both the total and average signal optical density values of HER2 MS and RGB images were analyzed, and all patients were divided into two groups based on the different 5-DFS. The quantification of HER2 MS images was negatively correlated with 5-DFS in lymph node–negative and –positive patients (P < .05), but RGB images were not in lymph node–positive patients (P = .101). Multivariate analysis indicated that the hazard ratio (HR) of HER2 MS was higher than that of HER2 RGB (HR = 2.454; 95% confidence interval [CI], 1.636-3.681 vs HR = 2.060; 95% CI, 1.361-3.119). Additionally, area under curve (AUC) by receiver operating characteristic analysis for HER2 MS was greater than that for HER2 RGB (AUC = 0.649; 95% CI, 0.577-0.722 vs AUC = 0.596; 95% CI, 0.522-0.670) in predicting the risk for recurrence. More importantly, the quantification of HER2 MS images has higher prediction accuracy than that of HER2 RGB images (69.6% vs 65.0%) on 5-DFS. Our study suggested that better information on BC prognosis could be obtained from the quantification of HER2 MS images and MS images might perform better in predicting BC prognosis than conventional RGB images.

Hyperspectral interventional imaging for enhanced tissue visualization and discrimination combining band selection methods

PURPOSE

Hyperspectral imaging is an emerging technology recently introduced in medical applications inasmuch as it provides a powerful tool for noninvasive tissue characterization. In this context, a new system was designed to be easily integrated in the operating room in order to detect anatomical tissues hardly noticed by the surgeon's naked eye.

METHOD

Our LCTF-based spectral imaging system is operative over visible, near- and middle-infrared spectral ranges (400-1700 nm). It is dedicated to enhance critical biological tissues such as the ureter and the facial nerve. We aim to find the best three relevant bands to create a RGB image to display during the intervention with maximal contrast between the target tissue and its surroundings. A comparative study is carried out between band selection methods and band transformation methods. Combined band selection methods are proposed. All methods are compared using different evaluation criteria.

RESULTS

Experimental results show that the proposed combined band selection methods provide the best performance with rich information, high tissue separability and short computational time. These methods yield a significant discrimination between biological tissues.

CONCLUSION

We developed a hyperspectral imaging system in order to enhance some biological tissue visualization. The proposed methods provided an acceptable trade-off between the evaluation criteria especially in SWIR spectral band that outperforms the naked eye's capacities.

Computational pathology to discriminate benign from malignant intraductal proliferations of the breast

The categorization of intraductal proliferative lesions of the breast based on routine light microscopic examination of histopathologic sections is in many cases challenging, even for experienced pathologists. The development of computational tools to aid pathologists in the characterization of these lesions would have great diagnostic and clinical value. As a first step to address this issue, we evaluated the ability of computational image analysis to accurately classify DCIS and UDH and to stratify nuclear grade within DCIS. Using 116 breast biopsies diagnosed as DCIS or UDH from the Massachusetts General Hospital (MGH), we developed a computational method to extract 392 features corresponding to the mean and standard deviation in nuclear size and shape, intensity, and texture across 8 color channels. We used L1-regularized logistic regression to build classification models to discriminate DCIS from UDH. The top-performing model contained 22 active features and achieved an AUC of 0.95 in cross-validation on the MGH data-set. We applied this model to an external validation set of 51 breast biopsies diagnosed as DCIS or UDH from the Beth Israel Deaconess Medical Center, and the model achieved an AUC of 0.86. The top-performing model contained active features from all color-spaces and from the three classes of features (morphology, intensity, and texture), suggesting the value of each for prediction. We built models to stratify grade within DCIS and obtained strong performance for stratifying low nuclear grade vs. high nuclear grade DCIS (AUC = 0.98 in cross-validation) with only moderate performance for discriminating low nuclear grade vs. intermediate nuclear grade and intermediate nuclear grade vs. high nuclear grade DCIS (AUC = 0.83 and 0.69, respectively). These data show that computational pathology models can robustly discriminate benign from malignant intraductal proliferative lesions of the breast and may aid pathologists in the diagnosis and classification of these lesions.