Multi-Pass Adaptive Voting for Nuclei Detection in Histopathological Images

Efficient Convolution Network to Assist Breast Cancer Diagnosis and Target Therapy

Breast cancer is the leading cause of cancer-related deaths among women worldwide, and early detection and treatment has been shown to significantly reduce fatality rates from severe illness. Moreover, determination of the human epidermal growth factor receptor-2 (HER2) gene amplification by Fluorescence in situ hybridization (FISH) and Dual in situ hybridization (DISH) is critical for the selection of appropriate breast cancer patients for HER2-targeted therapy. However, visual examination of microscopy is time-consuming, subjective and poorly reproducible due to high inter-observer variability among pathologists and cytopathologists. The lack of consistency in identifying carcinoma-like nuclei has led to divergences in the calculation of sensitivity and specificity. This manuscript introduces a highly efficient deep learning method with low computing cost. The experimental results demonstrate that the proposed framework achieves high precision and recall on three essential clinical applications, including breast cancer diagnosis and human epidermal receptor factor 2 (HER2) amplification detection on FISH and DISH slides for HER2 target therapy. Furthermore, the proposed method outperforms the majority of the benchmark methods in terms of IoU by a significant margin (p<0.001) on three essential clinical applications. Importantly, run time analysis shows that the proposed method obtains excellent segmentation results with notably reduced time for Artificial intelligence (AI) training (16.93%), AI inference (17.25%) and memory usage (18.52%), making the proposed framework feasible for practical clinical usage.

Optimized detection and segmentation of nuclei in gastric cancer images using stain normalization and blurred artifact removal

Histological analysis with microscopy is the gold standard to diagnose and stage cancer, where slides or whole slide images are analyzed for cell morphological and spatial features by pathologists. The nuclei of cancerous cells are characterized by nonuniform chromatin distribution, irregular shapes, and varying size. As nucleus area and shape alone carry prognostic value, detection and segmentation of nuclei are among the most important steps in disease grading. However, evaluation of nuclei is a laborious, time-consuming, and subjective process with large variation among pathologists. Recent advances in digital pathology have allowed significant applications in nuclei detection, segmentation, and classification, but automated image analysis is greatly affected by staining factors, scanner variability, and imaging artifacts, requiring robust image preprocessing, normalization, and segmentation methods for clinically satisfactory results. In this paper, we aimed to evaluate and compare the digital image analysis techniques used in clinical pathology and research in the setting of gastric cancer. A literature review was conducted to evaluate potential methods of improving nuclei detection. Digitized images of 35 patients from a retrospective cohort of gastric adenocarcinoma at Oulu University Hospital in 1987-2016 were annotated for nuclei (n = 9085) by expert pathologists and 14 images of different cancer types from public TCGA dataset with annotated nuclei (n = 7000) were used as a comparison to evaluate applicability in other cancer types. The detection and segmentation accuracy with the selected color normalization and stain separation techniques were compared between the methods. The extracted information can be supplemented by patient's medical data and fed to the existing statistical clinical tools or subjected to subsequent AI-assisted classification and prediction models. The performance of each method is evaluated by several metrics against the annotations done by expert pathologists. The F1-measure of 0.854 ± 0.068 is achieved with color normalization for the gastric cancer dataset, and 0.907 ± 0.044 with color deconvolution for the public dataset, showing comparable results to the earlier state-of-the-art works. The developed techniques serve as a basis for further research on application and interpretability of AI-assisted tools for gastric cancer diagnosis.

Deep Learning-Based Computational Cytopathologic Diagnosis of Metastatic Breast Carcinoma in Pleural Fluid

A Pleural effusion cytology is vital for treating metastatic breast cancer; however, concerns have arisen regarding the low accuracy and inter-observer variability in cytologic diagnosis. Although artificial intelligence-based image analysis has shown promise in cytopathology research, its application in diagnosing breast cancer in pleural fluid remains unexplored. To overcome these limitations, we evaluate the diagnostic accuracy of an artificial intelligence-based model using a large collection of cytopathological slides, to detect the malignant pleural effusion cytology associated with breast cancer. This study includes a total of 569 cytological slides of malignant pleural effusion of metastatic breast cancer from various institutions. We extracted 34,221 augmented image patches from whole-slide images and trained and validated a deep convolutional neural network model (DCNN) (Inception-ResNet-V2) with the images. Using this model, we classified 845 randomly selected patches, which were reviewed by three pathologists to compare their accuracy. The DCNN model outperforms the pathologists by demonstrating higher accuracy, sensitivity, and specificity compared to the pathologists (81.1% vs. 68.7%, 95.0% vs. 72.5%, and 98.6% vs. 88.9%, respectively). The pathologists reviewed the discordant cases of DCNN. After re-examination, the average accuracy, sensitivity, and specificity of the pathologists improved to 87.9, 80.2, and 95.7%, respectively. This study shows that DCNN can accurately diagnose malignant pleural effusion cytology in breast cancer and has the potential to support pathologists.

A Soft Label Deep Learning to Assist Breast Cancer Target Therapy and Thyroid Cancer Diagnosis

According to the World Health Organization Report 2022, cancer is the most common cause of death contributing to nearly one out of six deaths worldwide. Early cancer diagnosis and prognosis have become essential in reducing the mortality rate. On the other hand, cancer detection is a challenging task in cancer pathology. Trained pathologists can detect cancer, but their decisions are subjective to high intra- and inter-observer variability, which can lead to poor patient care owing to false-positive and false-negative results. In this study, we present a soft label fully convolutional network (SL-FCN) to assist in breast cancer target therapy and thyroid cancer diagnosis, using four datasets. To aid in breast cancer target therapy, the proposed method automatically segments human epidermal growth factor receptor 2 (HER2) amplification in fluorescence in situ hybridization (FISH) and dual in situ hybridization (DISH) images. To help in thyroid cancer diagnosis, the proposed method automatically segments papillary thyroid carcinoma (PTC) on Papanicolaou-stained fine needle aspiration and thin prep whole slide images (WSIs). In the evaluation of segmentation of HER2 amplification in FISH and DISH images, we compare the proposed method with thirteen deep learning approaches, including U-Net, U-Net with InceptionV5, Ensemble of U-Net with Inception-v4, Inception-Resnet-v2 encoder, and ResNet-34 encoder, SegNet, FCN, modified FCN, YOLOv5, CPN, SOLOv2, BCNet, and DeepLabv3+ with three different backbones, including MobileNet, ResNet, and Xception, on three clinical datasets, including two DISH datasets on two different magnification levels and a FISH dataset. The result on DISH breast dataset 1 shows that the proposed method achieves high accuracy of 87.77 ± 14.97%, recall of 91.20 ± 7.72%, and F1-score of 81.67 ± 17.76%, while, on DISH breast dataset 2, the proposed method achieves high accuracy of 94.64 ± 2.23%, recall of 83.78 ± 6.42%, and F1-score of 85.14 ± 6.61% and, on the FISH breast dataset, the proposed method achieves high accuracy of 93.54 ± 5.24%, recall of 83.52 ± 13.15%, and F1-score of 86.98 ± 9.85%, respectively. Furthermore, the proposed method outperforms most of the benchmark approaches by a significant margin (p <0.001). In evaluation of segmentation of PTC on Papanicolaou-stained WSIs, the proposed method is compared with three deep learning methods, including Modified FCN, U-Net, and SegNet. The experimental result demonstrates that the proposed method achieves high accuracy of 99.99 ± 0.01%, precision of 92.02 ± 16.6%, recall of 90.90 ± 14.25%, and F1-score of 89.82 ± 14.92% and significantly outperforms the baseline methods, including U-Net and FCN (p <0.001). With the high degree of accuracy, precision, and recall, the results show that the proposed method could be used in assisting breast cancer target therapy and thyroid cancer diagnosis with faster evaluation and minimizing human judgment errors.

Global Increase in Breast Cancer Incidence: Risk Factors and Preventive Measures

Breast cancer is a global cause for concern owing to its high incidence around the world. The alarming increase in breast cancer cases emphasizes the management of disease at multiple levels. The management should start from the beginning that includes stringent cancer screening or cancer registry to effective diagnostic and treatment strategies. Breast cancer is highly heterogeneous at morphology as well as molecular levels and needs different therapeutic regimens based on the molecular subtype. Breast cancer patients with respective subtype have different clinical outcome prognoses. Breast cancer heterogeneity emphasizes the advanced molecular testing that will help on-time diagnosis and improved survival. Emerging fields such as liquid biopsy and artificial intelligence would help to under the complexity of breast cancer disease and decide the therapeutic regimen that helps in breast cancer management. In this review, we have discussed various risk factors and advanced technology available for breast cancer diagnosis to combat the worst breast cancer status and areas that need to be focused for the better management of breast cancer.

Deep Learning on Histopathology Images for Breast Cancer Classification: A Bibliometric Analysis

Medical imaging is gaining significant attention in healthcare, including breast cancer. Breast cancer is the most common cancer-related death among women worldwide. Currently, histopathology image analysis is the clinical gold standard in cancer diagnosis. However, the manual process of microscopic examination involves laborious work and can be misleading due to human error. Therefore, this study explored the research status and development trends of deep learning on breast cancer image classification using bibliometric analysis. Relevant works of literature were obtained from the Scopus database between 2014 and 2021. The VOSviewer and Bibliometrix tools were used for analysis through various visualization forms. This study is concerned with the annual publication trends, co-authorship networks among countries, authors, and scientific journals. The co-occurrence network of the authors’ keywords was analyzed for potential future directions of the field. Authors started to contribute to publications in 2016, and the research domain has maintained its growth rate since. The United States and China have strong research collaboration strengths. Only a few studies use bibliometric analysis in this research area. This study provides a recent review on this fast-growing field to highlight status and trends using scientific visualization. It is hoped that the findings will assist researchers in identifying and exploring the potential emerging areas in the related field.

Evaluating Cancer-Related Biomarkers Based on Pathological Images: A Systematic Review

Many diseases are accompanied by changes in certain biochemical indicators called biomarkers in cells or tissues. A variety of biomarkers, including proteins, nucleic acids, antibodies, and peptides, have been identified. Tumor biomarkers have been widely used in cancer risk assessment, early screening, diagnosis, prognosis, treatment, and progression monitoring. For example, the number of circulating tumor cell (CTC) is a prognostic indicator of breast cancer overall survival, and tumor mutation burden (TMB) can be used to predict the efficacy of immune checkpoint inhibitors. Currently, clinical methods such as polymerase chain reaction (PCR) and next generation sequencing (NGS) are mainly adopted to evaluate these biomarkers, which are time-consuming and expansive. Pathological image analysis is an essential tool in medical research, disease diagnosis and treatment, functioning by extracting important physiological and pathological information or knowledge from medical images. Recently, deep learning-based analysis on pathological images and morphology to predict tumor biomarkers has attracted great attention from both medical image and machine learning communities, as this combination not only reduces the burden on pathologists but also saves high costs and time. Therefore, it is necessary to summarize the current process of processing pathological images and key steps and methods used in each process, including: (1) pre-processing of pathological images, (2) image segmentation, (3) feature extraction, and (4) feature model construction. This will help people choose better and more appropriate medical image processing methods when predicting tumor biomarkers.

Skin Melanoma Detection in Microscopic Images Using HMM-Based Asymmetric Analysis and Expectation Maximization

Melanoma is one of the deadliest types of skin cancer with increasing incidence. The most definitive diagnosis method is the histopathological examination of the tissue sample. In this paper, a melanoma detection algorithm is proposed based on decision-level fusion and a Hidden Markov Model (HMM), whose parameters are optimized using Expectation Maximization (EM) and asymmetric analysis. The texture heterogeneity of the samples is determined using asymmetric analysis. A fusion-based HMM classifier trained using EM is introduced. For this purpose, a novel texture feature is extracted based on two local binary patterns, namely local difference pattern (LDP) and statistical histogram features of the microscopic image. Extensive experiments demonstrate that the proposed melanoma detection algorithm yields a total error of less than 0.04%.

Feature-driven local cell graph (FLocK): New computational pathology-based descriptors for prognosis of lung cancer and HPV status of oropharyngeal cancers

Local spatial arrangement of nuclei in histopathology images of different cancer subtypes has been shown to have prognostic value. In order to capture localized nuclear architectural information, local cell cluster graph-based measurements have been proposed. However, conventional ways of cell graph construction only utilize nuclear spatial proximity, and do not differentiate between different cell types while constructing the graph. In this paper, we present feature-driven local cell cluster graph (FLocK), a new approach to constructing local cell graphs by simultaneously considering spatial proximity and attributes of the individual nuclei (e.g. shape, size, texture). In addition, we have designed a new set of quantitative graph-derived metrics to be extracted from FLocKs, in turn capturing the interplay between different proximally located clusters of nuclei. We have evaluated the efficacy of FLocK features extracted from H&E stained tissue images in two clinical applications: to classify short-term vs. long-term survival among patients of early stage non-small cell lung cancer (ES-NSCLC), and also to predict human papillomavirus (HPV) status of oropharyngeal squamous cell carcinoma (OP-SCCs). In the classification of long-term vs. short-term survival among patients of ES-NSCLC (training cohort, n = 434), the top 10 discriminative FLocK features related to the variation of FLocK size and intersected FLocK distance were identified, via Minimum Redundancy and Maximum Relevance (MRMR) selection, in 100 runs of 10-fold cross-validation, and in conjunction with a linear discriminant classifier yielded a mean AUC of 0.68 for predicting survival in the training cohort. This is better than other state-of-art histomorphometric and deep learning classifiers (cell cluster graphs (AUC = 0.62), global cell graph (AUC = 0.56), nuclear shape (AUC = 0.54), nuclear orientation (AUC = 0.61), AlexNet (AUC = 0.55), ResNet (AUC = 0.56)). The FLocK-based classifier yielded an AUC of 0.70 in an independent testing cohort (n = 150). The patients identified as "high-risk" had significantly poorer overall survival in the testing cohort, with a hazard ratio (95% confidence interval) of 2.24 (1.24-4.05), p = 0.01144). In the classification of HPV status of OP-SCC, the top three FLocK features pertaining to the portion of intersected FLocKs were used to construct a classifier, which yielded an AUC of 0.80 in the training cohort (n = 50), and an accuracy of 0.78 in an independent testing cohort (n = 35). The combination of FLocK measurements with cell cluster graphs, nuclear orientation, and nuclear shape improved the training AUC to 0.87, 0.91 and 0.85, respectively. Deep learning approaches yielded marginally better performance than the FLocK-based classifier in this application, with AUC = 0.78 for AlexNet, AUC = 0.81 for ResNet, and AUC = 0.76 for FLocK-based classifier in the testing cohort. However, the combination of two hand-crafted features: FLocK and nuclear orientation yielded a better performance (AUC = 0.84). FLocK provides a unique and quantitative way to analyze histology images of solid tumors and interrogate tumor morphology from a different aspect than existing histomorphometrics. The source code can be accessed at https://github.com/hacylu/FLocK.

Multi-aspect testing and ranking inference to quantify dimorphism in the cytoarchitecture of cerebellum of male, female and intersex individuals: a model applied to bovine brains

The dimorphism among male, female and freemartin intersex bovines, focusing on the vermal lobules VIII and IX, was analyzed using a novel data analytics approach to quantify morphometric differences in the cytoarchitecture of digitalized sections of the cerebellum. This methodology consists of multivariate and multi-aspect testing for cytoarchitecture-ranking, based on neuronal cell complexity among populations defined by factors, such as sex, age or pathology. In this context, we computed a set of shape descriptors of the neural cell morphology, categorized them into three domains named size, regularity and density, respectively. The output and results of our methodology are multivariate in nature, allowing an in-depth analysis of the cytoarchitectonic organization and morphology of cells. Interestingly, the Purkinje neurons and the underlying granule cells revealed the same morphological pattern: female possessed larger, denser and more irregular neurons than males. In the Freemartin, Purkinje neurons showed an intermediate setting between males and females, while the granule cells were the largest, most regular and dense. This methodology could be a powerful instrument to carry out morphometric analysis providing robust bases for objective tissue screening, especially in the field of neurodegenerative pathologies.

Artificial intelligence in digital breast pathology: Techniques and applications

Breast cancer is the most common cancer and second leading cause of cancer-related death worldwide. The mainstay of breast cancer workup is histopathological diagnosis - which guides therapy and prognosis. However, emerging knowledge about the complex nature of cancer and the availability of tailored therapies have exposed opportunities for improvements in diagnostic precision. In parallel, advances in artificial intelligence (AI) along with the growing digitization of pathology slides for the primary diagnosis are a promising approach to meet the demand for more accurate detection, classification and prediction of behaviour of breast tumours. In this article, we cover the current and prospective uses of AI in digital pathology for breast cancer, review the basics of digital pathology and AI, and outline outstanding challenges in the field.

Artificial intelligence in digital pathology - new tools for diagnosis and precision oncology

In the past decade, advances in precision oncology have resulted in an increased demand for predictive assays that enable the selection and stratification of patients for treatment. The enormous divergence of signalling and transcriptional networks mediating the crosstalk between cancer, stromal and immune cells complicates the development of functionally relevant biomarkers based on a single gene or protein. However, the result of these complex processes can be uniquely captured in the morphometric features of stained tissue specimens. The possibility of digitizing whole-slide images of tissue has led to the advent of artificial intelligence (AI) and machine learning tools in digital pathology, which enable mining of subvisual morphometric phenotypes and might, ultimately, improve patient management. In this Perspective, we critically evaluate various AI-based computational approaches for digital pathology, focusing on deep neural networks and 'hand-crafted' feature-based methodologies. We aim to provide a broad framework for incorporating AI and machine learning tools into clinical oncology, with an emphasis on biomarker development. We discuss some of the challenges relating to the use of AI, including the need for well-curated validation datasets, regulatory approval and fair reimbursement strategies. Finally, we present potential future opportunities for precision oncology.

Cell segmentation methods for label-free contrast microscopy: review and comprehensive comparison

BACKGROUND

Because of its non-destructive nature, label-free imaging is an important strategy for studying biological processes. However, routine microscopic techniques like phase contrast or DIC suffer from shadow-cast artifacts making automatic segmentation challenging. The aim of this study was to compare the segmentation efficacy of published steps of segmentation work-flow (image reconstruction, foreground segmentation, cell detection (seed-point extraction) and cell (instance) segmentation) on a dataset of the same cells from multiple contrast microscopic modalities.

RESULTS

We built a collection of routines aimed at image segmentation of viable adherent cells grown on the culture dish acquired by phase contrast, differential interference contrast, Hoffman modulation contrast and quantitative phase imaging, and we performed a comprehensive comparison of available segmentation methods applicable for label-free data. We demonstrated that it is crucial to perform the image reconstruction step, enabling the use of segmentation methods originally not applicable on label-free images. Further we compared foreground segmentation methods (thresholding, feature-extraction, level-set, graph-cut, learning-based), seed-point extraction methods (Laplacian of Gaussians, radial symmetry and distance transform, iterative radial voting, maximally stable extremal region and learning-based) and single cell segmentation methods. We validated suitable set of methods for each microscopy modality and published them online.

CONCLUSIONS

We demonstrate that image reconstruction step allows the use of segmentation methods not originally intended for label-free imaging. In addition to the comprehensive comparison of methods, raw and reconstructed annotated data and Matlab codes are provided.

Designing multifunctional cancer-targeted nanosystem for magnetic resonance molecular imaging-guided theranostics of lung cancer

The integration of diagnosis and therapy is an effective way to improve therapeutic effects for cancer patients, which has acquired widely attentions from researchers. Herein, a multifunctional drug-loaded nanosystem (F/A-PLGA@DOX/SPIO) has been designed and synthesized to reduce the side effects of traditional chemotherapy drugs and realize simultaneous tumor diagnosis and treatment. The surface modification of folic acid (FA) and activatable cell-penetrating peptide (ACPP) endows the nanosystem with excellent cancer targeting capabilities, thus reducing toxicity to normal organs. Besides, the F/A-PLGA@DOX/SPIO nanosystem can serve as an excellent magnetic resonance imaging (MRI) T2-negative contrast agent. More importantly, according to in vitro experiments, the F/A-PLGA@DOX/SPIO nanosystem can promote the overproduction of reactive oxygen species (ROS) within A549 lung cancer cells, inducing cell apoptosis, greatly enhancing the antineoplastic effect. Furthermore, with the help of MRI technology, the targeting imaging of the F/A-PLGA@DOX/SPIO nanosystem within tumors and the dynamic monitoring of medicine efficacy can be realized. Therefore, this study provided a multifunctional drug-loaded F/A-PLGA@DOX/SPIO targeted nanosystem for magnetic resonance molecular imaging-guided theranostics, which has excellent potential for the application in tumor diagnosis and therapy.

Nuclear shape and orientation features from H&E images predict survival in early-stage estrogen receptor-positive breast cancers

Early-stage estrogen receptor-positive (ER+) breast cancer (BCa) is the most common type of BCa in the United States. One critical question with these tumors is identifying which patients will receive added benefit from adjuvant chemotherapy. Nuclear pleomorphism (variance in nuclear shape and morphology) is an important constituent of breast grading schemes, and in ER+ cases, the grade is highly correlated with disease outcome. This study aimed to investigate whether quantitative computer-extracted image features of nuclear shape and orientation on digitized images of hematoxylin-stained and eosin-stained tissue of lymph node-negative (LN-), ER+ BCa could help stratify patients into discrete (<10 years short-term vs. >10 years long-term survival) outcome groups independent of standard clinical and pathological parameters. We considered a tissue microarray (TMA) cohort of 276 ER+, LN- patients comprising 150 patients with long-term and 126 patients with short-term overall survival, wherein 177 randomly chosen cases formed the modeling set, and 99 remaining cases the test set. Segmentation of individual nuclei was performed using multiresolution watershed; subsequently, 615 features relating to nuclear shape/texture and orientation disorder were extracted from each TMA spot. The Wilcoxon's rank-sum test identified the 15 most prognostic quantitative histomorphometric features within the modeling set. These features were then subsequently combined via a linear discriminant analysis classifier and evaluated on the test set to assign a probability of long-term vs. short-term disease-specific survival. In univariate survival analysis, patients identified by the image classifier as high risk had significantly poorer survival outcome: hazard ratio (95% confident interval) = 2.91(1.23-6.92), p = 0.02786. Multivariate analysis controlling for T-stage, histology grade, and nuclear grade showed the classifier to be independently predictive of poorer survival: hazard ratio (95% confident interval) = 3.17(0.33-30.46), p = 0.01039. Our results suggest that quantitative histomorphometric features of nuclear shape and orientation are strongly and independently predictive of patient survival in ER+, LN- BCa.

Object-Oriented Segmentation of Cell Nuclei in Fluorescence Microscopy Images

Cell nucleus segmentation remains an open and challenging problem especially to segment nuclei in cell clumps. Splitting a cell clump would be straightforward if the gradients of boundary pixels in-between the nuclei were always higher than the others. However, imperfections may exist: inhomogeneities of pixel intensities in a nucleus may cause to define spurious boundaries whereas insufficient pixel intensity differences at the border of overlapping nuclei may cause to miss some true boundary pixels. In contrast, these imperfections are typically observed at the pixel-level, causing local changes in pixel values without changing the semantics on a large scale. In response to these issues, this article introduces a new nucleus segmentation method that relies on using gradient information not at the pixel level but at the object level. To this end, it proposes to decompose an image into smaller homogeneous subregions, define edge-objects at four different orientations to encode the gradient information at the object level, and devise a merging algorithm, in which the edge-objects vote for subregion pairs along their orientations and the pairs are iteratively merged if they get sufficient votes from multiple orientations. Our experiments on fluorescence microscopy images reveal that this high-level representation and the design of a merging algorithm using edge-objects (gradients at the object level) improve the segmentation results.

Deep Learning Nuclei Detection in Digitized Histology Images by Superpixels

Background

Advances in image analysis and computational techniques have facilitated automatic detection of critical features in histopathology images. Detection of nuclei is critical for squamous epithelium cervical intraepithelial neoplasia (CIN) classification into normal, CIN1, CIN2, and CIN3 grades.

Methods

In this study, a deep learning (DL)-based nuclei segmentation approach is investigated based on gathering localized information through the generation of superpixels using a simple linear iterative clustering algorithm and training with a convolutional neural network.

Results

The proposed approach was evaluated on a dataset of 133 digitized histology images and achieved an overall nuclei detection (object-based) accuracy of 95.97%, with demonstrated improvement over imaging-based and clustering-based benchmark techniques.

Conclusions

The proposed DL-based nuclei segmentation Method with superpixel analysis has shown improved segmentation results in comparison to state-of-the-art methods.