An automatic nuclei segmentation method based on deep convolutional neural networks for histopathology images

Enhancing nuclei segmentation in breast histopathology images using U-Net with backbone architectures

Breast cancer remains a leading cause of mortality among women worldwide, underscoring the need for accurate and timely diagnostic methods. Precise segmentation of nuclei in breast histopathology images is crucial for effective diagnosis and prognosis, offering critical insights into tumor characteristics and informing treatment strategies. This paper presents an enhanced U-Net architecture utilizing ResNet-34 as an advanced backbone, aimed at improving nuclei segmentation performance. The proposed model is evaluated and compared with standard U-Net and its other variants, including U-Net with VGG-16 and Inception-v3 backbones, using the BreCaHad dataset with nuclei masks generated through ImageJ software. The U-Net model with ResNet-34 backbone achieved superior performance, recording an Intersection over Union (IoU) score of 0.795, significantly outperforming the basic U-Net's IoU score of 0.725. The integration of advanced backbones and data augmentation techniques substantially improved segmentation accuracy, especially on limited medical imaging datasets. Comparative analysis demonstrated that ResNet-34 consistently surpassed other configurations across multiple metrics, including IoU, accuracy, precision, and F1 score. Further validation on the BNS and MoNuSeg-2018 datasets confirmed the robustness of the proposed model. This study highlights the potential of advanced deep learning architectures combined with augmentation methods to address challenges in nuclei segmentation, contributing to the development of more effective clinical diagnostic tools and improved patient care outcomes.

Comparative Study of Cell Nuclei Segmentation Based on Computational and Handcrafted Features Using Machine Learning Algorithms

Background: Nuclei segmentation is the first stage of automated microscopic image analysis. The cell nucleus is a crucial aspect in segmenting to gain more insight into cell characteristics and functions that enable computer-aided pathology for early disease detection, such as prostate cancer, breast cancer, brain tumors, and other diagnoses. Nucleus segmentation remains a challenging task despite significant advancements in automated methods. Traditional techniques, such as Otsu thresholding and watershed approaches, are ineffective in challenging scenarios. However, deep learning-based methods exhibit remarkable results across various biological imaging modalities, including computational pathology. Methods: This work explores machine learning approaches for nuclei segmentation by evaluating the quality of nuclei image segmentation. We employed several methods, including K-means clustering, Random Forest (RF), Support Vector Machine (SVM) with handcrafted features, and Logistic Regression (LR) using features derived from Convolutional Neural Networks (CNNs). Handcrafted features extract attributes like the shape, texture, and intensity of nuclei and are meticulously developed based on specialized knowledge. Conversely, CNN-based features are automatically acquired representations that identify complex patterns in nuclei images. To assess how effectively these techniques segment cell nuclei, their performance is evaluated. Results: Experimental results show that Logistic Regression based on CNN-derived features outperforms the other techniques, achieving an accuracy of 96.90%, a Dice coefficient of 74.24, and a Jaccard coefficient of 55.61. In contrast, the Random Forest, Support Vector Machine, and K-means algorithms yielded lower segmentation performance metrics. Conclusions: The conclusions suggest that leveraging CNN-based features in conjunction with Logistic Regression significantly enhances the accuracy of cell nuclei segmentation in pathological images. This approach holds promise for refining computer-aided pathology workflows, potentially leading to more reliable and earlier disease diagnoses.

Improving generalization capability of deep learning-based nuclei instance segmentation by non-deterministic train time and deterministic test time stain normalization

With the advent of digital pathology and microscopic systems that can scan and save whole slide histological images automatically, there is a growing trend to use computerized methods to analyze acquired images. Among different histopathological image analysis tasks, nuclei instance segmentation plays a fundamental role in a wide range of clinical and research applications. While many semi- and fully-automatic computerized methods have been proposed for nuclei instance segmentation, deep learning (DL)-based approaches have been shown to deliver the best performances. However, the performance of such approaches usually degrades when tested on unseen datasets. In this work, we propose a novel method to improve the generalization capability of a DL-based automatic segmentation approach. Besides utilizing one of the state-of-the-art DL-based models as a baseline, our method incorporates non-deterministic train time and deterministic test time stain normalization, and ensembling to boost the segmentation performance. We trained the model with one single training set and evaluated its segmentation performance on seven test datasets. Our results show that the proposed method provides up to 4.9%, 5.4%, and 5.9% better average performance in segmenting nuclei based on Dice score, aggregated Jaccard index, and panoptic quality score, respectively, compared to the baseline segmentation model.

Artificial intelligence: illuminating the depths of the tumor microenvironment

Artificial intelligence (AI) can acquire characteristics that are not yet known to humans through extensive learning, enabling to handle large amounts of pathology image data. Divided into machine learning and deep learning, AI has the advantage of handling large amounts of data and processing image analysis, consequently it also has a great potential in accurately assessing tumour microenvironment (TME) models. With the complex composition of the TME, in-depth study of TME contributes to new ideas for treatment, assessment of patient response to postoperative therapy and prognostic prediction. This leads to a review of the development of AI's application in TME assessment in this study, provides an overview of AI techniques applied to medicine, delves into the application of AI in analysing the quantitative and spatial location characteristics of various cells (tumour cells, immune and non-immune cells) in the TME, reveals the predictive prognostic value of TME and provides new ideas for tumour therapy, highlights the great potential for clinical applications. In addition, a discussion of its limitations and encouraging future directions for its practical clinical application is presented.

Localization and Risk Stratification of Thyroid Nodules in Ultrasound Images Through Deep Learning

OBJECTIVE

Deep learning algorithms have commonly been used for the differential diagnosis between benign and malignant thyroid nodules. The aim of the study described here was to develop an integrated system that combines a deep learning model and a clinical standard Thyroid Imaging Reporting and Data System (TI-RADS) for the simultaneous segmentation and risk stratification of thyroid nodules.

METHODS

Three hundred four ultrasound images from two independent sites with TI-RADS 4 thyroid nodules were collected. The edge connection and Criminisi algorithm were used to remove manually induced markers in ultrasound images. An integrated system based on TI-RADS and a mask region-based convolution neural network (Mask R-CNN) was proposed to stratify subclasses of TI-RADS 4 thyroid nodules and to segment thyroid nodules in the ultrasound images. Accuracy and the precision-recall curve were used to evaluate stratification performance, and the Dice similarity coefficient (DSC) between the segmentation of Mask R-CNN and the radiologist's contour was used to evaluate the segmentation performance of the model.

RESULTS

The combined approach could significantly enhance the performance of the proposed integrated system. Overall stratification accuracy of TI-RADS 4 thyroid nodules, mean average precision and mean DSC of the proposed model in the independent test set was 90.79%, 0.8579 and 0.83, respectively. Specifically, stratification accuracy values for TI-RADS 4a, 4b and 4c thyroid nodules were 95.83%, 84.21% and 77.78%, respectively.

CONCLUSION

An integrated system combining TI-RADS and a deep learning model was developed. The system can provide clinicians with not only diagnostic assistance from TI-RADS but also accurate segmentation of thyroid nodules, which improves the applicability of the system in clinical practice.

CD20highCD138low tumor-infiltrating lymphocytes predominantly related to cytokine‒cytokine receptor interactions are associated with favorable outcomes in neuroblastoma patients

Recent advances have revealed that the role of the immune system is prominent in the antitumor response. In the present study, it is aimed to provide an expression profile of tumor-infiltrating lymphocytes (TILs), including mature B cells, plasma cells, and their clinical relevance in neuroblastoma. The expression of CD20 and CD138 was analyzed in the Cangelosi786 dataset (n = 769) as a training dataset and in our cohort (n = 120) as a validation cohort. CD20 high expression was positively associated with favorable overall survival (OS) and event-free survival (EFS) (OS: P < 0.001; EFS: P < 0.001) in the training dataset, whereas CD138 high expression was associated with poor OS and EFS (OS: P < 0.001; EFS: P < 0.001) in both the training and validation datasets. Accordingly, a combined pattern of CD20 and CD138 expression was developed, whereby neuroblastoma patients with CD20highCD138low expression had a consistently favorable OS and EFS compared with those with CD20lowCD138high expression in both the training and validation cohorts (P < 0.0001 and P < 0.01, respectively). Examination of potential molecular functions revealed that signaling pathways, including cytokine‒cytokine receptor interactions, chemokine, and the NF-kappa B signaling pathways, were involved. Differentially expressed genes, such as BMP7, IL7R, BIRC3, CCR7, CXCR5, CCL21, and CCL19, predominantly play important roles in predicting the survival of neuroblastoma patients. Our study proposes that a new combination of CD20 and CD138 signatures is associated with neuroblastoma patient survival. The related signaling pathways reflect the close associations among the number of TILs, cytokine abundance and patient outcomes and provide therapeutic insights into neuroblastoma.

Built to Last? Reproducibility and Reusability of Deep Learning Algorithms in Computational Pathology

Recent progress in computational pathology has been driven by deep learning. While code and data availability are essential to reproduce findings from preceding publications, ensuring a deep learning model's reusability is more challenging. For that, the codebase should be well-documented and easy to integrate into existing workflows and models should be robust toward noise and generalizable toward data from different sources. Strikingly, only a few computational pathology algorithms have been reused by other researchers so far, let alone employed in a clinical setting. To assess the current state of reproducibility and reusability of computational pathology algorithms, we evaluated peer-reviewed articles available in PubMed, published between January 2019 and March 2021, in 5 use cases: stain normalization; tissue type segmentation; evaluation of cell-level features; genetic alteration prediction; and inference of grading, staging, and prognostic information. We compiled criteria for data and code availability and statistical result analysis and assessed them in 160 publications. We found that only one-quarter (41 of 160 publications) made code publicly available. Among these 41 studies, three-quarters (30 of 41) analyzed their results statistically, half of them (20 of 41) released their trained model weights, and approximately a third (16 of 41) used an independent cohort for evaluation. Our review is intended for both pathologists interested in deep learning and researchers applying algorithms to computational pathology challenges. We provide a detailed overview of publications with published code in the field, list reusable data handling tools, and provide criteria for reproducibility and reusability.

Histological classification of canine and feline lymphoma using a modular approach based on deep learning and advanced image processing

Histopathological examination of tissue samples is essential for identifying tumor malignancy and the diagnosis of different types of tumor. In the case of lymphoma classification, nuclear size of the neoplastic lymphocytes is one of the key features to differentiate the different subtypes. Based on the combination of artificial intelligence and advanced image processing, we provide a workflow for the classification of lymphoma with regards to their nuclear size (small, intermediate, and large). As the baseline for our workflow testing, we use a Unet++ model trained on histological images of canine lymphoma with individually labeled nuclei. As an alternative to the Unet++, we also used a publicly available pre-trained and unmodified instance segmentation model called Stardist to demonstrate that our modular classification workflow can be combined with different types of segmentation models if they can provide proper nuclei segmentation. Subsequent to nuclear segmentation, we optimize algorithmic parameters for accurate classification of nuclear size using a newly derived reference size and final image classification based on a pathologists-derived ground truth. Our image classification module achieves a classification accuracy of up to 92% on canine lymphoma data. Compared to the accuracy ranging from 66.67 to 84% achieved using measurements provided by three individual pathologists, our algorithm provides a higher accuracy level and reproducible results. Our workflow also demonstrates a high transferability to feline lymphoma, as shown by its accuracy of up to 84.21%, even though our workflow was not optimized for feline lymphoma images. By determining the nuclear size distribution in tumor areas, our workflow can assist pathologists in subtyping lymphoma based on the nuclei size and potentially improve reproducibility. Our proposed approach is modular and comprehensible, thus allowing adaptation for specific tasks and increasing the users' trust in computer-assisted image classification.

Breast Cancer Histopathological Images Segmentation Using Deep Learning

Hospitals generate a significant amount of medical data every day, which constitute a very rich database for research. Today, this database is still not exploitable because to make its valorization possible, the images require an annotation which remains a costly and difficult task. Thus, the use of an unsupervised segmentation method could facilitate the process. In this article, we propose two approaches for the semantic segmentation of breast cancer histopathology images. On the one hand, an autoencoder architecture for unsupervised segmentation is proposed, and on the other hand, an improvement U-Net architecture for supervised segmentation is proposed. We evaluate these models on a public dataset of histological images of breast cancer. In addition, the performance of our segmentation methods is measured using several evaluation metrics such as accuracy, recall, precision and F1 score. The results are competitive with those of other modern methods.

Biomarkers of Tumor Heterogeneity in Glioblastoma Multiforme Cohort of TCGA

Tumor Whole Slide Images (WSI) are often heterogeneous, which hinders the discovery of biomarkers in the presence of confounding clinical factors. In this study, we present a pipeline for identifying biomarkers from the Glioblastoma Multiforme (GBM) cohort of WSIs from TCGA archive. The GBM cohort endures many technical artifacts while the discovery of GBM biomarkers is challenged because "age" is the single most confounding factor for predicting outcomes. The proposed approach relies on interpretable features (e.g., nuclear morphometric indices), effective similarity metrics for heterogeneity analysis, and robust statistics for identifying biomarkers. The pipeline first removes artifacts (e.g., pen marks) and partitions each WSI into patches for nuclear segmentation via an extended U-Net for subsequent quantitative representation. Given the variations in fixation and staining that can artificially modulate hematoxylin optical density (HOD), we extended Navab's Lab method to normalize images and reduce the impact of batch effects. The heterogeneity of each WSI is then represented either as probability density functions (PDF) per patient or as the composition of a dictionary predicted from the entire cohort of WSIs. For PDF- or dictionary-based methods, morphometric subtypes are constructed based on distances computed from optimal transport and linkage analysis or consensus clustering with Euclidean distances, respectively. For each inferred subtype, Kaplan-Meier and/or the Cox regression model are used to regress the survival time. Since age is the single most important confounder for predicting survival in GBM and there is an observed violation of the proportionality assumption in the Cox model, we use both age and age-squared coupled with the Likelihood ratio test and forest plots for evaluating competing statistics. Next, the PDF- and dictionary-based methods are combined to identify biomarkers that are predictive of survival. The combined model has the advantage of integrating global (e.g., cohort scale) and local (e.g., patient scale) attributes of morphometric heterogeneity, coupled with robust statistics, to reveal stable biomarkers. The results indicate that, after normalization of the GBM cohort, mean HOD, eccentricity, and cellularity are predictive of survival. Finally, we also stratified the GBM cohort as a function of EGFR expression and published genomic subtypes to reveal genomic-dependent morphometric biomarkers.

Different approaches to Imaging Mass Cytometry data analysis

Imaging Mass Cytometry (IMC) is a novel, high multiplexing imaging platform capable of simultaneously detecting and visualizing up to 40 different protein targets. It is a strong asset available for in-depth study of histology and pathophysiology of the tissues. Bearing in mind the robustness of this technique and the high spatial context of the data it gives, it is especially valuable in studying the biology of cancer and tumor microenvironment. IMC-derived data are not classical micrographic images, and due to the characteristics of the data obtained using IMC, the image analysis approach, in this case, can diverge to a certain degree from the classical image analysis pipelines. As the number of publications based on the IMC is on the rise, this trend is also followed by an increase in the number of available methodologies designated solely to IMC-derived data analysis. This review has for an aim to give a systematic synopsis of all the available classical image analysis tools and pipelines useful to be employed for IMC data analysis and give an overview of tools intentionally developed solely for this purpose, easing the choice to researchers of selecting the most suitable methodologies for a specific type of analysis desired.

A survey on recent trends in deep learning for nucleus segmentation from histopathology images

Nucleus segmentation is an imperative step in the qualitative study of imaging datasets, considered as an intricate task in histopathology image analysis. Segmenting a nucleus is an important part of diagnosing, staging, and grading cancer, but overlapping regions make it hard to separate and tell apart independent nuclei. Deep Learning is swiftly paving its way in the arena of nucleus segmentation, attracting quite a few researchers with its numerous published research articles indicating its efficacy in the field. This paper presents a systematic survey on nucleus segmentation using deep learning in the last five years (2017-2021), highlighting various segmentation models (U-Net, SCPP-Net, Sharp U-Net, and LiverNet) and exploring their similarities, strengths, datasets utilized, and unfolding research areas.

Clinical Significance of a CD3/CD8-Based Immunoscore in Neuroblastoma Patients Using Digital Pathology

Background

Infiltrating immune cells have been reported as prognostic markers in many cancer types. We aimed to evaluate the prognostic role of tumor-infiltrating lymphocytes, namely CD3+ T cells, CD8+ cytotoxic T cells and memory T cells (CD45RO+), in neuroblastoma.

Patients and Methods

Immunohistochemistry was used to determine the expression of CD3, CD8 and CD45RO in the tumor samples of 244 neuroblastoma patients. We then used digital pathology to calculate the densities of these markers and derived an immunoscore based on such densities.

Results

Densities of CD3+ and CD8+ T cells in tumor were positively associated with the overall survival (OS) and event-free survival (EFS), whereas density of CD45RO+ T cells in tumor was negatively associated with OS but not EFS. An immunoscore with low density of CD3 and CD8 (CD3-CD8-) was indictive of a greater risk of death (hazard ratio 6.39, 95% confidence interval 3.09-13.20) and any event (i.e., relapse at any site, progressive disease, second malignancy, or death) (hazard ratio 4.65, 95% confidence interval 2.73-7.93). Multivariable analysis revealed that the CD3-CD8- immunoscore was an independent prognostic indicator for OS, even after adjusting for other known prognostic indicators.

Conclusions

The new immunoscore based on digital pathology evaluated densities of tumor-infiltrating CD3+ and CD8+ T cells contributes to the prediction of prognosis in neuroblastoma patients.

Convolutional Blur Attention Network for Cell Nuclei Segmentation

Accurately segmented nuclei are important, not only for cancer classification, but also for predicting treatment effectiveness and other biomedical applications. However, the diversity of cell types, various external factors, and illumination conditions make nucleus segmentation a challenging task. In this work, we present a new deep learning-based method for cell nucleus segmentation. The proposed convolutional blur attention (CBA) network consists of downsampling and upsampling procedures. A blur attention module and a blur pooling operation are used to retain the feature salience and avoid noise generation in the downsampling procedure. A pyramid blur pooling (PBP) module is proposed to capture the multi-scale information in the upsampling procedure. The superiority of the proposed method has been compared with a few prior segmentation models, namely U-Net, ENet, SegNet, LinkNet, and Mask RCNN on the 2018 Data Science Bowl (DSB) challenge dataset and the multi-organ nucleus segmentation (MoNuSeg) at MICCAI 2018. The Dice similarity coefficient and some evaluation matrices, such as F1 score, recall, precision, and average Jaccard index (AJI) were used to evaluate the segmentation efficiency of these models. Overall, the proposal method in this paper has the best performance, the AJI indicator on the DSB dataset and MoNuSeg is 0.8429, 0.7985, respectively.

Compound computer vision workflow for efficient and automated immunohistochemical analysis of whole slide images

AIMS

Immunohistochemistry (IHC) assessment of tissue is a central component of the modern pathology workflow, but quantification is challenged by subjective estimates by pathologists or manual steps in semi-automated digital tools. This study integrates various computer vision tools to develop a fully automated workflow for quantifying Ki-67, a standard IHC test used to assess cell proliferation on digital whole slide images (WSIs).

METHODS

We create an automated nuclear segmentation strategy by deploying a Mask R-CNN classifier to recognise and count 3,3'-diaminobenzidine positive and negative nuclei. To further improve automation, we replaced manual selection of regions of interest (ROIs) by aligning Ki-67 WSIs with corresponding H&E-stained sections, using scale-invariant feature transform (SIFT) and a conventional histomorphological convolutional neural networks to define tumour-rich areas for quantification.

RESULTS

The Mask R-CNN was tested on 147 images generated from 34 brain tumour Ki-67 WSIs and showed a high concordance with aggregate pathologists' estimates ([Formula: see text] assessors; [Formula: see text] r=0.9750). Concordance of each assessor's Ki-67 estimates was higher when compared with the Mask R-CNN than between individual assessors (r_avg=0.9322 vs 0.8703; p=0.0213). Coupling the Mask R-CNN with SIFT-CNN workflow demonstrated ROIs can be automatically chosen and partially sampled to improve automation and dramatically decrease computational time (average: 88.55-19.28 min; p<0.0001).

CONCLUSIONS

We show how innovations in computer vision can be serially compounded to automate and improve implementation in clinical workflows. Generalisation of this approach to other ancillary studies has significant implications for computational pathology.

Automated detection and staging of malaria parasites from cytological smears using convolutional neural networks

Microscopic examination of blood smears remains the gold standard for laboratory inspection and diagnosis of malaria. Smear inspection is, however, time-consuming and dependent on trained microscopists with results varying in accuracy. We sought to develop an automated image analysis method to improve accuracy and standardization of smear inspection that retains capacity for expert confirmation and image archiving. Here, we present a machine learning method that achieves red blood cell (RBC) detection, differentiation between infected/uninfected cells, and parasite life stage categorization from unprocessed, heterogeneous smear images. Based on a pretrained Faster Region-Based Convolutional Neural Networks (R-CNN) model for RBC detection, our model performs accurately, with an average precision of 0.99 at an intersection-over-union threshold of 0.5. Application of a residual neural network-50 model to infected cells also performs accurately, with an area under the receiver operating characteristic curve of 0.98. Finally, combining our method with a regression model successfully recapitulates intraerythrocytic developmental cycle with accurate lifecycle stage categorization. Combined with a mobile-friendly web-based interface, called PlasmoCount, our method permits rapid navigation through and review of results for quality assurance. By standardizing assessment of Giemsa smears, our method markedly improves inspection reproducibility and presents a realistic route to both routine lab and future field-based automated malaria diagnosis.

Deep Learning Approaches and Applications in Toxicologic Histopathology: Current Status and Future Perspectives

Whole slide imaging enables the use of a wide array of digital image analysis tools that are revolutionizing pathology. Recent advances in digital pathology and deep convolutional neural networks have created an enormous opportunity to improve workflow efficiency, provide more quantitative, objective, and consistent assessments of pathology datasets, and develop decision support systems. Such innovations are already making their way into clinical practice. However, the progress of machine learning - in particular, deep learning (DL) - has been rather slower in nonclinical toxicology studies. Histopathology data from toxicology studies are critical during the drug development process that is required by regulatory bodies to assess drug-related toxicity in laboratory animals and its impact on human safety in clinical trials. Due to the high volume of slides routinely evaluated, low-throughput, or narrowly performing DL methods that may work well in small-scale diagnostic studies or for the identification of a single abnormality are tedious and impractical for toxicologic pathology. Furthermore, regulatory requirements around good laboratory practice are a major hurdle for the adoption of DL in toxicologic pathology. This paper reviews the major DL concepts, emerging applications, and examples of DL in toxicologic pathology image analysis. We end with a discussion of specific challenges and directions for future research.

AI in spotting high-risk characteristics of medical imaging and molecular pathology

Medical imaging provides a comprehensive perspective and rich information for disease diagnosis. Combined with artificial intelligence technology, medical imaging can be further mined for detailed pathological information. Many studies have shown that the macroscopic imaging characteristics of tumors are closely related to microscopic gene, protein and molecular changes. In order to explore the function of artificial intelligence algorithms in in-depth analysis of medical imaging information, this paper reviews the articles published in recent years from three perspectives: medical imaging analysis method, clinical applications and the development of medical imaging in the direction of pathological molecular prediction. We believe that AI-aided medical imaging analysis will be extensively contributing to precise and efficient clinical decision.

Breast histopathological image analysis using image processing techniques for diagnostic puposes: A methodological review

Breast cancer in women is the second most common cancer worldwide. Early detection of breast cancer can reduce the risk of human life. Non-invasive techniques such as mammograms and ultrasound imaging are popularly used to detect the tumour. However, histopathological analysis is necessary to determine the malignancy of the tumour as it analyses the image at the cellular level. Manual analysis of these slides is time consuming, tedious, subjective and are susceptible to human errors. Also, at times the interpretation of these images are inconsistent between laboratories. Hence, a Computer-Aided Diagnostic system that can act as a decision support system is need of the hour. Moreover, recent developments in computational power and memory capacity led to the application of computer tools and medical image processing techniques to process and analyze breast cancer histopathological images. This review paper summarizes various traditional and deep learning based methods developed to analyze breast cancer histopathological images. Initially, the characteristics of breast cancer histopathological images are discussed. A detailed discussion on the various potential regions of interest is presented which is crucial for the development of Computer-Aided Diagnostic systems. We summarize the recent trends and choices made during the selection of medical image processing techniques. Finally, a detailed discussion on the various challenges involved in the analysis of BCHI is presented along with the future scope.

Skin Melanoma Detection in Microscopic Images Using HMM-Based Asymmetric Analysis and Expectation Maximization

Melanoma is one of the deadliest types of skin cancer with increasing incidence. The most definitive diagnosis method is the histopathological examination of the tissue sample. In this paper, a melanoma detection algorithm is proposed based on decision-level fusion and a Hidden Markov Model (HMM), whose parameters are optimized using Expectation Maximization (EM) and asymmetric analysis. The texture heterogeneity of the samples is determined using asymmetric analysis. A fusion-based HMM classifier trained using EM is introduced. For this purpose, a novel texture feature is extracted based on two local binary patterns, namely local difference pattern (LDP) and statistical histogram features of the microscopic image. Extensive experiments demonstrate that the proposed melanoma detection algorithm yields a total error of less than 0.04%.

Efficient deep learning architecture with dimension-wise pyramid pooling for nuclei segmentation of histopathology images

Image segmentation remains to be one of the most vital tasks in the area of computer vision and more so in the case of medical image processing. Image segmentation quality is the main metric that is often considered with memory and computation efficiency overlooked, limiting the use of power hungry models for practical use. In this paper, we propose a novel framework (Kidney-SegNet) that combines the effectiveness of an attention based encoder-decoder architecture with atrous spatial pyramid pooling with highly efficient dimension-wise convolutions. The segmentation results of the proposed Kidney-SegNet architecture have been shown to outperform existing state-of-the-art deep learning methods by evaluating them on two publicly available kidney and TNBC breast H&E stained histopathology image datasets. Further, our simulation experiments also reveal that the computational complexity and memory requirement of our proposed architecture is very efficient compared to existing deep learning state-of-the-art methods for the task of nuclei segmentation of H&E stained histopathology images. The source code of our implementation will be available at https://github.com/Aaatresh/Kidney-SegNet.

Machine Learning Algorithms, Applied to Intact Islets of Langerhans, Demonstrate Significantly Enhanced Insulin Staining at the Capillary Interface of Human Pancreatic β Cells

Pancreatic β cells secrete the hormone insulin into the bloodstream and are critical in the control of blood glucose concentrations. β cells are clustered in the micro-organs of the islets of Langerhans, which have a rich capillary network. Recent work has highlighted the intimate spatial connections between β cells and these capillaries, which lead to the targeting of insulin secretion to the region where the β cells contact the capillary basement membrane. In addition, β cells orientate with respect to the capillary contact point and many proteins are differentially distributed at the capillary interface compared with the rest of the cell. Here, we set out to develop an automated image analysis approach to identify individual β cells within intact islets and to determine if the distribution of insulin across the cells was polarised. Our results show that a U-Net machine learning algorithm correctly identified β cells and their orientation with respect to the capillaries. Using this information, we then quantified insulin distribution across the β cells to show enrichment at the capillary interface. We conclude that machine learning is a useful analytical tool to interrogate large image datasets and analyse sub-cellular organisation.

Automatic detection and characterization of quantitative phase images of thalassemic red blood cells using a mask region-based convolutional neural network

SIGNIFICANCE

Label-free quantitative phase imaging is a promising technique for the automatic detection of abnormal red blood cells (RBCs) in real time. Although deep-learning techniques can accurately detect abnormal RBCs from quantitative phase images efficiently, their applications in diagnostic testing are limited by the lack of transparency. More interpretable results such as morphological and biochemical characteristics of individual RBCs are highly desirable.

AIM

An end-to-end deep-learning model was developed to efficiently discriminate thalassemic RBCs (tRBCs) from healthy RBCs (hRBCs) in quantitative phase images and segment RBCs for single-cell characterization.

APPROACH

Two-dimensional quantitative phase images of hRBCs and tRBCs were acquired using digital holographic microscopy. A mask region-based convolutional neural network (Mask R-CNN) model was trained to discriminate tRBCs and segment individual RBCs. Characterization of tRBCs was achieved utilizing SHapley Additive exPlanation analysis and canonical correlation analysis on automatically segmented RBC phase images.

RESULTS

The implemented model achieved 97.8% accuracy in detecting tRBCs. Phase-shift statistics showed the highest influence on the correct classification of tRBCs. Associations between the phase-shift features and three-dimensional morphological features were revealed.

CONCLUSIONS

The implemented Mask R-CNN model accurately identified tRBCs and segmented RBCs to provide single-RBC characterization, which has the potential to aid clinical decision-making.

Mathematical Modelling of Ground Truth Image for 3D Microscopic Objects Using Cascade of Convolutional Neural Networks Optimized with Parameters’ Combinations Generators

Mathematical modelling to compute ground truth from 3D images is an area of research that can strongly benefit from machine learning methods. Deep neural networks (DNNs) are state-of-the-art methods design for solving these kinds of difficulties. Convolutional neural networks (CNNs), as one class of DNNs, can overcome special requirements of quantitative analysis especially when image segmentation is needed. This article presents a system that uses a cascade of CNNs with symmetric blocks of layers in chain, dedicated to 3D image segmentation from microscopic images of 3D nuclei. The system is designed through eight experiments that differ in following aspects: number of training slices and 3D samples for training, usage of pre-trained CNNs and number of slices and 3D samples for validation. CNNs parameters are optimized using linear, brute force, and random combinatorics, followed by voter and median operations. Data augmentation techniques such as reflection, translation and rotation are used in order to produce sufficient training set for CNNs. Optimal CNN parameters are reached by defining 11 standard and two proposed metrics. Finally, benchmarking demonstrates that CNNs improve segmentation accuracy, reliability and increased annotation accuracy, confirming the relevance of CNNs to generate high-throughput mathematical ground truth 3D images.