Segmentation methods of H&E-stained histological images of lymphoma: A review

Review of computer-assisted diagnosis model to classify follicular lymphoma histology

The field of image processing is experiencing significant advancements to support professionals in analyzing histological images obtained from biopsies. The primary objective is to enhance the process of diagnosis and prognostic evaluations. Various forms of cancer can be diagnosed by employing different segmentation techniques followed by postprocessing approaches that can identify distinct neoplastic areas. Using computer approaches facilitates a more objective and efficient study of experts. The progressive advancement of histological image analysis holds significant importance in modern medicine. This paper provides an overview of the current advances in segmentation and classification approaches for images of follicular lymphoma. This research analyzes the primary image processing techniques utilized in the various stages of preprocessing, segmentation of the region of interest, classification, and postprocessing as described in the existing literature. The study also examines the strengths and weaknesses associated with these approaches. Additionally, this study encompasses an examination of validation procedures and an exploration of prospective future research roads in the segmentation of neoplasias.

HoLy-Net: Segmentation of histological images of diffuse large B-cell lymphoma

Over the last years, there has been large progress in automated segmentation and classification methods in histological whole slide images (WSIs) stained with hematoxylin and eosin (H&E). Current state-of-the-art (SOTA) techniques are based on diverse datasets of H&E-stained WSIs of different types of predominantly solid cancer. However, there is a scarcity of methods and datasets enabling segmentation of tumors of the lymphatic system (lymphomas). Here, we propose a solution for segmentation of diffuse large B-cell lymphoma (DLBCL), the most common non-Hodgkin's lymphoma. Our method applies to both H&E-stained slides and to a broad range of markers stained with immunohistochemistry (IHC). While IHC staining is an important tool in cancer diagnosis and treatment decisions, there are few automated segmentation and classification methods for IHC-stained WSIs. To address the challenges of nuclei segmentation in H&E- and IHC-stained DLBCL images, we propose HoLy-Net - a HoVer-Net-based deep learning model for lymphoma segmentation. We train two different models, one for segmenting H&E- and one for IHC-stained images and compare the test results with the SOTA methods as well as with the original version of HoVer-Net. Subsequently, we segment patient WSIs and perform single cell-level analysis of different cell types to identify patient-specific tumor characteristics such as high level of immune infiltration. Our method outperforms general-purpose segmentation methods for H&E staining in lymphoma WSIs (with an F1 score of 0.899) and is also a unique automated method for IHC slide segmentation (with an F1 score of 0.913). With our solution, we provide a new dataset we denote LyNSeC (lymphoma nuclear segmentation and classification) containing 73,931 annotated cell nuclei from H&E and 87,316 from IHC slides. Our method and dataset open up new avenues for quantitative, large-scale studies of morphology and microenvironment of lymphomas overlooked by the current automated segmentation methods.

Towards Realistic 3D Models of Tumor Vascular Networks

For reliable in silico or in vitro investigations in, for example, biosensing and drug delivery applications, accurate models of tumor vascular networks down to the capillary size are essential. Compared to images acquired with conventional medical imaging techniques, digitalized histological tumor slices have a higher resolution, enabling the delineation of capillaries. Volume rendering procedures can then be used to generate a 3D model. However, the preparation of such slices leads to misalignments in relative slice orientation between consecutive slices. Thus, image registration algorithms are necessary to re-align the slices. Here, we present an algorithm for the registration and reconstruction of a vascular network from histologic slices applied to 169 tumor slices. The registration includes two steps. First, consecutive images are incrementally pre-aligned using feature- and area-based transformations. Second, using the previous transformations, parallel registration for all images is enabled. Combining intensity- and color-based thresholds along with heuristic analysis, vascular structures are segmented. A 3D interpolation technique is used for volume rendering. This results in a 3D vascular network with approximately 400-450 vessels with diameters down to 25-30 µm. A delineation of vessel structures with close distance was limited in areas of high structural density. Improvement can be achieved by using images with higher resolution and or machine learning techniques.

A Deep Multi-Label Segmentation Network For Eosinophilic Esophagitis Whole Slide Biopsy Diagnostics

Eosinophilic esophagitis (EoE) is an allergic inflammatory condition of the esophagus associated with elevated numbers of eosinophils. Disease diagnosis and monitoring require determining the concentration of eosinophils in esophageal biopsies, a time-consuming, tedious and somewhat subjective task currently performed by pathologists. Here, we developed a machine learning pipeline to identify, quantitate and diagnose EoE patients' at the whole slide image level. We propose a platform that combines multi-label segmentation deep network decision support system with dynamics convolution that is able to process whole biopsy slide. Our network is able to segment both intact and not-intact eosinophils with a mean intersection over union (mIoU) of 0.93. This segmentation enables the local quantification of intact eosinophils with a mean absolute error of 0.611 eosinophils. We examined a cohort of 1066 whole slide images from 400 patients derived from multiple institutions. Using this set, our model achieved a global accuracy of 94.75%, sensitivity of 94.13%, and specificity of 95.25% in reporting EoE disease activity. Our work provides state-of-the-art performances on the largest EoE cohort to date, and successfully addresses two of the main challenges in EoE diagnostics and digital pathology, the need to detect several types of small features simultaneously, and the ability to analyze whole slides efficiently. Our results pave the way for an automated diagnosis of EoE and can be utilized for other conditions with similar challenges.

Development of Planar Illumination Strategies for Solving Mysteries in the Sub-Cellular Realm

Optical microscopy has vastly expanded the frontiers of structural and functional biology, due to the non-invasive probing of dynamic volumes in vivo. However, traditional widefield microscopy illuminating the entire field of view (FOV) is adversely affected by out-of-focus light scatter. Consequently, standard upright or inverted microscopes are inept in sampling diffraction-limited volumes smaller than the optical system's point spread function (PSF). Over the last few decades, several planar and structured (sinusoidal) illumination modalities have offered unprecedented access to sub-cellular organelles and 4D (3D + time) image acquisition. Furthermore, these optical sectioning systems remain unaffected by the size of biological samples, providing high signal-to-noise (SNR) ratios for objective lenses (OLs) with long working distances (WDs). This review aims to guide biologists regarding planar illumination strategies, capable of harnessing sub-micron spatial resolution with a millimeter depth of penetration.

Whale Optimizer-Based Clustering for Breast Histopathology Image Segmentation

Breast histopathology image segmentation is a complex task due to indiscernibly correlated and noisy regions of interest. Breast histopathological images are composed of different types of cells. Some of these cells can be harmful for humans due to the presence of cancer. Under such circumstances, many segmentation techniques for automatic detection of cancer cells have been proposed considering clustering schemes. However, such clustering methodologies are sensitive to initial cluster centers, which promote false-positive solutions. This paper presents the use of the Whale Optimization Algorithm (WOA) for proper clustering segmentation of breast histopathological images to overcome clustering issues. Also, a rigorous comparative study is conducted among the proposed approach and several state-of-art Nature-Inspired Optimization Algorithms (NIOAs) and traditional clustering techniques. The numerical results indicate that the proposed approach outperforms the other utilized clustering methods in terms of precision, robustness, and quality of the segmented outputs.

Harnessing artificial intelligence to infer novel spatial biomarkers for the diagnosis of eosinophilic esophagitis

Eosinophilic esophagitis (EoE) is a chronic allergic inflammatory condition of the esophagus associated with elevated esophageal eosinophils. Second only to gastroesophageal reflux disease, EoE is one of the leading causes of chronic refractory dysphagia in adults and children. EoE is a clinicopathologic disorder and the histological portion of the diagnosis requires enumerating the density of esophageal eosinophils in esophageal biopsies, and evaluating additional features such as basal zone hyperplasia is helpful. However, this task requires time-consuming, somewhat subjective manual analysis, thus reducing the ability to process the complex tissue structure and infer its relationship with the patient's clinical status. Previous artificial intelligence (AI) approaches that aimed to improve histology-based diagnosis focused on recapitulating identification and quantification of the area of maximal eosinophil density, the gold standard manual metric for determining EoE disease activity. However, this metric does not account for the distribution of eosinophils or other histological features, over the whole slide image. Here, we developed an artificial intelligence platform that infers local and spatial biomarkers based on semantic segmentation of intact eosinophils and basal zone distributions. Besides the maximal density of eosinophils [referred to as Peak Eosinophil Count (PEC)] and a maximal basal zone fraction, we identify the value of two additional metrics that reflect the distribution of eosinophils and basal zone fractions. This approach enables a decision support system that predicts EoE activity and potentially classifies the histological severity of EoE patients. We utilized a cohort that includes 1,066 biopsy slides from 400 subjects to validate the system's performance and achieved a histological severity classification accuracy of 86.70%, sensitivity of 84.50%, and specificity of 90.09%. Our approach highlights the importance of systematically analyzing the distribution of biopsy features over the entire slide and paves the way toward a personalized decision support system that will assist not only in counting cells but can also potentially improve diagnosis and provide treatment prediction.

Cluster Analysis of Cell Nuclei in H&E-Stained Histological Sections of Prostate Cancer and Classification Based on Traditional and Modern Artificial Intelligence Techniques

Biomarker identification is very important to differentiate the grade groups in the histopathological sections of prostate cancer (PCa). Assessing the cluster of cell nuclei is essential for pathological investigation. In this study, we present a computer-based method for cluster analyses of cell nuclei and performed traditional (i.e., unsupervised method) and modern (i.e., supervised method) artificial intelligence (AI) techniques for distinguishing the grade groups of PCa. Two datasets on PCa were collected to carry out this research. Histopathology samples were obtained from whole slides stained with hematoxylin and eosin (H&E). In this research, state-of-the-art approaches were proposed for color normalization, cell nuclei segmentation, feature selection, and classification. A traditional minimum spanning tree (MST) algorithm was employed to identify the clusters and better capture the proliferation and community structure of cell nuclei. K-medoids clustering and stacked ensemble machine learning (ML) approaches were used to perform traditional and modern AI-based classification. The binary and multiclass classification was derived to compare the model quality and results between the grades of PCa. Furthermore, a comparative analysis was carried out between traditional and modern AI techniques using different performance metrics (i.e., statistical parameters). Cluster features of the cell nuclei can be useful information for cancer grading. However, further validation of cluster analysis is required to accomplish astounding classification results.

Machine Learning Methods for Histopathological Image Analysis: A Review

Histopathological images (HIs) are the gold standard for evaluating some types of tumors for cancer diagnosis. The analysis of such images is time and resource-consuming and very challenging even for experienced pathologists, resulting in inter-observer and intra-observer disagreements. One of the ways of accelerating such an analysis is to use computer-aided diagnosis (CAD) systems. This paper presents a review on machine learning methods for histopathological image analysis, including shallow and deep learning methods. We also cover the most common tasks in HI analysis, such as segmentation and feature extraction. Besides, we present a list of publicly available and private datasets that have been used in HI research.

Efficient Regularization Framework for Histopathological Image Classification Using Convolutional Neural Networks

Deep learning methods are characterized by their capacity to learn data representation compared to the traditional machine learning algorithms. However, these methods are prone to overfitting on small volumes of data. The objective of this research is to overcome this limitation by improving the generalization in the proposed deep learning framework based on various techniques: data augmentation, small models, optimizer selection, and ensemble learning. For ensembling, the authors used selected models from different checkpoints and both voting and unweighted average methods for combination. The experimental study on the lymphomas histopathological dataset highlights the efficiency of the MobileNet2 network combined with the stochastic gradient descent (SGD) optimizer in terms of generalization. The best results have been achieved by the combination of the best three checkpoint models (98.67% of accuracy). These findings provide important insights into the efficiency of the checkpoint ensemble learning method for histopathological image classification.

Pixel precise unsupervised detection of viral particle proliferation in cellular imaging data

Cellular and molecular imaging techniques and models have been developed to characterize single stages of viral proliferation after focal infection of cells in vitro. The fast and automatic classification of cell imaging data may prove helpful prior to any further comparison of representative experimental data to mathematical models of viral propagation in host cells. Here, we use computer generated images drawn from a reproduction of an imaging model from a previously published study of experimentally obtained cell imaging data representing progressive viral particle proliferation in host cell monolayers. Inspired by experimental time-based imaging data, here in this study viral particle increase in time is simulated by a one-by-one increase, across images, in black or gray single pixels representing dead or partially infected cells, and hypothetical remission by a one-by-one increase in white pixels coding for living cells in the original image model. The image simulations are submitted to unsupervised learning by a Self-Organizing Map (SOM) and the Quantization Error in the SOM output (SOM-QE) is used for automatic classification of the image simulations as a function of the represented extent of viral particle proliferation or cell recovery. Unsupervised classification by SOM-QE of 160 model images, each with more than three million pixels, is shown to provide a statistically reliable, pixel precise, and fast classification model that outperforms human computer-assisted image classification by RGB image mean computation. The automatic classification procedure proposed here provides a powerful approach to understand finely tuned mechanisms in the infection and proliferation of virus in cell lines in vitro or other cells.

•

Automatic classification of cell imaging data for in vitro viral propagation in host cells.

•

Economical in terms of computation times for rapid change/no change detection in large image data prior to human decision making.

•

Simulating viral proliferation/cell recovery by progressive and selective single pixel changes in contrast polarity and/or intensity.

Enhancing Multi-tissue and Multi-scale Cell Nuclei Segmentation with Deep Metric Learning

(1) Background: The segmentation of cell nuclei is an essential task in a wide range of biomedical studies and clinical practices. The full automation of this process remains a challenge due to intra- and internuclear variations across a wide range of tissue morphologies, differences in staining protocols and imaging procedures. (2) Methods: A deep learning model with metric embeddings such as contrastive loss and triplet loss with semi-hard negative mining is proposed in order to accurately segment cell nuclei in a diverse set of microscopy images. The effectiveness of the proposed model was tested on a large-scale multi-tissue collection of microscopy image sets. (3) Results: The use of deep metric learning increased the overall segmentation prediction by 3.12% in the average value of Dice similarity coefficients as compared to no metric learning. In particular, the largest gain was observed for segmenting cell nuclei in H&E -stained images when deep learning network and triplet loss with semi-hard negative mining were considered for the task. (4) Conclusion: We conclude that deep metric learning gives an additional boost to the overall learning process and consequently improves the segmentation performance. Notably, the improvement ranges approximately between 0.13% and 22.31% for different types of images in the terms of Dice coefficients when compared to no metric deep learning.

ki67 nuclei detection and ki67-index estimation: a novel automatic approach based on human vision modeling

BACKGROUND

The protein ki67 (pki67) is a marker of tumor aggressiveness, and its expression has been proven to be useful in the prognostic and predictive evaluation of several types of tumors. To numerically quantify the pki67 presence in cancerous tissue areas, pathologists generally analyze histochemical images to count the number of tumor nuclei marked for pki67. This allows estimating the ki67-index, that is the percentage of tumor nuclei positive for pki67 over all the tumor nuclei. Given the high image resolution and dimensions, its estimation by expert clinicians is particularly laborious and time consuming. Though automatic cell counting techniques have been presented so far, the problem is still open.

RESULTS

In this paper we present a novel automatic approach for the estimations of the ki67-index. The method starts by exploiting the STRESS algorithm to produce a color enhanced image where all pixels belonging to nuclei are easily identified by thresholding, and then separated into positive (i.e. pixels belonging to nuclei marked for pki67) and negative by a binary classification tree. Next, positive and negative nuclei pixels are processed separately by two multiscale procedures identifying isolated nuclei and separating adjoining nuclei. The multiscale procedures exploit two Bayesian classification trees to recognize positive and negative nuclei-shaped regions.

CONCLUSIONS

The evaluation of the computed results, both through experts' visual assessments and through the comparison of the computed indexes with those of experts, proved that the prototype is promising, so that experts believe in its potential as a tool to be exploited in the clinical practice as a valid aid for clinicians estimating the ki67-index. The MATLAB source code is open source for research purposes.

Resolution-agnostic tissue segmentation in whole-slide histopathology images with convolutional neural networks

Modern pathology diagnostics is being driven toward large scale digitization of microscopic tissue sections. A prerequisite for its safe implementation is the guarantee that all tissue present on a glass slide can also be found back in the digital image. Whole-slide scanners perform a tissue segmentation in a low resolution overview image to prevent inefficient high-resolution scanning of empty background areas. However, currently applied algorithms can fail in detecting all tissue regions.

In this study, we developed convolutional neural networks to distinguish tissue from background. We collected 100 whole-slide images of 10 tissue samples—staining categories from five medical centers for development and testing. Additionally, eight more images of eight unfamiliar categories were collected for testing only. We compared our fully-convolutional neural networks to three traditional methods on a range of resolution levels using Dice score and sensitivity.

We also tested whether a single neural network can perform equivalently to multiple networks, each specialized in a single resolution. Overall, our solutions outperformed the traditional methods on all the tested resolutions. The resolution-agnostic network achieved average Dice scores between 0.97 and 0.98 across the tested resolution levels, only 0.0069 less than the resolution-specific networks. Finally, its excellent generalization performance was demonstrated by achieving averages of 0.98 Dice score and 0.97 sensitivity on the eight unfamiliar images. A future study should test this network prospectively.

An automated method for segmentation and quantification of blood vessels in histology images

Alzheimer's disease (AD) is a chronic neuro-degenerative disease that adversely affect many people on a global scale. Despite different diagnostic and therapeutic treatment, there is no cure for AD. The brain is one of the most complex organ and researchers are still trying to understand so as to find a cure.

OBJECTIVE

To complement the efforts of clinical researchers engaged in research in alzheimer's disease, accurate segmentation and quantification of blood vessels in brain images is required.

METHOD

For robust segmentation of blood vessels even in the presence of colour variation, we introduce a fully automated morphological tool that can extract and quantify vessels from haematoxylin and diaminobenzidine stained histology brain image. The method, exploits saturation channel of stained image slides, ISODATA threshold method is applied to obtain a binary image. This helps in eliminating background and remaining with only blood vessels. A one-stage procedure that includes eliminating small artefacts is performed on the binary mask. The intensity of the image is transformed. Joining is performed to deal with fragmentation of intact blood vessels on the images, and artefactual appearance of the blood vessel structures. The artefactual fragments based on measured incoherence with neighbouring tissue are removed. The vessels are then labelled to facilitate quantification. Morphometric measurements are used during the vessel quantification assess both vessels with lumen and vessels without lumen. We have quantified the diameter of blood vessels.

RESULTS

The image processing technique is developed in close collaboration with neuroscientist experts to help clinician. We have evaluated our proposed approach qualitatively. The method was validated against their manual quantification results. Qualitative results show that the method can indeed segment the blood vessels in the presence of colour variations and artefacts. The quantitative method produces fairly better results.

In Vivo Anticancer Activity of a Rhenium(I) Tricarbonyl Complex

The rhenium(I) complex fac-[Re(CO)₃(2,9-dimethyl-1,10-phenanthroline)(OH₂)]⁺ (1) was previously shown to exhibit potent in vitro anticancer activity in a manner distinct from conventional platinum-based drugs (J. Am. Chem. Soc. 2017, 139, 14302-14314). In this study, we report further efforts to explore its aqueous speciation and antitumor activity. The cellular uptake of 1 was measured in A2780 and cisplatin-resistant A2780CP70 ovarian cancer cells by inductively coupled plasma mass spectrometry, revealing similar uptake efficiency in both cell lines. High accumulation in the mitochondria was observed, contradicting prior fluorescence microscopy studies. The luminescence of 1 is highly dependent on pH and coordination environment, making fluorescence microscopy somewhat unreliable for determining compound localization. The in vivo anticancer activity of 1 was evaluated in mice bearing patient-derived ovarian cancer tumor xenografts. These studies conclusively show that 1 is capable of inhibiting tumor growth, providing further credibility for the use of these compounds as anticancer agents.

Proteome Imaging: From Classic to Modern Mass Spectrometry-Based Molecular Histology

In order to overcome the limitations of classic imaging in Histology during the actually era of multiomics, the multi-color "molecular microscope" by its emerging "molecular pictures" offers quantitative and spatial information about thousands of molecular profiles without labeling of potential targets. Healthy and diseased human tissues, as well as those of diverse invertebrate and vertebrate animal models, including genetically engineered species and cultured cells, can be easily analyzed by histology-directed MALDI imaging mass spectrometry. The aims of this review are to discuss a range of proteomic information emerging from MALDI mass spectrometry imaging comparative to classic histology, histochemistry and immunohistochemistry, with applications in biology and medicine, concerning the detection and distribution of structural proteins and biological active molecules, such as antimicrobial peptides and proteins, allergens, neurotransmitters and hormones, enzymes, growth factors, toxins and others. The molecular imaging is very well suited for discovery and validation of candidate protein biomarkers in neuroproteomics, oncoproteomics, aging and age-related diseases, parasitoproteomics, forensic, and ecotoxicology. Additionally, in situ proteome imaging may help to elucidate the physiological and pathological mechanisms involved in developmental biology, reproductive research, amyloidogenesis, tumorigenesis, wound healing, neural network regeneration, matrix mineralization, apoptosis and oxidative stress, pain tolerance, cell cycle and transformation under oncogenic stress, tumor heterogeneity, behavior and aggressiveness, drugs bioaccumulation and biotransformation, organism's reaction against environmental penetrating xenobiotics, immune signaling, assessment of integrity and functionality of tissue barriers, behavioral biology, and molecular origins of diseases. MALDI MSI is certainly a valuable tool for personalized medicine and "Eco-Evo-Devo" integrative biology in the current context of global environmental challenges.

A novel computational method for automatic segmentation, quantification and comparative analysis of immunohistochemically labeled tissue sections

Background

In the clinical practice, the objective quantification of histological results is essential not only to define objective and well-established protocols for diagnosis, treatment, and assessment, but also to ameliorate disease comprehension.

Software

The software MIAQuant_Learn presented in this work segments, quantifies and analyzes markers in histochemical and immunohistochemical images obtained by different biological procedures and imaging tools. MIAQuant_Learn employs supervised learning techniques to customize the marker segmentation process with respect to any marker color appearance. Our software expresses the location of the segmented markers with respect to regions of interest by mean-distance histograms, which are numerically compared by measuring their intersection. When contiguous tissue sections stained by different markers are available, MIAQuant_Learn aligns them and overlaps the segmented markers in a unique image enabling a visual comparative analysis of the spatial distribution of each marker (markers’ relative location). Additionally, it computes novel measures of markers’ co-existence in tissue volumes depending on their density.

Conclusions

Applications of MIAQuant_Learn in clinical research studies have proven its effectiveness as a fast and efficient tool for the automatic extraction, quantification and analysis of histological sections. It is robust with respect to several deficits caused by image acquisition systems and produces objective and reproducible results. Thanks to its flexibility, MIAQuant_Learn represents an important tool to be exploited in basic research where needs are constantly changing.