Automated segmentation of the melanocytes in skin histopathological images

Deep Hyperspherical Clustering for Skin Lesion Medical Image Segmentation

Diagnosis of skin lesions based on imaging techniques remains a challenging task because data (knowledge) uncertainty may reduce accuracy and lead to imprecise results. This paper investigates a new deep hyperspherical clustering (DHC) method for skin lesion medical image segmentation by combining deep convolutional neural networks and the theory of belief functions (TBF). The proposed DHC aims to eliminate the dependence on labeled data, improve segmentation performance, and characterize the imprecision caused by data (knowledge) uncertainty. First, the SLIC superpixel algorithm is employed to group the image into multiple meaningful superpixels, aiming to maximize the use of context without destroying the boundary information. Second, an autoencoder network is designed to transform the superpixels' information into potential features. Third, a hypersphere loss is developed to train the autoencoder network. The loss is defined to map the input to a pair of hyperspheres so that the network can perceive tiny differences. Finally, the result is redistributed to characterize the imprecision caused by data (knowledge) uncertainty based on the TBF. The proposed DHC method can well characterize the imprecision between skin lesions and non-lesions, which is particularly important for the medical procedures. A series of experiments on four dermoscopic benchmark datasets demonstrate that the proposed DHC yields better segmentation performance, increasing the accuracy of the predictions while can perceive imprecise regions compared to other typical methods.

Automated analysis of whole slide digital skin biopsy images

A rapidly increasing rate of melanoma diagnosis has been noted over the past three decades, and nearly 1 in 4 skin biopsies are diagnosed as melanocytic lesions. The gold standard for diagnosis of melanoma is the histopathological examination by a pathologist to analyze biopsy material at both the cellular and structural levels. A pathologist's diagnosis is often subjective and prone to variability, while deep learning image analysis methods may improve and complement current diagnostic and prognostic capabilities. Mitoses are important entities when reviewing skin biopsy cases as their presence carries prognostic information; thus, their precise detection is an important factor for clinical care. In addition, semantic segmentation of clinically important structures in skin biopsies might help the diagnosis pipeline with an accurate classification. We aim to provide prognostic and diagnostic information on skin biopsy images, including the detection of cellular level entities, segmentation of clinically important tissue structures, and other important factors toward the accurate diagnosis of skin biopsy images. This paper is an overview of our work on analysis of digital whole slide skin biopsy images, including mitotic figure (mitosis) detection, semantic segmentation, diagnosis, and analysis of pathologists' viewing patterns, and with new work on melanocyte detection. Deep learning has been applied to our methods for all the detection, segmentation, and diagnosis work. In our studies, deep learning is proven superior to prior approaches to skin biopsy analysis. Our work on analysis of pathologists' viewing patterns is the only such work in the skin biopsy literature. Our work covers the whole spectrum from low-level entities through diagnosis and understanding what pathologists do in performing their diagnoses.

Entropy and Gaussian Filter-Based Adaptive Active Contour for Segmentation of Skin Lesions

Malignant melanoma is considered one of the deadliest skin diseases if ignored without treatment. The mortality rate caused by melanoma is more than two times that of other skin malignancy diseases. These facts encourage computer scientists to find automated methods to discover skin cancers. Nowadays, the analysis of skin images is widely used by assistant physicians to discover the first stage of the disease automatically. One of the challenges the computer science researchers faced when developing such a system is the un-clarity of the existing images, such as noise like shadows, low contrast, hairs, and specular reflections, which complicates detecting the skin lesions in that images. This paper proposes the solution to the problem mentioned earlier using the active contour method. Still, seed selection in the dynamic contour method has the main drawback of where it should start the segmentation process. This paper uses Gaussian filter-based maximum entropy and morphological processing methods to find automatic seed points for active contour. By incorporating this, it can segment the lesion from dermoscopic images automatically. Our proposed methodology tested quantitative and qualitative measures on standard dataset dermis and used to test the proposed method's reliability which shows encouraging results.

A review: The detection of cancer cells in histopathology based on machine vision

Machine vision is being employed in defect detection, size measurement, pattern recognition, image fusion, target tracking and 3D reconstruction. Traditional cancer detection methods are dominated by manual detection, which wastes time and manpower, and heavily relies on the pathologists' skill and work experience. Therefore, these manual detection approaches are not convenient for the inheritance of domain knowledge, and are not suitable for the rapid development of medical care in the future. The emergence of machine vision can iteratively update and learn the domain knowledge of cancer cell pathology detection to achieve automated, high-precision, and consistent detection. Consequently, this paper reviews the use of machine vision to detect cancer cells in histopathology images, as well as the benefits and drawbacks of various detection approaches. First, we review the application of image preprocessing and image segmentation in histopathology for the detection of cancer cells, and compare the benefits and drawbacks of different algorithms. Secondly, for the characteristics of histopathological cancer cell images, the research progress of shape, color and texture features and other methods is mainly reviewed. Furthermore, for the classification methods of histopathological cancer cell images, the benefits and drawbacks of traditional machine vision approaches and deep learning methods are compared and analyzed. Finally, the above research is discussed and forecasted, with the expected future development tendency serving as a guide for future research.

Detection of malignant melanoma in H&E-stained images using deep learning techniques

Histopathological images are widely used to diagnose diseases including skin cancer. As digital histopathological images are typically of very large size, in the order of several billion pixels, automated identification of all abnormal cell nuclei and their distribution within multiple tissue sections would assist rapid comprehensive diagnostic assessment. In this paper, we propose a deep learning-based technique to segment the melanoma regions in Hematoxylin and Eosin (H&E) stained histopathological images. In this technique, the nuclei in the image are first segmented using a Convolutional Neural Network (CNN). The segmented nuclei are then used to generate melanoma region masks. Experimental results with a small melanoma dataset show that the proposed method can potentially segment the nuclei with more than 94 % accuracy and segment the melanoma regions with a Dice coefficient of around 85 %. The proposed technique also has a small execution time making it suitable for clinical diagnosis with a fast turnaround time.

Automated Diagnosis and Localization of Melanoma from Skin Histopathology Slides Using Deep Learning: A Multicenter Study

In traditional hospital systems, diagnosis and localization of melanoma are the critical challenges for pathological analysis, treatment instructions, and prognosis evaluation particularly in skin diseases. In literature, various studies have been reported to address these issues; however, a prominent smart diagnosis system is needed to be developed for the smart healthcare system. In this study, a deep learning-enabled diagnostic system is proposed and implemented that it has the capacity to automatically detect malignant melanoma in whole slide images (WSIs). In this system, the convolutional neural network (CNN), sophisticated statistical method, and image processing algorithms were integrated and implemented to locate benign and malignant lesions which are extremely useful in the diagnoses process of melanoma disease. To verify the exceptional performance of the proposed scheme, it is implemented in a multicenter database, which has 701 WSIs (641 WSIs from Central South University Xiangya Hospital (CSUXH) and 60 WSIs from the Cancer Genome Atlas (TCGA)). Experimental results have verified that the proposed system has achieved an area under the receiver operating characteristic curve (AUROC) of 0.971. Furthermore, the lesion area on the WSIs is represented by its degree of malignancy. These results show that the proposed system has the capacity to fully automate the diagnosis and localization problem of the melanoma in the smart healthcare systems.

An unsupervised method for histological image segmentation based on tissue cluster level graph cut

While deep learning models have demonstrated outstanding performance in medical image segmentation tasks, histological annotations for training deep learning models are usually challenging to obtain, due to the effort and experience required to carefully delineate tissue structures. In this study, we propose an unsupervised method, termed as tissue cluster level graph cut (TisCut), for segmenting histological images into meaningful compartments (e.g., tumor or non-tumor regions), which aims at assisting histological annotations for downstream supervised models. The TisCut consists of three modules. First, histological tissue objects are clustered based on their spatial proximity and morphological features. The Voronoi diagram is then constructed based on tissue object clustering. In the last module, morphological features computed from the Voronoi diagram are integrated into a region adjacency graph. Image partition is then performed to divide the image into meaningful compartments by using the graph cut algorithm. The TisCut has been evaluated on three histological image sets for necrosis and melanoma detections. Experiments show that the TisCut could provide a comparative performance with U-Net models, which achieves about 70% and 85% Jaccard index coefficients in partitioning brain and skin histological images, respectively. In addition, it shows the potential to be used for generating histological annotations when training masks are difficult to collect for supervised segmentation models.

Semi-Supervised Nests of Melanocytes Segmentation Method Using Convolutional Autoencoders

In this research, we present a semi-supervised segmentation solution using convolutional autoencoders to solve the problem of segmentation tasks having a small number of ground-truth images. We evaluate the proposed deep network architecture for the detection of nests of nevus cells in histopathological images of skin specimens is an important step in dermatopathology. The diagnostic criteria based on the degree of uniformity and symmetry of border irregularities are particularly vital in dermatopathology, in order to distinguish between benign and malignant skin lesions. However, to the best of our knowledge, it is the first described method to segment the nests region. The novelty of our approach is not only the area of research, but, furthermore, we address a problem with a small ground-truth dataset. We propose an effective computer-vision based deep learning tool that can perform the nests segmentation based on an autoencoder architecture with two learning steps. Experimental results verified the effectiveness of the proposed approach and its ability to segment nests areas with Dice similarity coefficient 0.81, sensitivity 0.76, and specificity 0.94, which is a state-of-the-art result.

A New Approach to Border Irregularity Assessment with Application in Skin Pathology

The border irregularity assessment of tissue structures is an important step in medical diagnostics (e.g., in dermatoscopy, pathology, and cardiology). The diagnostic criteria based on the degree of uniformity and symmetry of border irregularities are particularly vital in dermatopathology, to distinguish between benign and malignant skin lesions. We propose a new method for the segmentation of individual border projections and measuring their morphometry. It is based mainly on analyzing the curvature of the object’s border to identify endpoints of projection bases, and on analyzing object’s skeleton in the graph representation to identify bases of projections and their location along the object’s main axis. The proposed segmentation method has been tested on 25 skin whole slide images of common melanocytic lesions. In total, 825 out of 992 (83%) manually segmented retes (projections of epidermis) were detected correctly and the Jaccard similarity coefficient for the task of detecting retes was 0.798. Experimental results verified the effectiveness of the proposed approach. Our method is particularly well suited for assessing the border irregularity of human epidermis and thus could help develop computer-aided diagnostic algorithms for skin cancer detection.

Skin cancer detection using non-invasive techniques

Skin cancer is the most common form of cancer and is globally rising. Historically, the diagnosis of skin cancers has depended on various conventional techniques which are of an invasive manner. A variety of commercial diagnostic tools and auxiliary techniques are available to detect skin cancer. This article explains in detail the principles and approaches involved for non-invasive skin cancer diagnostic methods such as photography, dermoscopy, sonography, confocal microscopy, Raman spectroscopy, fluorescence spectroscopy, terahertz spectroscopy, optical coherence tomography, the multispectral imaging technique, thermography, electrical bio-impedance, tape stripping and computer-aided analysis. The characteristics of an ideal screening test are outlined, and the authors pose several points for clinicians and scientists to consider in the evaluation of current and future studies of skin cancer detection and diagnosis. This comprehensive review critically analyses the literature associated with the field and summarises the recent updates along with their merits and demerits.

Localization of Diagnostically Relevant Regions of Interest in Whole Slide Images: a Comparative Study

Whole slide digital imaging technology enables researchers to study pathologists' interpretive behavior as they view digital slides and gain new understanding of the diagnostic medical decision-making process. In this study, we propose a simple yet important analysis to extract diagnostically relevant regions of interest (ROIs) from tracking records using only pathologists' actions as they viewed biopsy specimens in the whole slide digital imaging format (zooming, panning, and fixating). We use these extracted regions in a visual bag-of-words model based on color and texture features to predict diagnostically relevant ROIs on whole slide images. Using a logistic regression classifier in a cross-validation setting on 240 digital breast biopsy slides and viewport tracking logs of three expert pathologists, we produce probability maps that show 74 % overlap with the actual regions at which pathologists looked. We compare different bag-of-words models by changing dictionary size, visual word definition (patches vs. superpixels), and training data (automatically extracted ROIs vs. manually marked ROIs). This study is a first step in understanding the scanning behaviors of pathologists and the underlying reasons for diagnostic errors.

A Containerized Software System for Generation, Management, and Exploration of Features from Whole Slide Tissue Images

Well-curated sets of pathology image features will be critical to clinical studies that aim to evaluate and predict treatment responses. Researchers require information synthesized across multiple biological scales, from the patient to the molecular scale, to more effectively study cancer. This article describes a suite of services and web applications that allow users to select regions of interest in whole slide tissue images, run a segmentation pipeline on the selected regions to extract nuclei and compute shape, size, intensity, and texture features, store and index images and analysis results, and visualize and explore images and computed features. All the services are deployed as containers and the user-facing interfaces as web-based applications. The set of containers and web applications presented in this article is used in cancer research studies of morphologic characteristics of tumor tissues. The software is free and open source. Cancer Res; 77(21); e79-82. ©2017 AACR.

An Assessment of Imaging Informatics for Precision Medicine in Cancer

Objectives: Precision medicine requires the measurement, quantification, and cataloging of medical characteristics to identify the most effective medical intervention. However, the amount of available data exceeds our current capacity to extract meaningful information. We examine the informatics needs to achieve precision medicine from the perspective of quantitative imaging and oncology. Methods: The National Cancer Institute (NCI) organized several workshops on the topic of medical imaging and precision medicine. The observations and recommendations are summarized herein. Results: Recommendations include: use of standards in data collection and clinical correlates to promote interoperability; data sharing and validation of imaging tools; clinician's feedback in all phases of research and development; use of open-source architecture to encourage reproducibility and reusability; use of challenges which simulate real-world situations to incentivize innovation; partnership with industry to facilitate commercialization; and education in academic communities regarding the challenges involved with translation of technology from the research domain to clinical utility and the benefits of doing so. Conclusions: This article provides a survey of the role and priorities for imaging informatics to help advance quantitative imaging in the era of precision medicine. While these recommendations were drawn from oncology, they are relevant and applicable to other clinical domains where imaging aids precision medicine.

Automatic measurement of melanoma depth of invasion in skin histopathological images

Measurement of melanoma depth of invasion (DoI) in skin tissues is of great significance in grading the severity of skin disease and planning patient's treatment. However, accurate and automatic measurement of melanocytic tumor depth is a challenging problem mainly due to the difficulty of skin granular identification and melanoma detection. In this paper, we propose a technique for measuring melanoma DoI in microscopic images digitized from MART1 (i.e., meleanoma-associated antigen recognized by T cells) stained skin histopathological sections. The technique consists of four modules. First, skin melanoma areas are detected by combining color features with the Mahalanobis distance measure. Next, skin epidermis is segmented by a multi-thresholding method. The skin granular layer is then identified based on Bayesian classification of segmented skin epidermis pixels. Finally, the melanoma DoI is computed using a multi-resolution approach with Hausdorff distance measurement. Experimental results show that the proposed technique provides a superior performance in measuring the melanoma DoI than two closely related techniques.

A Study on Automatic Segmentation and Classification of Skin Lesions in Dermoscopic Images

Malignant Melanoma is one of the rare and the deadliest form of skin cancer if left untreated. Death rate due to this cancer is three times more than all other skin-related malignancies combined. Incidence rates of melanoma have been increasing, especially among young adults, but survival rates are high if detected early. There is a need for an automated system to assess a patient's risk of melanoma using digital dermoscopy, that is, a skin imaging technique widely used for pigmented skin lesion inspection. Although many automated and semi-automated methods are available to diagnose skin cancer but each has its own limitations and there is no final, state-of-the art technique to date which is able to be implemented in real scenario. This survey paper is based on techniques used to segment the skin cancer, analysis of their merits and demerits and their applications on advanced imaging techniques.

Multi-Pass Adaptive Voting for Nuclei Detection in Histopathological Images

Nuclei detection is often a critical initial step in the development of computer aided diagnosis and prognosis schemes in the context of digital pathology images. While over the last few years, a number of nuclei detection methods have been proposed, most of these approaches make idealistic assumptions about the staining quality of the tissue. In this paper, we present a new Multi-Pass Adaptive Voting (MPAV) for nuclei detection which is specifically geared towards images with poor quality staining and noise on account of tissue preparation artifacts. The MPAV utilizes the symmetric property of nuclear boundary and adaptively selects gradient from edge fragments to perform voting for a potential nucleus location. The MPAV was evaluated in three cohorts with different staining methods: Hematoxylin &Eosin, CD31 &Hematoxylin, and Ki-67 and where most of the nuclei were unevenly and imprecisely stained. Across a total of 47 images and nearly 17,700 manually labeled nuclei serving as the ground truth, MPAV was able to achieve a superior performance, with an area under the precision-recall curve (AUC) of 0.73. Additionally, MPAV also outperformed three state-of-the-art nuclei detection methods, a single pass voting method, a multi-pass voting method, and a deep learning based method.

Differential diagnosis of squamous cell carcinoma in situ using skin histopathological images

Differential diagnosis of squamous cell carcinoma in situ is of great importance for prognosis and decision making in the disease treatment procedure. Currently, differential diagnosis is done by pathologists based on examination of the histopathological slides under the microscope, which is time consuming and prone to inter and intra observer variability. In this paper, we have proposed an automated method for differential diagnosis of SCC in situ from actinic keratosis, which is known to be a precursor of squamous cell carcinoma. The process begins with epidermis segmentation and cornified layer removal. Then, epidermis axis is specified using the paths in its skeleton and the granular layer is removed via connected components analysis. Finally, diagnosis is done based on the classification result of intensity profiles extracted from lines perpendicular to the epidermis axis. The results of the study are in agreement with the gold standards provided by expert pathologists.

An efficient technique for nuclei segmentation based on ellipse descriptor analysis and improved seed detection algorithm

In this paper, we propose an efficient method for segmenting cell nuclei in the skin histopathological images. The proposed technique consists of four modules. First, it separates the nuclei regions from the background with an adaptive threshold technique. Next, an elliptical descriptor is used to detect the isolated nuclei with elliptical shapes. This descriptor classifies the nuclei regions based on two ellipticity parameters. Nuclei clumps and nuclei with irregular shapes are then localized by an improved seed detection technique based on voting in the eroded nuclei regions. Finally, undivided nuclei regions are segmented by a marked watershed algorithm. Experimental results on 114 different image patches indicate that the proposed technique provides a superior performance in nuclei detection and segmentation.

Efficient epidermis segmentation for whole slide skin histopathological images

In order to develop a computer-aided diagnosis system for histopathological skin cancer diagnosis, segmentation of the epidermis area is the very first and crucial step. An improved computer-aided epidermis segmentation technique for the whole slide skin histopathological image is proposed in this paper. The proposed technique first obtains an initial segmentation result with the help of global thresholding and shape analysis. A template matching method, with adaptive template intensity value, is then applied. Finally, a threshold is calculated based on the probability density function of the processed image after template matching. The threshold is then used to obtain the final segmentation result. Experimental results show that the proposed technique overcomes the limitation of the existing technique and provides a superior performance with sensitivity at 97.99%, and precision at 96.00%.

Spatial normalization of human back images for dermatological studies

A large number of pigmented skin lesions (PSLs) are a strong predictor of malignant melanoma. Many dermatologists advocate total body photography for high-risk patients because detecting new-appearing, disappearing, and changing PSL is important for early detection of the disease. However, manual inspection and matching of PSL is a subjective, tedious, and error-prone task. A computer program for tracking the corresponding PSL will greatly improve the matching process. In this paper, we describe the construction of the first human back template (atlas), which is used to facilitate spatial normalization of the PSL during the matching process. Four pairs of anatomically meaningful landmarks (neck, shoulder, armpit, and hip points) are used as reference points on the back image. Using the landmarks, a grid with longitudes and latitudes is constructed and overlaid on each subject-specific back image. To perform spatial normalization, the grid is registered into the back template, a unit-square rectilinear grid. To demonstrate the benefits of using the back template, we apply several state-of-the-art point-matching algorithms on 56 pairs of real dermatological images and show that utilizing spatially normalized coordinates improves the PSL matching accuracies.