1
|
Lama N, Stanley RJ, Lama B, Maurya A, Nambisan A, Hagerty J, Phan T, Van Stoecker W. LAMA: Lesion-Aware Mixup Augmentation for Skin Lesion Segmentation. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:1812-1823. [PMID: 38409610 PMCID: PMC11300415 DOI: 10.1007/s10278-024-01000-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Revised: 12/18/2023] [Accepted: 12/21/2023] [Indexed: 02/28/2024]
Abstract
Deep learning can exceed dermatologists' diagnostic accuracy in experimental image environments. However, inaccurate segmentation of images with multiple skin lesions can be seen with current methods. Thus, information present in multiple-lesion images, available to specialists, is not retrievable by machine learning. While skin lesion images generally capture a single lesion, there may be cases in which a patient's skin variation may be identified as skin lesions, leading to multiple false positive segmentations in a single image. Conversely, image segmentation methods may find only one region and may not capture multiple lesions in an image. To remedy these problems, we propose a novel and effective data augmentation technique for skin lesion segmentation in dermoscopic images with multiple lesions. The lesion-aware mixup augmentation (LAMA) method generates a synthetic multi-lesion image by mixing two or more lesion images from the training set. We used the publicly available International Skin Imaging Collaboration (ISIC) 2017 Challenge skin lesion segmentation dataset to train the deep neural network with the proposed LAMA method. As none of the previous skin lesion datasets (including ISIC 2017) has considered multiple lesions per image, we created a new multi-lesion (MuLe) segmentation dataset utilizing publicly available ISIC 2020 skin lesion images with multiple lesions per image. MuLe was used as a test set to evaluate the effectiveness of the proposed method. Our test results show that the proposed method improved the Jaccard score 8.3% from 0.687 to 0.744 and the Dice score 5% from 0.7923 to 0.8321 over a baseline model on MuLe test images. On the single-lesion ISIC 2017 test images, LAMA improved the baseline model's segmentation performance by 0.08%, raising the Jaccard score from 0.7947 to 0.8013 and the Dice score 0.6% from 0.8714 to 0.8766. The experimental results showed that LAMA improved the segmentation accuracy on both single-lesion and multi-lesion dermoscopic images. The proposed LAMA technique warrants further study.
Collapse
Affiliation(s)
- Norsang Lama
- Missouri University of Science & Technology, Rolla, MO, 65409, USA
| | | | | | - Akanksha Maurya
- Missouri University of Science & Technology, Rolla, MO, 65409, USA
| | - Anand Nambisan
- Missouri University of Science & Technology, Rolla, MO, 65409, USA
| | | | - Thanh Phan
- Missouri University of Science & Technology, Rolla, MO, 65409, USA
| | | |
Collapse
|
2
|
Helkkula T, Christensen G, Ingvar C, Isaksson K, Harbst K, Persson B, Ingvar Å, Hafström A, Carneiro A, Gaspar V, Jönsson G, Nielsen K. BioMEL: a translational research biobank of melanocytic lesions and melanoma. BMJ Open 2024; 14:e069694. [PMID: 38309755 PMCID: PMC10840057 DOI: 10.1136/bmjopen-2022-069694] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/29/2022] [Accepted: 01/17/2024] [Indexed: 02/05/2024] Open
Abstract
INTRODUCTION Diagnosing invasive cutaneous melanoma (CM) can be challenging due to subjectivity in distinguishing equivocal nevi, melanoma in situ and thin CMs. The underlying molecular mechanisms of progression from nevus to melanoma must be better understood. Identifying biomarkers for treatment response, diagnostics and prognostics is crucial. Using biomedical data from biobanks and population-based healthcare data, translational research can improve patient care by implementing evidence-based findings. The BioMEL biobank is a prospective, multicentre, large-scale biomedical database on equivocal nevi and all stages of primary melanoma to metastases. Its purpose is to serve as a translational resource, enabling researchers to uncover objective molecular, genotypic, phenotypic and structural differences in nevi and all stages of melanoma. The main objective is to leverage BioMEL to significantly improve diagnostics, prognostics and therapy outcomes of patients with melanoma. METHODS AND ANALYSIS The BioMEL biobank contains biological samples, epidemiological information and medical data from adult patients who receive routine care for melanoma. BioMEL is focused on primary and metastatic melanoma, but equivocal pigmented lesions such as clinically atypical nevi and melanoma in situ are also included. BioMEL data are gathered by questionnaires, blood sampling, tumour imaging, tissue sampling, medical records and histopathological reports. ETHICS AND DISSEMINATION The BioMEL biobank project is approved by the national Swedish Ethical Review Authority (Dnr. 2013/101, 2013/339, 2020/00469, 2021/01432 and 2022/02421-02). The datasets generated are not publicly available due to regulations related to the ethical review authority. TRIAL REGISTRATION NUMBER NCT05446155.
Collapse
Affiliation(s)
- Teo Helkkula
- Dermatology and Venereology, Lund University Skin Cancer research group, Department of Clinical Sciences Lund, Lund University, Lund, Sweden
- Department of Dermatology and Venereology, Skåne University Hospital Lund, Lund, Sweden
- Lund Melanoma Study Group, Lund University, Lund, Sweden
| | - Gustav Christensen
- Dermatology and Venereology, Lund University Skin Cancer research group, Department of Clinical Sciences Lund, Lund University, Lund, Sweden
- Department of Dermatology and Venereology, Skåne University Hospital Lund, Lund, Sweden
- Lund Melanoma Study Group, Lund University, Lund, Sweden
| | - Christian Ingvar
- Lund Melanoma Study Group, Lund University, Lund, Sweden
- Department of Surgery, Department of Clinical Sciences Lund, Lund University, Lund, Sweden
| | - Karolin Isaksson
- Lund Melanoma Study Group, Lund University, Lund, Sweden
- Department of Surgery, Department of Clinical Sciences Lund, Lund University, Lund, Sweden
- Department of Surgery, Central Hospital in Kristianstad, Kristianstad, Sweden
| | - Katja Harbst
- Lund Melanoma Study Group, Lund University, Lund, Sweden
- Department of Oncology, Department of Clinical Sciences Lund, Lund University, Lund, Sweden
| | - Bertil Persson
- Dermatology and Venereology, Lund University Skin Cancer research group, Department of Clinical Sciences Lund, Lund University, Lund, Sweden
- Department of Dermatology and Venereology, Skåne University Hospital Lund, Lund, Sweden
- Lund Melanoma Study Group, Lund University, Lund, Sweden
| | - Åsa Ingvar
- Dermatology and Venereology, Lund University Skin Cancer research group, Department of Clinical Sciences Lund, Lund University, Lund, Sweden
- Department of Dermatology and Venereology, Skåne University Hospital Lund, Lund, Sweden
- Lund Melanoma Study Group, Lund University, Lund, Sweden
| | - Anna Hafström
- Lund Melanoma Study Group, Lund University, Lund, Sweden
- Department of Otorhinolaryngology, wtih Head and Neck Surgery, Skåne University Hospital Lund, Lund, Sweden
| | - Ana Carneiro
- Lund Melanoma Study Group, Lund University, Lund, Sweden
- Department of Oncology, Department of Clinical Sciences Lund, Lund University, Lund, Sweden
- Department of Oncology, Skåne University Hospital Lund, Lund, Sweden
| | - Viktoria Gaspar
- Dermatology and Venereology, Lund University Skin Cancer research group, Department of Clinical Sciences Lund, Lund University, Lund, Sweden
- Lund Melanoma Study Group, Lund University, Lund, Sweden
- Department of Pathology, Hospital in Helsingborg, Helsingborg, Sweden
| | - Göran Jönsson
- Lund Melanoma Study Group, Lund University, Lund, Sweden
- Department of Oncology, Department of Clinical Sciences Lund, Lund University, Lund, Sweden
| | - Kari Nielsen
- Dermatology and Venereology, Lund University Skin Cancer research group, Department of Clinical Sciences Lund, Lund University, Lund, Sweden
- Department of Dermatology and Venereology, Skåne University Hospital Lund, Lund, Sweden
- Lund Melanoma Study Group, Lund University, Lund, Sweden
- Department of Dermatology and Venereology, Hospital in Helsingborg, Helsingborg, Sweden
| |
Collapse
|
3
|
Lama N, Hagerty J, Nambisan A, Stanley RJ, Van Stoecker W. Skin Lesion Segmentation in Dermoscopic Images with Noisy Data. J Digit Imaging 2023; 36:1712-1722. [PMID: 37020149 PMCID: PMC10407008 DOI: 10.1007/s10278-023-00819-8] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Revised: 03/15/2023] [Accepted: 03/17/2023] [Indexed: 04/07/2023] Open
Abstract
We propose a deep learning approach to segment the skin lesion in dermoscopic images. The proposed network architecture uses a pretrained EfficientNet model in the encoder and squeeze-and-excitation residual structures in the decoder. We applied this approach on the publicly available International Skin Imaging Collaboration (ISIC) 2017 Challenge skin lesion segmentation dataset. This benchmark dataset has been widely used in previous studies. We observed many inaccurate or noisy ground truth labels. To reduce noisy data, we manually sorted all ground truth labels into three categories - good, mildly noisy, and noisy labels. Furthermore, we investigated the effect of such noisy labels in training and test sets. Our test results show that the proposed method achieved Jaccard scores of 0.807 on the official ISIC 2017 test set and 0.832 on the curated ISIC 2017 test set, exhibiting better performance than previously reported methods. Furthermore, the experimental results showed that the noisy labels in the training set did not lower the segmentation performance. However, the noisy labels in the test set adversely affected the evaluation scores. We recommend that the noisy labels should be avoided in the test set in future studies for accurate evaluation of the segmentation algorithms.
Collapse
Affiliation(s)
- Norsang Lama
- Missouri University of Science &Technology, Rolla, MO, 65409, USA
| | | | - Anand Nambisan
- Missouri University of Science &Technology, Rolla, MO, 65409, USA
| | | | | |
Collapse
|
4
|
Lama N, Kasmi R, Hagerty JR, Stanley RJ, Young R, Miinch J, Nepal J, Nambisan A, Stoecker WV. ChimeraNet: U-Net for Hair Detection in Dermoscopic Skin Lesion Images. J Digit Imaging 2023; 36:526-535. [PMID: 36385676 PMCID: PMC10039207 DOI: 10.1007/s10278-022-00740-6] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Revised: 11/02/2022] [Accepted: 11/09/2022] [Indexed: 11/18/2022] Open
Abstract
Hair and ruler mark structures in dermoscopic images are an obstacle preventing accurate image segmentation and detection of critical network features. Recognition and removal of hairs from images can be challenging, especially for hairs that are thin, overlapping, faded, or of similar color as skin or overlaid on a textured lesion. This paper proposes a novel deep learning (DL) technique to detect hair and ruler marks in skin lesion images. Our proposed ChimeraNet is an encoder-decoder architecture that employs pretrained EfficientNet in the encoder and squeeze-and-excitation residual (SERes) structures in the decoder. We applied this approach at multiple image sizes and evaluated it using the publicly available HAM10000 (ISIC2018 Task 3) skin lesion dataset. Our test results show that the largest image size (448 × 448) gave the highest accuracy of 98.23 and Jaccard index of 0.65 on the HAM10000 (ISIC 2018 Task 3) skin lesion dataset, exhibiting better performance than for two well-known deep learning approaches, U-Net and ResUNet-a. We found the Dice loss function to give the best results for all measures. Further evaluated on 25 additional test images, the technique yields state-of-the-art accuracy compared to 8 previously reported classical techniques. We conclude that the proposed ChimeraNet architecture may enable improved detection of fine image structures. Further application of DL techniques to detect dermoscopy structures is warranted.
Collapse
Affiliation(s)
- Norsang Lama
- Missouri University of Science & Technology, Rolla, MO, 65409, USA
| | | | | | - R Joe Stanley
- Missouri University of Science & Technology, Rolla, MO, 65409, USA.
| | - Reagan Young
- Missouri University of Science & Technology, Rolla, MO, 65409, USA
| | - Jessica Miinch
- Missouri University of Science & Technology, Rolla, MO, 65409, USA
| | | | - Anand Nambisan
- Missouri University of Science & Technology, Rolla, MO, 65409, USA
| | | |
Collapse
|
5
|
Jaklitsch E, Thames T, de Campos Silva T, Coll P, Oliviero M, Ferris LK. Clinical Utility of an AI-powered, Handheld Elastic Scattering Spectroscopy Device on the Diagnosis and Management of Skin Cancer by Primary Care Physicians. J Prim Care Community Health 2023; 14:21501319231205979. [PMID: 37933569 PMCID: PMC10631325 DOI: 10.1177/21501319231205979] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Revised: 09/18/2023] [Accepted: 09/20/2023] [Indexed: 11/08/2023] Open
Abstract
BACKGROUND Patients with lesions suspicious for skin cancer often present to primary care physicians (PCPs), who may have limited training in skin cancer diagnosis. OBJECTIVE To measure the impact of an adjunctive handheld device for PCPs that employs elastic scattering spectroscopy (ESS) on the diagnosis and management of skin cancer. METHODS Fifty-seven PCPs evaluated 50 clinical images of skin lesions (25 malignant and 25 benign), first without and then with knowledge of the handheld ESS device output, and in each case indicated if a lesion was likely to be benign or malignant. RESULTS The diagnostic sensitivity of the PCPs with and without the use of the ESS device was 88% (95% CI, 84%-92%) and 67% (95% CI, 62%-72%), respectively (P < .0001). In contrast, no significant difference was observed in the diagnostic specificity. The management sensitivity of the physicians with and without the use of the ESS device was 94% (95% CI, 91%-96%) and 81% (95% CI, 77%-85%), respectively (P = .0009). Similarly, no significant difference was observed in the management specificity. CONCLUSION The use of the ESS device may have the potential to help improve skin cancer diagnosis and confidence in management decision-making in a primary care setting.
Collapse
Affiliation(s)
- Erik Jaklitsch
- School of Medicine, University of Pittsburgh, Pittsburgh, PA, USA
| | | | | | | | - Margaret Oliviero
- Dermatology Department, Skin and Cancer Associates, Plantation, FL, USA
| | - Laura Korb Ferris
- Department of Dermatology, University of Pittsburgh, Pittsburgh, PA, USA
| |
Collapse
|
6
|
Talavera-Martínez L, Bibiloni P, González-Hidalgo M. Computational texture features of dermoscopic images and their link to the descriptive terminology: A survey. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2019; 182:105049. [PMID: 31494412 DOI: 10.1016/j.cmpb.2019.105049] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/31/2019] [Revised: 08/12/2019] [Accepted: 08/23/2019] [Indexed: 06/10/2023]
Abstract
Computer-extracted texture features are relevant to diagnose cutaneous lesions such as melanomas. Our goal is to set a relationship between a well-established descriptive terminology, which describes the attributes of dermoscopic structures based on their aspect rather than their underlying causes, and the computational methods to extract texture-based features. By tackling this problem, we can ascertain what indicators used by dermatologists are reflected in the extracted texture features. We first review the state-of-the-art models for texture extraction in dermoscopic images. By comparing the methods' performance and goals, we conclude that (I) a single color space does not seem to give performances as good as using several ones, thus the latter is reasonable (II) the optimal number of extracted features seems to vary depending on the method's goal, and extracting a large number of features can lead to a loss of models robustness (III) methods such as GLCM, Sobel or Law energy filters are mainly used to capture local properties to detect specific dermoscopic structures (IV) methods that extract local and global features, like Gabor wavelets or SPT, tend to be used to analyze the presence of certain patterns of dermoscopic structures, e.g. globular, reticular, etc.
Collapse
Affiliation(s)
- Lidia Talavera-Martínez
- Universitat de les Illes Balears, SCOPIA Research Group, Palma 07122, Spain; Balearic Islands Health Research Institute (IdISBa), Palma 07010, Spain.
| | - Pedro Bibiloni
- Universitat de les Illes Balears, SCOPIA Research Group, Palma 07122, Spain; Balearic Islands Health Research Institute (IdISBa), Palma 07010, Spain.
| | - Manuel González-Hidalgo
- Universitat de les Illes Balears, SCOPIA Research Group, Palma 07122, Spain; Balearic Islands Health Research Institute (IdISBa), Palma 07010, Spain.
| |
Collapse
|
7
|
Krieter M, Schultz E, Debus D. [Malignant melanoma - from diagnosis to follow-up care]. MMW Fortschr Med 2019; 161:42-50. [PMID: 31129855 DOI: 10.1007/s15006-019-0018-6] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
Affiliation(s)
- Manuel Krieter
- Universitätsklinik für Dermatologie, Paracelsus Medizinische Privatuniversität Nürnberg, Prof.-Ernst-Nathan-Str. 1, D-90419, Nürnberg, Deutschland.
| | - Erwin Schultz
- Universitätsklinik für Dermatologie, Paracelsus Medizinische Privatuniversität Nürnberg, Prof.-Ernst-Nathan-Str. 1, D-90419, Nürnberg, Deutschland
| | - Dirk Debus
- Universitätsklinik für Dermatologie, Paracelsus Medizinische Privatuniversität Nürnberg, Prof.-Ernst-Nathan-Str. 1, D-90419, Nürnberg, Deutschland
| |
Collapse
|
8
|
Turani Z, Fatemizadeh E, Blumetti T, Daveluy S, Moraes AF, Chen W, Mehregan D, Andersen PE, Nasiriavanaki M. Optical Radiomic Signatures Derived from Optical Coherence Tomography Images Improve Identification of Melanoma. Cancer Res 2019; 79:2021-2030. [PMID: 30777852 PMCID: PMC6836720 DOI: 10.1158/0008-5472.can-18-2791] [Citation(s) in RCA: 58] [Impact Index Per Article: 11.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2018] [Revised: 01/14/2019] [Accepted: 02/13/2019] [Indexed: 11/16/2022]
Abstract
The current gold standard for clinical diagnosis of melanoma is excisional biopsy and histopathologic analysis. Approximately 15-30 benign lesions are biopsied to diagnose each melanoma. In addition, biopsies are invasive and result in pain, anxiety, scarring, and disfigurement of patients, which can add additional burden to the health care system. Among several imaging techniques developed to enhance melanoma diagnosis, optical coherence tomography (OCT), with its high-resolution and intermediate penetration depth, can potentially provide required diagnostic information noninvasively. Here, we present an image analysis algorithm, "optical properties extraction (OPE)," which improves the specificity and sensitivity of OCT by identifying unique optical radiomic signatures pertinent to melanoma detection. We evaluated the performance of the algorithm using several tissue-mimicking phantoms and then tested the OPE algorithm on 69 human subjects. Our data show that benign nevi and melanoma can be differentiated with 97% sensitivity and 98% specificity. These findings suggest that the adoption of OPE algorithm in the clinic can lead to improvements in melanoma diagnosis and patient experience. SIGNIFICANCE: This study describes a noninvasive, safe, simple-to-implement, and accurate method for the detection and differentiation of malignant melanoma versus benign nevi.
Collapse
Affiliation(s)
- Zahra Turani
- Department of Electrical Engineering, Sharif University of Technology, Tehran, Iran
- Department of Biomedical Engineering, Wayne State University, Detroit, Michigan
| | - Emad Fatemizadeh
- Department of Electrical Engineering, Sharif University of Technology, Tehran, Iran
| | - Tatiana Blumetti
- Cutaneous Oncology Department, AC Camargo Cancer Center, São Paulo, Brazil
| | - Steven Daveluy
- Department of Dermatology, School of Medicine Wayne State University, Detroit, Michigan
| | - Ana Flavia Moraes
- Cutaneous Oncology Department, AC Camargo Cancer Center, São Paulo, Brazil
| | - Wei Chen
- Department of Oncology, Karmanos Cancer Institute, Detroit, Michigan
| | - Darius Mehregan
- Cutaneous Oncology Department, AC Camargo Cancer Center, São Paulo, Brazil
| | - Peter E Andersen
- Department of Health Technology, Technical University of Denmark, Lyngby, Denmark
| | - Mohammadreza Nasiriavanaki
- Department of Biomedical Engineering, Wayne State University, Detroit, Michigan.
- Department of Dermatology, School of Medicine Wayne State University, Detroit, Michigan
- Department of Oncology, Karmanos Cancer Institute, Detroit, Michigan
| |
Collapse
|
9
|
Mishra NK, Kaur R, Kasmi R, Hagerty JR, LeAnder R, Stanley RJ, Moss RH, Stoecker WV. Automatic lesion border selection in dermoscopy images using morphology and color features. Skin Res Technol 2019; 25:544-552. [PMID: 30868667 DOI: 10.1111/srt.12685] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2019] [Revised: 01/12/2019] [Accepted: 01/12/2019] [Indexed: 11/29/2022]
Abstract
PURPOSE We present a classifier for automatically selecting a lesion border for dermoscopy skin lesion images, to aid in computer-aided diagnosis of melanoma. Variation in photographic technique of dermoscopy images makes segmentation of skin lesions a difficult problem. No single algorithm provides an acceptable lesion border to allow further processing of skin lesions. METHODS We present a random forests border classifier model to select a lesion border from 12 segmentation algorithm borders, graded on a "good-enough" border basis. Morphology and color features inside and outside the automatic border are used to build the model. RESULTS For a random forests classifier applied to an 802-lesion test set, the model predicts a satisfactory border in 96.38% of cases, in comparison to the best single border algorithm, which detects a satisfactory border in 85.91% of cases. CONCLUSION The performance of the classifier-based automatic skin lesion finder is found to be better than any single algorithm used in this research.
Collapse
Affiliation(s)
| | - Ravneet Kaur
- Department of Electrical and Computer Engineering, Southern Illinois University Edwardsville, Edwardsville, Illinois
| | - Reda Kasmi
- Department of Electrical Engineering, University of Bejaia, Bejaia, Algeria.,Department of Electrical Engineering, University of Bouira, Bouira, Algeria
| | | | - Robert LeAnder
- Department of Electrical and Computer Engineering, Southern Illinois University Edwardsville, Edwardsville, Illinois
| | - Ronald J Stanley
- Department of Electrical and Computer Engineering, Missouri University of Science and Technology, Rolla, Missouri
| | - Randy H Moss
- Department of Electrical and Computer Engineering, Missouri University of Science and Technology, Rolla, Missouri
| | | |
Collapse
|
10
|
Hagerty JR, Stanley RJ, Almubarak HA, Lama N, Kasmi R, Guo P, Drugge RJ, Rabinovitz HS, Oliviero M, Stoecker WV. Deep Learning and Handcrafted Method Fusion: Higher Diagnostic Accuracy for Melanoma Dermoscopy Images. IEEE J Biomed Health Inform 2019; 23:1385-1391. [PMID: 30624234 DOI: 10.1109/jbhi.2019.2891049] [Citation(s) in RCA: 63] [Impact Index Per Article: 12.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
This paper presents an approach that combines conventional image processing with deep learning by fusing the features from the individual techniques. We hypothesize that the two techniques, with different error profiles, are synergistic. The conventional image processing arm uses three handcrafted biologically inspired image processing modules and one clinical information module. The image processing modules detect lesion features comparable to clinical dermoscopy information-atypical pigment network, color distribution, and blood vessels. The clinical module includes information submitted to the pathologist-patient age, gender, lesion location, size, and patient history. The deep learning arm utilizes knowledge transfer via a ResNet-50 network that is repurposed to predict the probability of melanoma classification. The classification scores of each individual module from both processing arms are then ensembled utilizing logistic regression to predict an overall melanoma probability. Using cross-validated results of melanoma classification measured by area under the receiver operator characteristic curve (AUC), classification accuracy of 0.94 was obtained for the fusion technique. In comparison, the ResNet-50 deep learning based classifier alone yields an AUC of 0.87 and conventional image processing based classifier yields an AUC of 0.90. Further study of fusion of conventional image processing techniques and deep learning is warranted.
Collapse
|
11
|
Prediction of Dermoscopy Patterns for Recognition of both Melanocytic and Non-Melanocytic Skin Lesions. COMPUTERS 2016. [DOI: 10.3390/computers5030013] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
12
|
Dermoscopic and clinical features of trunk melanomas. Postepy Dermatol Alergol 2014; 31:362-7. [PMID: 25610350 PMCID: PMC4293392 DOI: 10.5114/pdia.2014.47119] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2014] [Revised: 06/21/2014] [Accepted: 09/10/2014] [Indexed: 11/17/2022] Open
Abstract
INTRODUCTION Malignant melanomas account for 5% of all skin cancers and usually have a fatal clinical course. Additionally, the incidence of melanoma increases more rapidly than in any other cancer, and this has been attributed to the development of highly sensitive diagnostic techniques, mainly dermoscopy, which allows for early diagnosis. The phenotypic manifestations of gene/environment interactions, environmental factor and genetic factors may determine subtypes and anatomic localization of melanoma. Histopathologic subtypes, risk factors, and thickness of the skin are different in trunk melanomas. AIM To determine the frequency of dermatoscopic features in trunk melanomas. This study also investigates dermoscopic features according to the diameter of lesions. MATERIAL AND METHODS Seventy-one trunk melanomas were included. Their dermoscopic and clinical images, histopathological and clinical data were assessed. The relations between the diameter, Breslow thickness and dermoscopic characteristics were evaluated. RESULTS The most common dermoscopic findings of trunk melanomas were the multicomponent pattern (55 patients, 77.5%), asymmetry (62 patients; 87.3%), blue-gray veil (59 patients, 83.1%), and color variety (56 patients, 78.8%). When dermoscopic findings were compared, a multicomponent pattern (p = 0.03), milky-red areas (p = 0.001), blue-gray veils (p = 0.023), and regression structures (p = 0.037) were more common in large melanomas than in small melanomas. CONCLUSIONS The most common dermoscopic findings of trunk melanomas were the multicomponent pattern, asymmetry and blue-gray veil, color variety. The multicomponent pattern, milky-red areas, blue-gray veils, regression structures were statistically significant dermoscopic features in a group of large-diameter melanomas, compared to small melanomas.
Collapse
|
13
|
Silva CS, Marcal AR. Colour-based dermoscopy classification of cutaneous lesions: an alternative approach. COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING-IMAGING AND VISUALIZATION 2013. [DOI: 10.1080/21681163.2013.803683] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
|
14
|
Pralong P, Bathelier E, Dalle S, Poulalhon N, Debarbieux S, Thomas L. Dermoscopy of lentigo maligna melanoma: report of 125 cases. Br J Dermatol 2012; 167:280-7. [DOI: 10.1111/j.1365-2133.2012.10932.x] [Citation(s) in RCA: 104] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|
15
|
Xu J, Gupta K, Stoecker WV, Krishnamurthy Y, Rabinovitz HS, Bangert A, Calcara D, Oliviero M, Malters JM, Drugge R, Stanley RJ, Moss RH, Celebi ME. Analysis of globule types in malignant melanoma. ACTA ACUST UNITED AC 2009; 145:1245-51. [PMID: 19917953 DOI: 10.1001/archdermatol.2009.285] [Citation(s) in RCA: 13] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
OBJECTIVE To identify and analyze subtypes of globules based on size, shape, network connectedness, pigmentation, and distribution to determine which globule types and globule distributions are most frequently associated with a diagnosis of malignant melanoma. DESIGN Retrospective case series of dermoscopy images with globules. SETTING Private dermatology practices. PARTICIPANTS Patients in dermatology practices. Intervention Observation only. Main Outcome Measure Association of globule types with malignant melanoma. RESULTS The presence of large globules (odds ratio [OR], 5.25) and globules varying in size (4.72) or shape (5.37) had the highest ORs for malignant melanoma among all globule types and combinations studied. Classical globules (dark, discrete, convex, and 0.10-0.20 mm) had a higher risk (OR, 4.20) than irregularly shaped globules (dark, discrete, and not generally convex) (2.89). Globules connected to other structures were not significant in the diagnosis of malignant melanoma. Of the different configurations studied, asymmetric clusters have the highest risk (OR, 3.02). CONCLUSIONS The presence of globules of varying size or shape seems to be more associated with a diagnosis of malignant melanoma than any other globule type or distribution in this study. Large globules are of particular importance in the diagnosis of malignant melanoma.
Collapse
Affiliation(s)
- Jin Xu
- Stoecker and Associates, 1702 E 10th St, Rolla, MO 65401-4600, USA
| | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
16
|
Scope A, Benvenuto-Andrade C, Agero ALC, Marghoob AA. Nonmelanocytic Lesions Defying the Two-Step Dermoscopy Algorithm. Dermatol Surg 2006; 32:1398-406. [PMID: 17083595 DOI: 10.1111/j.1524-4725.2006.32312.x] [Citation(s) in RCA: 20] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
The first step of the two-step algorithm of dermoscopy aims at differentiating melanocytic from nonmelanocytic pigmented lesions, using a stepwise evaluation for the presence of specific dermoscopic criteria. The purpose of this article is to heighten awareness of clinicians to nonmelanocytic lesions that defy the two-step algorithm, thus simulating melanocytic lesions dermoscopically. Seborrheic keratosis, solar lentigo, dermatofibroma, and supernumerary accessory nipple may present with network-like structures. Seborrheic keratosis, dermatofibroma, subcorneal hemorrhage, basal cell carcinoma (BCC), and cutaneous metastases of breast and other cancers may contain pigmented globules. Peripheral streaks can also be seen in seborrheic keratosis and BCC. Homogenous bluish pigmentation, simulating a blue nevus, can also be seen in benign vascular lesions, Kaposi sarcoma, radiation tattoo, and BCC. This overlap of features between melanocytic and nonmelanocytic lesions suggests that integration of all dermoscopic features in the lesion, rather than a stepwise evaluation, may facilitate reaching the correct diagnosis in select cases as outlined in this article.
Collapse
Affiliation(s)
- Alon Scope
- Dermatology Service, Department of Medicine, Memorial Sloan-Kettering Cancer Center, New York, New York 10022, USA
| | | | | | | |
Collapse
|
17
|
|
18
|
Wyatt AJ, Agero ALC, Delgado R, Busam KJ, Marghoob AA. Cutaneous metastatic breast carcinoma with melanocyte colonization: a clinical and dermoscopic mimic of malignant melanoma. Dermatol Surg 2006; 32:949-54. [PMID: 16875480 DOI: 10.1111/j.1524-4725.2006.32202.x] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
Abstract
BACKGROUND Melanocyte colonization of breast carcinoma cells may occur in those tumors that breach the epidermal-dermal interface. The resultant melanin deposition in tumor cells rarely leads to clinical pigmentation of the tumor. Typically, selective staining methods are required to detect the pigment. OBSERVATION The authors describe a 60-year-old woman with a history of mammary carcinoma and an irregularly pigmented nodule with peripheral globules and a blue-white veil on dermoscopy, which was a clinical and dermoscopic mimic of malignant melanoma. CONCLUSIONS Awareness of melanocyte colonization of non-melanocytic tumor cells and the dermoscopic-histologic correlations can aid in avoiding a potential pitfall, and emphasize the importance of such relationships, when using this tool.
Collapse
Affiliation(s)
- Angela J Wyatt
- Department of Dermatology, New York Presbyterian Hospital, Weill Medical College of Cornell University, New York, New York 10022, USA
| | | | | | | | | |
Collapse
|
19
|
Cutaneous Metastatic Breast Carcinoma with Melanocyte Colonization. Dermatol Surg 2006. [DOI: 10.1097/00042728-200607000-00016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
|
20
|
Seidenari S, Pellacani G, Grana C. Pigment distribution in melanocytic lesion images: a digital parameter to be employed for computer-aided diagnosis. Skin Res Technol 2005; 11:236-41. [PMID: 16221139 DOI: 10.1111/j.0909-725x.2005.00123.x] [Citation(s) in RCA: 31] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
BACKGROUND/PURPOSE Since in early melanoma (MM) and especially in in situ MM differential structures, which are diagnostic for MM may be lacking, pigment distribution asymmetry represents an important diagnostic feature. Our aim was to automatically assess pigment distribution in images referring to MMs, atypical nevi (AN) and clearly benign nevi (BN), and to evaluate the diagnostic capability of numerical parameters describing a non homogeneous distribution of pigmentation. METHODS An image analysis program enabling the numerical assessment of pigment distribution in melanocytic lesions (ML), based on evaluation and comparison of red, green, blue (RGB) colour components inside image colour blocks, was employed on 459 videomicroscopic digital images, referring to 95 MMs, 76 AN and 288 BN. RESULTS Significant differences in pigment distribution parameters (mean RGB distance, variance and maximum distance) between the three ML populations were observed, permitting a good discrimination of MMs. On the test set comprising 230 lesion images, the area under the curve value of the receiver operating characteristic curve was 0.933. For a D score equal to 0, corresponding to the best diagnostic accuracy (86.6%), a sensitivity of 87.5% and a specificity of 85.7% were obtained. CONCLUSION This original evaluation method for digital pigment distribution, based on mathematical description and comparison of colours in different image blocks, provides numerical parameters to be implemented in image analysis programs for computer-aided MM diagnosis.
Collapse
Affiliation(s)
- S Seidenari
- Department of Dermatology, University of Modena and Reggio Emilia, Modena, Italy.
| | | | | |
Collapse
|
21
|
Seidenari S, Pellacani G, Grana C. Colors in atypical nevi: a computer description reproducing clinical assessment. Skin Res Technol 2005; 11:36-41. [PMID: 15691257 DOI: 10.1111/j.1600-0846.2005.00097.x] [Citation(s) in RCA: 15] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
BACKGROUND/PURPOSE Atypical nevi (AN) share some dermoscopic features with early melanoma (MM), and computer elaboration of digital images could represent a useful support to diagnosis to assess automatically colors in AN, and to compare the data with those referring to clearly benign nevi (BN) and MMs. METHODS An image analysis program enabling the numerical description of color areas in melanocytic lesions was used on 459 videomicroscopic images, referring to 76 AN, 288 clearly BN and 95 MMs. RESULTS Black, white and blue-gray were more frequently found in AN than in clearly BN, but less frequently than in MMs. Color area values significantly differed between the three groups. CONCLUSION The clinical-morphological interpretation of the numerical data, based on the mathematical description of the aspect and distribution of different color areas in different lesion types may contribute to the characterization of AN and their distinction from MMs.
Collapse
Affiliation(s)
- Stefania Seidenari
- Department of Dermatology, University of Modena and Reggio Emila, Italy.
| | | | | |
Collapse
|
22
|
Braun RP, Rabinovitz HS, Oliviero M, Kopf AW, Saurat JH. Dermoscopy of pigmented skin lesions. J Am Acad Dermatol 2005; 52:109-21. [DOI: 10.1016/j.jaad.2001.11.001] [Citation(s) in RCA: 222] [Impact Index Per Article: 11.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|
23
|
Pellacani G, Grana C, Seidenari S. Automated description of colours in polarized-light surface microscopy images of melanocytic lesions. Melanoma Res 2004; 14:125-30. [PMID: 15057042 DOI: 10.1097/00008390-200404000-00008] [Citation(s) in RCA: 16] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
The aim of this study was to develop a computerized method for the identification and description of colour areas in melanocytic lesion images based on an approach mimicking the human perception of colours. A colour palette comprising six colour groups (black, dark brown, light brown, blue-grey, red and white) was created by selecting single colour components within melanocytic lesion images acquired using a digital videomicroscope, and was implemented in the image analysis program. For each colour region, the area, the distance from the lesion centroid, the spread, the colour area distribution in the internal and the external part of the lesion, and asymmetries were assessed on 604 melanocytic lesion images in our image database. Black, white and blue-grey colour areas were detected more frequently in melanomas compared with naevi. Moreover, significant differences in colour descriptors were observed for each colour group, showing that colour areas are more unevenly distributed in melanomas compared with naevi. Using a discriminant analysis approach, the extension of dark, white and blue-grey areas and some descriptors of the distribution of the colour areas were identified as the most relevant colour parameters for differentiating between benign and malignant lesions. In conclusion, our automatic procedure breaks down the image into the colour areas used in the clinical examination process, and also supplies a description of their extension and distribution, with parameters that correlate with the clinical concepts of regularity and homogeneity.
Collapse
Affiliation(s)
- Giovanni Pellacani
- Department of Dermatology, University of Modena and Reggio Emila, 41100 Modena, Italy
| | | | | |
Collapse
|