1
|
Ding Y, Yi Z, Xiao J, Hu M, Guo Y, Liao Z, Wang Y. CTH-Net: A CNN and Transformer hybrid network for skin lesion segmentation. iScience 2024; 27:109442. [PMID: 38523786 PMCID: PMC10957498 DOI: 10.1016/j.isci.2024.109442] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2023] [Revised: 01/25/2024] [Accepted: 03/04/2024] [Indexed: 03/26/2024] Open
Abstract
Automatically and accurately segmenting skin lesions can be challenging, due to factors such as low contrast and fuzzy boundaries. This paper proposes a hybrid encoder-decoder model (CTH-Net) based on convolutional neural network (CNN) and Transformer, capitalizing on the advantages of these approaches. We propose three modules for skin lesion segmentation and seamlessly connect them with carefully designed model architecture. Better segmentation performance is achieved by introducing SoftPool in the CNN branch and sandglass block in the bottleneck layer. Extensive experiments were conducted on four publicly accessible skin lesion datasets, ISIC 2016, ISIC 2017, ISIC 2018, and PH2 to confirm the efficacy and benefits of the proposed strategy. Experimental results show that the proposed CTH-Net provides better skin lesion segmentation performance in both quantitative and qualitative testing when compared with state-of-the-art approaches. We believe the CTH-Net design is inspiring and can be extended to other applications/frameworks.
Collapse
Affiliation(s)
- Yuhan Ding
- School of Computer Science and Engineering, Central South University, Changsha 410083, China
| | - Zhenglin Yi
- Departments of Urology, Xiangya Hospital, Central South University, Changsha 410008, China
| | - Jiatong Xiao
- Departments of Urology, Xiangya Hospital, Central South University, Changsha 410008, China
| | - Minghui Hu
- Departments of Urology, Xiangya Hospital, Central South University, Changsha 410008, China
| | - Yu Guo
- Department of Burns and Plastic Surgery, Xiangya Hospital, Central South University, Changsha 410008, China
- National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Central South University, Changsha 410008, China
| | - Zhifang Liao
- School of Computer Science and Engineering, Central South University, Changsha 410083, China
| | - Yongjie Wang
- Department of Burns and Plastic Surgery, Xiangya Hospital, Central South University, Changsha 410008, China
- National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Central South University, Changsha 410008, China
| |
Collapse
|
2
|
Namburu A, Mohan S, Chakkaravarthy S, Selvaraj P. Skin Cancer Segmentation Based on Triangular Intuitionistic Fuzzy Sets. SN COMPUTER SCIENCE 2023; 4:228. [DOI: 10.1007/s42979-023-01701-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/12/2022] [Accepted: 01/20/2023] [Indexed: 09/15/2023]
|
3
|
Kaur R, GholamHosseini H, Sinha R, Lindén M. Automatic lesion segmentation using atrous convolutional deep neural networks in dermoscopic skin cancer images. BMC Med Imaging 2022; 22:103. [PMID: 35644612 PMCID: PMC9148511 DOI: 10.1186/s12880-022-00829-y] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2021] [Accepted: 04/13/2022] [Indexed: 12/16/2022] Open
Abstract
BACKGROUND Melanoma is the most dangerous and aggressive form among skin cancers, exhibiting a high mortality rate worldwide. Biopsy and histopathological analysis are standard procedures for skin cancer detection and prevention in clinical settings. A significant step in the diagnosis process is the deep understanding of the patterns, size, color, and structure of lesions based on images obtained through dermatoscopes for the infected area. However, the manual segmentation of the lesion region is time-consuming because the lesion evolves and changes its shape over time, making its prediction challenging. Moreover, it is challenging to predict melanoma at the initial stage as it closely resembles other skin cancer types that are not malignant as melanoma; thus, automatic segmentation techniques are required to design a computer-aided system for accurate and timely detection. METHODS As deep learning approaches have gained significant attention in recent years due to their remarkable performance, therefore, in this work, we proposed a novel design of a convolutional neural network (CNN) framework based on atrous convolutions for automatic lesion segmentation. This architecture is built based on the concept of atrous/dilated convolutions which are effective for semantic segmentation. A deep neural network is designed from scratch employing several building blocks consisting of convolutional, batch normalization, leakyReLU layer, and fine-tuned hyperparameters contributing altogether towards higher performance. CONCLUSION The network was tested on three benchmark datasets provided by International Skin Imaging Collaboration (ISIC), i.e., ISIC 2016, ISIC 2017, and ISIC 2018. The experimental results showed that the proposed network achieved an average Jaccard index of 90.4% on ISIC 2016, 81.8% on ISIC 2017, and 89.1% on ISIC 2018 datasets, respectively which is recorded as higher than the top three winners of the ISIC challenge and other state-of-the-art methods. Also, the model successfully extracts lesions from the whole image in one pass in less time, requiring no pre-processing step. The conclusions yielded that network is accurate in performing lesion segmentation on adopted datasets.
Collapse
Affiliation(s)
- Ranpreet Kaur
- School of Engineering, Computer, and Mathematical Sciences, Auckland University of Technology, 55 Wellesley street, 1010 Auckland, New Zealand
| | - Hamid GholamHosseini
- School of Engineering, Computer, and Mathematical Sciences, Auckland University of Technology, 55 Wellesley street, 1010 Auckland, New Zealand
| | - Roopak Sinha
- School of Engineering, Computer, and Mathematical Sciences, Auckland University of Technology, 55 Wellesley street, 1010 Auckland, New Zealand
| | - Maria Lindén
- School of Innovation Design and Engineering, Mälardalen University, Västerås, Sweden
| |
Collapse
|
4
|
Santos ESD, de M S Veras R, R T Aires K, M B F Portela H, Braz Junior G, Santos JD, Tavares JMR. Semi-automatic segmentation of skin lesions based on superpixels and hybrid texture information. Med Image Anal 2022; 77:102363. [DOI: 10.1016/j.media.2022.102363] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2021] [Revised: 12/13/2021] [Accepted: 01/10/2022] [Indexed: 10/19/2022]
|
5
|
Skin Lesion Extraction Using Multiscale Morphological Local Variance Reconstruction Based Watershed Transform and Fast Fuzzy C-Means Clustering. Symmetry (Basel) 2021. [DOI: 10.3390/sym13112085] [Citation(s) in RCA: 22] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022] Open
Abstract
Early identification of melanocytic skin lesions increases the survival rate for skin cancer patients. Automated melanocytic skin lesion extraction from dermoscopic images using the computer vision approach is a challenging task as the lesions present in the image can be of different colors, there may be a variation of contrast near the lesion boundaries, lesions may have different sizes and shapes, etc. Therefore, lesion extraction from dermoscopic images is a fundamental step for automated melanoma identification. In this article, a watershed transform based on the fast fuzzy c-means (FCM) clustering algorithm is proposed for the extraction of melanocytic skin lesion from dermoscopic images. Initially, the proposed method removes the artifacts from the dermoscopic images and enhances the texture regions. Further, it is filtered using a Gaussian filter and a local variance filter to enhance the lesion boundary regions. Later, the watershed transform based on MMLVR (multiscale morphological local variance reconstruction) is introduced to acquire the superpixels of the image with accurate boundary regions. Finally, the fast FCM clustering technique is implemented in the superpixels of the image to attain the final lesion extraction result. The proposed method is tested in the three publicly available skin lesion image datasets, i.e., ISIC 2016, ISIC 2017 and ISIC 2018. Experimental evaluation shows that the proposed method achieves a good result.
Collapse
|
6
|
Kassem MA, Hosny KM, Damaševičius R, Eltoukhy MM. Machine Learning and Deep Learning Methods for Skin Lesion Classification and Diagnosis: A Systematic Review. Diagnostics (Basel) 2021; 11:1390. [PMID: 34441324 PMCID: PMC8391467 DOI: 10.3390/diagnostics11081390] [Citation(s) in RCA: 63] [Impact Index Per Article: 21.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2021] [Revised: 07/25/2021] [Accepted: 07/27/2021] [Indexed: 12/04/2022] Open
Abstract
Computer-aided systems for skin lesion diagnosis is a growing area of research. Recently, researchers have shown an increasing interest in developing computer-aided diagnosis systems. This paper aims to review, synthesize and evaluate the quality of evidence for the diagnostic accuracy of computer-aided systems. This study discusses the papers published in the last five years in ScienceDirect, IEEE, and SpringerLink databases. It includes 53 articles using traditional machine learning methods and 49 articles using deep learning methods. The studies are compared based on their contributions, the methods used and the achieved results. The work identified the main challenges of evaluating skin lesion segmentation and classification methods such as small datasets, ad hoc image selection and racial bias.
Collapse
Affiliation(s)
- Mohamed A. Kassem
- Department of Robotics and Intelligent Machines, Faculty of Artificial Intelligence, Kaferelshiekh University, Kaferelshiekh 33511, Egypt;
| | - Khalid M. Hosny
- Department of Information Technology, Faculty of Computers and Informatics, Zagazig University, Zagazig 44519, Egypt
| | - Robertas Damaševičius
- Department of Applied Informatics, Vytautas Magnus University, 44404 Kaunas, Lithuania
| | - Mohamed Meselhy Eltoukhy
- Computer Science Department, Faculty of Computers and Informatics, Suez Canal University, Ismailia 41522, Egypt;
| |
Collapse
|
7
|
Level set approach based on Parzen Window and floor of log for edge computing object segmentation in digital images. Appl Soft Comput 2021. [DOI: 10.1016/j.asoc.2021.107273] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
|
8
|
|
9
|
Fast fully automatic skin lesions segmentation probabilistic with Parzen window. Comput Med Imaging Graph 2020; 85:101774. [PMID: 32835893 DOI: 10.1016/j.compmedimag.2020.101774] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2020] [Revised: 05/25/2020] [Accepted: 08/07/2020] [Indexed: 11/23/2022]
Abstract
Cutaneous melanoma accounts for over 90% of all melanoma, causing up to 55,500 annual deaths. However, it is a potentially curable type of cancer. Since melanoma is potentially curable, the disease's mortality rate is directly linked to late detection. This work proposes an approach that presents the balance between time and efficiency. This paper proposes the method of fast and automatic segmentation of skin lesions using probabilistic characteristics with the Parzen window (SPPW). The results obtained by the method were based on PH2 and ISIC datasets. The SPPW approach reached the following averages between the two datasets Specificity of 98.55%, Accuracy of 95.48%, Dice of 91.12%, Sensitivity of 88.45%, Mattheus of 87.86%, and Jaccard Index of 84.90%. The highlights of the proposed method are its short average segmentation time per image, and its metrics values, which are often higher than the ones obtained by other methods. Therefore, the SPPW method of segmentation is a quick, viable, and easily accessible option to aid in the diagnosis of diseased skin.
Collapse
|
10
|
Khatibi T, Rezaei N, Ataei Fashtami L, Totonchi M. Proposing a novel unsupervised stack ensemble of deep and conventional image segmentation (SEDCIS) method for localizing vitiligo lesions in skin images. Skin Res Technol 2020; 27:126-137. [PMID: 32662570 DOI: 10.1111/srt.12920] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2020] [Accepted: 06/20/2020] [Indexed: 11/26/2022]
Abstract
BACKGROUND Vitiligo is an acquired pigmentary skin disorder characterized by depigmented macules and patches which brings many challenges for the patients suffering from. For vitiligo severity assessment, several scoring methods have been proposed based on morphometry and colorimetry. But, all methods suffer from much inter- and intra-observer variations for estimating the depigmented area. For all mentioned assessment methods of vitiligo disorder, accurate segmentation of the skin images for lesion detection and localization is required. The image segmentation for localizing vitiligo skin lesions has many challenges because of illumination variation, different shapes and sizes of vitiligo lesions, vague lesion boundaries and skin hairs and vignette effects. The manual image segmentation is a tedious and time-consuming task. Therefore, using automatic image segmentation methods for lesion detection is necessarily required. MATERIALS AND METHODS In this study, a novel unsupervised stack ensemble of deep and conventional image segmentation (SEDCIS) methods is proposed for localizing vitiligo lesions in skin images. Unsupervised segmentation methods do not require prior manual segmentation of vitiligo lesions which is a tedious and time-consuming task with intra- and inter-observer variations. RESULTS Our collected dataset includes 877 images taken from 21 patients with the resolution of 5760*3840 pixels suffering from vitiligo disorder. Experimental results show that SEDCIS outperforms the compared methods with accuracy of 97%, sensitivity of 98%, specificity of 96%, area overlapping of 94%, and Dice index of 97%. CONCLUSION The proposed method can segment vitiligo lesions with highly reasonable performance and can be used for assessing the vitiligo lesion surface.
Collapse
Affiliation(s)
- Toktam Khatibi
- School of Industrial and Systems Engineering, Tarbiat Modares University (TMU), Tehran, Iran
| | - Niloofar Rezaei
- School of Industrial and Systems Engineering, Tarbiat Modares University (TMU), Tehran, Iran
| | - Leila Ataei Fashtami
- Department of Regenerative Medicine, Royan Institute for Stem Cell Biology & Technology, Tehran, Iran
| | - Mehdi Totonchi
- Department of Reproductive Imaging, Reproductive Biomedicine Research Center, Royan Institute for Reproductive Biomedicine, ACECR, Tehran, Iran
- Department of Andrology, Reproductive Biomedicine Research Center, Royan Institute for Reproductive Biomedicine, ACECR, Tehran, Iran
| |
Collapse
|