1
|
Din S, Mourad O, Serpedin E. LSCS-Net: A lightweight skin cancer segmentation network with densely connected multi-rate atrous convolution. Comput Biol Med 2024; 173:108303. [PMID: 38547653 DOI: 10.1016/j.compbiomed.2024.108303] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2023] [Revised: 01/18/2024] [Accepted: 03/12/2024] [Indexed: 04/17/2024]
Abstract
The rising occurrence and notable public health consequences of skin cancer, especially of the most challenging form known as melanoma, have created an urgent demand for more advanced approaches to disease management. The integration of modern computer vision methods into clinical procedures offers the potential for enhancing the detection of skin cancer . The UNet model has gained prominence as a valuable tool for this objective, continuously evolving to tackle the difficulties associated with the inherent diversity of dermatological images. These challenges stem from diverse medical origins and are further complicated by variations in lighting, patient characteristics, and hair density. In this work, we present an innovative end-to-end trainable network crafted for the segmentation of skin cancer . This network comprises an encoder-decoder architecture, a novel feature extraction block, and a densely connected multi-rate Atrous convolution block. We evaluated the performance of the proposed lightweight skin cancer segmentation network (LSCS-Net) on three widely used benchmark datasets for skin lesion segmentation: ISIC 2016, ISIC 2017, and ISIC 2018. The generalization capabilities of LSCS-Net are testified by the excellent performance on breast cancer and thyroid nodule segmentation datasets. The empirical findings confirm that LSCS-net attains state-of-the-art results, as demonstrated by a significantly elevated Jaccard index.
Collapse
Affiliation(s)
- Sadia Din
- Electrical and Computer Engineering Program, Texas A&M University, Doha, Qatar.
| | | | - Erchin Serpedin
- Department of Electrical and Computer Engineering, Texas A&M University, College Station, TX, USA
| |
Collapse
|
2
|
Zhi Y, Bie H, Wang J, Ren L. Masked autoencoders with generalizable self-distillation for skin lesion segmentation. Med Biol Eng Comput 2024:10.1007/s11517-024-03086-z. [PMID: 38653880 DOI: 10.1007/s11517-024-03086-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2023] [Accepted: 03/29/2024] [Indexed: 04/25/2024]
Abstract
In the field of skin lesion image segmentation, accurate identification and partitioning of diseased regions is of vital importance for in-depth analysis of skin cancer. Self-supervised learning, i.e., MAE, has emerged as a potent force in the medical imaging domain, which autonomously learns and extracts latent features from unlabeled data, thereby yielding pre-trained models that greatly assist downstream tasks. To encourage pre-trained models to more comprehensively learn the global structural and local detail information inherent in dermoscopy images, we introduce a Teacher-Student architecture, named TEDMAE, by incorporating a self-distillation mechanism, it learns holistic image feature information to improve the generalizable global knowledge learning of the student MAE model. To make the image features learned by the model suitable for unknown test images, two optimization strategies are, Exterior Conversion Augmentation (EC) utilizes random convolutional kernels and linear interpolation to effectively transform the input image into one with the same shape but altered intensities and textures, while Dynamic Feature Generation (DF) employs a nonlinear attention mechanism for feature merging, enhancing the expressive power of the features, are proposed to enhance the generalizability of global features learned by the teacher model, thereby improving the overall generalization capability of the pre-trained models. Experimental results from the three public skin disease datasets, ISIC2019, ISIC2017, and PH2 indicate that our proposed TEDMAE method outperforms several similar approaches. Specifically, TEDMAE demonstrated optimal segmentation and generalization performance on the ISIC2017 and PH2 datasets, with Dice scores reaching 82.1% and 91.2%, respectively. The best Jaccard values were 72.6% and 84.5%, while the optimal HD95% values were 13.0% and 8.9%, respectively.
Collapse
Affiliation(s)
- Yichen Zhi
- Department of Intelligent Media Computing Center, School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, 100876, People's Republic of China
| | - Hongxia Bie
- Department of Intelligent Media Computing Center, School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, 100876, People's Republic of China.
| | - Jiali Wang
- Department of Intelligent Media Computing Center, School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, 100876, People's Republic of China
| | - Lihan Ren
- Department of Intelligent Media Computing Center, School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, 100876, People's Republic of China
| |
Collapse
|
3
|
Li Y, Tian T, Hu J, Yuan C. SUTrans-NET: a hybrid transformer approach to skin lesion segmentation. PeerJ Comput Sci 2024; 10:e1935. [PMID: 38660200 PMCID: PMC11042008 DOI: 10.7717/peerj-cs.1935] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2023] [Accepted: 02/18/2024] [Indexed: 04/26/2024]
Abstract
Melanoma is a malignant skin tumor that threatens human life and health. Early detection is essential for effective treatment. However, the low contrast between melanoma lesions and normal skin and the irregularity in size and shape make skin lesions difficult to detect with the naked eye in the early stages, making the task of skin lesion segmentation challenging. Traditional encoder-decoder built with U-shaped networks using convolutional neural network (CNN) networks have limitations in establishing long-term dependencies and global contextual connections, while the Transformer architecture is limited in its application to small medical datasets. To address these issues, we propose a new skin lesion segmentation network, SUTrans-NET, which combines CNN and Transformer in a parallel fashion to form a dual encoder, where both CNN and Transformer branches perform dynamic interactive fusion of image information in each layer. At the same time, we introduce our designed multi-grouping module SpatialGroupAttention (SGA) to complement the spatial and texture information of the Transformer branch, and utilize the Focus idea of YOLOV5 to construct the Patch Embedding module in the Transformer to prevent the loss of pixel accuracy. In addition, we design a decoder with full-scale information fusion capability to fully fuse shallow and deep features at different stages of the encoder. The effectiveness of our method is demonstrated on the ISIC 2016, ISIC 2017, ISIC 2018 and PH2 datasets and its advantages over existing methods are verified.
Collapse
Affiliation(s)
- Yaqin Li
- School of Mathematics and Computer Science, Wuhan Polytechnic University School, Wuhan, Hubei, China
| | - Tonghe Tian
- School of Mathematics and Computer Science, Wuhan Polytechnic University School, Wuhan, Hubei, China
| | - Jing Hu
- School of Mathematics and Computer Science, Wuhan Polytechnic University School, Wuhan, Hubei, China
| | - Cao Yuan
- School of Mathematics and Computer Science, Wuhan Polytechnic University School, Wuhan, Hubei, China
| |
Collapse
|
4
|
Benčević M, Habijan M, Galić I, Babin D, Pižurica A. Understanding skin color bias in deep learning-based skin lesion segmentation. Comput Methods Programs Biomed 2024; 245:108044. [PMID: 38290289 DOI: 10.1016/j.cmpb.2024.108044] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/20/2023] [Revised: 01/17/2024] [Accepted: 01/21/2024] [Indexed: 02/01/2024]
Abstract
BACKGROUND The field of dermatological image analysis using deep neural networks includes the semantic segmentation of skin lesions, pivotal for lesion analysis, pathology inference, and diagnoses. While biases in neural network-based dermatoscopic image classification against darker skin tones due to dataset imbalance and contrast disparities are acknowledged, a comprehensive exploration of skin color bias in lesion segmentation models is lacking. It is imperative to address and understand the biases in these models. METHODS Our study comprehensively evaluates skin tone bias within prevalent neural networks for skin lesion segmentation. Since no information about skin color exists in widely used datasets, to quantify the bias we use three distinct skin color estimation methods: Fitzpatrick skin type estimation, Individual Typology Angle estimation as well as manual grouping of images by skin color. We assess bias across common models by training a variety of U-Net-based models on three widely-used datasets with 1758 different dermoscopic and clinical images. We also evaluate commonly suggested methods to mitigate bias. RESULTS Our findings expose a significant and large correlation between segmentation performance and skin color, revealing consistent challenges in segmenting lesions for darker skin tones across diverse datasets. Using various methods of skin color quantification, we have found significant bias in skin lesion segmentation against darker-skinned individuals when evaluated both in and out-of-sample. We also find that commonly used methods for bias mitigation do not result in any significant reduction in bias. CONCLUSIONS Our findings suggest a pervasive bias in most published lesion segmentation methods, given our use of commonly employed neural network architectures and publicly available datasets. In light of our findings, we propose recommendations for unbiased dataset collection, labeling, and model development. This presents the first comprehensive evaluation of fairness in skin lesion segmentation.
Collapse
Affiliation(s)
- Marin Benčević
- J. J. Strossmayer University of Osijek, Faculty of Electrical Engineering, Computer Science and Information Technology Osijek, Kneza Trpimira 2B, Osijek, 31000, Croatia; Ghent University, Department of Telecommunications and Information Processing, TELIN-GAIM, St-Pietersnieuwstraat 41, Ghent, 9000, Belgium.
| | - Marija Habijan
- J. J. Strossmayer University of Osijek, Faculty of Electrical Engineering, Computer Science and Information Technology Osijek, Kneza Trpimira 2B, Osijek, 31000, Croatia
| | - Irena Galić
- J. J. Strossmayer University of Osijek, Faculty of Electrical Engineering, Computer Science and Information Technology Osijek, Kneza Trpimira 2B, Osijek, 31000, Croatia
| | - Danilo Babin
- Ghent University, Department of Telecommunications and Information Processing, imec-TELIN-IPI, St-Pietersnieuwstraat 41, Ghent, 9000, Belgium
| | - Aleksandra Pižurica
- Ghent University, Department of Telecommunications and Information Processing, TELIN-GAIM, St-Pietersnieuwstraat 41, Ghent, 9000, Belgium
| |
Collapse
|
5
|
Luo X, Zhang H, Huang X, Gong H, Zhang J. DBNet-SI: Dual branch network of shift window attention and inception structure for skin lesion segmentation. Comput Biol Med 2024; 170:108090. [PMID: 38320341 DOI: 10.1016/j.compbiomed.2024.108090] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Revised: 12/27/2023] [Accepted: 01/27/2024] [Indexed: 02/08/2024]
Abstract
The U-shaped convolutional neural network (CNN) has attained remarkable achievements in the segmentation of skin lesion. However, given the inherent locality of convolution, this architecture cannot capture long-range pixel dependencies and multiscale global contextual information effectively. Moreover, repeated convolutions and downsampling operations can readily result in the omission of intricate local fine-grained details. In this paper, we proposed a U-shaped network (DBNet-SI) equipped with a dual-branch module that combines shift window attention and inception structures. First, we proposed a dual-branch module that combines shift window attention and inception structures (MSI) to better capture multiscale global contextual information and long-range pixel dependencies. Specifically, we have devised a cross-branch bidirectional interaction module within the MSI module to enable information complementarity between the two branches in the channel and spatial dimensions. Therefore, MSI is capable of extracting distinguishing and comprehensive features to accurately identify the skin lesion boundaries. Second, we have devised a progressive feature enhancement and information compensation module (PFEIC), which progressively compensates for fine-grained features through reconstructed skip connections and integrated global context attention modules. The results of the experiment show the superior segmentation performance of DBNet-SI compared with other deep learning models for skin lesion segmentation in the ISIC2017 and ISIC2018 datasets. Ablation studies demonstrate that our model can effectively extract rich multiscale global contextual information and compensate for the loss of local details.
Collapse
Affiliation(s)
- Xuqiong Luo
- School of Mathematics and Statistics, Changsha University of Science and Technology, ChangSha 410114, China
| | - Hao Zhang
- School of Mathematics and Statistics, Changsha University of Science and Technology, ChangSha 410114, China
| | - Xiaofei Huang
- School of Mathematics and Statistics, Changsha University of Science and Technology, ChangSha 410114, China
| | - Hongfang Gong
- School of Mathematics and Statistics, Changsha University of Science and Technology, ChangSha 410114, China.
| | - Jin Zhang
- School of Computer and Communication Engineering, Changsha University of Science and Technology, ChangSha 410114, China
| |
Collapse
|
6
|
Wu R, Lv H, Liang P, Cui X, Chang Q, Huang X. HSH-UNet: Hybrid selective high order interactive U-shaped model for automated skin lesion segmentation. Comput Biol Med 2024; 168:107798. [PMID: 38043470 DOI: 10.1016/j.compbiomed.2023.107798] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2023] [Revised: 10/31/2023] [Accepted: 11/29/2023] [Indexed: 12/05/2023]
Abstract
The use of computer-assisted clinical dermatologists to diagnose skin diseases is an important aid. And computer-assisted techniques mainly use deep neural networks. Recently, the proposal of higher-order spatial interaction operations in deep neural networks has attracted a lot of attention. It has the advantages of both convolution and transformers, and additionally has the advantages of efficient, extensible and translation-equivariant. However, the selection of the interaction order in higher-order interaction operations requires tedious manual selection of a suitable interaction order. In this paper, a hybrid selective higher-order interaction U-shaped model HSH-UNet is proposed to solve the problem that requires manual selection of the order. Specifically, we design a hybrid selective high-order interaction module HSHB embedded in the U-shaped model. The HSHB adaptively selects the appropriate order for the interaction operation channel-by-channel under the computationally obtained guiding features. The hybrid order interaction also solves the problem of fixed order of interaction at each level. We performed extensive experiments on three public skin lesion datasets and our own dataset to validate the effectiveness of our proposed method. The ablation experiments demonstrate the effectiveness of our hybrid selective higher order interaction module. The comparison with state-of-the-art methods also demonstrates the superiority of our proposed HSH-UNet performance. The code is available at https://github.com/wurenkai/HSH-UNet.
Collapse
Affiliation(s)
- Renkai Wu
- Department of Dermatology, RuiJin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China; Department of Surgery, Shanghai Key Laboratory of Gastric Neoplasms, Shanghai Institute of Digestive Surgery, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China; School of Microelectronics, Shanghai University, Shanghai, China
| | - Hongli Lv
- Department of Dermatology, Jiading District Central Hospital Affiliated Shanghai University of Medicine and Health Sciences, Shanghai, China
| | - Pengchen Liang
- Department of Surgery, Shanghai Key Laboratory of Gastric Neoplasms, Shanghai Institute of Digestive Surgery, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China; School of Microelectronics, Shanghai University, Shanghai, China
| | - Xiaoxu Cui
- Department of Surgery, Shanghai Key Laboratory of Gastric Neoplasms, Shanghai Institute of Digestive Surgery, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China; Department of Dermatology, Jiading District Central Hospital Affiliated Shanghai University of Medicine and Health Sciences, Shanghai, China
| | - Qing Chang
- Department of Surgery, Shanghai Key Laboratory of Gastric Neoplasms, Shanghai Institute of Digestive Surgery, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
| | - Xuan Huang
- Department of Dermatology, RuiJin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
| |
Collapse
|
7
|
Zhu W, Tian J, Chen M, Chen L, Chen J. MSS-UNet: A Multi-Spatial-Shift MLP-based UNet for skin lesion segmentation. Comput Biol Med 2024; 168:107719. [PMID: 38007976 DOI: 10.1016/j.compbiomed.2023.107719] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2023] [Revised: 10/17/2023] [Accepted: 11/15/2023] [Indexed: 11/28/2023]
Abstract
Multilayer perceptron (MLP) networks have become a popular alternative to convolutional neural networks and transformers because of fewer parameters. However, existing MLP-based models improve performance by increasing model depth, which adds computational complexity when processing local features of images. To meet this challenge, we propose MSS-UNet, a lightweight convolutional neural network (CNN) and MLP model for the automated segmentation of skin lesions from dermoscopic images. Specifically, MSS-UNet first uses the convolutional module to extract local information, which is essential for precisely segmenting the skin lesion. We propose an efficient double-spatial-shift MLP module, named DSS-MLP, which enhances the vanilla MLP by enabling communication between different spatial locations through double spatial shifts. We also propose a module named MSSEA with multiple spatial shifts of different strides and lighter external attention to enlarge the local receptive field and capture the boundary continuity of skin lesions. We extensively evaluated the MSS-UNet on ISIC 2017, 2018, and PH2 skin lesion datasets. On three datasets, the method achieves IoU metrics of 85.01%±0.65, 83.65%±1.05, and 92.71%±1.03, with a parameter size and computational complexity of 0.33M and 15.98G, respectively, outperforming most state-of-the-art methods.The code is publicly available at https://github.com/AirZWH/MSS-UNet.
Collapse
Affiliation(s)
- Wenhao Zhu
- Computer School, University of South China, Hengyang, China
| | - Jiya Tian
- School of Information Engineering, Xinjiang Institute of Technology, Aksu, China
| | - Mingzhi Chen
- School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Lingna Chen
- Computer School, University of South China, Hengyang, China.
| | - Junxi Chen
- Affiliated Nanhua Hospital, University of South China, Hengyang, China.
| |
Collapse
|
8
|
He Z, Li X, Chen Y, Lv N, Cai Y. Attention-based dual-path feature fusion network for automatic skin lesion segmentation. BioData Min 2023; 16:28. [PMID: 37807076 PMCID: PMC10561442 DOI: 10.1186/s13040-023-00345-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2023] [Accepted: 09/27/2023] [Indexed: 10/10/2023] Open
Abstract
Automatic segmentation of skin lesions is a critical step in Computer Aided Diagnosis (CAD) of melanoma. However, due to the blurring of the lesion boundary, uneven color distribution, and low image contrast, resulting in poor segmentation result. Aiming at the problem of difficult segmentation of skin lesions, this paper proposes an Attention-based Dual-path Feature Fusion Network (ADFFNet) for automatic skin lesion segmentation. Firstly, in the spatial path, a Boundary Refinement (BR) module is designed for the output of low-level features to filter out irrelevant background information and retain more boundary details of the lesion area. Secondly, in the context path, a Multi-scale Feature Selection (MFS) module is constructed for high-level feature output to capture multi-scale context information and use the attention mechanism to filter out redundant semantic information. Finally, we design a Dual-path Feature Fusion (DFF) module, which uses high-level global attention information to guide the step-by-step fusion of high-level semantic features and low-level detail features, which is beneficial to restore image detail information and further improve the pixel-level segmentation accuracy of skin lesion. In the experiment, the ISIC 2018 and PH2 datasets are employed to evaluate the effectiveness of the proposed method. It achieves a performance of 0.890/ 0.925 and 0.933 /0.954 on the F1-score and SE index, respectively. Comparative analysis with state-of-the-art segmentation methods reveals that the ADFFNet algorithm exhibits superior segmentation performance.
Collapse
Affiliation(s)
- Zhenxiang He
- School of Information Engineering, Southwest University of Science and Technology, Mianyang, China
- Tianfu College of Southwest University of Finance and Economics, Mianyang, China
| | - Xiaoxia Li
- School of Information Engineering, Southwest University of Science and Technology, Mianyang, China
- Robot Technology Used for Special Environment Key Laboratory of Sichuan Province, Mianyang, China
| | - Yuling Chen
- School of Information Engineering, Southwest University of Science and Technology, Mianyang, China
- Robot Technology Used for Special Environment Key Laboratory of Sichuan Province, Mianyang, China
| | - Nianzu Lv
- School of Information Engineering, Southwest University of Science and Technology, Mianyang, China
- Robot Technology Used for Special Environment Key Laboratory of Sichuan Province, Mianyang, China
| | - Yong Cai
- School of manufacturing science and Engineering, Southwest University of Science and Technology, Mianyang, China.
| |
Collapse
|
9
|
Zhou L, Liang L, Sheng X. GA-Net: Ghost convolution adaptive fusion skin lesion segmentation network. Comput Biol Med 2023; 164:107273. [PMID: 37562327 DOI: 10.1016/j.compbiomed.2023.107273] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2023] [Revised: 06/30/2023] [Accepted: 07/16/2023] [Indexed: 08/12/2023]
Abstract
Automatic segmentation of skin lesions is a pivotal task in computer-aided diagnosis, playing a crucial role in the early detection and treatment of skin cancer. Despite the existence of numerous deep learning-based segmentation methods, the extraction of lesion features remains inadequate as a result of the segmentation process. Consequently, skin lesion image segmentation continues to face challenges regarding missing detailed information and inaccurate segmentation of the lesion region. In this paper, we propose a ghost convolution adaptive fusion network for skin lesion segmentation. First, the neural network incorporates a ghost module instead of the ordinary convolution layer, generating a rich skin lesion feature map for comprehensive target feature extraction. Subsequently, the network employs an adaptive fusion module and bilateral attention module to connect the encoding and decoding layers, facilitating the integration of shallow and deep network information. Moreover, multi-level output patterns are used for pixel prediction. Layer feature fusion effectively combines output features of different scales, thus improving image segmentation accuracy. The proposed network was extensively evaluated on three publicly available datasets: ISIC2016, ISIC2017, and ISIC2018. The experimental results demonstrated accuracies of 96.42%, 94.07%, and 95.03%, and kappa coefficients of 90.41%, 81.08%, and 86.96%, respectively. The overall performance of our network surpassed that of existing networks. Simulation experiments further revealed that the ghost convolution adaptive fusion network exhibited superior segmentation results for skin lesion images, offering new possibilities for the diagnosis of skin diseases.
Collapse
Affiliation(s)
- Longsong Zhou
- School of Electrical Engineering and Automation, Jiangxi University of Science and Technology, Ganzhou, Jiangxi, 341000, China; Jinguan Copper Branch of Tongling Nonferrous Metals Group Co, Ltd, Tongling, Anhui, 244100, China
| | - Liming Liang
- School of Electrical Engineering and Automation, Jiangxi University of Science and Technology, Ganzhou, Jiangxi, 341000, China
| | - Xiaoqi Sheng
- School of Computer Science and Engineering, South China University of Technology, Guangzhou, Guangdong, 510006, China.
| |
Collapse
|
10
|
Yu Z, Yu L, Zheng W, Wang S. EIU-Net: Enhanced feature extraction and improved skip connections in U-Net for skin lesion segmentation. Comput Biol Med 2023; 162:107081. [PMID: 37301097 DOI: 10.1016/j.compbiomed.2023.107081] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2022] [Revised: 04/25/2023] [Accepted: 05/27/2023] [Indexed: 06/12/2023]
Abstract
Skin lesion segmentation is a computer-aided diagnosis method for quantitative analysis of melanoma that can improve efficiency and accuracy. Although many methods based on U-Net have achieved tremendous success, they still cannot handle challenging tasks well due to weak feature extraction. In response to skin lesion segmentation, a novel method called EIU-Net is proposed to tackle the challenging task. To capture the local and global contextual information, we employ inverted residual blocks and an efficient pyramid squeeze attention (EPSA) block as the main encoders at different stages, while atrous spatial pyramid pooling (ASPP) is utilized after the last encoder and the soft-pool method is introduced for downsampling. Also, we propose a novel method named multi-layer fusion (MLF) module to effectively fuse the feature distributions and capture significant boundary information of skin lesions in different encoders to improve the performance of the network. Furthermore, a reshaped decoders fusion module is used to obtain multi-scale information by fusing feature maps of different decoders to improve the final results of skin lesion segmentation. To validate the performance of our proposed network, we compare it with other methods on four public datasets, including the ISIC 2016, ISIC 2017, ISIC 2018, and PH2 datasets. And the main metric Dice scores achieved by our proposed EIU-Net are 0.919, 0.855, 0.902, and 0.916 on the four datasets, respectively, outperforming other methods. Ablation experiments also demonstrate the effectiveness of the main modules in our proposed network. Our code is available at https://github.com/AwebNoob/EIU-Net.
Collapse
Affiliation(s)
- Zimin Yu
- Department of Computer Science and Engineering, School of Information Science and Engineering, Yunnan University, Kunming, 650504, Yunnan, China
| | - Li Yu
- Department of Computer Science and Engineering, School of Information Science and Engineering, Yunnan University, Kunming, 650504, Yunnan, China
| | - Weihua Zheng
- Department of Computer Science and Engineering, School of Information Science and Engineering, Yunnan University, Kunming, 650504, Yunnan, China
| | - Shunfang Wang
- Department of Computer Science and Engineering, School of Information Science and Engineering, Yunnan University, Kunming, 650504, Yunnan, China; The Key Lab of Intelligent Systems and Computing of Yunnan Province, Yunnan University, Kunming, Yunnan, China.
| |
Collapse
|
11
|
Qin C, Zheng B, Zeng J, Chen Z, Zhai Y, Genovese A, Piuri V, Scotti F. Dynamically aggregating MLPs and CNNs for skin lesion segmentation with geometry regularization. Comput Methods Programs Biomed 2023; 238:107601. [PMID: 37210926 DOI: 10.1016/j.cmpb.2023.107601] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Revised: 04/24/2023] [Accepted: 05/13/2023] [Indexed: 05/23/2023]
Abstract
BACKGROUND AND OBJECTIVE Melanoma is a highly malignant skin tumor. Accurate segmentation of skin lesions from dermoscopy images is pivotal for computer-aided diagnosis of melanoma. However, blurred lesion boundaries, variable lesion shapes, and other interference factors pose a challenge in this regard. METHODS This work proposes a novel framework called CFF-Net (Cross Feature Fusion Network) for supervised skin lesion segmentation. The encoder of the network includes dual branches, where the CNNs branch aims to extract rich local features while MLPs branch is used to establish both the global-spatial-dependencies and global-channel-dependencies for precise delineation of skin lesions. Besides, a feature-interaction module between two branches is designed for strengthening the feature representation by allowing dynamic exchange of spatial and channel information, so as to retain more spatial details and inhibit irrelevant noise. Moreover, an auxiliary prediction task is introduced to learn the global geometric information, highlighting the boundary of the skin lesion. RESULTS Comprehensive experiments using four publicly available skin lesion datasets (i.e., ISIC 2018, ISIC 2017, ISIC 2016, and PH2) indicated that CFF-Net outperformed the state-of-the-art models. In particular, CFF-Net greatly increased the average Jaccard Index score from 79.71% to 81.86% in ISIC 2018, from 78.03% to 80.21% in ISIC 2017, from 82.58% to 85.38% in ISIC 2016, and from 84.18% to 89.71% in PH2 compared with U-Net. Ablation studies demonstrated the effectiveness of each proposed component. Cross-validation experiments in ISIC 2018 and PH2 datasets verified the generalizability of CFF-Net under different skin lesion data distributions. Finally, comparison experiments using three public datasets demonstrated the superior performance of our model. CONCLUSION The proposed CFF-Net performed well in four public skin lesion datasets, especially for challenging cases with blurred edges of skin lesions and low contrast between skin lesions and background. CFF-Net can be employed for other segmentation tasks with better prediction and more accurate delineation of boundaries.
Collapse
Affiliation(s)
- Chuanbo Qin
- Faculty of Intelligent Manufacturing, Wuyi University, Jiangmen 529020, China
| | - Bin Zheng
- Faculty of Intelligent Manufacturing, Wuyi University, Jiangmen 529020, China
| | - Junying Zeng
- Faculty of Intelligent Manufacturing, Wuyi University, Jiangmen 529020, China.
| | - Zhuyuan Chen
- Faculty of Intelligent Manufacturing, Wuyi University, Jiangmen 529020, China
| | - Yikui Zhai
- Faculty of Intelligent Manufacturing, Wuyi University, Jiangmen 529020, China
| | - Angelo Genovese
- Departimento di Information, Università degli Studi di Milano, 20133 Milano, Italy
| | - Vincenzo Piuri
- Departimento di Information, Università degli Studi di Milano, 20133 Milano, Italy
| | - Fabio Scotti
- Departimento di Information, Università degli Studi di Milano, 20133 Milano, Italy
| |
Collapse
|
12
|
Yang L, Fan C, Lin H, Qiu Y. Rema-Net: An efficient multi-attention convolutional neural network for rapid skin lesion segmentation. Comput Biol Med 2023; 159:106952. [PMID: 37084639 DOI: 10.1016/j.compbiomed.2023.106952] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Revised: 04/10/2023] [Accepted: 04/15/2023] [Indexed: 04/23/2023]
Abstract
For clinical treatment, the accurate segmentation of lesions from dermoscopic images is extremely valuable. Convolutional neural networks (such as U-Net and its numerous variants) have become the main methods for skin lesion segmentation in recent years. However, because these methods frequently have a large number of parameters and complicated algorithm structures, which results in high hardware requirements and long training time, it is difficult to effectively use them for fast training and segmentation tasks. For this reason, we proposed an efficient multi-attention convolutional neural network (Rema-Net) for rapid skin lesion segmentation. The down-sampling module of the network only uses a convolutional layer and a pooling layer, with spatial attention added to improve useful features. We also designed skip-connections between the down-sampling and up-sampling parts of the network, and used reverse attention operation on the skip-connections to strengthen segmentation performance of the network. We conducted extensive experiments on five publicly available datasets to validate the effectiveness of our method, including the ISIC-2016, ISIC-2017, ISIC-2018, PH2, and HAM10000 datasets. The results show that the proposed method reduced the number of parameters by nearly 40% when compared with U-Net. Furthermore, the segmentation metrics are significantly better than some previous methods, and the predictions are closer to the real lesion.
Collapse
Affiliation(s)
- Litao Yang
- School of Information Science and Engineering, Henan University of Technology, Zhengzhou City, Henan Province, 450001, China
| | - Chao Fan
- School of Artificial Intelligence and Big Data, Henan University of Technology, Zhengzhou City, Henan Province, 450001, China; Key Laboratory of Grain Information Processing and Control, Ministry of Education, Zhengzhou City, Henan Province, 450001, China.
| | - Hao Lin
- School of Information Science and Engineering, Henan University of Technology, Zhengzhou City, Henan Province, 450001, China
| | - Yingying Qiu
- School of Information Science and Engineering, Henan University of Technology, Zhengzhou City, Henan Province, 450001, China
| |
Collapse
|
13
|
Karri M, Annavarapu CSR, Acharya UR. Skin lesion segmentation using two-phase cross-domain transfer learning framework. Comput Methods Programs Biomed 2023; 231:107408. [PMID: 36805279 DOI: 10.1016/j.cmpb.2023.107408] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/25/2022] [Revised: 01/31/2023] [Accepted: 02/04/2023] [Indexed: 06/18/2023]
Abstract
BACKGROUND AND OBJECTIVE Deep learning (DL) models have been used for medical imaging for a long time but they did not achieve their full potential in the past because of insufficient computing power and scarcity of training data. In recent years, we have seen substantial growth in DL networks because of improved technology and an abundance of data. However, previous studies indicate that even a well-trained DL algorithm may struggle to generalize data from multiple sources because of domain shifts. Additionally, ineffectiveness of basic data fusion methods, complexity of segmentation target and low interpretability of current DL models limit their use in clinical decisions. To meet these challenges, we present a new two-phase cross-domain transfer learning system for effective skin lesion segmentation from dermoscopic images. METHODS Our system is based on two significant technical inventions. We examine a two- phase cross-domain transfer learning approach, including model-level and data-level transfer learning, by fine-tuning the system on two datasets, MoleMap and ImageNet. We then present nSknRSUNet, a high-performing DL network, for skin lesion segmentation using broad receptive fields and spatial edge attention feature fusion. We examine the trained model's generalization capabilities on skin lesion segmentation to quantify these two inventions. We cross-examine the model using two skin lesion image datasets, MoleMap and HAM10000, obtained from varied clinical contexts. RESULTS At data-level transfer learning for the HAM10000 dataset, the proposed model obtained 94.63% of DSC and 99.12% accuracy. In cross-examination at data-level transfer learning for the Molemap dataset, the proposed model obtained 93.63% of DSC and 97.01% of accuracy. CONCLUSION Numerous experiments reveal that our system produces excellent performance and improves upon state-of-the-art methods on both qualitative and quantitative measures.
Collapse
Affiliation(s)
- Meghana Karri
- Department of Computer Science and Engineering, Indian Institute of Technology (Indian School of Mines), Dhanbad, 826004, Jharkhand, India.
| | - Chandra Sekhara Rao Annavarapu
- Department of Computer Science and Engineering, Indian Institute of Technology (Indian School of Mines), Dhanbad, 826004, Jharkhand, India.
| | - U Rajendra Acharya
- Ngee Ann Polytechnic, Department of Electronics and Computer Engineering, 599489, Singapore; Department of Biomedical Engineering, School of science and Technology, SUSS university, Singapore; Department of Biomedical Informatics and Medical Engineering, Asia university, Taichung, Taiwan.
| |
Collapse
|
14
|
Zhou X, Tong T, Zhong Z, Fan H, Li Z. Saliency-CCE: Exploiting colour contextual extractor and saliency-based biomedical image segmentation. Comput Biol Med 2023; 154:106551. [PMID: 36716685 DOI: 10.1016/j.compbiomed.2023.106551] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2022] [Revised: 01/03/2023] [Accepted: 01/11/2023] [Indexed: 01/21/2023]
Abstract
Biomedical image segmentation is one critical component in computer-aided system diagnosis. However, various non-automatic segmentation methods are usually designed to segment target objects with single-task driven, ignoring the potential contribution of multi-task, such as the salient object detection (SOD) task and the image segmentation task. In this paper, we propose a novel dual-task framework for white blood cell (WBC) and skin lesion (SL) saliency detection and segmentation in biomedical images, called Saliency-CCE. Saliency-CCE consists of a preprocessing of hair removal for skin lesions images, a novel colour contextual extractor (CCE) module for the SOD task and an improved adaptive threshold (AT) paradigm for the image segmentation task. In the SOD task, we perform the CCE module to extract hand-crafted features through a novel colour channel volume (CCV) block and a novel colour activation mapping (CAM) block. We first exploit the CCV block to generate a target object's region of interest (ROI). After that, we employ the CAM block to yield a refined salient map as the final salient map from the extracted ROI. We propose a novel adaptive threshold (AT) strategy in the segmentation task to automatically segment the WBC and SL from the final salient map. We evaluate our proposed Saliency-CCE on the ISIC-2016, the ISIC-2017, and the SCISC datasets, which outperform representative state-of-the-art SOD and biomedical image segmentation approaches. Our code is available at https://github.com/zxg3017/Saliency-CCE.
Collapse
Affiliation(s)
- Xiaogen Zhou
- Fujian Provincial Key Laboratory of Information Processing and Intelligent Control, Minjiang University, Fuzhou, P.R. China; College of Physics and Information Engineering, Fuzhou University, Fuzhou, P.R. China
| | - Tong Tong
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, P.R. China
| | - Zhixiong Zhong
- Fujian Provincial Key Laboratory of Information Processing and Intelligent Control, Minjiang University, Fuzhou, P.R. China
| | - Haoyi Fan
- School of Computer and Artificial Intelligence, Zhengzhou University, Zhengzhou, P.R. China
| | - Zuoyong Li
- Fujian Provincial Key Laboratory of Information Processing and Intelligent Control, Minjiang University, Fuzhou, P.R. China.
| |
Collapse
|
15
|
Qiu S, Li C, Feng Y, Zuo S, Liang H, Xu A. GFANet: Gated Fusion Attention Network for skin lesion segmentation. Comput Biol Med 2023; 155:106462. [PMID: 36857942 DOI: 10.1016/j.compbiomed.2022.106462] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2022] [Revised: 12/13/2022] [Accepted: 12/19/2022] [Indexed: 02/21/2023]
Abstract
Automatic segmentation of skin lesions is crucial for diagnosing and treating skin diseases. Although current medical image segmentation methods have significantly improved the results of skin lesion segmentation, the following major challenges still affect the segmentation performance: (i) segmentation targets have irregular shapes and diverse sizes and (ii) low contrast or blurred boundaries between lesions and background. To address these issues, this study proposes a Gated Fusion Attention Network (GFANet) which designs two progressive relation decoders to accurately segment skin lesions images. First, we use a Context Features Gated Fusion Decoder (CGFD) to fuse multiple levels of contextual features, and then a prediction result is generated as the initial guide map. Then, it is optimized by a prediction decoder consisting of a shape flow and a final Gated Convolution Fusion (GCF) module, where we iteratively use a set of Channel Reverse Attention (CRA) modules and GCF modules in the shape flow to combine the features of the current layer and the prediction results of the adjacent next layer to gradually extract boundary information. Finally, to speed up network convergence and improve segmentation accuracy, we use GCF to fuse low-level features from the encoder and the final output of the shape flow. To verify the effectiveness and advantages of the proposed GFANet, we conduct extensive experiments on four publicly available skin lesion datasets (International Skin Imaging Collaboration [ISIC] 2016, ISIC 2017, ISIC 2018, and PH2) and compare them with state-of-the-art methods. The experimental results show that the proposed GFANet achieves excellent segmentation performance in commonly used evaluation metrics, and the segmentation results are stable. The source code is available at https://github.com/ShiHanQ/GFANet.
Collapse
Affiliation(s)
- Shihan Qiu
- Department of Intelligent Manufacturing, Wuyi University, Jiangmen, 529020, China
| | - Chengfei Li
- Department of Intelligent Manufacturing, Wuyi University, Jiangmen, 529020, China.
| | - Yue Feng
- Department of Intelligent Manufacturing, Wuyi University, Jiangmen, 529020, China
| | - Song Zuo
- Department of Hemangioma and Vascular Malformation, Henan Provincial People's Hospital, People's Hospital of Zhengzhou University, Zhengzhou, Henan, 450003, China.
| | - Huijie Liang
- Department of Intelligent Manufacturing, Wuyi University, Jiangmen, 529020, China
| | - Ao Xu
- Department of Intelligent Manufacturing, Wuyi University, Jiangmen, 529020, China
| |
Collapse
|
16
|
Jiang Y, Dong J, Zhang Y, Cheng T, Lin X, Liang J. PCF-Net: Position and context information fusion attention convolutional neural network for skin lesion segmentation. Heliyon 2023; 9:e13942. [PMID: 36923881 PMCID: PMC10009446 DOI: 10.1016/j.heliyon.2023.e13942] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2022] [Revised: 02/10/2023] [Accepted: 02/15/2023] [Indexed: 02/27/2023] Open
Abstract
Skin lesion segmentation is a crucial step in the process of skin cancer diagnosis and treatment. The variation in position, shape, size and edges of skin lesion areas poses a challenge for accurate segmentation of skin lesion areas through dermoscopic images. To meet these challenges, in this paper, using UNet as the baseline model, a convolutional neural network based on position and context information fusion attention is proposed, called PCF-Net. A novel two-branch attention mechanism is designed to aggregate Position and Context information, called Position and Context Information Aggregation Attention Module (PCFAM). A global context information complementary module (GCCM) was developed to obtain long-range dependencies. A multi-scale grouped dilated convolution feature extraction module (MSEM) was proposed to capture multi-scale feature information and place it in the bottleneck of UNet. On the ISIC2018 dataset, a large volume of ablation experiments demonstrated the superiority of PCF-Net for dermoscopic image segmentation after adding PCFAM, GCCM and MSEM. Compared with other state-of-the-art methods, the performance of PCF-Net achieves a competitive result in all metrics.
Collapse
Affiliation(s)
- Yun Jiang
- College of Computer Science and Engineering, Northwest Normal University, Lanzhou, China
| | - Jinkun Dong
- College of Computer Science and Engineering, Northwest Normal University, Lanzhou, China
| | - Yuan Zhang
- College of Computer Science and Engineering, Northwest Normal University, Lanzhou, China
| | - Tongtong Cheng
- College of Computer Science and Engineering, Northwest Normal University, Lanzhou, China
| | - Xin Lin
- College of Computer Science and Engineering, Northwest Normal University, Lanzhou, China
| | - Jing Liang
- College of Computer Science and Engineering, Northwest Normal University, Lanzhou, China
| |
Collapse
|
17
|
Zhang W, Lu F, Zhao W, Hu Y, Su H, Yuan M. ACCPG-Net: A skin lesion segmentation network with Adaptive Channel-Context-Aware Pyramid Attention and Global Feature Fusion. Comput Biol Med 2023; 154:106580. [PMID: 36716686 DOI: 10.1016/j.compbiomed.2023.106580] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2022] [Revised: 01/09/2023] [Accepted: 01/22/2023] [Indexed: 01/26/2023]
Abstract
The computer-aided diagnosis system based on dermoscopic images has played an important role in the clinical treatment of skin lesion. An accurate, efficient, and automatic skin lesion segmentation method is an important auxiliary tool for clinical diagnosis. At present, skin lesion segmentation still suffers from great challenges. Existing deep-learning-based automatic segmentation methods frequently use convolutional neural networks (CNN). However, the globally-sharing feature re-weighting vector may not be optimal for the prediction of lesion areas in dermoscopic images. The presence of hairs and spots in some samples aggravates the interference of similar categories, and reduces the segmentation accuracy. To solve this problem, this paper proposes a new deep network for precise skin lesion segmentation based on a U-shape structure. To be specific, two lightweight attention modules: adaptive channel-context-aware pyramid attention (ACCAPA) module and global feature fusion (GFF) module, are embedded in the network. The ACCAPA module can model the characteristics of the lesion areas by dynamically learning the channel information, contextual information and global structure information. GFF is used for different levels of semantic information interaction between encoder and decoder layers. To validate the effectiveness of the proposed method, we test the performance of ACCPG-Net on several public skin lesion datasets. The results show that our method achieves better segmentation performance compared to other state-of-the-art methods.
Collapse
Affiliation(s)
- Wenyu Zhang
- School of Information Science and Engineering, Lanzhou University, China
| | - Fuxiang Lu
- School of Information Science and Engineering, Lanzhou University, China.
| | - Wei Zhao
- School of Information Science and Engineering, Lanzhou University, China
| | - Yawen Hu
- School of Information Science and Engineering, Lanzhou University, China
| | - Hongjing Su
- School of Information Science and Engineering, Lanzhou University, China
| | - Min Yuan
- School of Information Science and Engineering, Lanzhou University, China
| |
Collapse
|
18
|
Dong C, Dai D, Zhang Y, Zhang C, Li Z, Xu S. Learning from dermoscopic images in association with clinical metadata for skin lesion segmentation and classification. Comput Biol Med 2023; 152:106321. [PMID: 36463792 DOI: 10.1016/j.compbiomed.2022.106321] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2022] [Revised: 11/03/2022] [Accepted: 11/14/2022] [Indexed: 11/18/2022]
Abstract
Automatic segmentation and classification of lesions are two clinically significant tasks in the computer-aided diagnosis of skin diseases. Both tasks are challenging due to the nonnegligible lesion differences in dermoscopic images from different patients. In this paper, we propose a novel pipeline to efficiently implement skin lesions' segmentation and classification tasks, which consists of a segmentation network and a classification network. To improve the performance of the segmentation network, we propose a novel module of Multi-Scale Holistic Feature Exploration (MSH) to thoroughly exploit perceptual clues latent among multi-scale feature maps as synthesized by the decoder. The MSH module enables holistic exploration of features across multiple scales to more effectively support downstream image analysis tasks. To boost the performance of the classification network, we propose a novel module of Cross-Modality Collaborative Feature Exploration (CMC) to discover latent discriminative features by collaboratively exploiting potential relationships between cross-modal features of dermoscopic images and clinical metadata. The CMC module enables dynamically capturing versatile interaction effects among cross-modal features during the model's representation learning procedure by discriminatively and adaptively learning the interaction weight associated with each crossmodality feature pair. In addition, to effectively reduce background noise and boost the lesion discrimination ability of the classification network, we crop the images based on lesion masks generated by the best segmentation model. We evaluate the proposed pipeline on the four public skin lesion datasets, where the ISIC 2018 and PH2 are for segmentation, and the ISIC 2019 and ISIC 2020 are combined into a new dataset, ISIC 2019&2020, for classification. It achieves a Jaccard index of 83.31% and 90.14% in skin lesion segmentation, an AUC of 97.98% and an Accuracy of 92.63% in skin lesion classification, which is superior to the performance of representative state-of-the-art skin lesion segmentation and classification methods. Last but not least, the new model for segmentation utilizes much fewer model parameters (3.3 M) than its peer approaches, leading to a greatly reduced number of labeled samples required for model training, which obtains substantially stronger robustness than its peers.
Collapse
|
19
|
Malik S, Islam SMR, Akram T, Naqvi SR, Alghamdi NS, Baryannis G. A novel hybrid meta-heuristic contrast stretching technique for improved skin lesion segmentation. Comput Biol Med 2022; 151:106222. [PMID: 36343406 DOI: 10.1016/j.compbiomed.2022.106222] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2022] [Revised: 10/04/2022] [Accepted: 10/15/2022] [Indexed: 12/27/2022]
Abstract
The high precedence of epidemiological examination of skin lesions necessitated the well-performing efficient classification and segmentation models. In the past two decades, various algorithms, especially machine/deep learning-based methods, replicated the classical visual examination to accomplish the above-mentioned tasks. These automated streams of models demand evident lesions with less background and noise affecting the region of interest. However, even after the proposal of these advanced techniques, there are gaps in achieving the efficacy of matter. Recently, many preprocessors proposed to enhance the contrast of lesions, which further aided the skin lesion segmentation and classification tasks. Metaheuristics are the methods used to support the search space optimisation problems. We propose a novel Hybrid Metaheuristic Differential Evolution-Bat Algorithm (DE-BA), which estimates parameters used in the brightness preserving contrast stretching transformation function. For extensive experimentation we tested our proposed algorithm on various publicly available databases like ISIC 2016, 2017, 2018 and PH2, and validated the proposed model with some state-of-the-art already existing segmentation models. The tabular and visual comparison of the results concluded that DE-BA as a preprocessor positively enhances the segmentation results.
Collapse
Affiliation(s)
- Shairyar Malik
- Department of Electrical and Computer Engineering, COMSATS University Islamabad, Wah Campus, G.T. Road, Wah Cantonment, 47040, Pakistan
| | - S M Riazul Islam
- Department of Computer Science, University of Huddersfield, Huddersfield, HD1 3DH, United Kingdom
| | - Tallha Akram
- Department of Electrical and Computer Engineering, COMSATS University Islamabad, Wah Campus, G.T. Road, Wah Cantonment, 47040, Pakistan.
| | - Syed Rameez Naqvi
- Department of Electrical and Computer Engineering, COMSATS University Islamabad, Wah Campus, G.T. Road, Wah Cantonment, 47040, Pakistan
| | - Norah Saleh Alghamdi
- Department of Computer Sciences, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, Riyadh, 11671, Saudi Arabia.
| | - George Baryannis
- Department of Computer Science, University of Huddersfield, Huddersfield, HD1 3DH, United Kingdom
| |
Collapse
|
20
|
Feng R, Zhuo L, Li X, Yin H, Wang Z. BLA-Net:Boundary learning assisted network for skin lesion segmentation. Comput Methods Programs Biomed 2022; 226:107190. [PMID: 36288686 DOI: 10.1016/j.cmpb.2022.107190] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/03/2022] [Revised: 10/14/2022] [Accepted: 10/17/2022] [Indexed: 06/16/2023]
Abstract
BACKGROUND AND OBJECTIVE Automatic skin lesion segmentation plays an important role in computer-aided diagnosis of skin diseases. However, current segmentation networks cannot accurately detect the boundaries of the skin lesion areas. METHODS In this paper, a boundary learning assisted network for skin lesion segmentation is proposed, namely BLA-Net, which adopts ResNet34 as backbone network under an encoder-decoder framework. The overall architecture is divided into two key components: Primary Segmentation Network (PSNet) and Auxiliary Boundary Learning Network (ABLNet). PSNet is to locate the skin lesion areas. Dynamic Deformable Convolution is introduced into the lower layer of the encoder, so that the network can effectively deal with complex skin lesion objects. And a Global Context Information Extraction Module is proposed and embedded into the high layer of the encoder to capture multi-receptive field and multi-scale global context features. ABLNet is to finely detect the boundaries of skin lesion area based on the low-level features of the encoder, in which an object regional attention mechanism is proposed to enhance the features of lesion object area and suppress those of irrelevant regions. ABLNet can assist the PSNet to realize accurate skin lesion segmentation. RESULTS We verified the segmentation performance of the proposed method on the two public dermoscopy datasets, namely ISBI 2016 and ISIC 2018. The experimental results show that our proposed method can achieve the Jaccard Index of 86.6%, 84.8% and the Dice Coefficient of 92.4%, 91.2% on ISBI 2016 and ISIC 2018 datasets, respectively. CONCLUSIONS Compared with existing methods, the proposed method can achieve the state-of-the-arts segmentation accuracy with less model parameters, which can assist dermatologists in clinical diagnosis and treatment.
Collapse
Affiliation(s)
- Ruiqi Feng
- Faculty of Information Technology, Beijing University of Technology, China; Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, China
| | - Li Zhuo
- Faculty of Information Technology, Beijing University of Technology, China; Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, China.
| | - Xiaoguang Li
- Faculty of Information Technology, Beijing University of Technology, China; Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, China
| | - Hongxia Yin
- Department of Radiology, Beijing Friendship Hospital, Capital Medical University, China
| | - Zhenchang Wang
- Department of Radiology, Beijing Friendship Hospital, Capital Medical University, China.
| |
Collapse
|
21
|
Jiang X, Jiang J, Wang B, Yu J, Wang J. SEACU-Net: Attentive ConvLSTM U-Net with squeeze-and-excitation layer for skin lesion segmentation. Comput Methods Programs Biomed 2022; 225:107076. [PMID: 36027859 DOI: 10.1016/j.cmpb.2022.107076] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/20/2022] [Revised: 07/08/2022] [Accepted: 08/14/2022] [Indexed: 06/15/2023]
Abstract
BACKGROUND AND OBJECTIVE Accurate segmentation of skin lesions is a pivotal step in dermoscopy image classification, which provides a powerful means for dermatologists to diagnose skin diseases. However, due to blurred boundaries, low contrast between the lesion and its surrounding skin, and changes in color and shape, most existing segmentation methods still face great challenges in obtaining receptive fields and extracting image feature information. To settle the above issues, we construct a new framework, named SEACU-Net, to analyze and segment skin lesion images. METHODS Inspired by the U-Net, we utilize dense convolution blocks to obtain more discriminative information. Then, at each encoding and decoding stage, a channel and spatial squeeze & excitation layer are designed after each convolution, to adaptively enhance useful information features and suppress low-value ones from different feature channels. In addition, the attention mechanism is integrated into the convolutional long short-term memory (ConvLSTM) structure, which improves sensitivity and prediction accuracy. Furthermore, this network introduces a novel loss based on binary cross-entropy and Jaccard losses, which can ensure more balanced segmentation. RESULTS The proposed method is applied to the ISIC 2017 and 2018 publicly image databases, then obtains a better performance in Dice, Jaccard, and Accuracy, with 89.11% and 87.58% Dice value, 80.50% and 78.12% Jaccard value, 95.01%, and 93.60% Accuracy value, respectively. CONCLUSION The results of quantitative and qualitative experiments show that our method reaches high-performance skin lesion segmentation, and can help radiologists make radiotherapy treatment plans in clinical practice.
Collapse
Affiliation(s)
- Xiaoliang Jiang
- College of Mechanical Engineering, Quzhou University, Quzhou, Zhejiang, 324000, China.
| | - Jinyun Jiang
- College of Mechanical Engineering, Quzhou University, Quzhou, Zhejiang, 324000, China
| | - Ban Wang
- School of Mechanical Engineering, Hangzhou Dianzi University, Hangzhou, Zhejiang, 310018, China
| | - Jianping Yu
- College of Mechanical Engineering, Quzhou University, Quzhou, Zhejiang, 324000, China
| | - Jun Wang
- College of Mechanical Engineering, Quzhou University, Quzhou, Zhejiang, 324000, China
| |
Collapse
|
22
|
Feng K, Ren L, Wang G, Wang H, Li Y. SLT-Net: A codec network for skin lesion segmentation. Comput Biol Med 2022; 148:105942. [PMID: 35964466 DOI: 10.1016/j.compbiomed.2022.105942] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2022] [Revised: 07/28/2022] [Accepted: 08/06/2022] [Indexed: 11/03/2022]
Abstract
Automatic segmentation of skin lesions is beneficial for improving the accuracy and efficiency of melanoma diagnosis. However, due to variation in the size and shape of the lesion areas and the low contrast between the edges of the lesion and the normal skin tissue, this task is very challenging. The traditional convolutional neural network based on codec structure lacks the capability of multi-scale context information modeling and cannot realize information interaction of skip connections at the various levels, which limits the segmentation performance. Therefore, a new codec structure of skin lesion Transformer network (SLT-Net) was proposed and applied to skin lesion segmentation in this study. Specifically, SLT-Net used CSwinUnet as the codec to model the long-distance dependence between features and used the multi-scale context Transformer (MCT) as the skip connection to realize information interaction between skip connections across levels in the channel dimension. We have performed extensive experiments to verify the effectiveness and superiority of our proposed method on three public skin lesion datasets, including the ISIC-2016, ISIC-2017, and ISIC-2018. The DSC values on the three data sets reached 90.45%, 79.87% and 82.85% respectively, higher than most of the state-of-the-art methods. The excellent performance of SLT-Net on these three datasets proved that it could improve the accuracy of skin lesion segmentation, providing a new benchmark reference for skin lesion segmentation tasks. The code is available at https://github.com/FengKaili-fkl/SLT-Net.git.
Collapse
Affiliation(s)
- Kaili Feng
- The School of Electronic Information Engineering, Hebei University, Hebei, 071002, China
| | - Lili Ren
- Affiliated Hospital of Hebei University, Hebei, 071030, China
| | - Guanglei Wang
- The School of Electronic Information Engineering, Hebei University, Hebei, 071002, China
| | - Hongrui Wang
- The School of Electronic Information Engineering, Hebei University, Hebei, 071002, China
| | - Yan Li
- The School of Electronic Information Engineering, Hebei University, Hebei, 071002, China.
| |
Collapse
|
23
|
Liu Q, Wang J, Zuo M, Cao W, Zheng J, Zhao H, Xie J. NCRNet: Neighborhood Context Refinement Network for skin lesion segmentation. Comput Biol Med 2022; 146:105545. [PMID: 35477048 DOI: 10.1016/j.compbiomed.2022.105545] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2022] [Revised: 04/11/2022] [Accepted: 04/17/2022] [Indexed: 12/24/2022]
Abstract
Accurate skin lesion segmentation plays a fundamental role in computer-aided melanoma analysis. Recently, some FCN-based methods have been proposed and achieved promising results in lesion segmentation tasks. However, due to the variable shapes, different scales, noise interference, and ambiguous boundaries of skin lesions, the capabilities of lesion location and boundary delineation of these works are still insufficient. To overcome the above challenges, in this paper, we propose a novel Neighborhood Context Refinement Network (NCRNet) by using the coarse-to-fine strategy to achieve accurate skin lesion segmentation. The proposed NCRNet contains a shared encoder and two different but closely related decoders for locating the skin lesions and refining the lesion boundaries. Specifically, we first design the Parallel Attention Decoder (PAD), which can effectively extract and fuse the local detail information and global semantic information on multiple levels to locate skin lesions of different sizes and shapes. Then, based on the initial lesion location, we further design the Neighborhood Context Refinement Decoder (NCRD), aiming at leveraging the fine-grained multi-stage neighborhood context cues to refine the lesion boundaries continuously. Furthermore, the neighborhood-based deep supervision used in the NCRD can make the shared encoder pay more attention to the lesion boundary areas and promote convergence of the segmentation network. The public skin lesion segmentation dataset ISIC2017 is adopted to validate the effectiveness of the proposed NCRNet. Comprehensive experiments prove that the proposed NCRNet reaches the state-of-the-art performance than the other nine competitive methods and gets 78.62%, 86.55%, and 94.01% on Jaccard, Dice, and Accuracy, respectively.
Collapse
Affiliation(s)
- Qi Liu
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, 230026, China; Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China
| | - Jingkun Wang
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, 230026, China; Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China
| | - Mengying Zuo
- Department of Cardiology, Children's Hospital of Soochow University, Suzhou, 215003, China
| | - Weiwei Cao
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, 230026, China; Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China.
| | - Jian Zheng
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, 230026, China; Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China; Jinan Guoke Medical Technology Development Co., Ltd, Jinan, 250101, China
| | - Hui Zhao
- The Wenzhou Third Clinical Institute Affiliated to Wenzhou Medical University, (Wenzhou People's Hospital), Wenzhou, 325000, China
| | - Jing Xie
- The Wenzhou Third Clinical Institute Affiliated to Wenzhou Medical University, (Wenzhou People's Hospital), Wenzhou, 325000, China.
| |
Collapse
|
24
|
Khadka R, Jha D, Hicks S, Thambawita V, Riegler MA, Ali S, Halvorsen P. Meta-learning with implicit gradients in a few-shot setting for medical image segmentation. Comput Biol Med 2022; 143:105227. [PMID: 35124439 DOI: 10.1016/j.compbiomed.2022.105227] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2021] [Revised: 01/05/2022] [Accepted: 01/05/2022] [Indexed: 12/26/2022]
Abstract
Widely used traditional supervised deep learning methods require a large number of training samples but often fail to generalize on unseen datasets. Therefore, a more general application of any trained model is quite limited for medical imaging for clinical practice. Using separately trained models for each unique lesion category or a unique patient population will require sufficiently large curated datasets, which is not practical to use in a real-world clinical set-up. Few-shot learning approaches can not only minimize the need for an enormous number of reliable ground truth labels that are labour-intensive and expensive, but can also be used to model on a dataset coming from a new population. To this end, we propose to exploit an optimization-based implicit model agnostic meta-learning (iMAML) algorithm under few-shot settings for medical image segmentation. Our approach can leverage the learned weights from diverse but small training samples to perform analysis on unseen datasets with high accuracy. We show that, unlike classical few-shot learning approaches, our method improves generalization capability. To our knowledge, this is the first work that exploits iMAML for medical image segmentation and explores the strength of the model on scenarios such as meta-training on unique and mixed instances of lesion datasets. Our quantitative results on publicly available skin and polyp datasets show that the proposed method outperforms the naive supervised baseline model and two recent few-shot segmentation approaches by large margins. In addition, our iMAML approach shows an improvement of 2%-4% in dice score compared to its counterpart MAML for most experiments.
Collapse
Affiliation(s)
- Rabindra Khadka
- SimulaMet, Oslo, Norway; Oslo Metropolitan University, Oslo, Norway
| | - Debesh Jha
- SimulaMet, Oslo, Norway; UiT the Arctic University of Norway, Tromsø, Norway.
| | - Steven Hicks
- SimulaMet, Oslo, Norway; Oslo Metropolitan University, Oslo, Norway
| | | | - Michael A Riegler
- SimulaMet, Oslo, Norway; UiT the Arctic University of Norway, Tromsø, Norway
| | - Sharib Ali
- Department of Engineering Science, Institute of Biomedical Engineering, University of Oxford, Oxford, UK; NIHR Oxford Biomedical Research Centre, University of Oxford, Oxford, UK.
| | - Pål Halvorsen
- SimulaMet, Oslo, Norway; Oslo Metropolitan University, Oslo, Norway
| |
Collapse
|
25
|
Wu H, Chen S, Chen G, Wang W, Lei B, Wen Z. FAT-Net: Feature adaptive transformers for automated skin lesion segmentation. Med Image Anal 2021; 76:102327. [PMID: 34923250 DOI: 10.1016/j.media.2021.102327] [Citation(s) in RCA: 82] [Impact Index Per Article: 27.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2021] [Revised: 11/17/2021] [Accepted: 12/01/2021] [Indexed: 12/23/2022]
Abstract
Skin lesion segmentation from dermoscopic image is essential for improving the quantitative analysis of melanoma. However, it is still a challenging task due to the large scale variations and irregular shapes of the skin lesions. In addition, the blurred lesion boundaries between the skin lesions and the surrounding tissues may also increase the probability of incorrect segmentation. Due to the inherent limitations of traditional convolutional neural networks (CNNs) in capturing global context information, traditional CNN-based methods usually cannot achieve a satisfactory segmentation performance. In this paper, we propose a novel feature adaptive transformer network based on the classical encoder-decoder architecture, named FAT-Net, which integrates an extra transformer branch to effectively capture long-range dependencies and global context information. Furthermore, we also employ a memory-efficient decoder and a feature adaptation module to enhance the feature fusion between the adjacent-level features by activating the effective channels and restraining the irrelevant background noise. We have performed extensive experiments to verify the effectiveness of our proposed method on four public skin lesion segmentation datasets, including the ISIC 2016, ISIC 2017, ISIC 2018, and PH2 datasets. Ablation studies demonstrate the effectiveness of our feature adaptive transformers and memory-efficient strategies. Comparisons with state-of-the-art methods also verify the superiority of our proposed FAT-Net in terms of both accuracy and inference speed. The code is available at https://github.com/SZUcsh/FAT-Net.
Collapse
Affiliation(s)
- Huisi Wu
- College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, 518060, China
| | - Shihuai Chen
- College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, 518060, China
| | - Guilian Chen
- College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, 518060, China
| | - Wei Wang
- College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, 518060, China
| | - Baiying Lei
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, 518060, ChinaChina.
| | - Zhenkun Wen
- College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, 518060, China
| |
Collapse
|
26
|
Dai D, Dong C, Xu S, Yan Q, Li Z, Zhang C, Luo N. Ms RED: A novel multi-scale residual encoding and decoding network for skin lesion segmentation. Med Image Anal 2021; 75:102293. [PMID: 34800787 DOI: 10.1016/j.media.2021.102293] [Citation(s) in RCA: 40] [Impact Index Per Article: 13.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2021] [Revised: 09/01/2021] [Accepted: 10/27/2021] [Indexed: 12/22/2022]
Abstract
Computer-Aided Diagnosis (CAD) for dermatological diseases offers one of the most notable showcases where deep learning technologies display their impressive performance in acquiring and surpassing human experts. In such the CAD process, a critical step is concerned with segmenting skin lesions from dermoscopic images. Despite remarkable successes attained by recent deep learning efforts, much improvement is still anticipated to tackle challenging cases, e.g., segmenting lesions that are irregularly shaped, bearing low contrast, or possessing blurry boundaries. To address such inadequacies, this study proposes a novel Multi-scale Residual Encoding and Decoding network (Ms RED) for skin lesion segmentation, which is able to accurately and reliably segment a variety of lesions with efficiency. Specifically, a multi-scale residual encoding fusion module (MsR-EFM) is employed in an encoder, and a multi-scale residual decoding fusion module (MsR-DFM) is applied in a decoder to fuse multi-scale features adaptively. In addition, to enhance the representation learning capability of the newly proposed pipeline, we propose a novel multi-resolution, multi-channel feature fusion module (M2F2), which replaces conventional convolutional layers in encoder and decoder networks. Furthermore, we introduce a novel pooling module (Soft-pool) to medical image segmentation for the first time, retaining more helpful information when down-sampling and getting better segmentation performance. To validate the effectiveness and advantages of the proposed network, we compare it with several state-of-the-art methods on ISIC 2016, 2017, 2018, and PH2. Experimental results consistently demonstrate that the proposed Ms RED attains significantly superior segmentation performance across five popularly used evaluation criteria. Last but not least, the new model utilizes much fewer model parameters than its peer approaches, leading to a greatly reduced number of labeled samples required for model training, which in turn produces a substantially faster converging training process than its peers. The source code is available at https://github.com/duweidai/Ms-RED.
Collapse
Affiliation(s)
- Duwei Dai
- Institute of Medical Artificial Intelligence, the Second Affiliated Hospital of Xi'an Jiaotong University, Xi'an, 710061, China
| | - Caixia Dong
- Institute of Medical Artificial Intelligence, the Second Affiliated Hospital of Xi'an Jiaotong University, Xi'an, 710061, China
| | - Songhua Xu
- Institute of Medical Artificial Intelligence, the Second Affiliated Hospital of Xi'an Jiaotong University, Xi'an, 710061, China.
| | - Qingsen Yan
- Australian Institute for Machine Learning, University of Adelaide, Adelaide, 5005, Australia
| | - Zongfang Li
- Institute of Medical Artificial Intelligence, the Second Affiliated Hospital of Xi'an Jiaotong University, Xi'an, 710061, China.
| | - Chunyan Zhang
- Institute of Medical Artificial Intelligence, the Second Affiliated Hospital of Xi'an Jiaotong University, Xi'an, 710061, China
| | - Nana Luo
- Affiliated Hospital of Jining Medical University, Jining, 272000, China
| |
Collapse
|
27
|
Abstract
Recently, the incidence of skin cancer has increased considerably and is seriously threatening human health. Automatic detection of this disease, where early detection is critical to human life, is quite challenging. Factors such as undesirable residues (hair, ruler markers), indistinct boundaries, variable contrast, shape differences, and color differences in the skin lesion images make automatic analysis quite difficult. To overcome these challenges, a highly effective segmentation method based on a fully convolutional network (FCN) is presented in this paper. The proposed improved FCN (iFCN) architecture is used for the segmentation of full-resolution skin lesion images without any pre- or post-processing. It is to support the residual structure of the FCN architecture with spatial information. This situation, which creates a more advanced residual system, enables more precise detection of details on the edges of the lesion, and an analysis independent of skin color can be performed. It offers two contributions: determining the center of the lesion and clarifying the edge details despite the undesirable effects. Two publicly available datasets, the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 Challenge and PH2 datasets, are used to evaluate the performance of the iFCN method. The mean Jaccard index is 78.34%, the mean Dice score is 88.64%, and the mean accuracy value is 95.30% for the proposed method for the ISBI 2017 test dataset. Furthermore, the mean Jaccard index is 87.1%, the mean Dice score is 93.02%, and the mean accuracy value is 96.92% for the proposed method for the PH2 test dataset.
Collapse
Affiliation(s)
- Şaban Öztürk
- Technology Faculty, Electrical and Electronics Engineering, Amasya University, Amasya, Turkey.
| | - Umut Özkaya
- Engineering and Natural Science Faculty, Electrical and Electronics Engineering, Konya Technical University, Konya, Turkey
| |
Collapse
|
28
|
Zhao C, Shuai R, Ma L, Liu W, Wu M. Segmentation of dermoscopy images based on deformable 3D convolution and ResU-NeXt +. Med Biol Eng Comput 2021; 59:1815-1832. [PMID: 34304370 DOI: 10.1007/s11517-021-02397-9] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2021] [Accepted: 06/16/2021] [Indexed: 11/25/2022]
Abstract
Melanoma is one of the most dangerous skin cancers. The current melanoma segmentation is mainly based on FCNs (fully connected networks) and U-Net. Nevertheless, these two kinds of neural networks are prone to parameter redundancy, and the gradient of neural networks disappears that occurs when the neural network backpropagates as the neural network gets deeper, which will reduce the Jaccard index of the skin lesion image segmentation model. To solve the above problems and improve the survival rate of melanoma patients, an improved skin lesion segmentation model based on deformable 3D convolution and ResU-NeXt++ (D3DC- ResU-NeXt++) is proposed in this paper. The new modules in D3DC-ResU-NeXt++ can replace ordinary modules in the existing 2D convolutional neural networks (CNNs) that can be trained efficiently through standard backpropagation with high segmentation accuracy. In particular, we introduce a new data preprocessing method with dilation, crop operation, resizing, and hair removal (DCRH), which improves the Jaccard index of skin lesion image segmentation. Because rectified Adam (RAdam) does not easily fall into a local optimal solution and can converge quickly in segmentation model training, we also introduce RAdam as the training optimizer. The experiments show that our model has excellent performance on the segmentation of the ISIC2018 Task I dataset, and the Jaccard index achieves 86.84%. The proposed method improves the Jaccard index of segmentation of skin lesion images and can also assist dermatological doctors in determining and diagnosing the types of skin lesions and the boundary between lesions and normal skin, so as to improve the survival rate of skin cancer patients. Overview of the proposed model. An improved skin lesion segmentation model based on deformable 3D convolution and ResU-NeXt++ (D3DC- ResU-NeXt++) is proposed in this paper. D3DC-ResU-NeXt++ has strong spatial geometry processing capabilities, it is used to segment the skin lesion sample image; DCRH and transfer learning are used to preprocess the data set and D3DC-ResU-NeXt++ respectively, which can highlight the difference between the lesion area and the normal skin, and enhance the segmentation efficiency and robustness of the neural network; RAdam is used to speed up the convergence speed of neural network and improve the efficiency of segmentation.
Collapse
Affiliation(s)
- Chen Zhao
- College of Computer Science and Technology, Nanjing Tech University, Nanjing, 211816, China
| | - Renjun Shuai
- College of Computer Science and Technology, Nanjing Tech University, Nanjing, 211816, China.
| | - Li Ma
- Nanjing Health Information Center, Nanjing, 210003, China
| | - Wenjia Liu
- Changzhou No. 2 People's Hospital affiliated with Nanjing Medical University, Changzhou, 213003, China
| | - Menglin Wu
- College of Computer Science and Technology, Nanjing Tech University, Nanjing, 211816, China
| |
Collapse
|
29
|
Hasan MK, Dahal L, Samarakoon PN, Tushar FI, Martí R. DSNet: Automatic dermoscopic skin lesion segmentation. Comput Biol Med 2020; 120:103738. [PMID: 32421644 DOI: 10.1016/j.compbiomed.2020.103738] [Citation(s) in RCA: 42] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2020] [Revised: 03/27/2020] [Accepted: 03/27/2020] [Indexed: 02/08/2023]
Abstract
BACKGROUND AND OBJECTIVE Automatic segmentation of skin lesions is considered a crucial step in Computer-aided Diagnosis (CAD) systems for melanoma detection. Despite its significance, skin lesion segmentation remains an unsolved challenge due to their variability in color, texture, and shapes and indistinguishable boundaries. METHODS Through this study, we present a new and automatic semantic segmentation network for robust skin lesion segmentation named Dermoscopic Skin Network (DSNet). In order to reduce the number of parameters to make the network lightweight, we used a depth-wise separable convolution in lieu of standard convolution to project the learned discriminating features onto the pixel space at different stages of the encoder. Additionally, we implemented both a U-Net and a Fully Convolutional Network (FCN8s) to compare against the proposed DSNet. RESULTS We evaluate our proposed model on two publicly available datasets, namely ISIC-20171 and PH22. The obtained mean Intersection over Union (mIoU) is 77.5% and 87.0% respectively for ISIC-2017 and PH2 datasets which outperformed the ISIC-2017 challenge winner by 1.0% with respect to mIoU. Our proposed network also outperformed U-Net and FCN8s respectively by 3.6% and 6.8% with respect to mIoU on the ISIC-2017 dataset. CONCLUSION Our network for skin lesion segmentation outperforms the other methods discussed in the article and is able to provide better-segmented masks on two different test datasets which can lead to better performance in melanoma detection. Our trained model along with the source code and predicted masks are made publicly available3.
Collapse
Affiliation(s)
- Md Kamrul Hasan
- Computer Vision and Robotics Institute, University of Girona, Spain.
| | - Lavsen Dahal
- Computer Vision and Robotics Institute, University of Girona, Spain.
| | | | | | - Robert Martí
- Computer Vision and Robotics Institute, University of Girona, Spain.
| |
Collapse
|
30
|
Xie F, Yang J, Liu J, Jiang Z, Zheng Y, Wang Y. Skin lesion segmentation using high-resolution convolutional neural network. Comput Methods Programs Biomed 2020; 186:105241. [PMID: 31837637 DOI: 10.1016/j.cmpb.2019.105241] [Citation(s) in RCA: 54] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/14/2019] [Revised: 11/25/2019] [Accepted: 11/25/2019] [Indexed: 06/10/2023]
Abstract
BACKGROUND AND OBJECTIVE Skin lesion segmentation is an important but challenging task in computer-aided diagnosis of dermoscopy images. Many segmentation methods based on convolutional neural networks often fail to extract accurate lesion boundaries because the spatial size of feature maps decreases as the maps are processed throughout the network layers. We propose skin lesion segmentation in dermoscopy images based on a convolutional neural network with an attention mechanism, which can preserve edge details. METHODS We devised a high-resolution feature block containing three branches, namely, main, spatial attention, and channel-wise attention branches. The main branch takes high-resolution feature maps as input to extract spatial details around boundaries. The other two attention branches boost the discriminative features in the main branch regarding the spatial and channel-wise dimensions. By fusing the branch outputs, robust features with detailed spatial information can be extracted, and accurate skin lesion boundaries can be obtained. RESULTS Experiments on datasets from the International Symposium on Biomedical Imaging in 2016 and 2017 and the PH2 dataset retrieved Jaccard indices of 0.783, 0.858, and 0.857, respectively, for the proposed method. Hence, our method can accurately extract skin lesion boundaries and is robust to hair fibers and artifacts in the images. Overall, our method outperforms two typical segmentation networks (FCN-8 s and U-Net) and other state-of-the-art skin lesion segmentation methods. CONCLUSIONS The proposed network endowed with high-resolution feature blocks preserves spatial details during feature extraction, and its attention mechanism enhances representative features while suppressing noise. Hence, the proposed approach provides high-performance skin lesion segmentation.
Collapse
Affiliation(s)
- Fengying Xie
- Image Processing Center, School of Astronautics, Beihang University, Beijing 100191, China; Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing 100191, China
| | - Jiawen Yang
- Image Processing Center, School of Astronautics, Beihang University, Beijing 100191, China; Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing 100191, China
| | - Jie Liu
- Department of Dermatology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100730, China.
| | - Zhiguo Jiang
- Image Processing Center, School of Astronautics, Beihang University, Beijing 100191, China; Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing 100191, China
| | - Yushan Zheng
- Image Processing Center, School of Astronautics, Beihang University, Beijing 100191, China; Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing 100191, China
| | - Yukun Wang
- Department of Dermatology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100730, China
| |
Collapse
|
31
|
Moradi N, Mahdavi-Amiri N. Kernel sparse representation based model for skin lesions segmentation and classification. Comput Methods Programs Biomed 2019; 182:105038. [PMID: 31437709 DOI: 10.1016/j.cmpb.2019.105038] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/27/2019] [Revised: 08/12/2019] [Accepted: 08/15/2019] [Indexed: 06/10/2023]
Abstract
BACKGROUND AND OBJECTIVES Melanoma is a dangerous kind of skin disease with a high death rate, and its prevalence has increased rapidly in recent years. Diagnosis of melanoma in a primary phase can be helpful for its cure. Due to costs for dermatology, we need an automatic system to diagnose melanoma through lesion images. METHODS Here, we propose a sparse representation based method for segmentation and classification of lesion images. The main idea of our framework is based on a kernel sparse representation, which produces discriminative sparse codes to represent features in a high-dimensional feature space. Our novel formulation for discriminative kernel sparse coding jointly learns a kernel-based dictionary and a linear classifier. We also present an adaptive K-SVD algorithm for kernel dictionary and classifier learning. RESULTS We test our approach for both segmentation and classification tasks. The evaluation results on both dermoscopic and digital datasets demonstrate our approach to be competitive as compared to the available state-of-the-art methods, with the advantage of not needing any pre-processing. CONCLUSIONS Our method is insensitive to noise and image conditions and can be used effectively for challenging skin lesions. Our approach is so extensive to be adapted to various medical image segmentations.
Collapse
Affiliation(s)
- Nooshin Moradi
- Faculty of Mathematical Sciences, Sharif University of Technology, Tehran, Iran.
| | - Nezam Mahdavi-Amiri
- Faculty of Mathematical Sciences, Sharif University of Technology, Tehran, Iran.
| |
Collapse
|
32
|
Tang P, Liang Q, Yan X, Xiang S, Sun W, Zhang D, Coppola G. Efficient skin lesion segmentation using separable-Unet with stochastic weight averaging. Comput Methods Programs Biomed 2019; 178:289-301. [PMID: 31416556 DOI: 10.1016/j.cmpb.2019.07.005] [Citation(s) in RCA: 40] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/29/2019] [Revised: 07/04/2019] [Accepted: 07/04/2019] [Indexed: 06/10/2023]
Abstract
BACKGROUND AND OBJECTIVE Efficient segmentation of skin lesion in dermoscopy images can improve the classification accuracy of skin diseases, which provides a powerful approach for the dermatologists in examining pigmented skin lesions. However, the segmentation is challenging due to the low contrast of skin lesions from a captured image, fuzzy and indistinct lesion boundaries, huge variety of interclass variation of melanomas, the existence of artifacts, etc. In this work, an efficient and accurate melanoma region segmentation method is proposed for computer-aided diagnostic systems. METHOD A skin lesion segmentation (SLS) method based on the separable-Unet with stochastic weight averaging is proposed in this work. Specifically, the proposed Separable-Unet framework takes advantage of the separable convolutional block and U-Net architectures, which can extremely capture the context feature channel correlation and higher semantic feature information to enhance the pixel-level discriminative representation capability of fully convolutional networks (FCN). Further, considering that the over-fitting is a local optimum (or sub-optimum) problem, a scheme based on stochastic weight averaging is introduced, which can obtain much broader optimum and better generalization. RESULTS The proposed method is evaluated in three publicly available datasets. The experimental results showed that the proposed approach segmented the skin lesions with an average Dice coefficient of 93.03% and Jaccard index of 89.25% for the International Skin Imaging Collaboration (ISIC) 2016 Skin Lesion Challenge (SLC) dataset, 86.93% and 79.26% for the ISIC 2017 SLC, and 94.13% and 89.40% for the PH2 dataset, respectively. The proposed approach is compared with other state-of-the-art methods, and the results demonstrate that the proposed approach outperforms them for SLS on both melanoma and non-melanoma cases. Segmentation of a potential lesion with the proposed approach in a dermoscopy image requires less than 0.05 s of processing time, which is roughly 30 times faster than the second best method (regarding the value of Jaccard index) for the ISIC 2017 dataset with the same hardware configuration. CONCLUSIONS We concluded that using the separable convolutional block and U-Net architectures with stochastic weight averaging strategy could enable to obtain better pixel-level discriminative representation capability. Moreover, the considerably decreased computation time suggests that the proposed approach has potential for practical computer-aided diagnose systems, besides provides a segmentation for the specific analysis with improved segmentation performance.
Collapse
Affiliation(s)
- Peng Tang
- College of Electrical and Information Engineering, Hunan University, Changsha 410082, China; Hunan Key Laboratory of Intelligent Robot Technology in Electronic Manufacturing, Hunan University, Changsha 410082, China; National Engineering Laboratory for Robot Vision Perception and Control, Hunan University, Changsha 410082, China
| | - Qiaokang Liang
- College of Electrical and Information Engineering, Hunan University, Changsha 410082, China; Hunan Key Laboratory of Intelligent Robot Technology in Electronic Manufacturing, Hunan University, Changsha 410082, China; National Engineering Laboratory for Robot Vision Perception and Control, Hunan University, Changsha 410082, China.
| | - Xintong Yan
- College of Electrical and Information Engineering, Hunan University, Changsha 410082, China
| | - Shao Xiang
- College of Electrical and Information Engineering, Hunan University, Changsha 410082, China; Hunan Key Laboratory of Intelligent Robot Technology in Electronic Manufacturing, Hunan University, Changsha 410082, China; National Engineering Laboratory for Robot Vision Perception and Control, Hunan University, Changsha 410082, China
| | - Wei Sun
- College of Electrical and Information Engineering, Hunan University, Changsha 410082, China; Hunan Key Laboratory of Intelligent Robot Technology in Electronic Manufacturing, Hunan University, Changsha 410082, China; National Engineering Laboratory for Robot Vision Perception and Control, Hunan University, Changsha 410082, China
| | - Dan Zhang
- Department of Mechanical Engineering, York University, Toronto, ON M3J 1P3, Canada
| | - Gianmarc Coppola
- Faculty of Engineering and Applied Science, University of Ontario Institute of Technology, Oshawa, ON L1H 7K4, Canada
| |
Collapse
|
33
|
Abstract
BACKGROUND Dermoscopy is one of the common and effective imaging techniques in diagnosis of skin cancer, especially for pigmented lesions. Accurate skin lesion border detection is the key to extract important dermoscopic features of the skin lesion. In current clinical settings, border delineation is performed manually by dermatologists. Operator based assessments lead to intra- and inter-observer variations due to its subjective nature. Moreover it is a tedious process. Because of aforementioned hurdles, the automation of lesion boundary detection in dermoscopic images is necessary. In this study, we address this problem by developing a novel skin lesion border detection method with a robust edge indicator function, which is based on a meshless method. RESULT Our results are compared with the other image segmentation methods. Our skin lesion border detection algorithm outperforms other state-of-the-art methods. Based on dermatologist drawn ground truth skin lesion borders, the results indicate that our method generates reasonable boundaries than other prominent methods having Dice score of 0.886 ±0.094 and Jaccard score of 0.807 ±0.133. CONCLUSION We prove that smoothed particle hydrodynamic (SPH) kernels can be used as edge features in active contours segmentation and probability map can be employed to avoid the evolving contour from leaking into the object of interest.
Collapse
Affiliation(s)
- Mustafa Bayraktar
- Bioinformatics, University of Arkansas Little Rock, 2804 S. University, Little Rock, 72204 AR USA
| | - Sinan Kockara
- Computer Science, University of Central Arkansas, 201 Donaghey Avenue, Conway, 72035 AR USA
| | - Tansel Halic
- Computer Science, University of Central Arkansas, 201 Donaghey Avenue, Conway, 72035 AR USA
| | - Mutlu Mete
- Computer Science, Texas A&M University-Commerce, 2200 Campbell, Commerce, 75428 TX USA
| | - Henry K. Wong
- Dermatology, University of Arkansas for Medical Sciences, 324 Campus Dr., Little Rock, 72205 AR USA
| | - Kamran Iqbal
- Systems Engineering, University of Arkansas for Medical Sciences, 2804 S. University, Little Rock, 72204 AR USA
| |
Collapse
|
34
|
Al-Masni MA, Al-Antari MA, Choi MT, Han SM, Kim TS. Skin lesion segmentation in dermoscopy images via deep full resolution convolutional networks. Comput Methods Programs Biomed 2018; 162:221-231. [PMID: 29903489 DOI: 10.1016/j.cmpb.2018.05.027] [Citation(s) in RCA: 103] [Impact Index Per Article: 17.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/20/2018] [Revised: 04/30/2018] [Accepted: 05/17/2018] [Indexed: 06/08/2023]
Abstract
BACKGROUND AND OBJECTIVE Automatic segmentation of skin lesions in dermoscopy images is still a challenging task due to the large shape variations and indistinct boundaries of the lesions. Accurate segmentation of skin lesions is a key prerequisite step for any computer-aided diagnostic system to recognize skin melanoma. METHODS In this paper, we propose a novel segmentation methodology via full resolution convolutional networks (FrCN). The proposed FrCN method directly learns the full resolution features of each individual pixel of the input data without the need for pre- or post-processing operations such as artifact removal, low contrast adjustment, or further enhancement of the segmented skin lesion boundaries. We evaluated the proposed method using two publicly available databases, the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 Challenge and PH2 datasets. To evaluate the proposed method, we compared the segmentation performance with the latest deep learning segmentation approaches such as the fully convolutional network (FCN), U-Net, and SegNet. RESULTS Our results showed that the proposed FrCN method segmented the skin lesions with an average Jaccard index of 77.11% and an overall segmentation accuracy of 94.03% for the ISBI 2017 test dataset and 84.79% and 95.08%, respectively, for the PH2 dataset. In comparison to FCN, U-Net, and SegNet, the proposed FrCN outperformed them by 4.94%, 15.47%, and 7.48% for the Jaccard index and 1.31%, 3.89%, and 2.27% for the segmentation accuracy, respectively. Furthermore, the proposed FrCN achieved a segmentation accuracy of 95.62% for some representative clinical benign cases, 90.78% for the melanoma cases, and 91.29% for the seborrheic keratosis cases in the ISBI 2017 test dataset, exhibiting better performance than those of FCN, U-Net, and SegNet. CONCLUSIONS We conclude that using the full spatial resolutions of the input image could enable to learn better specific and prominent features, leading to an improvement in the segmentation performance.
Collapse
Affiliation(s)
- Mohammed A Al-Masni
- Department of Biomedical Engineering, College of Electronics and Information, Kyung Hee University, Yongin, Republic of Korea.
| | - Mugahed A Al-Antari
- Department of Biomedical Engineering, College of Electronics and Information, Kyung Hee University, Yongin, Republic of Korea.
| | - Mun-Taek Choi
- School of Mechanical Engineering, Sungkyunkwan University, Republic of Korea.
| | - Seung-Moo Han
- Department of Biomedical Engineering, College of Electronics and Information, Kyung Hee University, Yongin, Republic of Korea.
| | - Tae-Seong Kim
- Department of Biomedical Engineering, College of Electronics and Information, Kyung Hee University, Yongin, Republic of Korea.
| |
Collapse
|