1
|
Lama N, Stanley RJ, Lama B, Maurya A, Nambisan A, Hagerty J, Phan T, Van Stoecker W. LAMA: Lesion-Aware Mixup Augmentation for Skin Lesion Segmentation. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:1812-1823. [PMID: 38409610 PMCID: PMC11300415 DOI: 10.1007/s10278-024-01000-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Revised: 12/18/2023] [Accepted: 12/21/2023] [Indexed: 02/28/2024]
Abstract
Deep learning can exceed dermatologists' diagnostic accuracy in experimental image environments. However, inaccurate segmentation of images with multiple skin lesions can be seen with current methods. Thus, information present in multiple-lesion images, available to specialists, is not retrievable by machine learning. While skin lesion images generally capture a single lesion, there may be cases in which a patient's skin variation may be identified as skin lesions, leading to multiple false positive segmentations in a single image. Conversely, image segmentation methods may find only one region and may not capture multiple lesions in an image. To remedy these problems, we propose a novel and effective data augmentation technique for skin lesion segmentation in dermoscopic images with multiple lesions. The lesion-aware mixup augmentation (LAMA) method generates a synthetic multi-lesion image by mixing two or more lesion images from the training set. We used the publicly available International Skin Imaging Collaboration (ISIC) 2017 Challenge skin lesion segmentation dataset to train the deep neural network with the proposed LAMA method. As none of the previous skin lesion datasets (including ISIC 2017) has considered multiple lesions per image, we created a new multi-lesion (MuLe) segmentation dataset utilizing publicly available ISIC 2020 skin lesion images with multiple lesions per image. MuLe was used as a test set to evaluate the effectiveness of the proposed method. Our test results show that the proposed method improved the Jaccard score 8.3% from 0.687 to 0.744 and the Dice score 5% from 0.7923 to 0.8321 over a baseline model on MuLe test images. On the single-lesion ISIC 2017 test images, LAMA improved the baseline model's segmentation performance by 0.08%, raising the Jaccard score from 0.7947 to 0.8013 and the Dice score 0.6% from 0.8714 to 0.8766. The experimental results showed that LAMA improved the segmentation accuracy on both single-lesion and multi-lesion dermoscopic images. The proposed LAMA technique warrants further study.
Collapse
Affiliation(s)
- Norsang Lama
- Missouri University of Science & Technology, Rolla, MO, 65409, USA
| | | | | | - Akanksha Maurya
- Missouri University of Science & Technology, Rolla, MO, 65409, USA
| | - Anand Nambisan
- Missouri University of Science & Technology, Rolla, MO, 65409, USA
| | | | - Thanh Phan
- Missouri University of Science & Technology, Rolla, MO, 65409, USA
| | | |
Collapse
|
2
|
Collenne J, Monnier J, Iguernaissi R, Nawaf M, Richard MA, Grob JJ, Gaudy-Marqueste C, Dubuisson S, Merad D. Fusion between an Algorithm Based on the Characterization of Melanocytic Lesions' Asymmetry with an Ensemble of Convolutional Neural Networks for Melanoma Detection. J Invest Dermatol 2024; 144:1600-1607.e2. [PMID: 38296020 DOI: 10.1016/j.jid.2023.09.289] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2023] [Revised: 09/21/2023] [Accepted: 09/26/2023] [Indexed: 02/26/2024]
Abstract
Melanoma is still a major health problem worldwide. Early diagnosis is the first step toward reducing its mortality, but it remains a challenge even for experienced dermatologists. Although computer-aided systems have been developed to help diagnosis, the lack of insight into their predictions is still a significant limitation toward acceptance by the medical community. To tackle this issue, we designed handcrafted expert features representing color asymmetry within the lesions, which are parts of the approach used by dermatologists in their daily practice. These features are given to an artificial neural network classifying between nevi and melanoma. We compare our results with an ensemble of 7 state-of-the-art convolutional neural networks and merge the 2 approaches by computing the average prediction. Our experiments are done on a subset of the International Skin Imaging Collaboration 2019 dataset (6296 nevi, 1361 melanomas). The artificial neural network based on asymmetry achieved an area under the curve of 0.873, sensitivity of 90%, and specificity of 67%; the convolutional neural network approach achieved an area under the curve of 0.938, sensitivity of 91%, and specificity of 82%; and the fusion of both approaches achieved an area under the curve of 0.942, sensitivity of 92%, and specificity of 82%. Merging the knowledge of dermatologists with convolutional neural networks showed high performance for melanoma detection, encouraging collaboration between computer science and medical fields.
Collapse
Affiliation(s)
- Jules Collenne
- Computer Science and Systems Laboratory, CNRS UMR 7020, Aix-Marseille University, Marseille, France.
| | - Jilliana Monnier
- Computer Science and Systems Laboratory, CNRS UMR 7020, Aix-Marseille University, Marseille, France; Cancer Research Center of Marseille, Inserm UMR1068, CNRS UMR7258, Aix-Marseille University, Marseille, France; Dermatology and Skin Cancer Department, La Timone Hospital, Assistance Publique Hôpitaux de Marseille, Aix-Marseille University, Marseille, France
| | - Rabah Iguernaissi
- Computer Science and Systems Laboratory, CNRS UMR 7020, Aix-Marseille University, Marseille, France
| | - Motasem Nawaf
- Computer Science and Systems Laboratory, CNRS UMR 7020, Aix-Marseille University, Marseille, France
| | - Marie-Aleth Richard
- Dermatology and Skin Cancer Department, La Timone Hospital, Assistance Publique Hôpitaux de Marseille, Aix-Marseille University, Marseille, France
| | - Jean-Jacques Grob
- Cancer Research Center of Marseille, Inserm UMR1068, CNRS UMR7258, Aix-Marseille University, Marseille, France; Dermatology and Skin Cancer Department, La Timone Hospital, Assistance Publique Hôpitaux de Marseille, Aix-Marseille University, Marseille, France
| | - Caroline Gaudy-Marqueste
- Cancer Research Center of Marseille, Inserm UMR1068, CNRS UMR7258, Aix-Marseille University, Marseille, France; Dermatology and Skin Cancer Department, La Timone Hospital, Assistance Publique Hôpitaux de Marseille, Aix-Marseille University, Marseille, France
| | - Séverine Dubuisson
- Computer Science and Systems Laboratory, CNRS UMR 7020, Aix-Marseille University, Marseille, France
| | - Djamal Merad
- Computer Science and Systems Laboratory, CNRS UMR 7020, Aix-Marseille University, Marseille, France
| |
Collapse
|
3
|
Xie F, Xu P, Xi X, Gu X, Zhang P, Wang H, Shen X. Oral mucosal disease recognition based on dynamic self-attention and feature discriminant loss. Oral Dis 2024; 30:3094-3107. [PMID: 37731172 DOI: 10.1111/odi.14732] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Revised: 07/22/2023] [Accepted: 08/25/2023] [Indexed: 09/22/2023]
Abstract
OBJECTIVES To develop a dynamic self-attention and feature discrimination loss function (DSDF) model for identifying oral mucosal diseases presented to solve the problems of data imbalance, complex image background, and high similarity and difference of visual characteristics among different types of lesion areas. METHODS In DSDF, dynamic self-attention network can fully mine the context information between adjacent areas, improve the visual representation of the network, and promote the network model to learn and locate the image area of interest. Then, the feature discrimination loss function is used to constrain the diversity of channel characteristics, so as to enhance the feature discrimination ability of local similar areas. RESULTS The experimental results show that the recognition accuracy of the proposed method for oral mucosal disease is the highest at 91.16%, and is about 6% ahead of other advanced methods. In addition, DSDF has recall of 90.87% and F1 of 90.60%. CONCLUSIONS Convolutional neural networks can effectively capture the visual features of the oral mucosal disease lesions, and the distinguished visual features of different oral lesions can be extracted better using dynamic self-attention and feature discrimination loss function, which is conducive to the auxiliary diagnosis of oral mucosal diseases.
Collapse
Affiliation(s)
- Fei Xie
- Xi'an Key Laboratory of Human-Machine Integration and Control Technology for Intelligent Rehabilitation, Xijing University, Xi'an, China
- School of AOAIR, Xidian University, Xi'an, China
| | - Pengfei Xu
- School of Information Science and Technology, Northwest University, Xi'an, China
| | - Xinyi Xi
- School of Information Science and Technology, Northwest University, Xi'an, China
| | - Xiaokang Gu
- School of Information Science and Technology, Northwest University, Xi'an, China
| | - Panpan Zhang
- School of Information Science and Technology, Northwest University, Xi'an, China
| | - Hexu Wang
- Xi'an Key Laboratory of Human-Machine Integration and Control Technology for Intelligent Rehabilitation, Xijing University, Xi'an, China
| | - Xuemin Shen
- Department of Oral Mucosal Diseases, Shanghai Ninth People's Hospital, College of Stomatology, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| |
Collapse
|
4
|
Maurya A, Stanley RJ, Aradhyula HY, Lama N, Nambisan AK, Patel G, Saeed D, Swinfard S, Smith C, Jagannathan S, Hagerty JR, Stoecker WV. Basal Cell Carcinoma Diagnosis with Fusion of Deep Learning and Telangiectasia Features. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:1137-1150. [PMID: 38332404 PMCID: PMC11169204 DOI: 10.1007/s10278-024-00969-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Revised: 09/22/2023] [Accepted: 10/27/2023] [Indexed: 02/10/2024]
Abstract
In recent years, deep learning (DL) has been used extensively and successfully to diagnose different cancers in dermoscopic images. However, most approaches lack clinical inputs supported by dermatologists that could aid in higher accuracy and explainability. To dermatologists, the presence of telangiectasia, or narrow blood vessels that typically appear serpiginous or arborizing, is a critical indicator of basal cell carcinoma (BCC). Exploiting the feature information present in telangiectasia through a combination of DL-based techniques could create a pathway for both, improving DL results as well as aiding dermatologists in BCC diagnosis. This study demonstrates a novel "fusion" technique for BCC vs non-BCC classification using ensemble learning on a combination of (a) handcrafted features from semantically segmented telangiectasia (U-Net-based) and (b) deep learning features generated from whole lesion images (EfficientNet-B5-based). This fusion method achieves a binary classification accuracy of 97.2%, with a 1.3% improvement over the corresponding DL-only model, on a holdout test set of 395 images. An increase of 3.7% in sensitivity, 1.5% in specificity, and 1.5% in precision along with an AUC of 0.99 was also achieved. Metric improvements were demonstrated in three stages: (1) the addition of handcrafted telangiectasia features to deep learning features, (2) including areas near telangiectasia (surround areas), (3) discarding the noisy lower-importance features through feature importance. Another novel approach to feature finding with weak annotations through the examination of the surrounding areas of telangiectasia is offered in this study. The experimental results show state-of-the-art accuracy and precision in the diagnosis of BCC, compared to three benchmark techniques. Further exploration of deep learning techniques for individual dermoscopy feature detection is warranted.
Collapse
Affiliation(s)
- Akanksha Maurya
- Missouri University of Science &Technology, Rolla, MO, 65209, USA
| | - R Joe Stanley
- Missouri University of Science &Technology, Rolla, MO, 65209, USA.
| | | | - Norsang Lama
- Missouri University of Science &Technology, Rolla, MO, 65209, USA
| | - Anand K Nambisan
- Missouri University of Science &Technology, Rolla, MO, 65209, USA
| | | | | | | | - Colin Smith
- A.T. Still University of Health Sciences, Kirksville, MO, USA
| | | | | | | |
Collapse
|
5
|
Zhang L, Wang L, Yu M, Wu R, Steffens DC, Potter GG, Liu M. Hybrid representation learning for cognitive diagnosis in late-life depression over 5 years with structural MRI. Med Image Anal 2024; 94:103135. [PMID: 38461654 PMCID: PMC11016377 DOI: 10.1016/j.media.2024.103135] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Revised: 07/14/2023] [Accepted: 03/05/2024] [Indexed: 03/12/2024]
Abstract
Late-life depression (LLD) is a highly prevalent mood disorder occurring in older adults and is frequently accompanied by cognitive impairment (CI). Studies have shown that LLD may increase the risk of Alzheimer's disease (AD). However, the heterogeneity of presentation of geriatric depression suggests that multiple biological mechanisms may underlie it. Current biological research on LLD progression incorporates machine learning that combines neuroimaging data with clinical observations. There are few studies on incident cognitive diagnostic outcomes in LLD based on structural MRI (sMRI). In this paper, we describe the development of a hybrid representation learning (HRL) framework for predicting cognitive diagnosis over 5 years based on T1-weighted sMRI data. Specifically, we first extract prediction-oriented MRI features via a deep neural network, and then integrate them with handcrafted MRI features via a Transformer encoder for cognitive diagnosis prediction. Two tasks are investigated in this work, including (1) identifying cognitively normal subjects with LLD and never-depressed older healthy subjects, and (2) identifying LLD subjects who developed CI (or even AD) and those who stayed cognitively normal over five years. We validate the proposed HRL on 294 subjects with T1-weighted MRIs from two clinically harmonized studies. Experimental results suggest that the HRL outperforms several classical machine learning and state-of-the-art deep learning methods in LLD identification and prediction tasks.
Collapse
Affiliation(s)
- Lintao Zhang
- School of Information Science and Engineering, Linyi University, Linyi, Shandong 27600, China; Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, United States
| | - Lihong Wang
- Department of Psychiatry, University of Connecticut School of Medicine, University of Connecticut, Farmington, CT 06030, United States
| | - Minhui Yu
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, United States
| | - Rong Wu
- Connecticut Convergence Institute for Translation in Regenerative Engineering, University of Connecticut Health, Farmington, CT 06030, United States
| | - David C Steffens
- Department of Psychiatry, University of Connecticut School of Medicine, University of Connecticut, Farmington, CT 06030, United States
| | - Guy G Potter
- Department of Psychiatry and Behavioral Sciences, Duke University Medical Center, Durham, NC 27710, United States.
| | - Mingxia Liu
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, United States.
| |
Collapse
|
6
|
Quishpe-Usca A, Cuenca-Dominguez S, Arias-Viñansaca A, Bosmediano-Angos K, Villalba-Meneses F, Ramírez-Cando L, Tirado-Espín A, Cadena-Morejón C, Almeida-Galárraga D, Guevara C. The effect of hair removal and filtering on melanoma detection: a comparative deep learning study with AlexNet CNN. PeerJ Comput Sci 2024; 10:e1953. [PMID: 38660169 PMCID: PMC11041978 DOI: 10.7717/peerj-cs.1953] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2023] [Accepted: 03/03/2024] [Indexed: 04/26/2024]
Abstract
Melanoma is the most aggressive and prevalent form of skin cancer globally, with a higher incidence in men and individuals with fair skin. Early detection of melanoma is essential for the successful treatment and prevention of metastasis. In this context, deep learning methods, distinguished by their ability to perform automated and detailed analysis, extracting melanoma-specific features, have emerged. These approaches excel in performing large-scale analysis, optimizing time, and providing accurate diagnoses, contributing to timely treatments compared to conventional diagnostic methods. The present study offers a methodology to assess the effectiveness of an AlexNet-based convolutional neural network (CNN) in identifying early-stage melanomas. The model is trained on a balanced dataset of 10,605 dermoscopic images, and on modified datasets where hair, a potential obstructive factor, was detected and removed allowing for an assessment of how hair removal affects the model's overall performance. To perform hair removal, we propose a morphological algorithm combined with different filtering techniques for comparison: Fourier, Wavelet, average blur, and low-pass filters. The model is evaluated through 10-fold cross-validation and the metrics of accuracy, recall, precision, and the F1 score. The results demonstrate that the proposed model performs the best for the dataset where we implemented both a Wavelet filter and hair removal algorithm. It has an accuracy of 91.30%, a recall of 87%, a precision of 95.19%, and an F1 score of 90.91%.
Collapse
Affiliation(s)
- Angélica Quishpe-Usca
- School of Biological Sciences and Engineering, Yachay Tech University, San Miguel de Urcuquí, Imbabura, Ecuador
| | - Stefany Cuenca-Dominguez
- School of Biological Sciences and Engineering, Yachay Tech University, San Miguel de Urcuquí, Imbabura, Ecuador
| | - Araceli Arias-Viñansaca
- School of Biological Sciences and Engineering, Yachay Tech University, San Miguel de Urcuquí, Imbabura, Ecuador
| | - Karen Bosmediano-Angos
- School of Biological Sciences and Engineering, Yachay Tech University, San Miguel de Urcuquí, Imbabura, Ecuador
| | - Fernando Villalba-Meneses
- School of Biological Sciences and Engineering, Yachay Tech University, San Miguel de Urcuquí, Imbabura, Ecuador
| | - Lenin Ramírez-Cando
- School of Biological Sciences and Engineering, Yachay Tech University, San Miguel de Urcuquí, Imbabura, Ecuador
| | - Andrés Tirado-Espín
- School of Mathematical and Computational Sciences, Yachay Tech University, San Miguel de Urcuquí, Imbabura, Ecuador
| | - Carolina Cadena-Morejón
- School of Mathematical and Computational Sciences, Yachay Tech University, San Miguel de Urcuquí, Imbabura, Ecuador
| | - Diego Almeida-Galárraga
- School of Biological Sciences and Engineering, Yachay Tech University, San Miguel de Urcuquí, Imbabura, Ecuador
| | - Cesar Guevara
- Quantitative Methods Department, CUNEF Universidad, Madrid, Madrid, Spain
| |
Collapse
|
7
|
Al-Gaashani MS, Samee NA, Alkanhel R, Atteia G, Abdallah HA, Ashurov A, Ali Muthanna MS. Deep transfer learning with gravitational search algorithm for enhanced plant disease classification. Heliyon 2024; 10:e28967. [PMID: 38601589 PMCID: PMC11004804 DOI: 10.1016/j.heliyon.2024.e28967] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2023] [Revised: 03/15/2024] [Accepted: 03/27/2024] [Indexed: 04/12/2024] Open
Abstract
Plant diseases annually cause damage and loss of much of the crop, if not its complete destruction, and this constitutes a significant challenge for farm owners, governments, and consumers alike. Therefore, identifying and classifying diseases at an early stage is very important in order to sustain local and global food security. In this research, we designed a new method to identify plant diseases by combining transfer learning and Gravitational Search Algorithm (GSA). Two state-of-the-art pretrained models have been adopted for extracting features in this study, which are MobileNetV2 and ResNe50V2. Multilayer feature extraction is applied in this study to ensure representations of plant leaves from different levels of abstraction for precise classification. These features are then concatenated and passed to GSA for optimizing them. Finally, optimized features are passed to Multinomial Logistic Regression (MLR) for final classification. This integration is essential for categorizing 18 different types of infected and healthy leaf samples. The performance of our approach is strengthened by a comparative analysis that incorporates features optimized by the Genetic Algorithm (GA). Additionally, the MLR algorithm is contrasted with K-Nearest Neighbors (KNN). The empirical findings indicate that our model, which has been refined using GSA, achieves very high levels of precision. Specifically, the average precision for MLR is 99.2%, while for KNN it is 98.6%. The resulting results significantly exceed those achieved with GA-optimized features, thereby highlighting the superiority of our suggested strategy. One important result of our study is that we were able to decrease the number of features by more than 50%. This reduction greatly reduces the processing requirements without sacrificing the quality of the diagnosis. This work presents a robust and efficient approach to the early detection of plant diseases. The work demonstrates the utilization of sophisticated computational methods in agriculture, enabling the development of novel data-driven strategies for plant health management, therefore enhancing worldwide food security.
Collapse
Affiliation(s)
- Mehdhar S.A.M. Al-Gaashani
- School of Resources and Environment, University of Electronic Science and Technology of China, 4 1st Ring Rd East 2 Section, Chenghua District, Chengdu, 610056, Sichuan, China
| | - Nagwan Abdel Samee
- Department of Information Technology, College of Computer and Information Sciences, Princess Nourah Bint Abdulrahman University, P.O. Box 84428, Riyadh, 11671, Saudi Arabia
| | - Reem Alkanhel
- Department of Information Technology, College of Computer and Information Sciences, Princess Nourah Bint Abdulrahman University, P.O. Box 84428, Riyadh, 11671, Saudi Arabia
| | - Ghada Atteia
- Department of Information Technology, College of Computer and Information Sciences, Princess Nourah Bint Abdulrahman University, P.O. Box 84428, Riyadh, 11671, Saudi Arabia
| | - Hanaa A. Abdallah
- Department of Information Technology, College of Computer and Information Sciences, Princess Nourah Bint Abdulrahman University, P.O. Box 84428, Riyadh, 11671, Saudi Arabia
| | - Asadulla Ashurov
- School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China
| | - Mohammed Saleh Ali Muthanna
- Institute of Computer Technologies and Information Security, Southern Federal University, 344006, Taganrog, Russia
| |
Collapse
|
8
|
Maurya A, Stanley RJ, Lama N, Nambisan AK, Patel G, Saeed D, Swinfard S, Smith C, Jagannathan S, Hagerty JR, Stoecker WV. Hybrid Topological Data Analysis and Deep Learning for Basal Cell Carcinoma Diagnosis. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:92-106. [PMID: 38343238 DOI: 10.1007/s10278-023-00924-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Revised: 10/04/2023] [Accepted: 10/05/2023] [Indexed: 03/02/2024]
Abstract
A critical clinical indicator for basal cell carcinoma (BCC) is the presence of telangiectasia (narrow, arborizing blood vessels) within the skin lesions. Many skin cancer imaging processes today exploit deep learning (DL) models for diagnosis, segmentation of features, and feature analysis. To extend automated diagnosis, recent computational intelligence research has also explored the field of Topological Data Analysis (TDA), a branch of mathematics that uses topology to extract meaningful information from highly complex data. This study combines TDA and DL with ensemble learning to create a hybrid TDA-DL BCC diagnostic model. Persistence homology (a TDA technique) is implemented to extract topological features from automatically segmented telangiectasia as well as skin lesions, and DL features are generated by fine-tuning a pre-trained EfficientNet-B5 model. The final hybrid TDA-DL model achieves state-of-the-art accuracy of 97.4% and an AUC of 0.995 on a holdout test of 395 skin lesions for BCC diagnosis. This study demonstrates that telangiectasia features improve BCC diagnosis, and TDA techniques hold the potential to improve DL performance.
Collapse
Affiliation(s)
- Akanksha Maurya
- Missouri University of Science &Technology, Rolla, MO, 65209, USA
| | - R Joe Stanley
- Missouri University of Science &Technology, Rolla, MO, 65209, USA.
| | - Norsang Lama
- Missouri University of Science &Technology, Rolla, MO, 65209, USA
| | - Anand K Nambisan
- Missouri University of Science &Technology, Rolla, MO, 65209, USA
| | | | | | | | - Colin Smith
- A.T. Still, University of Health Sciences, Kirksville, MO, USA
| | | | | | | |
Collapse
|
9
|
Veeramani N, Jayaraman P, Krishankumar R, Ravichandran KS, Gandomi AH. DDCNN-F: double decker convolutional neural network 'F' feature fusion as a medical image classification framework. Sci Rep 2024; 14:676. [PMID: 38182607 PMCID: PMC10770172 DOI: 10.1038/s41598-023-49721-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2023] [Accepted: 12/11/2023] [Indexed: 01/07/2024] Open
Abstract
Melanoma is a severe skin cancer that involves abnormal cell development. This study aims to provide a new feature fusion framework for melanoma classification that includes a novel 'F' Flag feature for early detection. This novel 'F' indicator efficiently distinguishes benign skin lesions from malignant ones known as melanoma. The article proposes an architecture that is built in a Double Decker Convolutional Neural Network called DDCNN future fusion. The network's deck one, known as a Convolutional Neural Network (CNN), finds difficult-to-classify hairy images using a confidence factor termed the intra-class variance score. These hirsute image samples are combined to form a Baseline Separated Channel (BSC). By eliminating hair and using data augmentation techniques, the BSC is ready for analysis. The network's second deck trains the pre-processed BSC and generates bottleneck features. The bottleneck features are merged with features generated from the ABCDE clinical bio indicators to promote classification accuracy. Different types of classifiers are fed to the resulting hybrid fused features with the novel 'F' Flag feature. The proposed system was trained using the ISIC 2019 and ISIC 2020 datasets to assess its performance. The empirical findings expose that the DDCNN feature fusion strategy for exposing malignant melanoma achieved a specificity of 98.4%, accuracy of 93.75%, precision of 98.56%, and Area Under Curve (AUC) value of 0.98. This study proposes a novel approach that can accurately identify and diagnose fatal skin cancer and outperform other state-of-the-art techniques, which is attributed to the DDCNN 'F' Feature fusion framework. Also, this research ascertained improvements in several classifiers when utilising the 'F' indicator, resulting in the highest specificity of + 7.34%.
Collapse
Affiliation(s)
- Nirmala Veeramani
- School of Computing, SASTRA Deemed to Be University, Thanjavur, India
| | | | - Raghunathan Krishankumar
- Information Technology Systems and Analytics Area, Indian Institute of Management Bodh Gaya, Bodh Gaya, Bihar, 824234, India
| | | | - Amir H Gandomi
- Faculty of Engineering and Information Technology, University of Technology Sydney, Ultimo, NSW, Australia.
- University Research and Innovation Center (EKIK), Obuda University, Buddapest, Hungary.
| |
Collapse
|
10
|
Longo LHDC, Roberto GF, Tosta TAA, de Faria PR, Loyola AM, Cardoso SV, Silva AB, do Nascimento MZ, Neves LA. Classification of Multiple H&E Images via an Ensemble Computational Scheme. ENTROPY (BASEL, SWITZERLAND) 2023; 26:34. [PMID: 38248160 PMCID: PMC10814107 DOI: 10.3390/e26010034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/27/2023] [Revised: 12/23/2023] [Accepted: 12/25/2023] [Indexed: 01/23/2024]
Abstract
In this work, a computational scheme is proposed to identify the main combinations of handcrafted descriptors and deep-learned features capable of classifying histological images stained with hematoxylin and eosin. The handcrafted descriptors were those representatives of multiscale and multidimensional fractal techniques (fractal dimension, lacunarity and percolation) applied to quantify the histological images with the corresponding representations via explainable artificial intelligence (xAI) approaches. The deep-learned features were obtained from different convolutional neural networks (DenseNet-121, EfficientNet-b2, Inception-V3, ResNet-50 and VGG-19). The descriptors were investigated through different associations. The most relevant combinations, defined through a ranking algorithm, were analyzed via a heterogeneous ensemble of classifiers with the support vector machine, naive Bayes, random forest and K-nearest neighbors algorithms. The proposed scheme was applied to histological samples representative of breast cancer, colorectal cancer, oral dysplasia and liver tissue. The best results were accuracy rates of 94.83% to 100%, with the identification of pattern ensembles for classifying multiple histological images. The computational scheme indicated solutions exploring a reduced number of features (a maximum of 25 descriptors) and with better performance values than those observed in the literature. The presented information in this study is useful to complement and improve the development of computer-aided diagnosis focused on histological images.
Collapse
Affiliation(s)
- Leonardo H. da Costa Longo
- Department of Computer Science and Statistics (DCCE), São Paulo State University (UNESP), Rua Cristóvão Colombo, 2265, São José do Rio Preto 15054-000, SP, Brazil
| | - Guilherme F. Roberto
- Department of Informatics Engineering, Faculty of Engineering, University of Porto, Dr. Roberto Frias, sn, 4200-465 Porto, Portugal;
| | - Thaína A. A. Tosta
- Science and Technology Institute, Federal University of São Paulo (UNIFESP), Avenida Cesare Mansueto Giulio Lattes, 1201, São José dos Campos 12247-014, SP, Brazil;
| | - Paulo R. de Faria
- Department of Histology and Morphology, Institute of Biomedical Science, Federal University of Uberlândia (UFU), Av. Amazonas, S/N, Uberlândia 38405-320, MG, Brazil;
| | - Adriano M. Loyola
- Area of Oral Pathology, School of Dentistry, Federal University of Uberlândia (UFU), R. Ceará—Umuarama, Uberlândia 38402-018, MG, Brazil; (A.M.L.)
| | - Sérgio V. Cardoso
- Area of Oral Pathology, School of Dentistry, Federal University of Uberlândia (UFU), R. Ceará—Umuarama, Uberlândia 38402-018, MG, Brazil; (A.M.L.)
| | - Adriano B. Silva
- Faculty of Computer Science (FACOM), Federal University of Uberlândia (UFU), Avenida João Naves de Ávila 2121, Bl.B, Uberlândia 38400-902, MG, Brazil
| | - Marcelo Z. do Nascimento
- Faculty of Computer Science (FACOM), Federal University of Uberlândia (UFU), Avenida João Naves de Ávila 2121, Bl.B, Uberlândia 38400-902, MG, Brazil
| | - Leandro A. Neves
- Department of Computer Science and Statistics (DCCE), São Paulo State University (UNESP), Rua Cristóvão Colombo, 2265, São José do Rio Preto 15054-000, SP, Brazil
| |
Collapse
|
11
|
Huang HY, Hsiao YP, Karmakar R, Mukundan A, Chaudhary P, Hsieh SC, Wang HC. A Review of Recent Advances in Computer-Aided Detection Methods Using Hyperspectral Imaging Engineering to Detect Skin Cancer. Cancers (Basel) 2023; 15:5634. [PMID: 38067338 PMCID: PMC10705122 DOI: 10.3390/cancers15235634] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2023] [Revised: 11/20/2023] [Accepted: 11/24/2023] [Indexed: 08/15/2024] Open
Abstract
Skin cancer, a malignant neoplasm originating from skin cell types including keratinocytes, melanocytes, and sweat glands, comprises three primary forms: basal cell carcinoma (BCC), squamous cell carcinoma (SCC), and malignant melanoma (MM). BCC and SCC, while constituting the most prevalent categories of skin cancer, are generally considered less aggressive compared to MM. Notably, MM possesses a greater capacity for invasiveness, enabling infiltration into adjacent tissues and dissemination via both the circulatory and lymphatic systems. Risk factors associated with skin cancer encompass ultraviolet (UV) radiation exposure, fair skin complexion, a history of sunburn incidents, genetic predisposition, immunosuppressive conditions, and exposure to environmental carcinogens. Early detection of skin cancer is of paramount importance to optimize treatment outcomes and preclude the progression of disease, either locally or to distant sites. In pursuit of this objective, numerous computer-aided diagnosis (CAD) systems have been developed. Hyperspectral imaging (HSI), distinguished by its capacity to capture information spanning the electromagnetic spectrum, surpasses conventional RGB imaging, which relies solely on three color channels. Consequently, this study offers a comprehensive exploration of recent CAD investigations pertaining to skin cancer detection and diagnosis utilizing HSI, emphasizing diagnostic performance parameters such as sensitivity and specificity.
Collapse
Affiliation(s)
- Hung-Yi Huang
- Department of Dermatology, Ditmanson Medical Foundation Chiayi Christian Hospital, Chia Yi City 60002, Taiwan;
| | - Yu-Ping Hsiao
- Department of Dermatology, Chung Shan Medical University Hospital, No.110, Sec. 1, Jianguo N. Rd., South District, Taichung City 40201, Taiwan;
- Institute of Medicine, School of Medicine, Chung Shan Medical University, No.110, Sec. 1, Jianguo N. Rd., South District, Taichung City 40201, Taiwan
| | - Riya Karmakar
- Department of Mechanical Engineering, National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi City 62102, Taiwan; (R.K.); (A.M.)
| | - Arvind Mukundan
- Department of Mechanical Engineering, National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi City 62102, Taiwan; (R.K.); (A.M.)
| | - Pramod Chaudhary
- Department of Aeronautical Engineering, Vel Tech Rangarajan Dr. Sagunthala R&D Institute of Science and Technology, Avadi, Chennai 600 062, India;
| | - Shang-Chin Hsieh
- Department of Plastic Surgery, Kaohsiung Armed Forces General Hospital, 2, Zhongzheng 1st. Rd., Lingya District, Kaohsiung 80284, Taiwan
| | - Hsiang-Chen Wang
- Department of Mechanical Engineering, National Chung Cheng University, 168, University Rd., Min Hsiung, Chia Yi City 62102, Taiwan; (R.K.); (A.M.)
- Department of Medical Research, Dalin Tzu Chi General Hospital, No. 2, Min-Sheng Rd., Dalin Town, Chia Yi City 62247, Taiwan
- Technology Development, Hitspectra Intelligent Technology Co., Ltd., Kaohsiung 80661, Taiwan
| |
Collapse
|
12
|
Patel RH, Foltz EA, Witkowski A, Ludzik J. Analysis of Artificial Intelligence-Based Approaches Applied to Non-Invasive Imaging for Early Detection of Melanoma: A Systematic Review. Cancers (Basel) 2023; 15:4694. [PMID: 37835388 PMCID: PMC10571810 DOI: 10.3390/cancers15194694] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Revised: 09/05/2023] [Accepted: 09/19/2023] [Indexed: 10/15/2023] Open
Abstract
BACKGROUND Melanoma, the deadliest form of skin cancer, poses a significant public health challenge worldwide. Early detection is crucial for improved patient outcomes. Non-invasive skin imaging techniques allow for improved diagnostic accuracy; however, their use is often limited due to the need for skilled practitioners trained to interpret images in a standardized fashion. Recent innovations in artificial intelligence (AI)-based techniques for skin lesion image interpretation show potential for the use of AI in the early detection of melanoma. OBJECTIVE The aim of this study was to evaluate the current state of AI-based techniques used in combination with non-invasive diagnostic imaging modalities including reflectance confocal microscopy (RCM), optical coherence tomography (OCT), and dermoscopy. We also aimed to determine whether the application of AI-based techniques can lead to improved diagnostic accuracy of melanoma. METHODS A systematic search was conducted via the Medline/PubMed, Cochrane, and Embase databases for eligible publications between 2018 and 2022. Screening methods adhered to the 2020 version of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Included studies utilized AI-based algorithms for melanoma detection and directly addressed the review objectives. RESULTS We retrieved 40 papers amongst the three databases. All studies directly comparing the performance of AI-based techniques with dermatologists reported the superior or equivalent performance of AI-based techniques in improving the detection of melanoma. In studies directly comparing algorithm performance on dermoscopy images to dermatologists, AI-based algorithms achieved a higher ROC (>80%) in the detection of melanoma. In these comparative studies using dermoscopic images, the mean algorithm sensitivity was 83.01% and the mean algorithm specificity was 85.58%. Studies evaluating machine learning in conjunction with OCT boasted accuracy of 95%, while studies evaluating RCM reported a mean accuracy rate of 82.72%. CONCLUSIONS Our results demonstrate the robust potential of AI-based techniques to improve diagnostic accuracy and patient outcomes through the early identification of melanoma. Further studies are needed to assess the generalizability of these AI-based techniques across different populations and skin types, improve standardization in image processing, and further compare the performance of AI-based techniques with board-certified dermatologists to evaluate clinical applicability.
Collapse
Affiliation(s)
- Raj H. Patel
- Edward Via College of Osteopathic Medicine, VCOM-Louisiana, 4408 Bon Aire Dr, Monroe, LA 71203, USA
- Department of Dermatology, Oregon Health & Science University, Portland, OR 97239, USA (A.W.); (J.L.)
| | - Emilie A. Foltz
- Department of Dermatology, Oregon Health & Science University, Portland, OR 97239, USA (A.W.); (J.L.)
- Elson S. Floyd College of Medicine, Washington State University, Spokane, WA 99202, USA
| | - Alexander Witkowski
- Department of Dermatology, Oregon Health & Science University, Portland, OR 97239, USA (A.W.); (J.L.)
| | - Joanna Ludzik
- Department of Dermatology, Oregon Health & Science University, Portland, OR 97239, USA (A.W.); (J.L.)
| |
Collapse
|
13
|
Lama N, Hagerty J, Nambisan A, Stanley RJ, Van Stoecker W. Skin Lesion Segmentation in Dermoscopic Images with Noisy Data. J Digit Imaging 2023; 36:1712-1722. [PMID: 37020149 PMCID: PMC10407008 DOI: 10.1007/s10278-023-00819-8] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Revised: 03/15/2023] [Accepted: 03/17/2023] [Indexed: 04/07/2023] Open
Abstract
We propose a deep learning approach to segment the skin lesion in dermoscopic images. The proposed network architecture uses a pretrained EfficientNet model in the encoder and squeeze-and-excitation residual structures in the decoder. We applied this approach on the publicly available International Skin Imaging Collaboration (ISIC) 2017 Challenge skin lesion segmentation dataset. This benchmark dataset has been widely used in previous studies. We observed many inaccurate or noisy ground truth labels. To reduce noisy data, we manually sorted all ground truth labels into three categories - good, mildly noisy, and noisy labels. Furthermore, we investigated the effect of such noisy labels in training and test sets. Our test results show that the proposed method achieved Jaccard scores of 0.807 on the official ISIC 2017 test set and 0.832 on the curated ISIC 2017 test set, exhibiting better performance than previously reported methods. Furthermore, the experimental results showed that the noisy labels in the training set did not lower the segmentation performance. However, the noisy labels in the test set adversely affected the evaluation scores. We recommend that the noisy labels should be avoided in the test set in future studies for accurate evaluation of the segmentation algorithms.
Collapse
Affiliation(s)
- Norsang Lama
- Missouri University of Science &Technology, Rolla, MO, 65409, USA
| | | | - Anand Nambisan
- Missouri University of Science &Technology, Rolla, MO, 65409, USA
| | | | | |
Collapse
|
14
|
Viriyasaranon T, Chun JW, Koh YH, Cho JH, Jung MK, Kim SH, Kim HJ, Lee WJ, Choi JH, Woo SM. Annotation-Efficient Deep Learning Model for Pancreatic Cancer Diagnosis and Classification Using CT Images: A Retrospective Diagnostic Study. Cancers (Basel) 2023; 15:3392. [PMID: 37444502 DOI: 10.3390/cancers15133392] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2023] [Revised: 06/26/2023] [Accepted: 06/26/2023] [Indexed: 07/15/2023] Open
Abstract
The aim of this study was to develop a novel deep learning (DL) model without requiring large-annotated training datasets for detecting pancreatic cancer (PC) using computed tomography (CT) images. This retrospective diagnostic study was conducted using CT images collected from 2004 and 2019 from 4287 patients diagnosed with PC. We proposed a self-supervised learning algorithm (pseudo-lesion segmentation (PS)) for PC classification, which was trained with and without PS and validated on randomly divided training and validation sets. We further performed cross-racial external validation using open-access CT images from 361 patients. For internal validation, the accuracy and sensitivity for PC classification were 94.3% (92.8-95.4%) and 92.5% (90.0-94.4%), and 95.7% (94.5-96.7%) and 99.3 (98.4-99.7%) for the convolutional neural network (CNN) and transformer-based DL models (both with PS), respectively. Implementing PS on a small-sized training dataset (randomly sampled 10%) increased accuracy by 20.5% and sensitivity by 37.0%. For external validation, the accuracy and sensitivity were 82.5% (78.3-86.1%) and 81.7% (77.3-85.4%) and 87.8% (84.0-90.8%) and 86.5% (82.3-89.8%) for the CNN and transformer-based DL models (both with PS), respectively. PS self-supervised learning can increase DL-based PC classification performance, reliability, and robustness of the model for unseen, and even small, datasets. The proposed DL model is potentially useful for PC diagnosis.
Collapse
Affiliation(s)
- Thanaporn Viriyasaranon
- Graduate Program in System Health Science and Engineering, Division of Mechanical and Biomedical Engineering, Ewha Womans University, Seoul 03760, Republic of Korea
| | - Jung Won Chun
- Center for Liver and Pancreatobiliary Cancer, National Cancer Center, Goyang 10408, Republic of Korea
| | - Young Hwan Koh
- Center for Liver and Pancreatobiliary Cancer, National Cancer Center, Goyang 10408, Republic of Korea
| | - Jae Hee Cho
- Department of Internal Medicine, Gangnam Severance Hospital, Yonsei University College of Medicine, Seoul 03722, Republic of Korea
| | - Min Kyu Jung
- Department of Internal Medicine, Kyungpook National University Hospital, Daegu 41944, Republic of Korea
| | - Seong-Hun Kim
- Department of Internal Medicine, Research Institute of Clinical Medicine of Jeonbuk National University-Biomedical Research Institute of Jeonbuk National University Hospital, Jeonju 54907, Republic of Korea
| | - Hyo Jung Kim
- Department of Gastroenterology, Korea University Guro Hospital, Seoul 10408, Republic of Korea
| | - Woo Jin Lee
- Center for Liver and Pancreatobiliary Cancer, National Cancer Center, Goyang 10408, Republic of Korea
| | - Jang-Hwan Choi
- Graduate Program in System Health Science and Engineering, Division of Mechanical and Biomedical Engineering, Ewha Womans University, Seoul 03760, Republic of Korea
| | - Sang Myung Woo
- Center for Liver and Pancreatobiliary Cancer, National Cancer Center, Goyang 10408, Republic of Korea
| |
Collapse
|
15
|
Liu N, Rejeesh MR, Sundararaj V, Gunasundari B. ACO-KELM: Anti Coronavirus Optimized Kernel-based Softplus Extreme Learning Machine for Classification of Skin Cancer. EXPERT SYSTEMS WITH APPLICATIONS 2023:120719. [PMID: 37362255 PMCID: PMC10268820 DOI: 10.1016/j.eswa.2023.120719] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/15/2022] [Revised: 04/28/2023] [Accepted: 06/03/2023] [Indexed: 06/28/2023]
Abstract
Due to the presence of redundant and irrelevant features in large-dimensional biomedical datasets, the prediction accuracy of disease diagnosis can often be decreased. Therefore, it is important to adopt feature extraction methodologies that can deal with problem structures and identify underlying data patterns. In this paper, we propose a novel approach called the Anti Coronavirus Optimized Kernel-based Softplus Extreme Learning Machine (ACO-KSELM) to accurately predict different types of skin cancer by analyzing high-dimensional datasets. To evaluate the proposed ACO-KSELM method, we used four different skin cancer image datasets: ISIC 2016, ACS, HAM10000, and PAD-UFES-20. These dermoscopic image datasets were preprocessed using Gaussian filters to remove noise and artifacts, and relevant features based on color, texture, and shape were extracted using color histogram, Haralick texture, and Hu moment extraction approaches, respectively. Finally, the proposed ACO-KSELM method accurately predicted and classified the extracted features into Basal Cell Carcinoma (BCC), Squamous Cell Carcinoma (SCC), Actinic Keratosis (ACK), Seborrheic Keratosis (SEK), Bowen's disease (BOD), Melanoma (MEL), and Nevus (NEV) categories. The analytical results showed that the proposed method achieved a higher rate of prediction accuracy of about 98.9%, 98.7%, 98.6%, and 97.9% for the ISIC 2016, ACS, HAM10000, and PAD-UFES-20 datasets, respectively.
Collapse
Affiliation(s)
- Nannan Liu
- School of Electronic and Information Engineering, Ningbo University of Technology, Ningbo, 315211, China
| | - M R Rejeesh
- REVIRE Intelligence LLP, Eraviputoorakadi, Tamilnadu India
| | | | - B Gunasundari
- Departmentof IT, REVIRE Intelligence LLP, Tamilnadu India
| |
Collapse
|
16
|
Jiang W, Wang H, Chen W, Zhao Y, Yan B, Chen D, Dong X, Cheng J, Lin Z, Zhuo S, Wang H, Yan J. Association of collagen deep learning classifier with prognosis and chemotherapy benefits in stage II-III colon cancer. Bioeng Transl Med 2023; 8:e10526. [PMID: 37206212 PMCID: PMC10189440 DOI: 10.1002/btm2.10526] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2022] [Revised: 03/30/2023] [Accepted: 04/03/2023] [Indexed: 05/21/2023] Open
Abstract
The current tumor-node-metastasis staging system does not provide sufficient prognostic prediction or adjuvant chemotherapy benefit information for stage II-III colon cancer (CC) patients. Collagen in the tumor microenvironment affects the biological behaviors and chemotherapy response of cancer cells. Hence, in this study, we proposed a collagen deep learning (collagenDL) classifier based on the 50-layer residual network model for predicting disease-free survival (DFS) and overall survival (OS). The collagenDL classifier was significantly associated with DFS and OS (P < 0.001). The collagenDL nomogram, integrating the collagenDL classifier and three clinicopathologic predictors, improved the prediction performance, which showed satisfactory discrimination and calibration. These results were independently validated in the internal and external validation cohorts. In addition, high-risk stage II and III CC patients with high-collagenDL classifier, rather than low-collagenDL classifier, exhibited a favorable response to adjuvant chemotherapy. In conclusion, the collagenDL classifier could predict prognosis and adjuvant chemotherapy benefits in stage II-III CC patients.
Collapse
Affiliation(s)
- Wei Jiang
- Department of General Surgery, Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor, Nanfang Hospital, The First School of Clinical MedicineSouthern Medical UniversityGuangzhouPeople's Republic of China
- School of ScienceJimei UniversityXiamenFujianPeople's Republic of China
| | - Huaiming Wang
- Department of Colorectal Surgery & Guangdong Institute of Gastroenterology, Guangdong Provincial Key Laboratory of Colorectal and Pelvic Floor Diseases, the Sixth Affiliated Hospital, Supported by National Key Clinical DisciplineSun Yat‐sen UniversityGuangzhouGuangdongPeople's Republic of China
| | - Weisheng Chen
- Department of General Surgery, Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor, Nanfang Hospital, The First School of Clinical MedicineSouthern Medical UniversityGuangzhouPeople's Republic of China
| | - Yandong Zhao
- Department of Pathology, the Sixth Affiliated HospitalSun Yat‐sen UniversityGuangzhouGuangdongPeople's Republic of China
| | - Botao Yan
- Department of General Surgery, Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor, Nanfang Hospital, The First School of Clinical MedicineSouthern Medical UniversityGuangzhouPeople's Republic of China
| | - Dexin Chen
- Department of General Surgery, Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor, Nanfang Hospital, The First School of Clinical MedicineSouthern Medical UniversityGuangzhouPeople's Republic of China
| | - Xiaoyu Dong
- Department of General Surgery, Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor, Nanfang Hospital, The First School of Clinical MedicineSouthern Medical UniversityGuangzhouPeople's Republic of China
| | - Jiaxin Cheng
- Department of General Surgery, Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor, Nanfang Hospital, The First School of Clinical MedicineSouthern Medical UniversityGuangzhouPeople's Republic of China
| | - Zexi Lin
- School of ScienceJimei UniversityXiamenFujianPeople's Republic of China
| | - Shuangmu Zhuo
- School of ScienceJimei UniversityXiamenFujianPeople's Republic of China
| | - Hui Wang
- Department of Colorectal Surgery & Guangdong Institute of Gastroenterology, Guangdong Provincial Key Laboratory of Colorectal and Pelvic Floor Diseases, the Sixth Affiliated Hospital, Supported by National Key Clinical DisciplineSun Yat‐sen UniversityGuangzhouGuangdongPeople's Republic of China
| | - Jun Yan
- Department of General Surgery, Guangdong Provincial Key Laboratory of Precision Medicine for Gastrointestinal Tumor, Nanfang Hospital, The First School of Clinical MedicineSouthern Medical UniversityGuangzhouPeople's Republic of China
| |
Collapse
|
17
|
Yang L, Fan C, Lin H, Qiu Y. Rema-Net: An efficient multi-attention convolutional neural network for rapid skin lesion segmentation. Comput Biol Med 2023; 159:106952. [PMID: 37084639 DOI: 10.1016/j.compbiomed.2023.106952] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Revised: 04/10/2023] [Accepted: 04/15/2023] [Indexed: 04/23/2023]
Abstract
For clinical treatment, the accurate segmentation of lesions from dermoscopic images is extremely valuable. Convolutional neural networks (such as U-Net and its numerous variants) have become the main methods for skin lesion segmentation in recent years. However, because these methods frequently have a large number of parameters and complicated algorithm structures, which results in high hardware requirements and long training time, it is difficult to effectively use them for fast training and segmentation tasks. For this reason, we proposed an efficient multi-attention convolutional neural network (Rema-Net) for rapid skin lesion segmentation. The down-sampling module of the network only uses a convolutional layer and a pooling layer, with spatial attention added to improve useful features. We also designed skip-connections between the down-sampling and up-sampling parts of the network, and used reverse attention operation on the skip-connections to strengthen segmentation performance of the network. We conducted extensive experiments on five publicly available datasets to validate the effectiveness of our method, including the ISIC-2016, ISIC-2017, ISIC-2018, PH2, and HAM10000 datasets. The results show that the proposed method reduced the number of parameters by nearly 40% when compared with U-Net. Furthermore, the segmentation metrics are significantly better than some previous methods, and the predictions are closer to the real lesion.
Collapse
Affiliation(s)
- Litao Yang
- School of Information Science and Engineering, Henan University of Technology, Zhengzhou City, Henan Province, 450001, China
| | - Chao Fan
- School of Artificial Intelligence and Big Data, Henan University of Technology, Zhengzhou City, Henan Province, 450001, China; Key Laboratory of Grain Information Processing and Control, Ministry of Education, Zhengzhou City, Henan Province, 450001, China.
| | - Hao Lin
- School of Information Science and Engineering, Henan University of Technology, Zhengzhou City, Henan Province, 450001, China
| | - Yingying Qiu
- School of Information Science and Engineering, Henan University of Technology, Zhengzhou City, Henan Province, 450001, China
| |
Collapse
|
18
|
Kasmi R, Hagerty J, Young R, Lama N, Nepal J, Miinch J, Stoecker W, Stanley RJ. SharpRazor: Automatic removal of hair and ruler marks from dermoscopy images. Skin Res Technol 2023; 29:e13203. [PMID: 37113095 PMCID: PMC10234178 DOI: 10.1111/srt.13203] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Accepted: 05/03/2022] [Indexed: 04/03/2023]
Abstract
BACKGROUND The removal of hair and ruler marks is critical in handcrafted image analysis of dermoscopic skin lesions. No other dermoscopic artifacts cause more problems in segmentation and structure detection. PURPOSE The aim of the work is to detect both white and black hair, artifacts and finally inpaint correctly the image. METHOD We introduce a new algorithm: SharpRazor, to detect hair and ruler marks and remove them from the image. Our multiple-filter approach detects hairs of varying widths within varying backgrounds, while avoiding detection of vessels and bubbles. The proposed algorithm utilizes grayscale plane modification, hair enhancement, segmentation using tri-directional gradients, and multiple filters for hair of varying widths. We develop an alternate entropy-based processing adaptive thresholding method. White or light-colored hair, and ruler marks are detected separately and added to the final hair mask. A classifier removes noise objects. Finally, a new technique of inpainting is presented, and this is utilized to remove the detected object from the lesion image. RESULTS The proposed algorithm is tested on two datasets, and compares with seven existing methods measuring accuracy, precision, recall, dice, and Jaccard scores. SharpRazor is shown to outperform existing methods. CONCLUSION The Shaprazor techniques show the promise to reach the purpose of removing and inpaint both dark and white hair in a wide variety of lesions.
Collapse
Affiliation(s)
- Reda Kasmi
- Faculty of Technology, Laboratoire de Technologie Industrielle et de l'Information (LTII)University of BejaiaBejaiaAlgeria
| | | | - Reagan Young
- Department of Electrical and Computer EngineeringMissouri University of Science and TechnologyRollaMissouriUSA
| | - Norsang Lama
- Department of Electrical and Computer EngineeringMissouri University of Science and TechnologyRollaMissouriUSA
| | - Januka Nepal
- Department of Electrical and Computer EngineeringMissouri University of Science and TechnologyRollaMissouriUSA
| | - Jessica Miinch
- Department of Electrical and Computer EngineeringMissouri University of Science and TechnologyRollaMissouriUSA
| | | | - R Joe Stanley
- Department of Electrical and Computer EngineeringMissouri University of Science and TechnologyRollaMissouriUSA
| |
Collapse
|
19
|
Lama N, Kasmi R, Hagerty JR, Stanley RJ, Young R, Miinch J, Nepal J, Nambisan A, Stoecker WV. ChimeraNet: U-Net for Hair Detection in Dermoscopic Skin Lesion Images. J Digit Imaging 2023; 36:526-535. [PMID: 36385676 PMCID: PMC10039207 DOI: 10.1007/s10278-022-00740-6] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Revised: 11/02/2022] [Accepted: 11/09/2022] [Indexed: 11/18/2022] Open
Abstract
Hair and ruler mark structures in dermoscopic images are an obstacle preventing accurate image segmentation and detection of critical network features. Recognition and removal of hairs from images can be challenging, especially for hairs that are thin, overlapping, faded, or of similar color as skin or overlaid on a textured lesion. This paper proposes a novel deep learning (DL) technique to detect hair and ruler marks in skin lesion images. Our proposed ChimeraNet is an encoder-decoder architecture that employs pretrained EfficientNet in the encoder and squeeze-and-excitation residual (SERes) structures in the decoder. We applied this approach at multiple image sizes and evaluated it using the publicly available HAM10000 (ISIC2018 Task 3) skin lesion dataset. Our test results show that the largest image size (448 × 448) gave the highest accuracy of 98.23 and Jaccard index of 0.65 on the HAM10000 (ISIC 2018 Task 3) skin lesion dataset, exhibiting better performance than for two well-known deep learning approaches, U-Net and ResUNet-a. We found the Dice loss function to give the best results for all measures. Further evaluated on 25 additional test images, the technique yields state-of-the-art accuracy compared to 8 previously reported classical techniques. We conclude that the proposed ChimeraNet architecture may enable improved detection of fine image structures. Further application of DL techniques to detect dermoscopy structures is warranted.
Collapse
Affiliation(s)
- Norsang Lama
- Missouri University of Science & Technology, Rolla, MO, 65409, USA
| | | | | | - R Joe Stanley
- Missouri University of Science & Technology, Rolla, MO, 65409, USA.
| | - Reagan Young
- Missouri University of Science & Technology, Rolla, MO, 65409, USA
| | - Jessica Miinch
- Missouri University of Science & Technology, Rolla, MO, 65409, USA
| | | | - Anand Nambisan
- Missouri University of Science & Technology, Rolla, MO, 65409, USA
| | | |
Collapse
|
20
|
Alphonse AS, Benifa JVB, Muaad AY, Chola C, Heyat MBB, Murshed BAH, Abdel Samee N, Alabdulhafith M, Al-antari MA. A Hybrid Stacked Restricted Boltzmann Machine with Sobel Directional Patterns for Melanoma Prediction in Colored Skin Images. Diagnostics (Basel) 2023; 13:diagnostics13061104. [PMID: 36980412 PMCID: PMC10047753 DOI: 10.3390/diagnostics13061104] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Revised: 03/07/2023] [Accepted: 03/09/2023] [Indexed: 03/17/2023] Open
Abstract
Melanoma, a kind of skin cancer that is very risky, is distinguished by uncontrolled cell multiplication. Melanoma detection is of the utmost significance in clinical practice because of the atypical border structure and the numerous types of tissue it can involve. The identification of melanoma is still a challenging process for color images, despite the fact that numerous approaches have been proposed in the research that has been done. In this research, we present a comprehensive system for the efficient and precise classification of skin lesions. The framework includes preprocessing, segmentation, feature extraction, and classification modules. Preprocessing with DullRazor eliminates skin-imaging hair artifacts. Next, Fully Connected Neural Network (FCNN) semantic segmentation extracts precise and obvious Regions of Interest (ROIs). We then extract relevant skin image features from ROIs using an enhanced Sobel Directional Pattern (SDP). For skin image analysis, Sobel Directional Pattern outperforms ABCD. Finally, a stacked Restricted Boltzmann Machine (RBM) classifies skin ROIs. Stacked RBMs accurately classify skin melanoma. The experiments have been conducted on five datasets: Pedro Hispano Hospital (PH2), International Skin Imaging Collaboration (ISIC 2016), ISIC 2017, Dermnet, and DermIS, and achieved an accuracy of 99.8%, 96.5%, 95.5%, 87.9%, and 97.6%, respectively. The results show that a stack of Restricted Boltzmann Machines is superior for categorizing skin cancer types using the proposed innovative SDP.
Collapse
Affiliation(s)
- A. Sherly Alphonse
- School of Computer Science and Engineering, Vellore Institute of Technology, Chennai 600127, India
| | - J. V. Bibal Benifa
- Department of Studies in Computer Science and Engineering, Indian Institute of Information Technology, Kottayam 686635, India
- Correspondence: (J.V.B.B.); (M.A.); (M.A.A.-a.)
| | - Abdullah Y. Muaad
- Department of Studies in Computer Science, University of Mysore, Manasagangothri, Mysore 570006, India
| | - Channabasava Chola
- Department of Studies in Computer Science and Engineering, Indian Institute of Information Technology, Kottayam 686635, India
| | - Md Belal Bin Heyat
- IoT Research Center, College of Computer Science and Software Engineering, Shenzhen University, Shenzhen 518060, China
| | | | - Nagwan Abdel Samee
- Department of Information Technology, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, P.O. Box 84428, Riyadh 11671, Saudi Arabia
| | - Maali Alabdulhafith
- Department of Information Technology, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, P.O. Box 84428, Riyadh 11671, Saudi Arabia
- Correspondence: (J.V.B.B.); (M.A.); (M.A.A.-a.)
| | - Mugahed A. Al-antari
- Department of Artificial Intelligence, College of Software and Convergence Technology, Daeyang AI Center, Sejong University, Seoul 05006, Republic of Korea
- Correspondence: (J.V.B.B.); (M.A.); (M.A.A.-a.)
| |
Collapse
|
21
|
Bharathi G, Malleswaran M, Muthupriya V. Detection and diagnosis of melanoma skin cancers in dermoscopic images using pipelined internal module architecture (PIMA) method. Microsc Res Tech 2023; 86:701-713. [PMID: 36860140 DOI: 10.1002/jemt.24307] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2022] [Revised: 02/10/2023] [Accepted: 02/13/2023] [Indexed: 03/03/2023]
Abstract
Detection and diagnosis of melanoma skin cancer is important to save the life of humans. The main objective of this article is to perform both detection and diagnosis of the skin cancers in dermoscopy images. Both skin cancer detection and diagnosis system uses deep learning architectures for the effective performance improvement as the main objective. The detection process involves by identifying the cancer affected skin dermoscopy images and the diagnosis process involves by estimating the severity levels of the segmented cancer regions in skin images. This article proposes parallel CNN architecture for the classification of skin images into either melanoma or healthy. Initially, color map histogram equalization (CMHE) method is proposed in this article to enhance the source skin images and then thick and thin edges are detected from the enhanced skin image using the Fuzzy system. The gray-level co-occurrence matrix (GLCM) and Law's texture features are extracted from the edge detected images and these features are optimized using genetic algorithm (GA) approach. Further, the optimized features are classified by the developed pipelined internal module architecture (PIMA) of deep learning structure. The cancer regions in the classified melanoma skin images are segmented using mathematical morphological process and these segmented cancer regions are diagnosed into either mild or severe using the proposed PIMA structure. The proposed PIMA-based skin cancer classification system is applied and tested on ISIC and HAM 10000 skin image datasets. RESEARCH HIGHLIGHTS: The melanoma skin cancer is detected and classified using dermoscopy images. The skin dermoscopy images are enhanced using color map histogram equalization. GLCM and Law's texture features are extracted from the enhanced skin images. To propose pipelined internal module architecture (PIMA) for the classification of skin images.
Collapse
Affiliation(s)
- G Bharathi
- Faculty of Department of Electronics and Communication Engineering, Ranippettai Engineering College, Ranipet, Tamil Nadu, India
| | - M Malleswaran
- Department of Electronics and Communication, Anna University, Chennai, India
| | - V Muthupriya
- Department of Computer Science Engineering, B.S AbdurRahman Crescent Institute of Science and Technology, Chennai, India
| |
Collapse
|
22
|
Zhang W, Lu F, Zhao W, Hu Y, Su H, Yuan M. ACCPG-Net: A skin lesion segmentation network with Adaptive Channel-Context-Aware Pyramid Attention and Global Feature Fusion. Comput Biol Med 2023; 154:106580. [PMID: 36716686 DOI: 10.1016/j.compbiomed.2023.106580] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2022] [Revised: 01/09/2023] [Accepted: 01/22/2023] [Indexed: 01/26/2023]
Abstract
The computer-aided diagnosis system based on dermoscopic images has played an important role in the clinical treatment of skin lesion. An accurate, efficient, and automatic skin lesion segmentation method is an important auxiliary tool for clinical diagnosis. At present, skin lesion segmentation still suffers from great challenges. Existing deep-learning-based automatic segmentation methods frequently use convolutional neural networks (CNN). However, the globally-sharing feature re-weighting vector may not be optimal for the prediction of lesion areas in dermoscopic images. The presence of hairs and spots in some samples aggravates the interference of similar categories, and reduces the segmentation accuracy. To solve this problem, this paper proposes a new deep network for precise skin lesion segmentation based on a U-shape structure. To be specific, two lightweight attention modules: adaptive channel-context-aware pyramid attention (ACCAPA) module and global feature fusion (GFF) module, are embedded in the network. The ACCAPA module can model the characteristics of the lesion areas by dynamically learning the channel information, contextual information and global structure information. GFF is used for different levels of semantic information interaction between encoder and decoder layers. To validate the effectiveness of the proposed method, we test the performance of ACCPG-Net on several public skin lesion datasets. The results show that our method achieves better segmentation performance compared to other state-of-the-art methods.
Collapse
Affiliation(s)
- Wenyu Zhang
- School of Information Science and Engineering, Lanzhou University, China
| | - Fuxiang Lu
- School of Information Science and Engineering, Lanzhou University, China.
| | - Wei Zhao
- School of Information Science and Engineering, Lanzhou University, China
| | - Yawen Hu
- School of Information Science and Engineering, Lanzhou University, China
| | - Hongjing Su
- School of Information Science and Engineering, Lanzhou University, China
| | - Min Yuan
- School of Information Science and Engineering, Lanzhou University, China
| |
Collapse
|
23
|
Baloi A, Costea C, Gutt R, Balacescu O, Turcu F, Belean B. Hexagonal-Grid-Layout Image Segmentation Using Shock Filters: Computational Complexity Case Study for Microarray Image Analysis Related to Machine Learning Approaches. SENSORS (BASEL, SWITZERLAND) 2023; 23:2582. [PMID: 36904788 PMCID: PMC10007319 DOI: 10.3390/s23052582] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/02/2023] [Revised: 02/17/2023] [Accepted: 02/21/2023] [Indexed: 06/18/2023]
Abstract
Hexagonal grid layouts are advantageous in microarray technology; however, hexagonal grids appear in many fields, especially given the rise of new nanostructures and metamaterials, leading to the need for image analysis on such structures. This work proposes a shock-filter-based approach driven by mathematical morphology for the segmentation of image objects disposed in a hexagonal grid. The original image is decomposed into a pair of rectangular grids, such that their superposition generates the initial image. Within each rectangular grid, the shock-filters are once again used to confine the foreground information for each image object into an area of interest. The proposed methodology was successfully applied for microarray spot segmentation, whereas its character of generality is underlined by the segmentation results obtained for two other types of hexagonal grid layouts. Considering the segmentation accuracy through specific quality measures for microarray images, such as the mean absolute error and the coefficient of variation, high correlations of our computed spot intensity features with the annotated reference values were found, indicating the reliability of the proposed approach. Moreover, taking into account that the shock-filter PDE formalism is targeting the one-dimensional luminance profile function, the computational complexity to determine the grid is minimized. The order of growth for the computational complexity of our approach is at least one order of magnitude lower when compared with state-of-the-art microarray segmentation approaches, ranging from classical to machine learning ones.
Collapse
Affiliation(s)
- Aurel Baloi
- Research Center for Integrated Analysis and Territorial Management, University of Bucharest, 4-12 Regina Elisabeta, 030018 Bucharest, Romania
- Faculty of Administration and Business, University of Bucharest, 030018 Bucharest, Romania
| | - Carmen Costea
- Department of Mathematics, Faculty of Automation and Computer Science, Technical University of Cluj-Napoca, 400114 Cluj-Napoca, Romania
| | - Robert Gutt
- Center of Advanced Research and Technologies for Alternative Energies, National Institute for Research and Development of Isotopic and Molecular Technologies, 400293 Cluj-Napoca, Romania
| | - Ovidiu Balacescu
- Department of Genetics, Genomics and Experimental Pathology, The Oncology Institute, Prof. Dr. Ion Chiricuta, 400015 Cluj-Napoca, Romania
| | - Flaviu Turcu
- Center of Advanced Research and Technologies for Alternative Energies, National Institute for Research and Development of Isotopic and Molecular Technologies, 400293 Cluj-Napoca, Romania
- Faculty of Physics, Babes-Bolyai University, 400084 Cluj-Napoca, Romania
| | - Bogdan Belean
- Center of Advanced Research and Technologies for Alternative Energies, National Institute for Research and Development of Isotopic and Molecular Technologies, 400293 Cluj-Napoca, Romania
| |
Collapse
|
24
|
Duan Y, Huo J, Chen M, Hou F, Yan G, Li S, Wang H. Early prediction of sepsis using double fusion of deep features and handcrafted features. APPL INTELL 2023; 53:1-17. [PMID: 36685641 PMCID: PMC9843111 DOI: 10.1007/s10489-022-04425-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/21/2022] [Indexed: 01/19/2023]
Abstract
Sepsis is a life-threatening medical condition that is characterized by the dysregulated immune system response to infections, having both high morbidity and mortality rates. Early prediction of sepsis is critical to the decrease of mortality. This paper presents a novel early warning model called Double Fusion Sepsis Predictor (DFSP) for sepsis onset. DFSP is a double fusion framework that combines the benefits of early and late fusion strategies. First, a hybrid deep learning model that combines both the convolutional and recurrent neural networks to extract deep features is proposed. Second, deep features and handcrafted features, such as clinical scores, are concatenated to build the joint feature representation (early fusion). Third, several tree-based models based on joint feature representation are developed to generate the risk scores of sepsis onset that are combined with an End-to-End neural network for final sepsis detection (late fusion). To evaluate DFSP, a retrospective study was conducted, which included patients admitted to the ICUs of a hospital in Shanghai China. The results demonstrate that the DFSP outperforms state-of-the-art approaches in early sepsis prediction.
Collapse
Affiliation(s)
- Yongrui Duan
- School of Economics & Management, Tongji University, Shanghai, China
| | - Jiazhen Huo
- School of Economics & Management, Tongji University, Shanghai, China
| | - Mingzhou Chen
- School of Economics & Management, Tongji University, Shanghai, China
| | - Fenggang Hou
- Department of Oncology, Shanghai Municipal Hospital of Traditional Chinese Medicine, Shanghai, China
| | - Guoliang Yan
- Department of Geriatrics, Shanghai Municipal Hospital of Traditional Chinese Medicine, Shanghai, China
| | - Shufang Li
- Emergency Department, Shuguang Hospital Affiliated to Shanghai University of Traditional Chinese Medicine, Shanghai, China
| | - Haihui Wang
- Department of Geriatrics, Shanghai Municipal Hospital of Traditional Chinese Medicine, Shanghai, China
| |
Collapse
|
25
|
Meng H, Liu X, Niu J, Wang Y, Liao J, Li Q, Chen C. DGANet: A Dual Global Attention Neural Network for Breast Lesion Detection in Ultrasound Images. ULTRASOUND IN MEDICINE & BIOLOGY 2023; 49:31-44. [PMID: 36202677 DOI: 10.1016/j.ultrasmedbio.2022.07.006] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/31/2021] [Revised: 06/20/2022] [Accepted: 07/13/2022] [Indexed: 06/16/2023]
Abstract
Deep learning-based breast lesion detection in ultrasound images has demonstrated great potential to provide objective suggestions for radiologists and improve their accuracy in diagnosing breast diseases. However, the lack of an effective feature enhancement approach limits the performance of deep learning models. Therefore, in this study, we propose a novel dual global attention neural network (DGANet) to improve the accuracy of breast lesion detection in ultrasound images. Specifically, we designed a bilateral spatial attention module and a global channel attention module to enhance features in spatial and channel dimensions, respectively. The bilateral spatial attention module enhances features by capturing supporting information in regions neighboring breast lesions and reducing integration of noise signal. The global channel attention module enhances features of important channels by weighted calculation, where the weights are decided by the learned interdependencies among all channels. To verify the performance of the DGANet, we conduct breast lesion detection experiments on our collected data set of 7040 ultrasound images and a public data set of breast ultrasound images. YOLOv3, RetinaNet, Faster R-CNN, YOLOv5, and YOLOX are used as comparison models. The results indicate that DGANet outperforms the comparison methods by 0.2%-5.9% in total mean average precision.
Collapse
Affiliation(s)
- Hui Meng
- School of Intelligent Science and Technology, Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, 1 Sub-lane Xiangshan, Hangzhou 310024, China
| | - Xuefeng Liu
- State Key Laboratory of Virtual Reality Technology and Systems, School of Computer Science and Engineering, Beihang University, Haidian District, Beijing, China
| | - Jianwei Niu
- State Key Laboratory of Virtual Reality Technology and Systems, School of Computer Science and Engineering, Beihang University, Haidian District, Beijing, China.
| | - Yong Wang
- Department of Diagnostic Ultrasound, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Jintang Liao
- Department of Ultrasound, Xiangya Hospital of Central South University, Changsha, China
| | - Qingfeng Li
- Research Center of Big Data and Computational Intelligence, Hangzhou Innovation Institute of Beihang University, Hangzhou, China
| | - Chen Chen
- State Key Laboratory of Virtual Reality Technology and Systems, School of Computer Science and Engineering, Beihang University, Haidian District, Beijing, China
| |
Collapse
|
26
|
Nancy Jane Y, Charanya SK, Amsaprabhaa M, Jayashanker P, Nehemiah H K. 2-HDCNN: A two-tier hybrid dual convolution neural network feature fusion approach for diagnosing malignant melanoma. Comput Biol Med 2023; 152:106333. [PMID: 36463793 DOI: 10.1016/j.compbiomed.2022.106333] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2022] [Revised: 11/11/2022] [Accepted: 11/15/2022] [Indexed: 11/20/2022]
Abstract
Melanoma is a fatal form of skin cancer, which causes excess skin cell growth in the body. The objective of this work is to develop a two-tier hybrid dual convolution neural network (2-HDCNN) feature fusion approach for malignant melanoma prediction. The first-tier baseline Convolutional Neural Network (CNN) extracts the hard to classify samples based on the confidence factor (class probability variance score) and generates a Baseline Segregated Dataset (BSD). The BSD is then preprocessed using hair removal and data augmentation techniques. The preprocessed BSD is trained with the second-tier CNN that yields the bottleneck features. These features are then combined with the derived features from the ABCD (Asymmetry, Border, Color and Diameter) medical rule to improve classification accuracy. The generated hybrid fused features are fed to different classifiers like Gradient boosting classifiers, Bagging classifiers, XGBoost classifiers, Decision trees, Support Vector Machine, Logistic regression and Multi-layer perceptron. For performance assessment, the proposed framework is trained on the ISIC 2018 dataset. The experimental results prove that the presented 2-HDCNN feature fusion approach has reached an accuracy of 92.15%, precision of 96.96%, specificity of 96.8%, sensitivity of 86.48%, and AUC (Area Under Curve) value of 0.96 for diagnosing malignant melanoma.
Collapse
Affiliation(s)
- Y Nancy Jane
- Department of Computer Technology, Madras Institute of Technology (Anna University), Chennai, 600044, India
| | - S K Charanya
- Department of Computer Technology, Madras Institute of Technology (Anna University), Chennai, 600044, India
| | - M Amsaprabhaa
- Department of Computer Technology, Madras Institute of Technology (Anna University), Chennai, 600044, India
| | - Preetiha Jayashanker
- Department of Computer Technology, Madras Institute of Technology (Anna University), Chennai, 600044, India
| | | |
Collapse
|
27
|
BF2SkNet: best deep learning features fusion-assisted framework for multiclass skin lesion classification. Neural Comput Appl 2022. [DOI: 10.1007/s00521-022-08084-6] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
|
28
|
Wang S, Yin Y, Wang D, Wang Y, Jin Y. Interpretability-Based Multimodal Convolutional Neural Networks for Skin Lesion Diagnosis. IEEE TRANSACTIONS ON CYBERNETICS 2022; 52:12623-12637. [PMID: 34546933 DOI: 10.1109/tcyb.2021.3069920] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Skin lesion diagnosis is a key step for skin cancer screening, which requires high accuracy and interpretability. Though many computer-aided methods, especially deep learning methods, have made remarkable achievements in skin lesion diagnosis, their generalization and interpretability are still a challenge. To solve this issue, we propose an interpretability-based multimodal convolutional neural network (IM-CNN), which is a multiclass classification model with skin lesion images and metadata of patients as input for skin lesion diagnosis. The structure of IM-CNN consists of three main paths to deal with metadata, features extracted from segmented skin lesion with domain knowledge, and skin lesion images, respectively. We add interpretable visual modules to provide explanations for both images and metadata. In addition to area under the ROC curve (AUC), sensitivity, and specificity, we introduce a new indicator, an AUC curve with a sensitivity larger than 80% (AUC_SEN_80) for performance evaluation. Extensive experimental studies are conducted on the popular HAM10000 dataset, and the results indicate that the proposed model has overwhelming advantages compared with popular deep learning models, such as DenseNet, ResNet, and other state-of-the-art models for melanoma diagnosis. The proposed multimodal model also achieves on average 72% and 21% improvement in terms of sensitivity and AUC_SEN_80, respectively, compared with the single-modal model. The visual explanations can also help gain trust from dermatologists and realize man-machine collaborations, effectively reducing the limitation of black-box models in supporting medical decision making.
Collapse
|
29
|
Attention Cost-Sensitive Deep Learning-Based Approach for Skin Cancer Detection and Classification. Cancers (Basel) 2022; 14:cancers14235872. [PMID: 36497355 PMCID: PMC9735681 DOI: 10.3390/cancers14235872] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2022] [Revised: 11/21/2022] [Accepted: 11/23/2022] [Indexed: 12/03/2022] Open
Abstract
Deep learning-based models have been employed for the detection and classification of skin diseases through medical imaging. However, deep learning-based models are not effective for rare skin disease detection and classification. This is mainly due to the reason that rare skin disease has very a smaller number of data samples. Thus, the dataset will be highly imbalanced, and due to the bias in learning, most of the models give better performances. The deep learning models are not effective in detecting the affected tiny portions of skin disease in the overall regions of the image. This paper presents an attention-cost-sensitive deep learning-based feature fusion ensemble meta-classifier approach for skin cancer detection and classification. Cost weights are included in the deep learning models to handle the data imbalance during training. To effectively learn the optimal features from the affected tiny portions of skin image samples, attention is integrated into the deep learning models. The features from the finetuned models are extracted and the dimensionality of the features was further reduced by using a kernel-based principal component (KPCA) analysis. The reduced features of the deep learning-based finetuned models are fused and passed into ensemble meta-classifiers for skin disease detection and classification. The ensemble meta-classifier is a two-stage model. The first stage performs the prediction of skin disease and the second stage performs the classification by considering the prediction of the first stage as features. Detailed analysis of the proposed approach is demonstrated for both skin disease detection and skin disease classification. The proposed approach demonstrated an accuracy of 99% on skin disease detection and 99% on skin disease classification. In all the experimental settings, the proposed approach outperformed the existing methods and demonstrated a performance improvement of 4% accuracy for skin disease detection and 9% accuracy for skin disease classification. The proposed approach can be used as a computer-aided diagnosis (CAD) tool for the early diagnosis of skin cancer detection and classification in healthcare and medical environments. The tool can accurately detect skin diseases and classify the skin disease into their skin disease family.
Collapse
|
30
|
Tiwari A, Tripathi S, Pandey DC, Sharma N, Sharma S. Detection of COVID-19 Infection in CT and X-ray images using transfer learning approach. Technol Health Care 2022; 30:1273-1286. [PMID: 36093719 DOI: 10.3233/thc-220114] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
BACKGROUND The infection caused by the SARS-CoV-2 (COVID-19) pandemic is a threat to human lives. An early and accurate diagnosis is necessary for treatment. OBJECTIVE The study presents an efficient classification methodology for precise identification of infection caused by COVID-19 using CT and X-ray images. METHODS The depthwise separable convolution-based model of MobileNet V2 was exploited for feature extraction. The features of infection were supplied to the SVM classifier for training which produced accurate classification results. RESULT The accuracies for CT and X-ray images are 99.42% and 98.54% respectively. The MCC score was used to avoid any mislead caused by accuracy and F1 score as it is more mathematically balanced metric. The MCC scores obtained for CT and X-ray were 0.9852 and 0.9657, respectively. The Youden's index showed a significant improvement of more than 2% for both imaging techniques. CONCLUSION The proposed transfer learning-based approach obtained the best results for all evaluation metrics and produced reliable results for the accurate identification of COVID-19 symptoms. This study can help in reducing the time in diagnosis of the infection.
Collapse
Affiliation(s)
- Alok Tiwari
- School of Biomedical Engineering, Indian Institute of Technology (Banaras Hindu University), Varanasi, India.,School of Biomedical Engineering, Indian Institute of Technology (Banaras Hindu University), Varanasi, India
| | - Sumit Tripathi
- Department of Electronics and Communication Engineering, Graphic Era Deemed to be University, Dehradun, Uttarakhand, India.,School of Biomedical Engineering, Indian Institute of Technology (Banaras Hindu University), Varanasi, India
| | - Dinesh Chandra Pandey
- Department of Management Studies, Graphic Era Deemed to be University, Dehradun, Uttarakhand, India
| | - Neeraj Sharma
- School of Biomedical Engineering, Indian Institute of Technology (Banaras Hindu University), Varanasi, India
| | - Shiru Sharma
- School of Biomedical Engineering, Indian Institute of Technology (Banaras Hindu University), Varanasi, India
| |
Collapse
|
31
|
Lee JRH, Pavlova M, Famouri M, Wong A. Cancer-Net SCa: tailored deep neural network designs for detection of skin cancer from dermoscopy images. BMC Med Imaging 2022; 22:143. [PMID: 35945505 PMCID: PMC9364616 DOI: 10.1186/s12880-022-00871-w] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2021] [Accepted: 07/26/2022] [Indexed: 11/25/2022] Open
Abstract
Background Skin cancer continues to be the most frequently diagnosed form of cancer in the U.S., with not only significant effects on health and well-being but also significant economic costs associated with treatment. A crucial step to the treatment and management of skin cancer is effective early detection with key screening approaches such as dermoscopy examinations, leading to stronger recovery prognoses. Motivated by the advances of deep learning and inspired by the open source initiatives in the research community, in this study we introduce Cancer-Net SCa, a suite of deep neural network designs tailored for the detection of skin cancer from dermoscopy images that is open source and available to the general public. To the best of the authors’ knowledge, Cancer-Net SCa comprises the first machine-driven design of deep neural network architectures tailored specifically for skin cancer detection, one of which leverages attention condensers for an efficient self-attention design. Results We investigate and audit the behaviour of Cancer-Net SCa in a responsible and transparent manner through explainability-driven performance validation. All the proposed designs achieved improved accuracy when compared to the ResNet-50 architecture while also achieving significantly reduced architectural and computational complexity. In addition, when evaluating the decision making process of the networks, it can be seen that diagnostically relevant critical factors are leveraged rather than irrelevant visual indicators and imaging artifacts. Conclusion The proposed Cancer-Net SCa designs achieve strong skin cancer detection performance on the International Skin Imaging Collaboration (ISIC) dataset, while providing a strong balance between computation and architectural efficiency and accuracy. While Cancer-Net SCa is not a production-ready screening solution, the hope is that the release of Cancer-Net SCa in open source, open access form will encourage researchers, clinicians, and citizen data scientists alike to leverage and build upon them.
Collapse
Affiliation(s)
- James Ren Hou Lee
- Vision and Image Processing Research Group, University of Waterloo, Waterloo, Canada.
| | - Maya Pavlova
- Vision and Image Processing Research Group, University of Waterloo, Waterloo, Canada.,DarwinAI Corp, Waterloo, Canada
| | | | - Alexander Wong
- Vision and Image Processing Research Group, University of Waterloo, Waterloo, Canada.,Waterloo Artificial Intelligence Institute, University of Waterloo, Waterloo, Canada.,DarwinAI Corp, Waterloo, Canada
| |
Collapse
|
32
|
Computational Intelligence-Based Melanoma Detection and Classification Using Dermoscopic Images. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2022; 2022:2370190. [PMID: 35685142 PMCID: PMC9173896 DOI: 10.1155/2022/2370190] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/09/2022] [Revised: 04/18/2022] [Accepted: 05/09/2022] [Indexed: 11/21/2022]
Abstract
Melanoma is a kind of skin cancer caused by the irregular development of pigment-producing cells. Since melanoma detection efficiency is limited to different factors such as poor contrast among lesions and nearby skin regions, and visual resemblance among melanoma and non-melanoma lesions, intelligent computer-aided diagnosis (CAD) models are essential. Recently, computational intelligence (CI) and deep learning (DL) techniques are utilized for effective decision-making in the biomedical field. In addition, the fast-growing advancements in computer-aided surgeries and recent progress in molecular, cellular, and tissue engineering research have made CI an inevitable part of biomedical applications. In this view, the research work here develops a novel computational intelligence-based melanoma detection and classification technique using dermoscopic images (CIMDC-DIs). The proposed CIMDC-DI model encompasses different subprocesses. Primarily, bilateral filtering with fuzzy k-means (FKM) clustering-based image segmentation is applied as a preprocessing step. Besides, NasNet-based feature extractor with stochastic gradient descent is applied for feature extraction. Finally, the manta ray foraging optimization (MRFO) algorithm with a cascaded neural network (CNN) is exploited for the classification process. To ensure the potential efficiency of the CIMDC-DI technique, we conducted a wide-ranging simulation analysis, and the results reported its effectiveness over the existing recent algorithms with the maximum accuracy of 97.50%.
Collapse
|
33
|
Galaz Z, Drotar P, Mekyska J, Gazda M, Mucha J, Zvoncak V, Smekal Z, Faundez-Zanuy M, Castrillon R, Orozco-Arroyave JR, Rapcsak S, Kincses T, Brabenec L, Rektorova I. Comparison of CNN-Learned vs. Handcrafted Features for Detection of Parkinson's Disease Dysgraphia in a Multilingual Dataset. Front Neuroinform 2022; 16:877139. [PMID: 35722168 PMCID: PMC9198652 DOI: 10.3389/fninf.2022.877139] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2022] [Accepted: 04/19/2022] [Indexed: 11/13/2022] Open
Abstract
Parkinson's disease dysgraphia (PDYS), one of the earliest signs of Parkinson's disease (PD), has been researched as a promising biomarker of PD and as the target of a noninvasive and inexpensive approach to monitoring the progress of the disease. However, although several approaches to supportive PDYS diagnosis have been proposed (mainly based on handcrafted features (HF) extracted from online handwriting or the utilization of deep neural networks), it remains unclear which approach provides the highest discrimination power and how these approaches can be transferred between different datasets and languages. This study aims to compare classification performance based on two types of features: features automatically extracted by a pretrained convolutional neural network (CNN) and HF designed by human experts. Both approaches are evaluated on a multilingual dataset collected from 143 PD patients and 151 healthy controls in the Czech Republic, United States, Colombia, and Hungary. The subjects performed the spiral drawing task (SDT; a language-independent task) and the sentence writing task (SWT; a language-dependent task). Models based on logistic regression and gradient boosting were trained in several scenarios, specifically single language (SL), leave one language out (LOLO), and all languages combined (ALC). We found that the HF slightly outperformed the CNN-extracted features in all considered evaluation scenarios for the SWT. In detail, the following balanced accuracy (BACC) scores were achieved: SL—0.65 (HF), 0.58 (CNN); LOLO—0.65 (HF), 0.57 (CNN); and ALC—0.69 (HF), 0.66 (CNN). However, in the case of the SDT, features extracted by a CNN provided competitive results: SL—0.66 (HF), 0.62 (CNN); LOLO—0.56 (HF), 0.54 (CNN); and ALC—0.60 (HF), 0.60 (CNN). In summary, regarding the SWT, the HF outperformed the CNN-extracted features over 6% (mean BACC of 0.66 for HF, and 0.60 for CNN). In the case of the SDT, both feature sets provided almost identical classification performance (mean BACC of 0.60 for HF, and 0.58 for CNN).
Collapse
Affiliation(s)
- Zoltan Galaz
- Department of Telecommunications, Brno University of Technology, Brno, Czechia
| | - Peter Drotar
- Intelligent Information Systems Laboratory, Faculty of Electrical Engineering and Informatics, Technical University of Kosice, Košice, Slovakia
| | - Jiri Mekyska
- Department of Telecommunications, Brno University of Technology, Brno, Czechia
| | - Matej Gazda
- Intelligent Information Systems Laboratory, Faculty of Electrical Engineering and Informatics, Technical University of Kosice, Košice, Slovakia
| | - Jan Mucha
- Department of Telecommunications, Brno University of Technology, Brno, Czechia
| | - Vojtech Zvoncak
- Department of Telecommunications, Brno University of Technology, Brno, Czechia
| | - Zdenek Smekal
- Department of Telecommunications, Brno University of Technology, Brno, Czechia
| | | | - Reinel Castrillon
- Faculty of Engineering, Universidad de Antioquia—UdeA, Medellín, Colombia
- Faculty of Engineering, Universidad Católica de Oriente, Rionegro, Colombia
| | - Juan Rafael Orozco-Arroyave
- Faculty of Engineering, Universidad de Antioquia—UdeA, Medellín, Colombia
- Pattern Recognition Lab, Friedrich-Alexander-Universität, Erlangen, Germany
| | - Steven Rapcsak
- Department of Neurology, College of Medicine, University of Arizona, Tucson, AZ, United States
| | - Tamas Kincses
- Department of Neurology, University of Szeged, Szeged, Hungary
| | - Lubos Brabenec
- Applied Neuroscience Research Group, Central European Institute of Technology—CEITEC, Masaryk University, Brno, Czechia
| | - Irena Rektorova
- Applied Neuroscience Research Group, Central European Institute of Technology—CEITEC, Masaryk University, Brno, Czechia
- First Department of Neurology, Faculty of Medicine and St. Anne's University Hospital, Masaryk University, Brno, Czechia
- *Correspondence: Irena Rektorova
| |
Collapse
|
34
|
Maurya A, Stanley RJ, Lama N, Jagannathan S, Saeed D, Swinfard S, Hagerty JR, Stoecker WV. A deep learning approach to detect blood vessels in basal cell carcinoma. Skin Res Technol 2022; 28:571-576. [PMID: 35611797 PMCID: PMC9907638 DOI: 10.1111/srt.13150] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2021] [Accepted: 03/09/2022] [Indexed: 11/29/2022]
Abstract
PURPOSE Blood vessels called telangiectasia are visible in skin lesions with the aid of dermoscopy. Telangiectasia are a pivotal identifying feature of basal cell carcinoma. These vessels appear thready, serpiginous, and may also appear arborizing, that is, wide vessels branch into successively thinner vessels. Due to these intricacies, their detection is not an easy task, neither with manual annotation nor with computerized techniques. In this study, we automate the segmentation of telangiectasia in dermoscopic images with a deep learning U-Net approach. METHODS We apply a combination of image processing techniques and a deep learning-based U-Net approach to detect telangiectasia in digital basal cell carcinoma skin cancer images. We compare loss functions and optimize the performance by using a combination loss function to manage class imbalance of skin versus vessel pixels. RESULTS We establish a baseline method for pixel-based telangiectasia detection in skin cancer lesion images. An analysis and comparison for human observer variability in annotation is also presented. CONCLUSION Our approach yields Jaccard score within the variation of human observers as it addresses a new aspect of the rapidly evolving field of deep learning: automatic identification of cancer-specific structures. Further application of DL techniques to detect dermoscopic structures and handle noisy labels is warranted.
Collapse
Affiliation(s)
- A Maurya
- Missouri University of Science &Technology, Rolla, Missouri
| | - R J Stanley
- Missouri University of Science &Technology, Rolla, Missouri
| | - N Lama
- Missouri University of Science &Technology, Rolla, Missouri
| | | | - D Saeed
- St. Louis University, St. Louis, Missouri
| | - S Swinfard
- Missouri University of Science &Technology, Rolla, Missouri
| | | | | |
Collapse
|
35
|
Multi-Class CNN for Classification of Multispectral and Autofluorescence Skin Lesion Clinical Images. J Clin Med 2022; 11:jcm11102833. [PMID: 35628958 PMCID: PMC9144655 DOI: 10.3390/jcm11102833] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2022] [Revised: 05/11/2022] [Accepted: 05/13/2022] [Indexed: 12/04/2022] Open
Abstract
In this work, we propose to use an artificial neural network to classify limited data of clinical multispectral and autofluorescence images of skin lesions. Although the amount of data is limited, the deep convolutional neural network classification of skin lesions using a multi-modal image set is studied and proposed for the first time. The unique dataset consists of spectral reflectance images acquired under 526 nm, 663 nm, 964 nm, and autofluorescence images under 405 nm LED excitation. The augmentation algorithm was applied for multi-modal clinical images of different skin lesion groups to expand the training datasets. It was concluded from saliency maps that the classification performed by the convolutional neural network is based on the distribution of the major skin chromophores and endogenous fluorophores. The resulting classification confusion matrices, as well as the performance of trained neural networks, have been investigated and discussed.
Collapse
|
36
|
Biomimetic Nanoscale Materials for Skin Cancer Therapy and Detection. J Skin Cancer 2022; 2022:2961996. [PMID: 35433050 PMCID: PMC9010180 DOI: 10.1155/2022/2961996] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2021] [Accepted: 03/29/2022] [Indexed: 02/06/2023] Open
Abstract
Skin cancer has developed as one of the most common types of cancer in the world, with a significant impact on public health impact and the economy. Nanotechnology methods for cancer treatment are appealing since they allow for the effective transport of medicines and other biologically active substances to specific tissues while minimizing harmful consequences. It is one of the most significant fields of research for treating skin cancer. Various nanomaterials have been employed in skin cancer therapy. The current review will summarize numerous methods of treating and diagnosing skin cancer in the earliest stages. There are numerous skin cancer indicators available for the prompt diagnosis of this type of disease. Traditional approaches to skin cancer diagnosis are explored, as are their shortcomings. Electrochemical and optical biosensors for skin cancer diagnosis and management were also discussed. Finally, various difficulties concerning the cost and ease of use of innovative methods should be addressed and overcome.
Collapse
|
37
|
Hsu BWY, Tseng VS. Hierarchy-aware contrastive learning with late fusion for skin lesion classification. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 216:106666. [PMID: 35124480 DOI: 10.1016/j.cmpb.2022.106666] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/07/2021] [Revised: 01/18/2022] [Accepted: 01/24/2022] [Indexed: 06/14/2023]
Abstract
BACKGROUND AND OBJECTIVE The incidence rate of skin cancers is increasing worldwide annually. Using machine learning and deep learning for skin lesion classification is one of the essential research topics. In this study, we formulate a major-type misclassification problem that previous studies did not consider in the multi-class skin lesion classification. Moreover, addressing the major-type misclassification problem is significant for real-world computer-aided diagnosis. METHODS This study presents a novel method, namely Hierarchy-Aware Contrastive Learning with Late Fusion (HAC-LF), to improve the overall performance of multi-class skin classification. In HAC-LF, we design a new loss function, Hierarchy-Aware Contrastive Loss (HAC Loss), to reduce the impact of the major-type misclassification problem. The late fusion method is applied to balance the major-type and multi-class classification performance. RESULTS We conduct a series of experiments with the ISIC 2019 Challenges dataset, which consists of three skin lesion datasets, to verify the performance of our methods. The results show that our proposed method surpasses the representative deep learning methods for skin lesion classification in all evaluation metrics used in this study. HAC-LF achieves 0.871, 0.842, 0.889 for accuracy, sensitivity, and specificity in the major-type classification, respectively. With the imbalanced class distribution, HAC-LF outperforms the baseline model regarding the sensitivity of minority classes. CONCLUSIONS This research formulates a major-type misclassification problem. We propose HAC-LF to deal with it and boost the multi-class skin lesion classification performance. According to the results, the advantage of HAC-LF is that the proposed HAC Loss can beneficially reduce the impact of the major-type misclassification by decreasing the major-type error rate. Besides the medical field HAC-LF is promising to be applied to other domains possessing the data with the hierarchical structure.
Collapse
Affiliation(s)
- Benny Wei-Yun Hsu
- Institute of Computer Science and Engineering, National Yang Ming Chiao Tung University, Hsinchu 30010, Taiwan
| | - Vincent S Tseng
- Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu 30010, Taiwan.
| |
Collapse
|
38
|
Dual attention based network for skin lesion classification with auxiliary learning. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103549] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
|
39
|
Li G, Jimenez G. Optimal diagnosis of the skin cancer using a hybrid deep neural network and grasshopper optimization algorithm. Open Med (Wars) 2022; 17:508-517. [PMID: 35350832 PMCID: PMC8919846 DOI: 10.1515/med-2022-0439] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2021] [Revised: 01/25/2022] [Accepted: 01/26/2022] [Indexed: 11/15/2022] Open
Abstract
When skin cells divide abnormally, it can cause a tumor or abnormal lymph fluid or blood. The masses appear benign and malignant, with the benign being limited to one area and not spreading, but some can spread throughout the body through the body’s lymphatic system. Skin cancer is easier to diagnose than other cancers because its symptoms can be seen with the naked eye. This makes us to provide an artificial intelligence-based methodology to diagnose this cancer with higher accuracy. This article proposes a new non-destructive testing method based on the AlexNet and Extreme Learning Machine network to provide better results of the diagnosis. The method is then optimized based on a new improved version of the Grasshopper optimization algorithm (GOA). Simulation of the proposed method is then compared with some different state-of-the-art methods and the results showed that the proposed method with 98% accuracy and 93% sensitivity has the highest efficiency.
Collapse
Affiliation(s)
- Gengluo Li
- School of Information Engineering, NanChang University , NanChang , Jiangxi 330031 , China
| | | |
Collapse
|
40
|
Melanoma segmentation using deep learning with test-time augmentations and conditional random fields. Sci Rep 2022; 12:3948. [PMID: 35273282 PMCID: PMC8913825 DOI: 10.1038/s41598-022-07885-y] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2021] [Accepted: 01/31/2022] [Indexed: 11/08/2022] Open
Abstract
In a computer-aided diagnostic (CAD) system for skin lesion segmentation, variations in shape and size of the skin lesion makes the segmentation task more challenging. Lesion segmentation is an initial step in CAD schemes as it leads to low error rates in quantification of the structure, boundary, and scale of the skin lesion. Subjective clinical assessment of the skin lesion segmentation results provided by current state-of-the-art deep learning segmentation techniques does not offer the required results as per the inter-observer agreement of expert dermatologists. This study proposes a novel deep learning-based, fully automated approach to skin lesion segmentation, including sophisticated pre and postprocessing approaches. We use three deep learning models, including UNet, deep residual U-Net (ResUNet), and improved ResUNet (ResUNet++). The preprocessing phase combines morphological filters with an inpainting algorithm to eliminate unnecessary hair structures from the dermoscopic images. Finally, we used test time augmentation (TTA) and conditional random field (CRF) in the postprocessing stage to improve segmentation accuracy. The proposed method was trained and evaluated on ISIC-2016 and ISIC-2017 skin lesion datasets. It achieved an average Jaccard Index of 85.96% and 80.05% for ISIC-2016 and ISIC-2017 datasets, when trained individually. When trained on combined dataset (ISIC-2016 and ISIC-2017), the proposed method achieved an average Jaccard Index of 80.73% and 90.02% on ISIC-2017 and ISIC-2016 testing datasets. The proposed methodological framework can be used to design a fully automated computer-aided skin lesion diagnostic system due to its high scalability and robustness.
Collapse
|
41
|
Razzak I, Naz S. Unit-Vise: Deep Shallow Unit-Vise Residual Neural Networks With Transition Layer For Expert Level Skin Cancer Classification. IEEE/ACM TRANSACTIONS ON COMPUTATIONAL BIOLOGY AND BIOINFORMATICS 2022; 19:1225-1234. [PMID: 33211666 DOI: 10.1109/tcbb.2020.3039358] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Many modern neural network architectures with over parameterized regime have been used for identification of skin cancer. Recent work showed that network, where the hidden units are polynomially smaller in size, showed better performance than overparameterized models. Hence, in this paper, we present multistage unit-vise deep dense residual network with transition and additional supervision blocks that enforces the shorter connections resulting in better feature representation. Unlike ResNet, We divided the network into several stages, and each stage consists of several dense connected residual units that support residual learning with dense connectivity and limited the skip connectivity. Thus, each stage can consider the features from its earlier layers locally as well as less complicated in comparison to its counter network. Evaluation results on ISIC-2018 challenge consisting of 10,015 training images show considerable improvement over other approaches achieving 98.05 percent accuracy and improving on the best results achieved in comparison to state of the art methods. The code of Unit-vise network is publicly available.1.
Collapse
|
42
|
AlZubi AA, Tiwari S, Walia K, Alanazi JM, AlZobi FI, Verma R. An Efficient Stacked Deep Transfer Learning Model for Automated Diagnosis of Lyme Disease. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2022; 2022:2933015. [PMID: 35265109 PMCID: PMC8901315 DOI: 10.1155/2022/2933015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/02/2022] [Revised: 01/26/2022] [Accepted: 02/07/2022] [Indexed: 12/05/2022]
Abstract
Lyme disease is one of the most common vector-borne infections. It typically causes cardiac illnesses, neurologic illnesses, musculoskeletal disorders, and dermatologic conditions. However, most of the time, it is poorly diagnosed due to many similarities with other diseases such as drug rash. Given the potentially serious consequences of unnecessary antimicrobial treatments, it is essential to understand frequent and uncommon diagnoses that explain symptoms in this population. Recently, deep learning models have been used for the diagnosis of various rash-related diseases. However, these models suffer from overfitting and color variation problems. To overcome these problems, an efficient stacked deep transfer learning model is proposed that can efficiently distinguish between patients infected with Lyme (+) or infected with other infections. 2nd order edge-based color constancy is used as a preprocessing approach to reduce the impact of multisource light from images acquired under different setups. The AlexNet pretrained learning model is used for building the Lyme disease diagnosis model. To prevent overfitting, data augmentation techniques are also used to augment the dataset. In addition, 5-fold cross-validation is also used. Comparative analysis indicates that the proposed model outperforms the existing models in terms of accuracy, f-measure, sensitivity, specificity, and area under the curve.
Collapse
Affiliation(s)
- Ahmad Ali AlZubi
- Computer Science Department, Community College, King Saud University, Riyadh, Saudi Arabia
| | - Shailendra Tiwari
- Department of Computer Science and Engineering, Thapar Institute of Engineering & Technology, Patiala, India
| | - Kuldeep Walia
- Sri Guru Granth Sahib World University, Fatehgarh Sahib, Punjab, India
| | - Jazem Mutared Alanazi
- Computer Science Department, Community College, King Saud University, Riyadh, Saudi Arabia
| | - Firas Ibrahim AlZobi
- Department of Information Systems and Networks, Faculty of Information Technology, The World Islamic Sciences & Education University, Amman, Jordan
| | - Rohit Verma
- School of Computing, National College of Ireland, Dublin, Ireland
| |
Collapse
|
43
|
Xia T, Kumar A, Fulham M, Feng D, Wang Y, Kim EY, Jung Y, Kim J. Fused feature signatures to probe tumour radiogenomics relationships. Sci Rep 2022; 12:2173. [PMID: 35140267 PMCID: PMC8828715 DOI: 10.1038/s41598-022-06085-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2021] [Accepted: 01/14/2022] [Indexed: 11/09/2022] Open
Abstract
Radiogenomics relationships (RRs) aims to identify statistically significant correlations between medical image features and molecular characteristics from analysing tissue samples. Previous radiogenomics studies mainly relied on a single category of image feature extraction techniques (ETs); these are (i) handcrafted ETs that encompass visual imaging characteristics, curated from knowledge of human experts and, (ii) deep ETs that quantify abstract-level imaging characteristics from large data. Prior studies therefore failed to leverage the complementary information that are accessible from fusing the ETs. In this study, we propose a fused feature signature (FFSig): a selection of image features from handcrafted and deep ETs (e.g., transfer learning and fine-tuning of deep learning models). We evaluated the FFSig's ability to better represent RRs compared to individual ET approaches with two public datasets: the first dataset was used to build the FFSig using 89 patients with non-small cell lung cancer (NSCLC) comprising of gene expression data and CT images of the thorax and the upper abdomen for each patient; the second NSCLC dataset comprising of 117 patients with CT images and RNA-Seq data and was used as the validation set. Our results show that our FFSig encoded complementary imaging characteristics of tumours and identified more RRs with a broader range of genes that are related to important biological functions such as tumourigenesis. We suggest that the FFSig has the potential to identify important RRs that may assist cancer diagnosis and treatment in the future.
Collapse
Affiliation(s)
- Tian Xia
- School of Computer Science, Faculty of Engineering, The University of Sydney, Sydney, NSW, 2006, Australia.
| | - Ashnil Kumar
- School of Biomedical Engineering, Faculty of Engineering, The University of Sydney, Sydney, NSW, 2006, Australia
| | - Michael Fulham
- Department of Molecular Imaging, Royal Prince Alfred Hospital, Camperdown, NSW, 2050, Australia
| | - Dagan Feng
- School of Computer Science, Faculty of Engineering, The University of Sydney, Sydney, NSW, 2006, Australia
| | - Yue Wang
- Department of Electrical and Computer Engineering, Virginia Polytechnic Institute and State University, Arlington, VA, 22203, USA
| | - Eun Young Kim
- Department of Radiology, Gil Medical Center, Gachon University College of Medicine, Incheon, Republic of Korea
| | - Younhyun Jung
- School of Computing, Gachon University, Seongnam, Republic of Korea
| | - Jinman Kim
- School of Computer Science, Faculty of Engineering, The University of Sydney, Sydney, NSW, 2006, Australia
| |
Collapse
|
44
|
Oukil S, Kasmi R, Mokrani K, García-Zapirain B. Automatic segmentation and melanoma detection based on color and texture features in dermoscopic images. Skin Res Technol 2021; 28:203-211. [PMID: 34779062 PMCID: PMC9907597 DOI: 10.1111/srt.13111] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2021] [Accepted: 09/25/2021] [Indexed: 02/06/2023]
Abstract
PURPOSE Melanoma is known as the most aggressive form of skin cancer and one of the fastest growing malignant tumors worldwide. Several computer-aided diagnosis systems for melanoma have been proposed, still, the algorithms encounter difficulties in the early stage of lesions. This paper aims to discriminate melanoma and benign skin lesion in dermoscopic images. METHODS The proposed algorithm is based on the color and texture of skin lesions by introducing a novel feature extraction technique. The algorithm uses an automatic segmentation based on k-means generating a fairly accurate mask for each lesion. The feature extraction consists of the existing and novel color and texture attributes measuring how color and texture vary inside the lesion. To find the optimal results, all the attributes are extracted from lesions in five different color spaces (RGB, HSV, Lab, XYZ, and YCbCr) and used as the inputs for three classifiers (K nearest neighbors, support vector machine , and artificial neural network). RESULTS The PH2 set is used to assess the performance of the proposed algorithm. The results of our algorithm are compared to the results of published articles that used the same dataset, and it shows that the proposed method outperforms the state of the art by attaining a sensitivity of 99.25%, specificity of 99.58%, and accuracy of 99.51%. CONCLUSION The final results show that the colors combined with texture are powerful and relevant attributes for melanoma detection and show improvement over the state of the art.
Collapse
Affiliation(s)
- S Oukil
- LTII Laboratory University of Bejaia-Algeria, Faculty of Technology, University of Bejaia, Bejaia, Algeria
| | - R Kasmi
- LTII Laboratory University of Bejaia-Algeria, Faculty of Technology, University of Bejaia, Bejaia, Algeria.,Electrical Engineering Department, University of Bouira, Bouira, Algeria
| | - K Mokrani
- LTII Laboratory University of Bejaia-Algeria, Faculty of Technology, University of Bejaia, Bejaia, Algeria
| | | |
Collapse
|
45
|
Intelligent fusion-assisted skin lesion localization and classification for smart healthcare. Neural Comput Appl 2021. [DOI: 10.1007/s00521-021-06490-w] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2022]
|
46
|
Pereira PMM, Thomaz LA, Tavora LMN, Assuncao PAA, Fonseca-Pinto RM, Paiva RP, Faria SMMD. Melanoma classification using light-Fields with morlet scattering transform and CNN: Surface depth as a valuable tool to increase detection rate. Med Image Anal 2021; 75:102254. [PMID: 34649195 DOI: 10.1016/j.media.2021.102254] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2021] [Revised: 07/27/2021] [Accepted: 09/22/2021] [Indexed: 11/15/2022]
Abstract
Medical image classification through learning-based approaches has been increasingly used, namely in the discrimination of melanoma. However, for skin lesion classification in general, such methods commonly rely on dermoscopic or other 2D-macro RGB images. This work proposes to exploit beyond conventional 2D image characteristics, by considering a third dimension (depth) that characterises the skin surface rugosity, which can be obtained from light-field images, such as those available in the SKINL2 dataset. To achieve this goal, a processing pipeline was deployed using a morlet scattering transform and a CNN model, allowing to perform a comparison between using 2D information, only 3D information, or both. Results show that discrimination between Melanoma and Nevus reaches an accuracy of 84.00, 74.00 or 94.00% when using only 2D, only 3D, or both, respectively. An increase of 14.29pp in sensitivity and 8.33pp in specificity is achieved when expanding beyond conventional 2D information by also using depth. When discriminating between Melanoma and all other types of lesions (a further imbalanced setting), an increase of 28.57pp in sensitivity and decrease of 1.19pp in specificity is achieved for the same test conditions. Overall the results of this work demonstrate significant improvements over conventional approaches.
Collapse
Affiliation(s)
- Pedro M M Pereira
- Instituto de Telecomunicações, Morro do Lena - Alto do Vieiro, Leiria 2411-901, Portugal; University of Coimbra, Centre for Informatics and Systems of the University of Coimbra, Department of Informatics Engineering, Pinhal de Marrocos, Coimbra 3030-290, Portugal.
| | - Lucas A Thomaz
- Instituto de Telecomunicações, Morro do Lena - Alto do Vieiro, Leiria 2411-901, Portugal; ESTG, Polytechnic of Leiria, Morro do Lena - Alto do Vieiro, Leiria 2411-901, Portugal
| | - Luis M N Tavora
- ESTG, Polytechnic of Leiria, Morro do Lena - Alto do Vieiro, Leiria 2411-901, Portugal
| | - Pedro A A Assuncao
- Instituto de Telecomunicações, Morro do Lena - Alto do Vieiro, Leiria 2411-901, Portugal; ESTG, Polytechnic of Leiria, Morro do Lena - Alto do Vieiro, Leiria 2411-901, Portugal
| | - Rui M Fonseca-Pinto
- Instituto de Telecomunicações, Morro do Lena - Alto do Vieiro, Leiria 2411-901, Portugal; ESTG, Polytechnic of Leiria, Morro do Lena - Alto do Vieiro, Leiria 2411-901, Portugal
| | - Rui Pedro Paiva
- University of Coimbra, Centre for Informatics and Systems of the University of Coimbra, Department of Informatics Engineering, Pinhal de Marrocos, Coimbra 3030-290, Portugal
| | - Sergio M M de Faria
- Instituto de Telecomunicações, Morro do Lena - Alto do Vieiro, Leiria 2411-901, Portugal; ESTG, Polytechnic of Leiria, Morro do Lena - Alto do Vieiro, Leiria 2411-901, Portugal
| |
Collapse
|
47
|
Wang Y, Cai J, Louie DC, Wang ZJ, Lee TK. Incorporating clinical knowledge with constrained classifier chain into a multimodal deep network for melanoma detection. Comput Biol Med 2021; 137:104812. [PMID: 34507158 DOI: 10.1016/j.compbiomed.2021.104812] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2021] [Revised: 08/25/2021] [Accepted: 08/25/2021] [Indexed: 10/20/2022]
Abstract
In recent years, vast developments in Computer-Aided Diagnosis (CAD) for skin diseases have generated much interest from clinicians and other eventual end-users of this technology. Introducing clinical domain knowledge to these machine learning strategies can help dispel the black box nature of these tools, strengthening clinician trust. Clinical domain knowledge also provides new information channels which can improve CAD diagnostic performance. In this paper, we propose a novel framework for malignant melanoma (MM) detection by fusing clinical images and dermoscopic images. The proposed method combines a multi-labeled deep feature extractor and clinically constrained classifier chain (CC). This allows the 7-point checklist, a clinician diagnostic algorithm, to be included in the decision level while maintaining the clinical importance of the major and minor criteria in the checklist. Our proposed framework achieved an average accuracy of 81.3% for detecting all criteria and melanoma when testing on a publicly available 7-point checklist dataset. This is the highest reported results, outperforming state-of-the-art methods in the literature by 6.4% or more. Analyses also show that the proposed system surpasses the single modality system of using either clinical images or dermoscopic images alone and the systems without adopting the approach of multi-label and clinically constrained classifier chain. Our carefully designed system demonstrates a substantial improvement over melanoma detection. By keeping the familiar major and minor criteria of the 7-point checklist and their corresponding weights, the proposed system may be more accepted by physicians as a human-interpretable CAD tool for automated melanoma detection.
Collapse
Affiliation(s)
- Yuheng Wang
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada; Department of Dermatology and Skin Science, University of British Columbia, Vancouver, BC, Canada; Photomedicine Institute, Vancouver Coast Health Research Institute, Vancouver, BC, Canada; Departments of Cancer Control Research and Integrative Oncology, BC Cancer, Vancouver, BC, Canada
| | - Jiayue Cai
- School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, China; Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC, Canada.
| | - Daniel C Louie
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada; Department of Dermatology and Skin Science, University of British Columbia, Vancouver, BC, Canada; Photomedicine Institute, Vancouver Coast Health Research Institute, Vancouver, BC, Canada; Departments of Cancer Control Research and Integrative Oncology, BC Cancer, Vancouver, BC, Canada
| | - Z Jane Wang
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada; Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC, Canada
| | - Tim K Lee
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada; Department of Dermatology and Skin Science, University of British Columbia, Vancouver, BC, Canada; Photomedicine Institute, Vancouver Coast Health Research Institute, Vancouver, BC, Canada; Departments of Cancer Control Research and Integrative Oncology, BC Cancer, Vancouver, BC, Canada
| |
Collapse
|
48
|
Machine Learning and Deep Learning Methods for Skin Lesion Classification and Diagnosis: A Systematic Review. Diagnostics (Basel) 2021; 11:diagnostics11081390. [PMID: 34441324 PMCID: PMC8391467 DOI: 10.3390/diagnostics11081390] [Citation(s) in RCA: 62] [Impact Index Per Article: 20.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2021] [Revised: 07/25/2021] [Accepted: 07/27/2021] [Indexed: 12/04/2022] Open
Abstract
Computer-aided systems for skin lesion diagnosis is a growing area of research. Recently, researchers have shown an increasing interest in developing computer-aided diagnosis systems. This paper aims to review, synthesize and evaluate the quality of evidence for the diagnostic accuracy of computer-aided systems. This study discusses the papers published in the last five years in ScienceDirect, IEEE, and SpringerLink databases. It includes 53 articles using traditional machine learning methods and 49 articles using deep learning methods. The studies are compared based on their contributions, the methods used and the achieved results. The work identified the main challenges of evaluating skin lesion segmentation and classification methods such as small datasets, ad hoc image selection and racial bias.
Collapse
|
49
|
Ursuleanu TF, Luca AR, Gheorghe L, Grigorovici R, Iancu S, Hlusneac M, Preda C, Grigorovici A. Deep Learning Application for Analyzing of Constituents and Their Correlations in the Interpretations of Medical Images. Diagnostics (Basel) 2021; 11:1373. [PMID: 34441307 PMCID: PMC8393354 DOI: 10.3390/diagnostics11081373] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2021] [Revised: 07/25/2021] [Accepted: 07/27/2021] [Indexed: 12/13/2022] Open
Abstract
The need for time and attention, given by the doctor to the patient, due to the increased volume of medical data to be interpreted and filtered for diagnostic and therapeutic purposes has encouraged the development of the option to support, constructively and effectively, deep learning models. Deep learning (DL) has experienced an exponential development in recent years, with a major impact on interpretations of the medical image. This has influenced the development, diversification and increase of the quality of scientific data, the development of knowledge construction methods and the improvement of DL models used in medical applications. All research papers focus on description, highlighting, classification of one of the constituent elements of deep learning models (DL), used in the interpretation of medical images and do not provide a unified picture of the importance and impact of each constituent in the performance of DL models. The novelty in our paper consists primarily in the unitary approach, of the constituent elements of DL models, namely, data, tools used by DL architectures or specifically constructed DL architecture combinations and highlighting their "key" features, for completion of tasks in current applications in the interpretation of medical images. The use of "key" characteristics specific to each constituent of DL models and the correct determination of their correlations, may be the subject of future research, with the aim of increasing the performance of DL models in the interpretation of medical images.
Collapse
Affiliation(s)
- Tudor Florin Ursuleanu
- Faculty of General Medicine, “Grigore T. Popa” University of Medicine and Pharmacy, 700115 Iasi, Romania; (T.F.U.); (R.G.); (S.I.); (M.H.); (C.P.); (A.G.)
- Department of Surgery VI, “Sf. Spiridon” Hospital, 700111 Iasi, Romania
- Department of Surgery I, Regional Institute of Oncology, 700483 Iasi, Romania
| | - Andreea Roxana Luca
- Faculty of General Medicine, “Grigore T. Popa” University of Medicine and Pharmacy, 700115 Iasi, Romania; (T.F.U.); (R.G.); (S.I.); (M.H.); (C.P.); (A.G.)
- Department Obstetrics and Gynecology, Integrated Ambulatory of Hospital “Sf. Spiridon”, 700106 Iasi, Romania
| | - Liliana Gheorghe
- Faculty of General Medicine, “Grigore T. Popa” University of Medicine and Pharmacy, 700115 Iasi, Romania; (T.F.U.); (R.G.); (S.I.); (M.H.); (C.P.); (A.G.)
- Department of Radiology, “Sf. Spiridon” Hospital, 700111 Iasi, Romania
| | - Roxana Grigorovici
- Faculty of General Medicine, “Grigore T. Popa” University of Medicine and Pharmacy, 700115 Iasi, Romania; (T.F.U.); (R.G.); (S.I.); (M.H.); (C.P.); (A.G.)
| | - Stefan Iancu
- Faculty of General Medicine, “Grigore T. Popa” University of Medicine and Pharmacy, 700115 Iasi, Romania; (T.F.U.); (R.G.); (S.I.); (M.H.); (C.P.); (A.G.)
| | - Maria Hlusneac
- Faculty of General Medicine, “Grigore T. Popa” University of Medicine and Pharmacy, 700115 Iasi, Romania; (T.F.U.); (R.G.); (S.I.); (M.H.); (C.P.); (A.G.)
| | - Cristina Preda
- Faculty of General Medicine, “Grigore T. Popa” University of Medicine and Pharmacy, 700115 Iasi, Romania; (T.F.U.); (R.G.); (S.I.); (M.H.); (C.P.); (A.G.)
- Department of Endocrinology, “Sf. Spiridon” Hospital, 700111 Iasi, Romania
| | - Alexandru Grigorovici
- Faculty of General Medicine, “Grigore T. Popa” University of Medicine and Pharmacy, 700115 Iasi, Romania; (T.F.U.); (R.G.); (S.I.); (M.H.); (C.P.); (A.G.)
- Department of Surgery VI, “Sf. Spiridon” Hospital, 700111 Iasi, Romania
| |
Collapse
|
50
|
Pedrosa M, Zuquete A, Costa C. A Pseudonymisation Protocol With Implicit and Explicit Consent Routes for Health Records in Federated Ledgers. IEEE J Biomed Health Inform 2021; 25:2172-2183. [PMID: 33006933 DOI: 10.1109/jbhi.2020.3028454] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Healthcare data for primary use (diagnosis) may be encrypted for confidentiality purposes; however, secondary uses such as feeding machine learning algorithms requires open access. Full anonymity has no traceable identifiers to report diagnosis results. Moreover, implicit and explicit consent routes are of practical importance under recent data protection regulations (GDPR), translating directly into break-the-glass requirements. Pseudonymisation is an acceptable compromise when dealing with such orthogonal requirements and is an advisable measure to protect data. Our work presents a pseudonymisation protocol that is compliant with implicit and explicit consent routes. The protocol is constructed on a (t,n)-threshold secret sharing scheme and public key cryptography. The pseudonym is safely derived from a fragment of public information without requiring any data-subject's secret. The method is proven secure under reasonable cryptographic assumptions and scalable from the experimental results.
Collapse
|