1
|
Hatamikia S, George G, Schwarzhans F, Mahbod A, Woitek R. Breast MRI radiomics and machine learning-based predictions of response to neoadjuvant chemotherapy - How are they affected by variations in tumor delineation? Comput Struct Biotechnol J 2024; 23:52-63. [PMID: 38125296 PMCID: PMC10730996 DOI: 10.1016/j.csbj.2023.11.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2023] [Revised: 11/08/2023] [Accepted: 11/08/2023] [Indexed: 12/23/2023] Open
Abstract
Manual delineation of volumes of interest (VOIs) by experts is considered the gold-standard method in radiomics analysis. However, it suffers from inter- and intra-operator variability. A quantitative assessment of the impact of variations in these delineations on the performance of the radiomics predictors is required to develop robust radiomics based prediction models. In this study, we developed radiomics models for the prediction of pathological complete response to neoadjuvant chemotherapy in patients with two different breast cancer subtypes based on contrast-enhanced magnetic resonance imaging acquired prior to treatment (baseline MRI scans). Different mathematical operations such as erosion, smoothing, dilation, randomization, and ellipse fitting were applied to the original VOIs delineated by experts to simulate variations of segmentation masks. The effects of such VOI modifications on various steps of the radiomics workflow, including feature extraction, feature selection, and prediction performance, were evaluated. Using manual tumor VOIs and radiomics features extracted from baseline MRI scans, an AUC of up to 0.96 and 0.89 was achieved for human epidermal growth receptor 2 positive and triple-negative breast cancer, respectively. For smoothing and erosion, VOIs yielded the highest number of robust features and the best prediction performance, while ellipse fitting and dilation lead to the lowest robustness and prediction performance for both breast cancer subtypes. At most 28% of the selected features were similar to manual VOIs when different VOI delineation data were used. Differences in VOI delineation affect different steps of radiomics analysis, and their quantification is therefore important for development of standardized radiomics research.
Collapse
Affiliation(s)
- Sepideh Hatamikia
- Danube Private University, Krems, Rathausplatz 1, Krems-Stein, AT-3500, Austria
- Austrian Center for Medical Innovation and Technology (ACMIT), Viktor Kaplan-Straße 2/1, Wiener Neustadt 2700, Austria
| | - Geevarghese George
- Danube Private University, Krems, Rathausplatz 1, Krems-Stein, AT-3500, Austria
| | - Florian Schwarzhans
- Danube Private University, Krems, Rathausplatz 1, Krems-Stein, AT-3500, Austria
| | - Amirreza Mahbod
- Danube Private University, Krems, Rathausplatz 1, Krems-Stein, AT-3500, Austria
| | - Ramona Woitek
- Danube Private University, Krems, Rathausplatz 1, Krems-Stein, AT-3500, Austria
| |
Collapse
|
2
|
Primiero CA, Rezze GG, Caffery LJ, Carrera C, Podlipnik S, Espinosa N, Puig S, Janda M, Soyer HP, Malvehy J. A Narrative Review: Opportunities and Challenges in Artificial Intelligence Skin Image Analyses Using Total Body Photography. J Invest Dermatol 2024; 144:1200-1207. [PMID: 38231164 DOI: 10.1016/j.jid.2023.11.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Revised: 09/19/2023] [Accepted: 11/09/2023] [Indexed: 01/18/2024]
Abstract
Artificial intelligence (AI) algorithms for skin lesion classification have reported accuracy at par with and even outperformance of expert dermatologists in experimental settings. However, the majority of algorithms do not represent real-world clinical approach where skin phenotype and clinical background information are considered. We review the current state of AI for skin lesion classification and present opportunities and challenges when applied to total body photography (TBP). AI in TBP analysis presents opportunities for intrapatient assessment of skin phenotype and holistic risk assessment by incorporating patient-level metadata, although challenges exist for protecting patient privacy in algorithm development and improving explainable AI methods.
Collapse
Affiliation(s)
- Clare A Primiero
- Dermatology Department, Hospital Clinic and Fundació Clínic per la Recerca Biomèdica - Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain; Dermatology Research Centre, Frazer Institute, The University of Queensland, Brisbane, Australia
| | - Gisele Gargantini Rezze
- Dermatology Department, Hospital Clinic and Fundació Clínic per la Recerca Biomèdica - Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain
| | - Liam J Caffery
- Dermatology Research Centre, Frazer Institute, The University of Queensland, Brisbane, Australia; Centre of Health Services Research, Faculty of Medicine, The University of Queensland, Brisbane, Australia; Centre for Online Health, The University of Queensland, Brisbane, Australia
| | - Cristina Carrera
- Dermatology Department, Hospital Clinic and Fundació Clínic per la Recerca Biomèdica - Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain; Medicine Department, University of Barcelona, Barcelona, Spain; CIBER de Enfermedades raras, Instituto de Salud Carlos III, Barcelona, Spain
| | - Sebastian Podlipnik
- Dermatology Department, Hospital Clinic and Fundació Clínic per la Recerca Biomèdica - Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain; CIBER de Enfermedades raras, Instituto de Salud Carlos III, Barcelona, Spain
| | - Natalia Espinosa
- Dermatology Department, Hospital Clinic and Fundació Clínic per la Recerca Biomèdica - Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain
| | - Susana Puig
- Dermatology Department, Hospital Clinic and Fundació Clínic per la Recerca Biomèdica - Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain; Medicine Department, University of Barcelona, Barcelona, Spain; CIBER de Enfermedades raras, Instituto de Salud Carlos III, Barcelona, Spain
| | - Monika Janda
- Centre of Health Services Research, Faculty of Medicine, The University of Queensland, Brisbane, Australia
| | - H Peter Soyer
- Dermatology Research Centre, Frazer Institute, The University of Queensland, Brisbane, Australia; Dermatology Department, Princess Alexandra Hospital, Brisbane, Australia
| | - Josep Malvehy
- Dermatology Department, Hospital Clinic and Fundació Clínic per la Recerca Biomèdica - Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain; Medicine Department, University of Barcelona, Barcelona, Spain; CIBER de Enfermedades raras, Instituto de Salud Carlos III, Barcelona, Spain.
| |
Collapse
|
3
|
Xu Z, Guo X, Wang J. Enhancing skin lesion segmentation with a fusion of convolutional neural networks and transformer models. Heliyon 2024; 10:e31395. [PMID: 38807881 PMCID: PMC11130697 DOI: 10.1016/j.heliyon.2024.e31395] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2023] [Revised: 05/11/2024] [Accepted: 05/15/2024] [Indexed: 05/30/2024] Open
Abstract
Accurate segmentation is crucial in diagnosing and analyzing skin lesions. However, automatic segmentation of skin lesions is extremely challenging because of their variable sizes, uneven color distributions, irregular shapes, hair occlusions, and blurred boundaries. Owing to the limited range of convolutional networks receptive fields, shallow convolution cannot extract the global features of images and thus has limited segmentation performance. Because medical image datasets are small in scale, the use of excessively deep networks could cause overfitting and increase computational complexity. Although transformer networks can focus on extracting global information, they cannot extract sufficient local information and accurately segment detailed lesion features. In this study, we designed a dual-branch encoder that combines a convolution neural network (CNN) and a transformer. The CNN branch of the encoder comprises four layers, which learn the local features of images through layer-wise downsampling. The transformer branch also comprises four layers, enabling the learning of global image information through attention mechanisms. The feature fusion module in the network integrates local features and global information, emphasizes important channel features through the channel attention mechanism, and filters irrelevant feature expressions. The information exchange between the decoder and encoder is finally achieved through skip connections to supplement the information lost during the sampling process, thereby enhancing segmentation accuracy. The data used in this paper are from four public datasets, including images of melanoma, basal cell tumor, fibroma, and benign nevus. Because of the limited size of the image data, we enhanced them using methods such as random horizontal flipping, random vertical flipping, random brightness enhancement, random contrast enhancement, and rotation. The segmentation accuracy is evaluated through intersection over union and duration, integrity, commitment, and effort indicators, reaching 87.7 % and 93.21 %, 82.05 % and 89.19 %, 86.81 % and 92.72 %, and 92.79 % and 96.21 %, respectively, on the ISIC 2016, ISIC 2017, ISIC 2018, and PH2 datasets, respectively (code: https://github.com/hyjane/CCT-Net).
Collapse
Affiliation(s)
- Zhijian Xu
- School of Electronic Information Engineering, China West Normal University, No. 1 Shida Road, Nanchong, Sichuan, 637009, China
| | - Xingyue Guo
- School of Computer Science, China West Normal University, No. 1 Shida Road, Nanchong, Sichuan, 637009, China
| | - Juan Wang
- School of Computer Science, China West Normal University, No. 1 Shida Road, Nanchong, Sichuan, 637009, China
| |
Collapse
|
4
|
Ali R, Manikandan A, Lei R, Xu J. A novel SpaSA based hyper-parameter optimized FCEDN with adaptive CNN classification for skin cancer detection. Sci Rep 2024; 14:9336. [PMID: 38653997 DOI: 10.1038/s41598-024-57393-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2023] [Accepted: 03/18/2024] [Indexed: 04/25/2024] Open
Abstract
Skin cancer is the most prevalent kind of cancer in people. It is estimated that more than 1 million people get skin cancer every year in the world. The effectiveness of the disease's therapy is significantly impacted by early identification of this illness. Preprocessing is the initial detecting stage in enhancing the quality of skin images by removing undesired background noise and objects. This study aims is to compile preprocessing techniques for skin cancer imaging that are currently accessible. Researchers looking into automated skin cancer diagnosis might use this article as an excellent place to start. The fully convolutional encoder-decoder network and Sparrow search algorithm (FCEDN-SpaSA) are proposed in this study for the segmentation of dermoscopic images. The individual wolf method and the ensemble ghosting technique are integrated to generate a neighbour-based search strategy in SpaSA for stressing the correct balance between navigation and exploitation. The classification procedure is accomplished by using an adaptive CNN technique to discriminate between normal skin and malignant skin lesions suggestive of disease. Our method provides classification accuracies comparable to commonly used incremental learning techniques while using less energy, storage space, memory access, and training time (only network updates with new training samples, no network sharing). In a simulation, the segmentation performance of the proposed technique on the ISBI 2017, ISIC 2018, and PH2 datasets reached accuracies of 95.28%, 95.89%, 92.70%, and 98.78%, respectively, on the same dataset and assessed the classification performance. It is accurate 91.67% of the time. The efficiency of the suggested strategy is demonstrated through comparisons with cutting-edge methodologies.
Collapse
Affiliation(s)
- Rizwan Ali
- Department of Plastic Surgery, The First Affiliated Hospital, School of Medicine, Zhejiang University, No. 79 Qingchun Road, Hangzhou, 310003, China
| | - A Manikandan
- Department of ECE, SRM Institute of Science and Technology, Kattankulathur, Chennai, 603 203, India
| | - Rui Lei
- Department of Plastic Surgery, The First Affiliated Hospital, School of Medicine, Zhejiang University, No. 79 Qingchun Road, Hangzhou, 310003, China.
| | - Jinghong Xu
- Department of Plastic Surgery, The First Affiliated Hospital, School of Medicine, Zhejiang University, No. 79 Qingchun Road, Hangzhou, 310003, China.
| |
Collapse
|
5
|
Ding Y, Yi Z, Xiao J, Hu M, Guo Y, Liao Z, Wang Y. CTH-Net: A CNN and Transformer hybrid network for skin lesion segmentation. iScience 2024; 27:109442. [PMID: 38523786 PMCID: PMC10957498 DOI: 10.1016/j.isci.2024.109442] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2023] [Revised: 01/25/2024] [Accepted: 03/04/2024] [Indexed: 03/26/2024] Open
Abstract
Automatically and accurately segmenting skin lesions can be challenging, due to factors such as low contrast and fuzzy boundaries. This paper proposes a hybrid encoder-decoder model (CTH-Net) based on convolutional neural network (CNN) and Transformer, capitalizing on the advantages of these approaches. We propose three modules for skin lesion segmentation and seamlessly connect them with carefully designed model architecture. Better segmentation performance is achieved by introducing SoftPool in the CNN branch and sandglass block in the bottleneck layer. Extensive experiments were conducted on four publicly accessible skin lesion datasets, ISIC 2016, ISIC 2017, ISIC 2018, and PH2 to confirm the efficacy and benefits of the proposed strategy. Experimental results show that the proposed CTH-Net provides better skin lesion segmentation performance in both quantitative and qualitative testing when compared with state-of-the-art approaches. We believe the CTH-Net design is inspiring and can be extended to other applications/frameworks.
Collapse
Affiliation(s)
- Yuhan Ding
- School of Computer Science and Engineering, Central South University, Changsha 410083, China
| | - Zhenglin Yi
- Departments of Urology, Xiangya Hospital, Central South University, Changsha 410008, China
| | - Jiatong Xiao
- Departments of Urology, Xiangya Hospital, Central South University, Changsha 410008, China
| | - Minghui Hu
- Departments of Urology, Xiangya Hospital, Central South University, Changsha 410008, China
| | - Yu Guo
- Department of Burns and Plastic Surgery, Xiangya Hospital, Central South University, Changsha 410008, China
- National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Central South University, Changsha 410008, China
| | - Zhifang Liao
- School of Computer Science and Engineering, Central South University, Changsha 410083, China
| | - Yongjie Wang
- Department of Burns and Plastic Surgery, Xiangya Hospital, Central South University, Changsha 410008, China
- National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Central South University, Changsha 410008, China
| |
Collapse
|
6
|
Li D, Sun Y, Ren X, Zhou N, Li L, He G, Ma S, Wang Y. Dynamic evaluation of pathological changes in a mouse acne model by optical imaging technology. Exp Dermatol 2023; 32:1350-1360. [PMID: 37183301 DOI: 10.1111/exd.14826] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2023] [Revised: 03/27/2023] [Accepted: 04/19/2023] [Indexed: 05/16/2023]
Abstract
Acne vulgaris is a disorder of the pilosebaceous unit that is primarily caused by hyperseborrhoea, colonization with Propionibacterium acnes, hyperkeratosis and an inflammatory response. Existing pharmacodynamic assessment methods primarily focus on a single causative factor at a certain time point, making it difficult to assess multiple factors simultaneously in real time. Therefore, it is crucial to establish a dynamic and nondestructive method for the assessment of acne in vivo. This study utilized four-dimensional optical imaging techniques to assess the pathogenic factors and pathological progression of acne. LSCI was employed to measure blood flow; TPEF was used to observe inflammatory changes (NAD(P)H) in epidermal granular layer cells and structural changes in collagen fibres in the dermal layer. Additionally, the dermatoscope was used to investigate the micro-characterization of the lesions. We observed that the epidermis in the lesion area was thickened, hair follicles were keratinized, and there was obvious inflammation and blood flow aggregation by optical imaging technology. Based on these findings, the pathological progression of this acne model could be divided into the inflammation phase, accompanied by bacterial colonization, and the reparative phase. These results provide a new perspective for the assessment of acne and offer an experimental basis for the selection of precise drugs for clinical use.
Collapse
Affiliation(s)
- Dongying Li
- Experimental Research Center, China Academy of Chinese Medical Sciences, Beijing, China
| | - Yanan Sun
- Experimental Research Center, China Academy of Chinese Medical Sciences, Beijing, China
| | - Xing Ren
- Experimental Research Center, China Academy of Chinese Medical Sciences, Beijing, China
| | - Na Zhou
- Department of Immunology and Microbiology, School of Life Science, Beijing University of Chinese Medicine, Beijing, China
| | - Lu Li
- Experimental Research Center, China Academy of Chinese Medical Sciences, Beijing, China
| | - Gaiying He
- Experimental Research Center, China Academy of Chinese Medical Sciences, Beijing, China
| | - Shuhua Ma
- Experimental Research Center, China Academy of Chinese Medical Sciences, Beijing, China
| | - Yi Wang
- Experimental Research Center, China Academy of Chinese Medical Sciences, Beijing, China
| |
Collapse
|
7
|
Riaz Z, Khan B, Abdullah S, Khan S, Islam MS. Lung Tumor Image Segmentation from Computer Tomography Images Using MobileNetV2 and Transfer Learning. Bioengineering (Basel) 2023; 10:981. [PMID: 37627866 PMCID: PMC10451633 DOI: 10.3390/bioengineering10080981] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2023] [Revised: 08/14/2023] [Accepted: 08/17/2023] [Indexed: 08/27/2023] Open
Abstract
BACKGROUND Lung cancer is one of the most fatal cancers worldwide, and malignant tumors are characterized by the growth of abnormal cells in the tissues of lungs. Usually, symptoms of lung cancer do not appear until it is already at an advanced stage. The proper segmentation of cancerous lesions in CT images is the primary method of detection towards achieving a completely automated diagnostic system. METHOD In this work, we developed an improved hybrid neural network via the fusion of two architectures, MobileNetV2 and UNET, for the semantic segmentation of malignant lung tumors from CT images. The transfer learning technique was employed and the pre-trained MobileNetV2 was utilized as an encoder of a conventional UNET model for feature extraction. The proposed network is an efficient segmentation approach that performs lightweight filtering to reduce computation and pointwise convolution for building more features. Skip connections were established with the Relu activation function for improving model convergence to connect the encoder layers of MobileNetv2 to decoder layers in UNET that allow the concatenation of feature maps with different resolutions from the encoder to decoder. Furthermore, the model was trained and fine-tuned on the training dataset acquired from the Medical Segmentation Decathlon (MSD) 2018 Challenge. RESULTS The proposed network was tested and evaluated on 25% of the dataset obtained from the MSD, and it achieved a dice score of 0.8793, recall of 0.8602 and precision of 0.93. It is pertinent to mention that our technique outperforms the current available networks, which have several phases of training and testing.
Collapse
Affiliation(s)
- Zainab Riaz
- Hong Kong Center for Cerebro-Cardiovascular Health Engineering (COCHE), Hong Kong SAR, China; (Z.R.); (B.K.); (M.S.I.)
| | - Bangul Khan
- Hong Kong Center for Cerebro-Cardiovascular Health Engineering (COCHE), Hong Kong SAR, China; (Z.R.); (B.K.); (M.S.I.)
- Department of Biomedical Engineering, City University Hongkong, Hong Kong SAR, China
| | - Saad Abdullah
- Division of Intelligent Future Technologies, School of Innovation, Design and Engineering, Mälardalen University, P.O. Box 883, 721 23 Västerås, Sweden
| | - Samiullah Khan
- Center for Eye & Vision Research, 17W Science Park, Hong Kong SAR, China;
| | - Md Shohidul Islam
- Hong Kong Center for Cerebro-Cardiovascular Health Engineering (COCHE), Hong Kong SAR, China; (Z.R.); (B.K.); (M.S.I.)
| |
Collapse
|
8
|
Diaz Resendiz JL, Ponomaryov V, Reyes Reyes R, Sadovnychiy S. Explainable CAD System for Classification of Acute Lymphoblastic Leukemia Based on a Robust White Blood Cell Segmentation. Cancers (Basel) 2023; 15:3376. [PMID: 37444486 DOI: 10.3390/cancers15133376] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2023] [Revised: 06/25/2023] [Accepted: 06/26/2023] [Indexed: 07/15/2023] Open
Abstract
Leukemia is a significant health challenge, with high incidence and mortality rates. Computer-aided diagnosis (CAD) has emerged as a promising approach. However, deep-learning methods suffer from the "black box problem", leading to unreliable diagnoses. This research proposes an Explainable AI (XAI) Leukemia classification method that addresses this issue by incorporating a robust White Blood Cell (WBC) nuclei segmentation as a hard attention mechanism. The segmentation of WBC is achieved by combining image processing and U-Net techniques, resulting in improved overall performance. The segmented images are fed into modified ResNet-50 models, where the MLP classifier, activation functions, and training scheme have been tested for leukemia subtype classification. Additionally, we add visual explainability and feature space analysis techniques to offer an interpretable classification. Our segmentation algorithm achieves an Intersection over Union (IoU) of 0.91, in six databases. Furthermore, the deep-learning classifier achieves an accuracy of 99.9% on testing. The Grad CAM methods and clustering space analysis confirm improved network focus when classifying segmented images compared to non-segmented images. Overall, the proposed visual explainable CAD system has the potential to assist physicians in diagnosing leukemia and improving patient outcomes.
Collapse
Affiliation(s)
- Jose Luis Diaz Resendiz
- Instituto Politecnico Nacional, Escuela Superior de Ingenieria Mecanica y Electrica-Culhuacan, Av. Sta. Ana 1000, Mexico City 04440, Mexico
| | - Volodymyr Ponomaryov
- Instituto Politecnico Nacional, Escuela Superior de Ingenieria Mecanica y Electrica-Culhuacan, Av. Sta. Ana 1000, Mexico City 04440, Mexico
| | - Rogelio Reyes Reyes
- Instituto Politecnico Nacional, Escuela Superior de Ingenieria Mecanica y Electrica-Culhuacan, Av. Sta. Ana 1000, Mexico City 04440, Mexico
| | - Sergiy Sadovnychiy
- Instituto Mexicano del Petroleo, Eje Central Lazaro Cardenas Norte 152, Mexico City 07730, Mexico
| |
Collapse
|
9
|
Mirikharaji Z, Abhishek K, Bissoto A, Barata C, Avila S, Valle E, Celebi ME, Hamarneh G. A survey on deep learning for skin lesion segmentation. Med Image Anal 2023; 88:102863. [PMID: 37343323 DOI: 10.1016/j.media.2023.102863] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2022] [Revised: 02/01/2023] [Accepted: 05/31/2023] [Indexed: 06/23/2023]
Abstract
Skin cancer is a major public health problem that could benefit from computer-aided diagnosis to reduce the burden of this common disease. Skin lesion segmentation from images is an important step toward achieving this goal. However, the presence of natural and artificial artifacts (e.g., hair and air bubbles), intrinsic factors (e.g., lesion shape and contrast), and variations in image acquisition conditions make skin lesion segmentation a challenging task. Recently, various researchers have explored the applicability of deep learning models to skin lesion segmentation. In this survey, we cross-examine 177 research papers that deal with deep learning-based segmentation of skin lesions. We analyze these works along several dimensions, including input data (datasets, preprocessing, and synthetic data generation), model design (architecture, modules, and losses), and evaluation aspects (data annotation requirements and segmentation performance). We discuss these dimensions both from the viewpoint of select seminal works, and from a systematic viewpoint, examining how those choices have influenced current trends, and how their limitations should be addressed. To facilitate comparisons, we summarize all examined works in a comprehensive table as well as an interactive table available online3.
Collapse
Affiliation(s)
- Zahra Mirikharaji
- Medical Image Analysis Lab, School of Computing Science, Simon Fraser University, Burnaby V5A 1S6, Canada
| | - Kumar Abhishek
- Medical Image Analysis Lab, School of Computing Science, Simon Fraser University, Burnaby V5A 1S6, Canada
| | - Alceu Bissoto
- RECOD.ai Lab, Institute of Computing, University of Campinas, Av. Albert Einstein 1251, Campinas 13083-852, Brazil
| | - Catarina Barata
- Institute for Systems and Robotics, Instituto Superior Técnico, Avenida Rovisco Pais, Lisbon 1049-001, Portugal
| | - Sandra Avila
- RECOD.ai Lab, Institute of Computing, University of Campinas, Av. Albert Einstein 1251, Campinas 13083-852, Brazil
| | - Eduardo Valle
- RECOD.ai Lab, School of Electrical and Computing Engineering, University of Campinas, Av. Albert Einstein 400, Campinas 13083-952, Brazil
| | - M Emre Celebi
- Department of Computer Science and Engineering, University of Central Arkansas, 201 Donaghey Ave., Conway, AR 72035, USA.
| | - Ghassan Hamarneh
- Medical Image Analysis Lab, School of Computing Science, Simon Fraser University, Burnaby V5A 1S6, Canada.
| |
Collapse
|
10
|
Obayya M, Alhebri A, Maashi M, S. Salama A, Mustafa Hilal A, Alsaid MI, Osman AE, Alneil AA. Henry Gas Solubility Optimization Algorithm based Feature Extraction in Dermoscopic Images Analysis of Skin Cancer. Cancers (Basel) 2023; 15:cancers15072146. [PMID: 37046806 PMCID: PMC10093373 DOI: 10.3390/cancers15072146] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Revised: 03/27/2023] [Accepted: 03/30/2023] [Indexed: 04/08/2023] Open
Abstract
Artificial Intelligence (AI) techniques have changed the general perceptions about medical diagnostics, especially after the introduction and development of Convolutional Neural Networks (CNN) and advanced Deep Learning (DL) and Machine Learning (ML) approaches. In general, dermatologists visually inspect the images and assess the morphological variables such as borders, colors, and shapes to diagnose the disease. In this background, AI techniques make use of algorithms and computer systems to mimic the cognitive functions of the human brain and assist clinicians and researchers. In recent years, AI has been applied extensively in the domain of dermatology, especially for the detection and classification of skin cancer and other general skin diseases. In this research article, the authors propose an Optimal Multi-Attention Fusion Convolutional Neural Network-based Skin Cancer Diagnosis (MAFCNN-SCD) technique for the detection of skin cancer in dermoscopic images. The primary aim of the proposed MAFCNN-SCD technique is to classify skin cancer on dermoscopic images. In the presented MAFCNN-SCD technique, the data pre-processing is performed at the initial stage. Next, the MAFNet method is applied as a feature extractor with Henry Gas Solubility Optimization (HGSO) algorithm as a hyperparameter optimizer. Finally, the Deep Belief Network (DBN) method is exploited for the detection and classification of skin cancer. A sequence of simulations was conducted to establish the superior performance of the proposed MAFCNN-SCD approach. The comprehensive comparative analysis outcomes confirmed the supreme performance of the proposed MAFCNN-SCD technique over other methodologies.
Collapse
Affiliation(s)
- Marwa Obayya
- Department of Biomedical Engineering, College of Engineering, Princess Nourah bint Abdulrahman University, Riyadh 11671, Saudi Arabia
| | - Adeeb Alhebri
- Department of Accounting, Applied College, King Khalid University, Mohail Asser 63311, Saudi Arabia
| | - Mashael Maashi
- Department of Software Engineering, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
| | - Ahmed S. Salama
- Department of Electrical Engineering, Faculty of Engineering & Technology, Future University in Egypt, New Cairo 11845, Egypt
| | - Anwer Mustafa Hilal
- Department of Computer and Self Development, Preparatory Year Deanship, Prince Sattam bin Abdulaziz University, Al Kharj 11942, Saudi Arabia
| | - Mohamed Ibrahim Alsaid
- Department of Computer and Self Development, Preparatory Year Deanship, Prince Sattam bin Abdulaziz University, Al Kharj 11942, Saudi Arabia
| | - Azza Elneil Osman
- Department of Computer and Self Development, Preparatory Year Deanship, Prince Sattam bin Abdulaziz University, Al Kharj 11942, Saudi Arabia
| | - Amani A. Alneil
- Department of Computer and Self Development, Preparatory Year Deanship, Prince Sattam bin Abdulaziz University, Al Kharj 11942, Saudi Arabia
| |
Collapse
|
11
|
Hasan MK, Ahamad MA, Yap CH, Yang G. A survey, review, and future trends of skin lesion segmentation and classification. Comput Biol Med 2023; 155:106624. [PMID: 36774890 DOI: 10.1016/j.compbiomed.2023.106624] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2022] [Revised: 01/04/2023] [Accepted: 01/28/2023] [Indexed: 02/03/2023]
Abstract
The Computer-aided Diagnosis or Detection (CAD) approach for skin lesion analysis is an emerging field of research that has the potential to alleviate the burden and cost of skin cancer screening. Researchers have recently indicated increasing interest in developing such CAD systems, with the intention of providing a user-friendly tool to dermatologists to reduce the challenges encountered or associated with manual inspection. This article aims to provide a comprehensive literature survey and review of a total of 594 publications (356 for skin lesion segmentation and 238 for skin lesion classification) published between 2011 and 2022. These articles are analyzed and summarized in a number of different ways to contribute vital information regarding the methods for the development of CAD systems. These ways include: relevant and essential definitions and theories, input data (dataset utilization, preprocessing, augmentations, and fixing imbalance problems), method configuration (techniques, architectures, module frameworks, and losses), training tactics (hyperparameter settings), and evaluation criteria. We intend to investigate a variety of performance-enhancing approaches, including ensemble and post-processing. We also discuss these dimensions to reveal their current trends based on utilization frequencies. In addition, we highlight the primary difficulties associated with evaluating skin lesion segmentation and classification systems using minimal datasets, as well as the potential solutions to these difficulties. Findings, recommendations, and trends are disclosed to inform future research on developing an automated and robust CAD system for skin lesion analysis.
Collapse
Affiliation(s)
- Md Kamrul Hasan
- Department of Bioengineering, Imperial College London, UK; Department of Electrical and Electronic Engineering (EEE), Khulna University of Engineering & Technology (KUET), Khulna 9203, Bangladesh.
| | - Md Asif Ahamad
- Department of Electrical and Electronic Engineering (EEE), Khulna University of Engineering & Technology (KUET), Khulna 9203, Bangladesh.
| | - Choon Hwai Yap
- Department of Bioengineering, Imperial College London, UK.
| | - Guang Yang
- National Heart and Lung Institute, Imperial College London, UK; Cardiovascular Research Centre, Royal Brompton Hospital, UK.
| |
Collapse
|
12
|
Maqsood S, Damaševičius R. Multiclass skin lesion localization and classification using deep learning based features fusion and selection framework for smart healthcare. Neural Netw 2023; 160:238-258. [PMID: 36701878 DOI: 10.1016/j.neunet.2023.01.022] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2022] [Revised: 11/13/2022] [Accepted: 01/19/2023] [Indexed: 01/27/2023]
Abstract
BACKGROUND The idea of smart healthcare has gradually gained attention as a result of the information technology industry's rapid development. Smart healthcare uses next-generation technologies i.e., artificial intelligence (AI) and Internet of Things (IoT), to intelligently transform current medical methods to make them more efficient, dependable and individualized. One of the most prominent uses of telemedicine and e-health in medical image analysis is teledermatology. Telecommunications technologies are used in this industry to send medical information to professionals. Teledermatology is a useful method for the identification of skin lesions, particularly in rural locations, because the skin is visually perceptible. One of the most recent tools for diagnosing skin cancer is dermoscopy. To classify skin malignancies, numerous computational approaches have been proposed in the literature. However, difficulties still exist i.e., lesions with low contrast, imbalanced datasets, high level of memory complexity, and the extraction of redundant features. METHODS In this work, a unified CAD model is proposed based on a deep learning framework for skin lesion segmentation and classification. In the proposed approach, the source dermoscopic images are initially pre-processed using a contrast enhancement based modified bio-inspired multiple exposure fusion approach. In the second stage, a custom 26-layered convolutional neural network (CNN) architecture is designed to segment the skin lesion regions. In the third stage, four pre-trained CNN models (Xception, ResNet-50, ResNet-101 and VGG16) are modified and trained using transfer learning on the segmented lesion images. In the fourth stage, the deep features vectors are extracted from all the CNN models and fused using the convolutional sparse image decomposition fusion approach. In the fifth stage, the univariate measurement and Poisson distribution feature selection approach is used for the best features selection for classification. Finally, the selected features are fed to the multi-class support vector machine (MC-SVM) for the final classification. RESULTS The proposed approach employed to the HAM10000, ISIC2018, ISIC2019, and PH2 datasets and achieved an accuracy of 98.57%, 98.62%, 93.47%, and 98.98% respectively which are better than previous works. CONCLUSION When compared to renowned state-of-the-art methods, experimental results show that the proposed skin lesion detection and classification approach achieved higher performance in terms of both visually and enhanced quantitative evaluation with enhanced accuracy.
Collapse
Affiliation(s)
- Sarmad Maqsood
- Department of Software Engineering, Faculty of Informatics Engineering, Kaunas University of Technology, LT-51386 Kaunas, Lithuania.
| | - Robertas Damaševičius
- Department of Software Engineering, Faculty of Informatics Engineering, Kaunas University of Technology, LT-51386 Kaunas, Lithuania.
| |
Collapse
|
13
|
Wang Y, Su J, Xu Q, Zhong Y. A Collaborative Learning Model for Skin Lesion Segmentation and Classification. Diagnostics (Basel) 2023; 13:diagnostics13050912. [PMID: 36900056 PMCID: PMC10001355 DOI: 10.3390/diagnostics13050912] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Revised: 02/19/2023] [Accepted: 02/24/2023] [Indexed: 03/06/2023] Open
Abstract
The automatic segmentation and classification of skin lesions are two essential tasks in computer-aided skin cancer diagnosis. Segmentation aims to detect the location and boundary of the skin lesion area, while classification is used to evaluate the type of skin lesion. The location and contour information of lesions provided by segmentation is essential for the classification of skin lesions, while the skin disease classification helps generate target localization maps to assist the segmentation task. Although the segmentation and classification are studied independently in most cases, we find meaningful information can be explored using the correlation of dermatological segmentation and classification tasks, especially when the sample data are insufficient. In this paper, we propose a collaborative learning deep convolutional neural networks (CL-DCNN) model based on the teacher-student learning method for dermatological segmentation and classification. To generate high-quality pseudo-labels, we provide a self-training method. The segmentation network is selectively retrained through classification network screening pseudo-labels. Specially, we obtain high-quality pseudo-labels for the segmentation network by providing a reliability measure method. We also employ class activation maps to improve the location ability of the segmentation network. Furthermore, we provide the lesion contour information by using the lesion segmentation masks to improve the recognition ability of the classification network. Experiments are carried on the ISIC 2017 and ISIC Archive datasets. The CL-DCNN model achieved a Jaccard of 79.1% on the skin lesion segmentation task and an average AUC of 93.7% on the skin disease classification task, which is superior to the advanced skin lesion segmentation methods and classification methods.
Collapse
Affiliation(s)
- Ying Wang
- School of Information Science and Engineering, University of Jinan, Jinan 250022, China
- Shandong Provincial Key Laboratory of Network Based Intelligent Computing, University of Jinan, Jinan 250022, China
| | - Jie Su
- School of Information Science and Engineering, University of Jinan, Jinan 250022, China
- Shandong Provincial Key Laboratory of Network Based Intelligent Computing, University of Jinan, Jinan 250022, China
- Correspondence: ; Tel.: +86-15054125550
| | - Qiuyu Xu
- School of Information Science and Engineering, University of Jinan, Jinan 250022, China
- Shandong Provincial Key Laboratory of Network Based Intelligent Computing, University of Jinan, Jinan 250022, China
| | - Yixin Zhong
- School of Information Science and Engineering, University of Jinan, Jinan 250022, China
- Artificial Intelligence Research Institute, University of Jinan, Jinan 250022, China
| |
Collapse
|
14
|
Learning from dermoscopic images in association with clinical metadata for skin lesion segmentation and classification. Comput Biol Med 2023; 152:106321. [PMID: 36463792 DOI: 10.1016/j.compbiomed.2022.106321] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2022] [Revised: 11/03/2022] [Accepted: 11/14/2022] [Indexed: 11/18/2022]
Abstract
Automatic segmentation and classification of lesions are two clinically significant tasks in the computer-aided diagnosis of skin diseases. Both tasks are challenging due to the nonnegligible lesion differences in dermoscopic images from different patients. In this paper, we propose a novel pipeline to efficiently implement skin lesions' segmentation and classification tasks, which consists of a segmentation network and a classification network. To improve the performance of the segmentation network, we propose a novel module of Multi-Scale Holistic Feature Exploration (MSH) to thoroughly exploit perceptual clues latent among multi-scale feature maps as synthesized by the decoder. The MSH module enables holistic exploration of features across multiple scales to more effectively support downstream image analysis tasks. To boost the performance of the classification network, we propose a novel module of Cross-Modality Collaborative Feature Exploration (CMC) to discover latent discriminative features by collaboratively exploiting potential relationships between cross-modal features of dermoscopic images and clinical metadata. The CMC module enables dynamically capturing versatile interaction effects among cross-modal features during the model's representation learning procedure by discriminatively and adaptively learning the interaction weight associated with each crossmodality feature pair. In addition, to effectively reduce background noise and boost the lesion discrimination ability of the classification network, we crop the images based on lesion masks generated by the best segmentation model. We evaluate the proposed pipeline on the four public skin lesion datasets, where the ISIC 2018 and PH2 are for segmentation, and the ISIC 2019 and ISIC 2020 are combined into a new dataset, ISIC 2019&2020, for classification. It achieves a Jaccard index of 83.31% and 90.14% in skin lesion segmentation, an AUC of 97.98% and an Accuracy of 92.63% in skin lesion classification, which is superior to the performance of representative state-of-the-art skin lesion segmentation and classification methods. Last but not least, the new model for segmentation utilizes much fewer model parameters (3.3 M) than its peer approaches, leading to a greatly reduced number of labeled samples required for model training, which obtains substantially stronger robustness than its peers.
Collapse
|
15
|
BF2SkNet: best deep learning features fusion-assisted framework for multiclass skin lesion classification. Neural Comput Appl 2022. [DOI: 10.1007/s00521-022-08084-6] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
|
16
|
Hum YC, Tan HR, Tee YK, Yap WS, Tan TS, Salim MIM, Lai KW. The development of skin lesion detection application in smart handheld devices using deep neural networks. MULTIMEDIA TOOLS AND APPLICATIONS 2022; 81:41579-41610. [DOI: 10.1007/s11042-021-11013-9] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/08/2020] [Revised: 04/29/2021] [Accepted: 05/05/2021] [Indexed: 07/26/2024]
|
17
|
Li Z, Koban KC, Schenck TL, Giunta RE, Li Q, Sun Y. Artificial Intelligence in Dermatology Image Analysis: Current Developments and Future Trends. J Clin Med 2022; 11:jcm11226826. [PMID: 36431301 PMCID: PMC9693628 DOI: 10.3390/jcm11226826] [Citation(s) in RCA: 32] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2022] [Revised: 10/24/2022] [Accepted: 10/28/2022] [Indexed: 11/22/2022] Open
Abstract
BACKGROUND Thanks to the rapid development of computer-based systems and deep-learning-based algorithms, artificial intelligence (AI) has long been integrated into the healthcare field. AI is also particularly helpful in image recognition, surgical assistance and basic research. Due to the unique nature of dermatology, AI-aided dermatological diagnosis based on image recognition has become a modern focus and future trend. Key scientific concepts of review: The use of 3D imaging systems allows clinicians to screen and label skin pigmented lesions and distributed disorders, which can provide an objective assessment and image documentation of lesion sites. Dermatoscopes combined with intelligent software help the dermatologist to easily correlate each close-up image with the corresponding marked lesion in the 3D body map. In addition, AI in the field of prosthetics can assist in the rehabilitation of patients and help to restore limb function after amputation in patients with skin tumors. THE AIM OF THE STUDY For the benefit of patients, dermatologists have an obligation to explore the opportunities, risks and limitations of AI applications. This study focuses on the application of emerging AI in dermatology to aid clinical diagnosis and treatment, analyzes the current state of the field and summarizes its future trends and prospects so as to help dermatologists realize the impact of new technological innovations on traditional practices so that they can embrace and use AI-based medical approaches more quickly.
Collapse
Affiliation(s)
- Zhouxiao Li
- Department of Plastic and Reconstructive Surgery, Shanghai 9th People’s Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200023, China
- Division of Hand, Plastic and Aesthetic Surgery, University Hospital, LMU Munich, 80339 Munich, Germany
| | | | - Thilo Ludwig Schenck
- Division of Hand, Plastic and Aesthetic Surgery, University Hospital, LMU Munich, 80339 Munich, Germany
| | - Riccardo Enzo Giunta
- Division of Hand, Plastic and Aesthetic Surgery, University Hospital, LMU Munich, 80339 Munich, Germany
| | - Qingfeng Li
- Department of Plastic and Reconstructive Surgery, Shanghai 9th People’s Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200023, China
- Correspondence: (Q.L.); (Y.S.)
| | - Yangbai Sun
- Department of Plastic and Reconstructive Surgery, Shanghai 9th People’s Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200023, China
- Correspondence: (Q.L.); (Y.S.)
| |
Collapse
|
18
|
Nguyen VD, Bui ND, Do HK. Skin Lesion Classification on Imbalanced Data Using Deep Learning with Soft Attention. SENSORS (BASEL, SWITZERLAND) 2022; 22:7530. [PMID: 36236628 PMCID: PMC9572097 DOI: 10.3390/s22197530] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Revised: 09/16/2022] [Accepted: 09/23/2022] [Indexed: 06/16/2023]
Abstract
Today, the rapid development of industrial zones leads to an increased incidence of skin diseases because of polluted air. According to a report by the American Cancer Society, it is estimated that in 2022 there will be about 100,000 people suffering from skin cancer and more than 7600 of these people will not survive. In the context that doctors at provincial hospitals and health facilities are overloaded, doctors at lower levels lack experience, and having a tool to support doctors in the process of diagnosing skin diseases quickly and accurately is essential. Along with the strong development of artificial intelligence technologies, many solutions to support the diagnosis of skin diseases have been researched and developed. In this paper, a combination of one Deep Learning model (DenseNet, InceptionNet, ResNet, etc) with Soft-Attention, which unsupervisedly extract a heat map of main skin lesions. Furthermore, personal information including age and gender are also used. It is worth noting that a new loss function that takes into account the data imbalance is also proposed. Experimental results on data set HAM10000 show that using InceptionResNetV2 with Soft-Attention and the new loss function gives 90 percent accuracy, mean of precision, F1-score, recall, and AUC of 0.81, 0.81, 0.82, and 0.99, respectively. Besides, using MobileNetV3Large combined with Soft-Attention and the new loss function, even though the number of parameters is 11 times less and the number of hidden layers is 4 times less, it achieves an accuracy of 0.86 and 30 times faster diagnosis than InceptionResNetV2.
Collapse
Affiliation(s)
- Viet Dung Nguyen
- School of Electrical and Electronic Engineering, Hanoi University of Science and Technology, Dai Co Viet, Ha Noi 100000, Vietnam
| | - Ngoc Dung Bui
- Faculty of Information Technology, University of Transport and Communications, Ha Noi 100000, Vietnam
| | - Hoang Khoi Do
- School of Electrical and Electronic Engineering, Hanoi University of Science and Technology, Dai Co Viet, Ha Noi 100000, Vietnam
| |
Collapse
|
19
|
Venugopal V, Joseph J, Vipin Das M, Kumar Nath M. An EfficientNet-based modified sigmoid transform for enhancing dermatological macro-images of melanoma and nevi skin lesions. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 222:106935. [PMID: 35724474 DOI: 10.1016/j.cmpb.2022.106935] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/15/2022] [Revised: 04/28/2022] [Accepted: 06/03/2022] [Indexed: 06/15/2023]
Abstract
BACKGROUND AND OBJECTIVE During the initial stages, skin lesions may not have sufficient intensity difference or contrast from the background region on dermatological macro-images. The lack of proper light exposure at the time of capturing the image also reduces the contrast. Low contrast between lesion and background regions adversely impacts segmentation. Enhancement techniques for improving the contrast between lesion and background skin on dermatological macro-images are limited in the literature. An EfficientNet-based modified sigmoid transform for enhancing the contrast on dermatological macro-images is proposed to address this issue. METHODS A modified sigmoid transform is applied in the HSV color space. The crossover point in the modified sigmoid transform that divides the macro-image into lesion and background is predicted using a modified EfficientNet regressor to exclude manual intervention and subjectivity. The Modified EfficientNet regressor is constructed by replacing the classifier layer in the conventional EfficientNet with a regression layer. Transfer learning is employed to reduce the training time and size of the dataset required to train the modified EfficientNet regressor. For training the modified EfficientNet regressor, a set of value components extracted from the HSV color space representation of the macro-images in the training dataset is fed as input. The corresponding set of ideal crossover points at which the values of Dice similarity coefficient (DSC) between the ground-truth images and the segmented output images obtained from Otsu's thresholding are maximum, is defined as the target. RESULTS On images enhanced with the proposed framework, the DSC of segmented results obtained by Otsu's thresholding increased from 0.68 ± 0.34 to 0.81 ± 0.17. CONCLUSIONS The proposed algorithm could consistently improve the contrast between lesion and background on a comprehensive set of test images, justifying its applications in automated analysis of dermatological macro-images.
Collapse
Affiliation(s)
- Vipin Venugopal
- Department of Electronics and Communication Engineering, National Institute of Technology Puducherry, Karaikal, Puducherry 609609, India.
| | - Justin Joseph
- School of Bioengineering, VIT Bhopal University, Sehore, Madhya Pradesh 466114, India.
| | - M Vipin Das
- Department of Dermatology, Kerala Health Services, Trivandrum, Kerala 695035, India.
| | - Malaya Kumar Nath
- Department of Electronics and Communication Engineering, National Institute of Technology Puducherry, Karaikal, Puducherry 609609, India.
| |
Collapse
|
20
|
Lucieri A, Bajwa MN, Braun SA, Malik MI, Dengel A, Ahmed S. ExAID: A multimodal explanation framework for computer-aided diagnosis of skin lesions. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 215:106620. [PMID: 35033756 DOI: 10.1016/j.cmpb.2022.106620] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/25/2021] [Revised: 12/01/2021] [Accepted: 01/03/2022] [Indexed: 06/14/2023]
Abstract
BACKGROUND AND OBJECTIVES One principal impediment in the successful deployment of Artificial Intelligence (AI) based Computer-Aided Diagnosis (CAD) systems in everyday clinical workflows is their lack of transparent decision-making. Although commonly used eXplainable AI (XAI) methods provide insights into these largely opaque algorithms, such explanations are usually convoluted and not readily comprehensible. The explanation of decisions regarding the malignancy of skin lesions from dermoscopic images demands particular clarity, as the underlying medical problem definition is ambiguous in itself. This work presents ExAID (Explainable AI for Dermatology), a novel XAI framework for biomedical image analysis that provides multi-modal concept-based explanations, consisting of easy-to-understand textual explanations and visual maps, to justify the predictions. METHODS Our framework relies on Concept Activation Vectors to map human-understandable concepts to those learned by an arbitrary Deep Learning (DL) based algorithm, and Concept Localisation Maps to highlight those concepts in the input space. This identification of relevant concepts is then used to construct fine-grained textual explanations supplemented by concept-wise location information to provide comprehensive and coherent multi-modal explanations. All decision-related information is presented in a diagnostic interface for use in clinical routines. Moreover, the framework includes an educational mode providing dataset-level explanation statistics as well as tools for data and model exploration to aid medical research and education processes. RESULTS Through rigorous quantitative and qualitative evaluation of our framework on a range of publicly available dermoscopic image datasets, we show the utility of multi-modal explanations for CAD-assisted scenarios even in case of wrong disease predictions. We demonstrate that concept detectors for the explanation of pre-trained networks reach accuracies of up to 81.46%, which is comparable to supervised networks trained end-to-end. CONCLUSIONS We present a new end-to-end framework for the multi-modal explanation of DL-based biomedical image analysis in Melanoma classification and evaluate its utility on an array of datasets. Since perspicuous explanation is one of the cornerstones of any CAD system, we believe that ExAID will accelerate the transition from AI research to practice by providing dermatologists and researchers with an effective tool that they can both understand and trust. ExAID can also serve as the basis for similar applications in other biomedical fields.
Collapse
Affiliation(s)
- Adriano Lucieri
- German Research Center for Artificial Intelligence (DFKI) GmbH, Trippstadter Straße 122, 67663 Kaiserslautern, Germany; Technical University Kaiserslautern, Erwin-Schrödinger-Straße 52, 67663 Kaiserslautern, Germany.
| | - Muhammad Naseer Bajwa
- German Research Center for Artificial Intelligence (DFKI) GmbH, Trippstadter Straße 122, 67663 Kaiserslautern, Germany; Technical University Kaiserslautern, Erwin-Schrödinger-Straße 52, 67663 Kaiserslautern, Germany.
| | - Stephan Alexander Braun
- University Hospital Münster, Albert-Schweitzer-Campus 1, 48149 Münster, Germany; University Hospital of Düsseldorf, Moorenstraße 5, 40225 Düsseldorf, Germany.
| | - Muhammad Imran Malik
- School of Electrical Engineering and Computer Science (SEECS), National University of Sciences and Technology (NUST), Islamabad, Pakistan; Deep Learning Laboratory, National Center of Artificial Intelligence, Islamabad, Pakistan.
| | - Andreas Dengel
- German Research Center for Artificial Intelligence (DFKI) GmbH, Trippstadter Straße 122, 67663 Kaiserslautern, Germany; Technical University Kaiserslautern, Erwin-Schrödinger-Straße 52, 67663 Kaiserslautern, Germany.
| | - Sheraz Ahmed
- German Research Center for Artificial Intelligence (DFKI) GmbH, Trippstadter Straße 122, 67663 Kaiserslautern, Germany.
| |
Collapse
|
21
|
Abstract
Any cancer type is one of the leading death causes around the world. Skin cancer is a condition where malignant cells are formed in the tissues of the skin, such as melanoma, known as the most aggressive and deadly skin cancer type. The mortality rates of melanoma are associated with its high potential for metastasis in later stages, spreading to other body sites such as the lungs, bones, or the brain. Thus, early detection and diagnosis are closely related to survival rates. Computer Aided Design (CAD) systems carry out a pre-diagnosis of a skin lesion based on clinical criteria or global patterns associated with its structure. A CAD system is essentially composed by three modules: (i) lesion segmentation, (ii) feature extraction, and (iii) classification. In this work, a methodology is proposed for a CAD system development that detects global patterns using texture descriptors based on statistical measurements that allow melanoma detection from dermoscopic images. Image analysis was carried out using spatial domain methods, statistical measurements were used for feature extraction, and a classifier based on cellular automata (ACA) was used for classification. The proposed model was applied to dermoscopic images obtained from the PH2 database, and it was compared with other models using accuracy, sensitivity, and specificity as metrics. With the proposed model, values of 0.978, 0.944, and 0.987 of accuracy, sensitivity and specificity, respectively, were obtained. The results of the evaluated metrics show that the proposed method is more effective than other state-of-the-art methods for melanoma detection in dermoscopic images.
Collapse
|
22
|
Bektaş J, Bektaş Y, Ersin Kangal E. Integrating a novel SRCRN network for segmentation with representative batch-mode experiments for detecting melanoma. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2021.103218] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
|
23
|
Spyridonos P, Gaitanis G, Likas A, Bassukas I. Characterizing Malignant Melanoma Clinically Resembling Seborrheic Keratosis Using Deep Knowledge Transfer. Cancers (Basel) 2021; 13:cancers13246300. [PMID: 34944920 PMCID: PMC8699430 DOI: 10.3390/cancers13246300] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2021] [Revised: 12/10/2021] [Accepted: 12/13/2021] [Indexed: 12/26/2022] Open
Abstract
Simple Summary Malignant melanomas (MMs) with aypical clinical presentation constitute a diagnostic pitfall, and false negatives carry the risk of a diagnostic delay and improper disease management. Among the most common, challenging presentation forms of MMs are those that clinically resemble seborrheic keratosis (SK). On the other hand, SK may mimic melanoma, producing ‘false positive overdiagnosis’ and leading to needless excisions. The evolving efficiency of deep learning algorithms in image recognition and the availability of large image databases have accelerated the development of advanced computer-aided systems for melanoma detection. In the present study, we used image data from the International Skin Image Collaboration archive to explore the capacity of deep knowledge transfer in the challenging diagnostic task of the atypical skin tumors of MM and SK. Abstract Malignant melanomas resembling seborrheic keratosis (SK-like MMs) are atypical, challenging to diagnose melanoma cases that carry the risk of delayed diagnosis and inadequate treatment. On the other hand, SK may mimic melanoma, producing a ‘false positive’ with unnecessary lesion excisions. The present study proposes a computer-based approach using dermoscopy images for the characterization of SΚ-like MMs. Dermoscopic images were retrieved from the International Skin Imaging Collaboration archive. Exploiting image embeddings from pretrained convolutional network VGG16, we trained a support vector machine (SVM) classification model on a data set of 667 images. SVM optimal hyperparameter selection was carried out using the Bayesian optimization method. The classifier was tested on an independent data set of 311 images with atypical appearance: MMs had an absence of pigmented network and had an existence of milia-like cysts. SK lacked milia-like cysts and had a pigmented network. Atypical MMs were characterized with a sensitivity and specificity of 78.6% and 84.5%, respectively. The advent of deep learning in image recognition has attracted the interest of computer science towards improved skin lesion diagnosis. Open-source, public access archives of skin images empower further the implementation and validation of computer-based systems that might contribute significantly to complex clinical diagnostic problems such as the characterization of SK-like MMs.
Collapse
Affiliation(s)
- Panagiota Spyridonos
- Department of Medical Physics, Faculty of Medicine, School of Health Sciences, University of Ioannina, 45110 Ioannina, Greece
- Correspondence: (P.S.); (I.B.)
| | - George Gaitanis
- Department of Skin and Venereal Diseases, Faculty of Medicine, School of Health Sciences, University of Ioannina, 45110 Ioannina, Greece;
| | - Aristidis Likas
- Department of Computer Science & Engineering, School of Engineering, University of Ioannina, 45110 Ioannina, Greece;
| | - Ioannis Bassukas
- Department of Skin and Venereal Diseases, Faculty of Medicine, School of Health Sciences, University of Ioannina, 45110 Ioannina, Greece;
- Correspondence: (P.S.); (I.B.)
| |
Collapse
|
24
|
Daneshjou R, Smith MP, Sun MD, Rotemberg V, Zou J. Lack of Transparency and Potential Bias in Artificial Intelligence Data Sets and Algorithms: A Scoping Review. JAMA Dermatol 2021; 157:1362-1369. [PMID: 34550305 DOI: 10.1001/jamadermatol.2021.3129] [Citation(s) in RCA: 97] [Impact Index Per Article: 32.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
Importance Clinical artificial intelligence (AI) algorithms have the potential to improve clinical care, but fair, generalizable algorithms depend on the clinical data on which they are trained and tested. Objective To assess whether data sets used for training diagnostic AI algorithms addressing skin disease are adequately described and to identify potential sources of bias in these data sets. Data Sources In this scoping review, PubMed was used to search for peer-reviewed research articles published between January 1, 2015, and November 1, 2020, with the following paired search terms: deep learning and dermatology, artificial intelligence and dermatology, deep learning and dermatologist, and artificial intelligence and dermatologist. Study Selection Studies that developed or tested an existing deep learning algorithm for triage, diagnosis, or monitoring using clinical or dermoscopic images of skin disease were selected, and the articles were independently reviewed by 2 investigators to verify that they met selection criteria. Consensus Process Data set audit criteria were determined by consensus of all authors after reviewing existing literature to highlight data set transparency and sources of bias. Results A total of 70 unique studies were included. Among these studies, 1 065 291 images were used to develop or test AI algorithms, of which only 257 372 (24.2%) were publicly available. Only 14 studies (20.0%) included descriptions of patient ethnicity or race in at least 1 data set used. Only 7 studies (10.0%) included any information about skin tone in at least 1 data set used. Thirty-six of the 56 studies developing new AI algorithms for cutaneous malignant neoplasms (64.3%) met the gold standard criteria for disease labeling. Public data sets were cited more often than private data sets, suggesting that public data sets contribute more to new development and benchmarks. Conclusions and Relevance This scoping review identified 3 issues in data sets that are used to develop and test clinical AI algorithms for skin disease that should be addressed before clinical translation: (1) sparsity of data set characterization and lack of transparency, (2) nonstandard and unverified disease labels, and (3) inability to fully assess patient diversity used for algorithm development and testing.
Collapse
Affiliation(s)
- Roxana Daneshjou
- Stanford Department of Dermatology, Stanford School of Medicine, Redwood City, California.,Stanford Department of Biomedical Data Science, Stanford School of Medicine, Stanford, California
| | - Mary P Smith
- Department of Medicine, Memorial Sloan Kettering Cancer Center, New York, New York
| | - Mary D Sun
- currently a medical student at Icahn School of Medicine at Mount Sinai, New York, New York
| | - Veronica Rotemberg
- Dermatology Service, Memorial Sloan Kettering Cancer Center, New York, New York
| | - James Zou
- Department of Electrical Engineering, Stanford University, Stanford, California.,Department of Biomedical Data Science, Stanford University, Stanford, California.,Chan Zuckerberg Biohub, San Francisco, California
| |
Collapse
|
25
|
Attique Khan M, Sharif M, Akram T, Kadry S, Hsu C. A two‐stream deep neural network‐based intelligent system for complex skin cancer types classification. INT J INTELL SYST 2021. [DOI: 10.1002/int.22691] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/25/2023]
Affiliation(s)
- Muhammad Attique Khan
- Department of Computer Science COMSATS University Islamabad Wah Campus Wah Cantt Pakistan
| | - Muhammad Sharif
- Department of Computer Science COMSATS University Islamabad Wah Campus Wah Cantt Pakistan
| | - Tallha Akram
- Department of Electrical and Computer Engineering COMSATS University Islamabad Wah Campus Wah Cantt Pakistan
| | - Seifedine Kadry
- Faculty of Applied Computing and Technology Noroff University College Kristiansand Norway
| | - Ching‐Hsien Hsu
- Guangdong‐Hong Kong‐Macao Joint Laboratory for Intelligent Micro‐Nano Optoelectronic Technology, School of Mathematics and Big Data Foshan University Foshan China
- Department of Computer Science and Information Engineering Asia University Taichung Taiwan
- Department of Medical Research China Medical University Hospital China Medical University Taichung Taiwan
| |
Collapse
|
26
|
Zhao C, Shuai R, Ma L, Liu W, Wu M. Segmentation of dermoscopy images based on deformable 3D convolution and ResU-NeXt +. Med Biol Eng Comput 2021; 59:1815-1832. [PMID: 34304370 DOI: 10.1007/s11517-021-02397-9] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2021] [Accepted: 06/16/2021] [Indexed: 11/25/2022]
Abstract
Melanoma is one of the most dangerous skin cancers. The current melanoma segmentation is mainly based on FCNs (fully connected networks) and U-Net. Nevertheless, these two kinds of neural networks are prone to parameter redundancy, and the gradient of neural networks disappears that occurs when the neural network backpropagates as the neural network gets deeper, which will reduce the Jaccard index of the skin lesion image segmentation model. To solve the above problems and improve the survival rate of melanoma patients, an improved skin lesion segmentation model based on deformable 3D convolution and ResU-NeXt++ (D3DC- ResU-NeXt++) is proposed in this paper. The new modules in D3DC-ResU-NeXt++ can replace ordinary modules in the existing 2D convolutional neural networks (CNNs) that can be trained efficiently through standard backpropagation with high segmentation accuracy. In particular, we introduce a new data preprocessing method with dilation, crop operation, resizing, and hair removal (DCRH), which improves the Jaccard index of skin lesion image segmentation. Because rectified Adam (RAdam) does not easily fall into a local optimal solution and can converge quickly in segmentation model training, we also introduce RAdam as the training optimizer. The experiments show that our model has excellent performance on the segmentation of the ISIC2018 Task I dataset, and the Jaccard index achieves 86.84%. The proposed method improves the Jaccard index of segmentation of skin lesion images and can also assist dermatological doctors in determining and diagnosing the types of skin lesions and the boundary between lesions and normal skin, so as to improve the survival rate of skin cancer patients. Overview of the proposed model. An improved skin lesion segmentation model based on deformable 3D convolution and ResU-NeXt++ (D3DC- ResU-NeXt++) is proposed in this paper. D3DC-ResU-NeXt++ has strong spatial geometry processing capabilities, it is used to segment the skin lesion sample image; DCRH and transfer learning are used to preprocess the data set and D3DC-ResU-NeXt++ respectively, which can highlight the difference between the lesion area and the normal skin, and enhance the segmentation efficiency and robustness of the neural network; RAdam is used to speed up the convergence speed of neural network and improve the efficiency of segmentation.
Collapse
Affiliation(s)
- Chen Zhao
- College of Computer Science and Technology, Nanjing Tech University, Nanjing, 211816, China
| | - Renjun Shuai
- College of Computer Science and Technology, Nanjing Tech University, Nanjing, 211816, China.
| | - Li Ma
- Nanjing Health Information Center, Nanjing, 210003, China
| | - Wenjia Liu
- Changzhou No. 2 People's Hospital affiliated with Nanjing Medical University, Changzhou, 213003, China
| | - Menglin Wu
- College of Computer Science and Technology, Nanjing Tech University, Nanjing, 211816, China
| |
Collapse
|
27
|
|
28
|
Mahbod A, Schaefer G, Löw C, Dorffner G, Ecker R, Ellinger I. Investigating the Impact of the Bit Depth of Fluorescence-Stained Images on the Performance of Deep Learning-Based Nuclei Instance Segmentation. Diagnostics (Basel) 2021; 11:967. [PMID: 34072131 PMCID: PMC8230326 DOI: 10.3390/diagnostics11060967] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2021] [Revised: 05/05/2021] [Accepted: 05/25/2021] [Indexed: 11/25/2022] Open
Abstract
Nuclei instance segmentation can be considered as a key point in the computer-mediated analysis of histological fluorescence-stained (FS) images. Many computer-assisted approaches have been proposed for this task, and among them, supervised deep learning (DL) methods deliver the best performances. An important criterion that can affect the DL-based nuclei instance segmentation performance of FS images is the utilised image bit depth, but to our knowledge, no study has been conducted so far to investigate this impact. In this work, we released a fully annotated FS histological image dataset of nuclei at different image magnifications and from five different mouse organs. Moreover, by different pre-processing techniques and using one of the state-of-the-art DL-based methods, we investigated the impact of image bit depth (i.e., eight bits vs. sixteen bits) on the nuclei instance segmentation performance. The results obtained from our dataset and another publicly available dataset showed very competitive nuclei instance segmentation performances for the models trained with 8 bit and 16 bit images. This suggested that processing 8 bit images is sufficient for nuclei instance segmentation of FS images in most cases. The dataset including the raw image patches, as well as the corresponding segmentation masks is publicly available in the published GitHub repository.
Collapse
Affiliation(s)
- Amirreza Mahbod
- Institute for Pathophysiology and Allergy Research, Medical University of Vienna, A-1090 Vienna, Austria; (C.L.); (I.E.)
| | - Gerald Schaefer
- Department of Computer Science, Loughborough University, Loughborough LE11 3TT, UK;
| | - Christine Löw
- Institute for Pathophysiology and Allergy Research, Medical University of Vienna, A-1090 Vienna, Austria; (C.L.); (I.E.)
| | - Georg Dorffner
- Section for Artificial Intelligence and Decision Support, Medical University of Vienna, 1090 Vienna, Austria;
| | - Rupert Ecker
- Department of Research and Development, TissueGnostics GmbH, 1020 Vienna, Austria;
| | - Isabella Ellinger
- Institute for Pathophysiology and Allergy Research, Medical University of Vienna, A-1090 Vienna, Austria; (C.L.); (I.E.)
| |
Collapse
|
29
|
Khan MA, Sharif M, Akram T, Damaševičius R, Maskeliūnas R. Skin Lesion Segmentation and Multiclass Classification Using Deep Learning Features and Improved Moth Flame Optimization. Diagnostics (Basel) 2021; 11:811. [PMID: 33947117 PMCID: PMC8145295 DOI: 10.3390/diagnostics11050811] [Citation(s) in RCA: 73] [Impact Index Per Article: 24.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2021] [Revised: 04/23/2021] [Accepted: 04/26/2021] [Indexed: 11/18/2022] Open
Abstract
Manual diagnosis of skin cancer is time-consuming and expensive; therefore, it is essential to develop automated diagnostics methods with the ability to classify multiclass skin lesions with greater accuracy. We propose a fully automated approach for multiclass skin lesion segmentation and classification by using the most discriminant deep features. First, the input images are initially enhanced using local color-controlled histogram intensity values (LCcHIV). Next, saliency is estimated using a novel Deep Saliency Segmentation method, which uses a custom convolutional neural network (CNN) of ten layers. The generated heat map is converted into a binary image using a thresholding function. Next, the segmented color lesion images are used for feature extraction by a deep pre-trained CNN model. To avoid the curse of dimensionality, we implement an improved moth flame optimization (IMFO) algorithm to select the most discriminant features. The resultant features are fused using a multiset maximum correlation analysis (MMCA) and classified using the Kernel Extreme Learning Machine (KELM) classifier. The segmentation performance of the proposed methodology is analyzed on ISBI 2016, ISBI 2017, ISIC 2018, and PH2 datasets, achieving an accuracy of 95.38%, 95.79%, 92.69%, and 98.70%, respectively. The classification performance is evaluated on the HAM10000 dataset and achieved an accuracy of 90.67%. To prove the effectiveness of the proposed methods, we present a comparison with the state-of-the-art techniques.
Collapse
Affiliation(s)
- Muhammad Attique Khan
- Department of Computer Science, Wah Campus, COMSATS University Islamabad, Wah Cantonment 47040, Pakistan;
| | - Muhammad Sharif
- Department of Computer Science, Wah Campus, COMSATS University Islamabad, Wah Cantonment 47040, Pakistan;
| | - Tallha Akram
- Department of Electrical Engineering, Wah Campus, COMSATS University Islamabad, Islamabad 45550, Pakistan;
| | - Robertas Damaševičius
- Faculty of Applied Mathematics, Silesian University of Technology, 44-100 Gliwice, Poland
| | - Rytis Maskeliūnas
- Department of Applied Informatics, Vytautas Magnus University, 44404 Kaunas, Lithuania;
| |
Collapse
|
30
|
CryoNuSeg: A dataset for nuclei instance segmentation of cryosectioned H&E-stained histological images. Comput Biol Med 2021; 132:104349. [PMID: 33774269 DOI: 10.1016/j.compbiomed.2021.104349] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2020] [Revised: 02/28/2021] [Accepted: 03/16/2021] [Indexed: 12/31/2022]
Abstract
Nuclei instance segmentation plays an important role in the analysis of hematoxylin and eosin (H&E)-stained images. While supervised deep learning (DL)-based approaches represent the state-of-the-art in automatic nuclei instance segmentation, annotated datasets are required to train these models. There are two main types of tissue processing protocols resulting in formalin-fixed paraffin-embedded samples (FFPE) and frozen tissue samples (FS), respectively. Although FFPE-derived H&E stained tissue sections are the most widely used samples, H&E staining of frozen sections derived from FS samples is a relevant method in intra-operative surgical sessions as it can be performed more rapidly. Due to differences in the preparation of these two types of samples, the derived images and in particular the nuclei appearance may be different in the acquired whole slide images. Analysis of FS-derived H&E stained images can be more challenging as rapid preparation, staining, and scanning of FS sections may lead to deterioration in image quality. In this paper, we introduce CryoNuSeg, the first fully annotated FS-derived cryosectioned and H&E-stained nuclei instance segmentation dataset. The dataset contains images from 10 human organs that were not exploited in other publicly available datasets, and is provided with three manual mark-ups to allow measuring intra-observer and inter-observer variabilities. Moreover, we investigate the effects of tissue fixation/embedding protocol (i.e., FS or FFPE) on the automatic nuclei instance segmentation performance and provide a baseline segmentation benchmark for the dataset that can be used in future research. A step-by-step guide to generate the dataset as well as the full dataset and other detailed information are made available to fellow researchers at https://github.com/masih4/CryoNuSeg.
Collapse
|
31
|
Caffery LJ, Rotemberg V, Weber J, Soyer HP, Malvehy J, Clunie D. The Role of DICOM in Artificial Intelligence for Skin Disease. Front Med (Lausanne) 2021; 7:619787. [PMID: 33644087 PMCID: PMC7902872 DOI: 10.3389/fmed.2020.619787] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2020] [Accepted: 12/31/2020] [Indexed: 01/19/2023] Open
Abstract
There is optimism that artificial intelligence (AI) will result in positive clinical outcomes, which is driving research and investment in the use of AI for skin disease. At present, AI for skin disease is embedded in research and development and not practiced widely in clinical dermatology. Clinical dermatology is also undergoing a technological transformation in terms of the development and adoption of standards that optimizes the quality use of imaging. Digital Imaging and Communications in Medicine (DICOM) is the international standard for medical imaging. DICOM is a continually evolving standard. There is considerable effort being invested in developing dermatology-specific extensions to the DICOM standard. The ability to encode relevant metadata and afford interoperability with the digital health ecosystem (e.g., image repositories, electronic medical records) has driven the initial impetus in the adoption of DICOM for dermatology. DICOM has a dedicated working group whose role is to develop a mechanism to support AI workflows and encode AI artifacts. DICOM can improve AI workflows by encoding derived objects (e.g., secondary images, visual explainability maps, AI algorithm output) and the efficient curation of multi-institutional datasets for machine learning training, testing, and validation. This can be achieved using DICOM mechanisms such as standardized image formats and metadata, metadata-based image retrieval, and de-identification protocols. DICOM can address several important technological and workflow challenges for the implementation of AI. However, many other technological, ethical, regulatory, medicolegal, and workforce barriers will need to be addressed before DICOM and AI can be used effectively in dermatology.
Collapse
Affiliation(s)
- Liam J. Caffery
- Centre for Online, Centre for Health Services Research, The University of Queensland, Brisbane, QLD, Australia
- Dermatology Research Centre, The University of Queensland Diamantina Institute, The University of Queensland, Brisbane, QLD, Australia
| | - Veronica Rotemberg
- Dermatology Service, Department of Medicine, Memorial Sloan Kettering Cancer Center, New York, NY, United States
| | - Jochen Weber
- Dermatology Service, Department of Medicine, Memorial Sloan Kettering Cancer Center, New York, NY, United States
| | - H. Peter Soyer
- Dermatology Research Centre, The University of Queensland Diamantina Institute, The University of Queensland, Brisbane, QLD, Australia
- Department of Dermatology, Princess Alexandra Hospital, Brisbane, QLD, Australia
| | - Josep Malvehy
- Department of Dermatology, Institut d'Investigacions Biomèdiques August Pi i Sunyer, Hospital Clinic of Barcelona, University of Barcelona, Barcelona, Spain
| | | |
Collapse
|