The impact of patient clinical information on automated skin cancer detection

Leveraging AI and patient metadata to develop a novel risk score for skin cancer detection

Melanoma of the skin is the 17th most common cancer worldwide. Early detection of suspicious skin lesions (melanoma) can increase 5-year survival rates by 20%. The 7-point checklist (7PCL) has been extensively used to suggest urgent referrals for patients with a possible melanoma. However, the 7PCL method only considers seven meta-features to calculate a risk score and is only relevant for patients with suspected melanoma. There are limited studies on the extensive use of patient metadata for the detection of all skin cancer subtypes. This study investigates artificial intelligence (AI) models that utilise patient metadata consisting of 23 attributes for suspicious skin lesion detection. We have identified a new set of most important risk factors, namely "C4C risk factors", which is not just for melanoma, but for all types of skin cancer. The performance of the C4C risk factors for suspicious skin lesion detection is compared to that of the 7PCL and the Williams risk factors that predict the lifetime risk of melanoma. Our proposed AI framework ensembles five machine learning models and identifies seven new skin cancer risk factors: lesion pink, lesion size, lesion colour, lesion inflamed, lesion shape, lesion age, and natural hair colour, which achieved a sensitivity of 80.46 ± 2.50 % and a specificity of 62.09 ± 1.90 % in detecting suspicious skin lesions when evaluated using the metadata of 53,601 skin lesions collected from different skin cancer diagnostic clinics across the UK, significantly outperforming the 7PCL-based method (sensitivity 68.09 ± 2.10 % , specificity 61.07 ± 0.90 % ) and the Williams risk factors (sensitivity 66.32 ± 1.90 % , specificity 61.71 ± 0.6 % ). Furthermore, through weighting the seven new risk factors we came up with a new risk score, namely "C4C risk score", which alone achieved a sensitivity of 76.09 ± 1.20 % and a specificity of 61.71 ± 0.50 % , significantly outperforming the 7PCL-based risk score (sensitivity 73.91 ± 1.10 % , specificity 49.49 ± 0.50 % ) and the Williams risk score (sensitivity 60.68 ± 1.30 % , specificity 60.87 ± 0.80 % ). Finally, fusing the C4C risk factors with the 7PCL and Williams risk factors achieved the best performance, with a sensitivity of 85.24 ± 2.20 % and a specificity of 61.12 ± 0.90 % . We believe that fusing these newly found risk factors and new risk score with image data will further boost the AI model performance for suspicious skin lesion detection. Hence, the new set of skin cancer risk factors has the potential to be used to modify current skin cancer referral guidelines for all skin cancer subtypes, including melanoma.

The Role of Artificial Intelligence in the Diagnosis of Melanoma

The incidence of melanoma, the most aggressive form of skin cancer, continues to rise globally, particularly among fair-skinned populations (type I and II). Early detection is crucial for improving patient outcomes, and recent advancements in artificial intelligence (AI) have shown promise in enhancing the accuracy and efficiency of melanoma diagnosis and management. This review examines the role of AI in skin lesion diagnostics, highlighting two main approaches: machine learning, particularly convolutional neural networks (CNNs), and expert systems. AI techniques have demonstrated high accuracy in classifying dermoscopic images, often matching or surpassing dermatologists' performance. Integrating AI into dermatology has improved tasks, such as lesion classification, segmentation, and risk prediction, facilitating earlier and more accurate interventions. Despite these advancements, challenges remain, including biases in training data, interpretability issues, and integration of AI into clinical workflows. Ensuring diverse data representation and maintaining high standards of image quality are essential for reliable AI performance. Future directions involve the development of more sophisticated models, such as vision-language and multimodal models, and federated learning to address data privacy and generalizability concerns. Continuous validation and ethical integration of AI into clinical practice are vital for realizing its full potential for improving melanoma diagnosis and patient care.

Systematic review of approaches to detection and classification of skin cancer using artificial intelligence: Development and prospects

In recent years, there has been a significant improvement in the accuracy of the classification of pigmented skin lesions using artificial intelligence algorithms. Intelligent analysis and classification systems are significantly superior to visual diagnostic methods used by dermatologists and oncologists. However, the application of such systems in clinical practice is severely limited due to a lack of generalizability and risks of potential misclassification. Successful implementation of artificial intelligence-based tools into clinicopathological practice requires a comprehensive study of the effectiveness and performance of existing models, as well as further promising areas for potential research development. The purpose of this systematic review is to investigate and evaluate the accuracy of artificial intelligence technologies for detecting malignant forms of pigmented skin lesions. For the study, 10,589 scientific research and review articles were selected from electronic scientific publishers, of which 171 articles were included in the presented systematic review. All selected scientific articles are distributed according to the proposed neural network algorithms from machine learning to multimodal intelligent architectures and are described in the corresponding sections of the manuscript. This research aims to explore automated skin cancer recognition systems, from simple machine learning algorithms to multimodal ensemble systems based on advanced encoder-decoder models, visual transformers (ViT), and generative and spiking neural networks. In addition, as a result of the analysis, future directions of research, prospects, and potential for further development of automated neural network systems for classifying pigmented skin lesions are discussed.

Emerging Trends in the Treatment of Skin Disorders by Herbal Drugs: Traditional and Nanotechnological Approach

Since the earliest days, people have been employing herbal treatments extensively around the world. The development of phytochemical and phytopharmacological sciences has made it possible to understand the chemical composition and biological properties of a number of medicinal plant products. Due to certain challenges like large molecular weight and low bioavailability, some components of herbal extracts are not utilized for therapeutic purposes. It has been suggested that herbal medicine and nanotechnology can be combined to enhance the benefits of plant extracts by lowering dosage requirements and adverse effects and increasing therapeutic activity. Using nanotechnology, the active ingredient can be delivered in an adequate concentration and transported to the targeted site of action. Conventional therapy does not fulfill these requirements. This review focuses on different skin diseases and nanotechnology-based herbal medicines that have been utilized to treat them.

From data to diagnosis: skin cancer image datasets for artificial intelligence

Artificial intelligence (AI) solutions for skin cancer diagnosis continue to gain momentum, edging closer towards broad clinical use. These AI models, particularly deep-learning architectures, require large digital image datasets for development. This review provides an overview of the datasets used to develop AI algorithms and highlights the importance of dataset transparency for the evaluation of algorithm generalizability across varying populations and settings. Current challenges for curation of clinically valuable datasets are detailed, which include dataset shifts arising from demographic variations and differences in data collection methodologies, along with inconsistencies in labelling. These shifts can lead to differential algorithm performance, compromise of clinical utility, and the propagation of discriminatory biases when developed algorithms are implemented in mismatched populations. Limited representation of rare skin cancers and minoritized groups in existing datasets are highlighted, which can further skew algorithm performance. Strategies to address these challenges are presented, which include improving transparency, representation and interoperability. Federated learning and generative methods, which may improve dataset size and diversity without compromising privacy, are also examined. Lastly, we discuss model-level techniques that may address biases entrained through the use of datasets derived from routine clinical care. As the role of AI in skin cancer diagnosis becomes more prominent, ensuring the robustness of underlying datasets is increasingly important.

A novel multi-task learning network for skin lesion classification based on multi-modal clues and label-level fusion

In this paper, we propose a multi-task learning (MTL) network based on the label-level fusion of metadata and hand-crafted features by unsupervised clustering to generate new clustering labels as an optimization goal. We propose a MTL module (MTLM) that incorporates an attention mechanism to enable the model to learn more integrated, variable information. We propose a dynamic strategy to adjust the loss weights of different tasks, and trade off the contributions of multiple branches. Instead of feature-level fusion, we propose label-level fusion and combine the results of our proposed MTLM with the results of the image classification network to achieve better lesion prediction on multiple dermatological datasets. We verify the effectiveness of the proposed model by quantitative and qualitative measures. The MTL network using multi-modal clues and label-level fusion can yield the significant performance improvement for skin lesion classification.

Performance evaluation of E-VGG19 model: Enhancing real-time skin cancer detection and classification

Skin cancer is a pervasive and potentially life-threatening disease. Early detection plays a crucial role in improving patient outcomes. Machine learning (ML) techniques, particularly when combined with pre-trained deep learning models, have shown promise in enhancing the accuracy of skin cancer detection. In this paper, we enhanced the VGG19 pre-trained model with max pooling and dense layer for the prediction of skin cancer. Moreover, we also explored the pre-trained models such as Visual Geometry Group 19 (VGG19), Residual Network 152 version 2 (ResNet152v2), Inception-Residual Network version 2 (InceptionResNetV2), Dense Convolutional Network 201 (DenseNet201), Residual Network 50 (ResNet50), Inception version 3 (InceptionV3), For training, skin lesions dataset is used with malignant and benign cases. The models extract features and divide skin lesions into two categories: malignant and benign. The features are then fed into machine learning methods, including Linear Support Vector Machine (SVM), k-Nearest Neighbors (KNN), Decision Tree (DT), Logistic Regression (LR) and Support Vector Machine (SVM), our results demonstrate that combining E-VGG19 model with traditional classifiers significantly improves the overall classification accuracy for skin cancer detection and classification. Moreover, we have also compared the performance of baseline classifiers and pre-trained models with metrics (recall, F1 score, precision, sensitivity, and accuracy). The experiment results provide valuable insights into the effectiveness of various models and classifiers for accurate and efficient skin cancer detection. This research contributes to the ongoing efforts to create automated technologies for detecting skin cancer that can help healthcare professionals and individuals identify potential skin cancer cases at an early stage, ultimately leading to more timely and effective treatments.

A novel skin cancer detection model using modified finch deep CNN classifier model

Skin cancer is one of the most life-threatening diseases caused by the abnormal growth of the skin cells, when exposed to ultraviolet radiation. Early detection seems to be more crucial for reducing aberrant cell proliferation because the mortality rate is rapidly rising. Although multiple researches are available based on the skin cancer detection, there still exists challenges in improving the accuracy, reducing the computational time and so on. In this research, a novel skin cancer detection is performed using a modified falcon finch deep Convolutional neural network classifier (Modified Falcon finch deep CNN) that efficiently detects the disease with higher efficiency. The usage of modified falcon finch deep CNN classifier effectively analyzed the information relevant to the skin cancer and the errors are also minimized. The inclusion of the falcon finch optimization in the deep CNN classifier is necessary for efficient parameter tuning. This tuning enhanced the robustness and boosted the convergence of the classifier that detects the skin cancer in less stipulated time. The modified falcon finch deep CNN classifier achieved accuracy, sensitivity, and specificity values of 93.59%, 92.14%, and 95.22% regarding k-fold and 96.52%, 96.69%, and 96.54% regarding training percentage, proving more effective than literary works.

Artificial intelligence in the detection of skin cancer: State of the art

The incidence of melanoma is increasing rapidly. This cancer has a good prognosis if detected early. For this reason, various systems of skin lesion image analysis, which support imaging diagnostics of this neoplasm, are developing very dynamically. To detect and recognize neoplastic lesions, such systems use various artificial intelligence (AI) algorithms. This area of computer science applications has recently undergone dynamic development, abounding in several solutions that are effective tools supporting diagnosticians in many medical specialties. In this contribution, a number of applications of different classes of AI algorithms for the detection of this skin melanoma are presented and evaluated. Both classic systems based on the analysis of dermatoscopic images as well as total body systems, enabling the analysis of the patient's whole body to detect moles and pathologic changes, are discussed. These increasingly popular applications that allow the analysis of lesion images using smartphones are also described. The quantitative evaluation of the discussed systems with particular emphasis on the method of validation of the implemented algorithms is presented. The advantages and limitations of AI in the analysis of lesion images are also discussed, and problems requiring a solution for more effective use of AI in dermatology are identified.

Early automated detection system for skin cancer diagnosis using artificial intelligent techniques

Recently, skin cancer is one of the spread and dangerous cancers around the world. Early detection of skin cancer can reduce mortality. Traditional methods for skin cancer detection are painful, time-consuming, expensive, and may cause the disease to spread out. Dermoscopy is used for noninvasive diagnosis of skin cancer. Artificial Intelligence (AI) plays a vital role in diseases' diagnosis especially in biomedical engineering field. The automated detection systems based on AI reduce the complications in the traditional methods and can improve skin cancer's diagnosis rate. In this paper, automated early detection system for skin cancer dermoscopic images using artificial intelligent is presented. Adaptive snake (AS) and region growing (RG) algorithms are used for automated segmentation and compared with each other. The results show that AS is accurate and efficient (accuracy = 96%) more than RG algorithm (accuracy = 90%). Artificial Neural networks (ANN) and support vector machine (SVM) algorithms are used for automated classification compared with each other. The proposed system with ANN algorithm shows high accuracy (94%), precision (96%), specificity (95.83%), sensitivity (recall) (92.30%), and F1-score (0.94). The proposed system is easy to use, time consuming, enables patients to make early detection for skin cancer and has high efficiency.

Modelling a self-defined CNN for effectual classification of PCOS from ultrasound images

BACKGROUND

Polycystic Ovary Syndrome (PCOS) is a medical condition that causes hormonal disorders in women in their childbearing years. The hormonal imbalance leads to a delayed or even absent menstrual cycle. Women with PCOS mainly suffer from extreme weight gain, facial hair growth, acne, hair loss, skin darkening, and irregular periods, leading to infertility in rare cases. Doctors usually examine ultrasound images and conclude the affected ovary but are incapable of deciding whether it is a normal cyst, PCOS, or cancer cyst manually.

OBJECTIVE

To have access to the high-risk crucial PCOS and to detect the condition and the treatment aimed at mitigating health hazards such as endometrial hyperplasia/cancer, infertility, pregnancy complications, and the long-term burden of chronic diseases such as cardiometabolic disorders linked with PCOS.

METHODS

The proposed Self-Defined Convolution Neural Network method (SD_CNN) is used to extract the features and machine learning models such as SVM, Random Forest, and Logistic Regression are used to classify PCOS images. The parameter tuning is done with lesser parameters in order to overcome over-fitting issues. The self-defined model predicts the occurrence of the cyst based on the analyzed features and classifies the class labels effectively.

RESULTS

The Random Forest Classifier was found to be the most reliable and accurate among Support Vector Machine (SVM) and Logistic Regression (LR), with accuracy being 96.43%.

CONCLUSION

The proposed model establishes better trade-off compared to various other approaches and works effectually for PCOS prediction.

Automatic Skin Cancer Detection Using Clinical Images: A Comprehensive Review

Skin cancer has become increasingly common over the past decade, with melanoma being the most aggressive type. Hence, early detection of skin cancer and melanoma is essential in dermatology. Computational methods can be a valuable tool for assisting dermatologists in identifying skin cancer. Most research in machine learning for skin cancer detection has focused on dermoscopy images due to the existence of larger image datasets. However, general practitioners typically do not have access to a dermoscope and must rely on naked-eye examinations or standard clinical images. By using standard, off-the-shelf cameras to detect high-risk moles, machine learning has also proven to be an effective tool. The objective of this paper is to provide a comprehensive review of image-processing techniques for skin cancer detection using clinical images. In this study, we evaluate 51 state-of-the-art articles that have used machine learning methods to detect skin cancer over the past decade, focusing on clinical datasets. Even though several studies have been conducted in this field, there are still few publicly available clinical datasets with sufficient data that can be used as a benchmark, especially when compared to the existing dermoscopy databases. In addition, we observed that the available artifact removal approaches are not quite adequate in some cases and may also have a negative impact on the models. Moreover, the majority of the reviewed articles are working with single-lesion images and do not consider typical mole patterns and temporal changes in the lesions of each patient.

Federated and Transfer Learning Methods for the Classification of Melanoma and Nonmelanoma Skin Cancers: A Prospective Study

Skin cancer is considered a dangerous type of cancer with a high global mortality rate. Manual skin cancer diagnosis is a challenging and time-consuming method due to the complexity of the disease. Recently, deep learning and transfer learning have been the most effective methods for diagnosing this deadly cancer. To aid dermatologists and other healthcare professionals in classifying images into melanoma and nonmelanoma cancer and enabling the treatment of patients at an early stage, this systematic literature review (SLR) presents various federated learning (FL) and transfer learning (TL) techniques that have been widely applied. This study explores the FL and TL classifiers by evaluating them in terms of the performance metrics reported in research studies, which include true positive rate (TPR), true negative rate (TNR), area under the curve (AUC), and accuracy (ACC). This study was assembled and systemized by reviewing well-reputed studies published in eminent fora between January 2018 and July 2023. The existing literature was compiled through a systematic search of seven well-reputed databases. A total of 86 articles were included in this SLR. This SLR contains the most recent research on FL and TL algorithms for classifying malignant skin cancer. In addition, a taxonomy is presented that summarizes the many malignant and non-malignant cancer classes. The results of this SLR highlight the limitations and challenges of recent research. Consequently, the future direction of work and opportunities for interested researchers are established that help them in the automated classification of melanoma and nonmelanoma skin cancers.

The Promise of Semantic Segmentation in Detecting Actinic Keratosis Using Clinical Photography in the Wild

AK is a common precancerous skin condition that requires effective detection and treatment monitoring. To improve the monitoring of the AK burden in clinical settings with enhanced automation and precision, the present study evaluates the application of semantic segmentation based on the U-Net architecture (i.e., AKU-Net). AKU-Net employs transfer learning to compensate for the relatively small dataset of annotated images and integrates a recurrent process based on convLSTM to exploit contextual information and address the challenges related to the low contrast and ambiguous boundaries of AK-affected skin regions. We used an annotated dataset of 569 clinical photographs from 115 patients with actinic keratosis to train and evaluate the model. From each photograph, patches of 512 × 512 pixels were extracted using translation lesion boxes that encompassed lesions in different positions and captured different contexts of perilesional skin. In total, 16,488 translation-augmented crops were used for training the model, and 403 lesion center crops were used for testing. To demonstrate the improvements in AK detection, AKU-Net was compared with plain U-Net and U-Net++ architectures. The experimental results highlighted the effectiveness of AKU-Net, improving upon both automation and precision over existing approaches, paving the way for more effective and reliable evaluation of actinic keratosis in clinical settings.

A Multi-Stage Faster RCNN-Based iSPLInception for Skin Disease Classification Using Novel Optimization

Nowadays, skin cancer is considered a serious disorder in which early identification and treatment of the disease are essential to ensure the stability of the patients. Several existing skin cancer detection methods are introduced by employing deep learning (DL) to perform skin disease classification. Convolutional neural networks (CNNs) can classify melanoma skin cancer images. But, it suffers from an overfitting problem. Therefore, to overcome this problem and to classify both benign and malignant tumors efficiently, the multi-stage faster RCNN-based iSPLInception (MFRCNN-iSPLI) method is proposed. Then, the test dataset is used for evaluating the proposed model performance. The faster RCNN is employed directly to perform image classification. This may heavily raise computation time and network complications. So, the iSPLInception model is applied in the multi-stage classification. In this, the iSPLInception model is formulated using the Inception-ResNet design. For candidate box deletion, the prairie dog optimization algorithm is utilized. We have utilized two skin disease datasets, namely, ISIC 2019 Skin lesion image classification and the HAM10000 dataset for conducting experimental results. The methods' accuracy, precision, recall, and F1 score values are calculated, and the results are compared with the existing methods such as CNN, hybrid DL, Inception v3, and VGG19. With 95.82% accuracy, 96.85% precision, 96.52% recall, and 0.95% F1 score values, the output analysis of each measure verified the prediction and classification effectiveness of the method.

MSRNet: Multiclass Skin Lesion Recognition Using Additional Residual Block Based Fine-Tuned Deep Models Information Fusion and Best Feature Selection

Cancer is one of the leading significant causes of illness and chronic disease worldwide. Skin cancer, particularly melanoma, is becoming a severe health problem due to its rising prevalence. The considerable death rate linked with melanoma requires early detection to receive immediate and successful treatment. Lesion detection and classification are more challenging due to many forms of artifacts such as hairs, noise, and irregularity of lesion shape, color, irrelevant features, and textures. In this work, we proposed a deep-learning architecture for classifying multiclass skin cancer and melanoma detection. The proposed architecture consists of four core steps: image preprocessing, feature extraction and fusion, feature selection, and classification. A novel contrast enhancement technique is proposed based on the image luminance information. After that, two pre-trained deep models, DarkNet-53 and DensNet-201, are modified in terms of a residual block at the end and trained through transfer learning. In the learning process, the Genetic algorithm is applied to select hyperparameters. The resultant features are fused using a two-step approach named serial-harmonic mean. This step increases the accuracy of the correct classification, but some irrelevant information is also observed. Therefore, an algorithm is developed to select the best features called marine predator optimization (MPA) controlled Reyni Entropy. The selected features are finally classified using machine learning classifiers for the final classification. Two datasets, ISIC2018 and ISIC2019, have been selected for the experimental process. On these datasets, the obtained maximum accuracy of 85.4% and 98.80%, respectively. To prove the effectiveness of the proposed methods, a detailed comparison is conducted with several recent techniques and shows the proposed framework outperforms.

Metadata and Image Features Co-Aware Personalized Federated Learning for Smart Healthcare

Recently, artificial intelligence has been widely used in intelligent disease diagnosis and has achieved great success. However, most of the works mainly rely on the extraction of image features but ignore the use of clinical text information of patients, which may limit the diagnosis accuracy fundamentally. In this paper, we propose a metadata and image features co-aware personalized federated learning scheme for smart healthcare. Specifically, we construct an intelligent diagnosis model, by which users can obtain fast and accurate diagnosis services. Meanwhile, a personalized federated learning scheme is designed to utilize the knowledge learned from other edge nodes with larger contributions and customize high-quality personalized classification models for each edge node. Subsequently, a Naïve Bayes classifier is devised for classifying patient metadata. And then the image and metadata diagnosis results are jointly aggregated by different weights to improve the accuracy of intelligent diagnosis. Finally, the simulation results illustrate that, compared with the existing methods, our proposed algorithm achieves better classification accuracy, reaching about 97.16% on PAD-UFES-20 dataset.

Privacy-Aware Collaborative Learning for Skin Cancer Prediction

Cancer, including the highly dangerous melanoma, is marked by uncontrolled cell growth and the possibility of spreading to other parts of the body. However, the conventional approach to machine learning relies on centralized training data, posing challenges for data privacy in healthcare systems driven by artificial intelligence. The collection of data from diverse sensors leads to increased computing costs, while privacy restrictions make it challenging to employ traditional machine learning methods. Researchers are currently confronted with the formidable task of developing a skin cancer prediction technique that takes privacy concerns into account while simultaneously improving accuracy. In this work, we aimed to propose a decentralized privacy-aware learning mechanism to accurately predict melanoma skin cancer. In this research we analyzed federated learning from the skin cancer database. The results from the study showed that 92% accuracy was achieved by the proposed method, which was higher than baseline algorithms.

Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imaging

The collection and curation of large-scale medical datasets from multiple institutions is essential for training accurate deep learning models, but privacy concerns often hinder data sharing. Federated learning (FL) is a promising solution that enables privacy-preserving collaborative learning among different institutions, but it generally suffers from performance deterioration due to heterogeneous data distributions and a lack of quality labeled data. In this paper, we present a robust and label-efficient self-supervised FL framework for medical image analysis. Our method introduces a novel Transformer-based self-supervised pre-training paradigm that pre-trains models directly on decentralized target task datasets using masked image modeling, to facilitate more robust representation learning on heterogeneous data and effective knowledge transfer to downstream models. Extensive empirical results on simulated and real-world medical imaging non-IID federated datasets show that masked image modeling with Transformers significantly improves the robustness of models against various degrees of data heterogeneity. Notably, under severe data heterogeneity, our method, without relying on any additional pre-training data, achieves an improvement of 5.06%, 1.53% and 4.58% in test accuracy on retinal, dermatology and chest X-ray classification compared to the supervised baseline with ImageNet pre-training. In addition, we show that our federated self-supervised pre-training methods yield models that generalize better to out-of-distribution data and perform more effectively when fine-tuning with limited labeled data, compared to existing FL algorithms. The code is available at https://github.com/rui-yan/SSL-FL.

Recent Advances in Melanoma Diagnosis and Prognosis Using Machine Learning Methods

PURPOSE OF REVIEW

The purpose was to summarize the current role and state of artificial intelligence and machine learning in the diagnosis and management of melanoma.

RECENT FINDINGS

Deep learning algorithms can identify melanoma from clinical, dermoscopic, and whole slide pathology images with increasing accuracy. Efforts to provide more granular annotation to datasets and to identify new predictors are ongoing. There have been many incremental advances in both melanoma diagnostics and prognostic tools using artificial intelligence and machine learning. Higher quality input data will further improve these models' capabilities.

A Deep Learning Fusion Approach to Diagnosis the Polycystic Ovary Syndrome (PCOS)

One of the leading causes of female infertility is PCOS, which is a hormonal disorder affecting women of childbearing age. The common symptoms of PCOS include increased acne, irregular period, increase in body hair, and overweight. Early diagnosis of PCOS is essential to manage the symptoms and reduce the associated health risks. Nonetheless, the diagnosis is based on Rotterdam criteria, including a high level of androgen hormones, ovulation failure, and polycystic ovaries on the ultrasound image (PCOM). At present, doctors and radiologists manually perform PCOM detection using ovary ultrasound by counting the number of follicles and determining their volume in the ovaries, which is one of the challenging PCOS diagnostic criteria. Moreover, such physicians require more tests and checks for biochemical/clinical signs in addition to the patient’s symptoms in order to decide the PCOS diagnosis. Furthermore, clinicians do not utilize a single diagnostic test or specific method to examine patients. This paper introduces the data set that includes the ultrasound image of the ovary with clinical data related to the patient that has been classified as PCOS and non-PCOS. Next, we proposed a deep learning model that can diagnose the PCOM based on the ultrasound image, which achieved 84.81% accuracy using the Inception model. Then, we proposed a fusion model that includes the ultrasound image with clinical data to diagnose the patient if they have PCOS or not. The best model that has been developed achieved 82.46% accuracy by extracting the image features using MobileNet architecture and combine with clinical features.

A study on skin tumor classification based on dense convolutional networks with fused metadata

Skin cancer is the most common cause of death in humans. Statistics show that competent dermatologists have a diagnostic accuracy rate of less than 80%, while inexperienced dermatologists have a diagnostic accuracy rate of less than 60%. The higher rate of misdiagnosis will cause many patients to miss the most effective treatment window, risking the patients' life safety. However, the majority of the current study of neural network-based skin cancer diagnosis remains at the image level without patient clinical data. A deep convolutional network incorporating clinical patient metadata of skin cancer is presented to realize the classification model of skin cancer in order to further increase the accuracy of skin cancer diagnosis. There are three basic steps in the approach. First, the high-level features (edge features, color features, texture features, form features, etc.). Implied by the image were retrieved using the pre-trained DenseNet-169 model on the ImageNet dataset. Second, the MetaNet module is introduced, which uses metadata to control a certain portion of each feature channel in the DenseNet-169 network in order to produce weighted features. The MetaBlock module was added at the same time to improve the features retrieved from photos using metadata, choosing the most pertinent characteristics in accordance with the metadata data. The features of the MetaNet and MetaBlock modules were finally combined to create the MD-Net module, which was then used as input into the classifier to get the classification results for skin cancers. On the PAD-UFES-20 and ISIC 2019 datasets, the suggested methodology was assessed. The DenseNet-169 network model combined with this module, according to experimental data, obtains 81.4% in the balancing accuracy index, and its diagnostic accuracy is up between 8% and 15.6% compared to earlier efforts. Additionally, it solves the problem of actinic keratosis and poorly classified skin fibromas.

Classification of skin lesion images using modified Inception V3 model with transfer learning and augmentation techniques

In this article, a methodological approach to classifying malignant melanoma in dermoscopy images is presented. Early treatment of skin cancer increases the patient’s survival rate. The classification of melanoma skin cancer in the early stages is decided by dermatologists to treat the patient appropriately. Dermatologists need more time to diagnose affected skin lesions due to high resemblance between melanoma and benign. In this paper, a deep learning based Computer-Aided Diagnosis (CAD) system is developed to accurately classify skin lesions with a high classification rate. A new architecture has been framed to classify the skin lesion diseases using the Inception v3 model as a baseline architecture. The extracted features from the Inception Net are then flattened and are given to the DenseNet block to extracts more fine grained features of the lesion disease. The International Skin Imaging Collaboration (ISIC) archive datasets contains 3307 dermoscopy images which includes both benign and malignant skin images. The dataset images are trained using the proposed architecture with the learning rate of 0.0001, batch size 64 using various optimizer. The performance of the proposed model has also been evaluated using confusion matrix and ROC-AUC curves. The experimental results show that the proposed model attains a highest accuracy rate of 91.29 % compared to other state-of-the-art methods like ResNet, VGG-16, DenseNet, MobileNet. A confusion matrix and ROC curve are used to evaluate the performance analysis of skin images. The classification accuracy, sensitivity, specificity, testing accuracy, and AUC values were obtained at 90.33%, 82.87%, 91.29%, 87.12%, and 87.40% .

Melanoma Detection Using Deep Learning-Based Classifications

One of the most prevalent cancers worldwide is skin cancer, and it is becoming more common as the population ages. As a general rule, the earlier skin cancer can be diagnosed, the better. As a result of the success of deep learning (DL) algorithms in other industries, there has been a substantial increase in automated diagnosis systems in healthcare. This work proposes DL as a method for extracting a lesion zone with precision. First, the image is enhanced using Enhanced Super-Resolution Generative Adversarial Networks (ESRGAN) to improve the image's quality. Then, segmentation is used to segment Regions of Interest (ROI) from the full image. We employed data augmentation to rectify the data disparity. The image is then analyzed with a convolutional neural network (CNN) and a modified version of Resnet-50 to classify skin lesions. This analysis utilized an unequal sample of seven kinds of skin cancer from the HAM10000 dataset. With an accuracy of 0.86, a precision of 0.84, a recall of 0.86, and an F-score of 0.86, the proposed CNN-based Model outperformed the earlier study's results by a significant margin. The study culminates with an improved automated method for diagnosing skin cancer that benefits medical professionals and patients.

Skin Cancer Detection Based on Deep Learning

Background

The conventional procedure of skin-related disease detection is a visual inspection by a dermatologist or a primary care clinician, using a dermatoscope. The suspected patients with early signs of skin cancer are referred for biopsy and histopathological examination to ensure the correct diagnosis and the best treatment. Recent advancements in deep convolutional neural networks (CNNs) have achieved excellent performance in automated skin cancer classification with accuracy similar to that of dermatologists. However, such improvements are yet to bring about a clinically trusted and popular system for skin cancer detection.

Objective

This study aimed to propose viable deep learning (DL) based method for the detection of skin cancer in lesion images, to help physicians in diagnosis.

Material and Methods

In this analytical study, a novel DL based model was proposed, in which other than the lesion image, the patient's data, including the anatomical site of the lesion, age, and gender were used as the model input to predict the type of the lesion. An Inception-ResNet-v2 CNN pretrained for object recognition was employed in the proposed model.

Results

Based on the results, the proposed method achieved promising performance for various skin conditions, and also using the patient's metadata in addition to the lesion image for classification improved the classification accuracy by at least 5% in all cases investigated. On a dataset of 57536 dermoscopic images, the proposed approach achieved an accuracy of 89.3%±1.1% in the discrimination of 4 major skin conditions and 94.5%±0.9% in the classification of benign vs. malignant lesions.

Conclusion

The promising results highlight the efficacy of the proposed approach and indicate that the inclusion of the patient's metadata with the lesion image can enhance the skin cancer detection performance.

Attention Cost-Sensitive Deep Learning-Based Approach for Skin Cancer Detection and Classification

Deep learning-based models have been employed for the detection and classification of skin diseases through medical imaging. However, deep learning-based models are not effective for rare skin disease detection and classification. This is mainly due to the reason that rare skin disease has very a smaller number of data samples. Thus, the dataset will be highly imbalanced, and due to the bias in learning, most of the models give better performances. The deep learning models are not effective in detecting the affected tiny portions of skin disease in the overall regions of the image. This paper presents an attention-cost-sensitive deep learning-based feature fusion ensemble meta-classifier approach for skin cancer detection and classification. Cost weights are included in the deep learning models to handle the data imbalance during training. To effectively learn the optimal features from the affected tiny portions of skin image samples, attention is integrated into the deep learning models. The features from the finetuned models are extracted and the dimensionality of the features was further reduced by using a kernel-based principal component (KPCA) analysis. The reduced features of the deep learning-based finetuned models are fused and passed into ensemble meta-classifiers for skin disease detection and classification. The ensemble meta-classifier is a two-stage model. The first stage performs the prediction of skin disease and the second stage performs the classification by considering the prediction of the first stage as features. Detailed analysis of the proposed approach is demonstrated for both skin disease detection and skin disease classification. The proposed approach demonstrated an accuracy of 99% on skin disease detection and 99% on skin disease classification. In all the experimental settings, the proposed approach outperformed the existing methods and demonstrated a performance improvement of 4% accuracy for skin disease detection and 9% accuracy for skin disease classification. The proposed approach can be used as a computer-aided diagnosis (CAD) tool for the early diagnosis of skin cancer detection and classification in healthcare and medical environments. The tool can accurately detect skin diseases and classify the skin disease into their skin disease family.

Improvement of Mucosal Lesion Diagnosis with Machine Learning Based on Medical and Semiological Data: An Observational Study

Despite artificial intelligence used in skin dermatology diagnosis is booming, application in oral pathology remains to be developed. Early diagnosis and therefore early management, remain key points in the successful management of oral mucosa cancers. The objective was to develop and evaluate a machine learning algorithm that allows the prediction of oral mucosa lesions diagnosis. This cohort study included patients followed between January 2015 and December 2020 in the oral mucosal pathology consultation of the Toulouse University Hospital. Photographs and demographic and medical data were collected from each patient to constitute clinical cases. A machine learning model was then developed and optimized and compared to 5 models classically used in the field. A total of 299 patients representing 1242 records of oral mucosa lesions were used to train and evaluate machine learning models. Our model reached a mean accuracy of 0.84 for diagnostic prediction. The specificity and sensitivity range from 0.89 to 1.00 and 0.72 to 0.92, respectively. The other models were proven to be less efficient in performing this task. These results suggest the utility of machine learning-based tools in diagnosing oral mucosal lesions with high accuracy. Moreover, the results of this study confirm that the consideration of clinical data and medical history, in addition to the lesion itself, appears to play an important role.

Combination of Optical Biopsy with Patient Data for Improvement of Skin Tumor Identification

In this study, patient data were combined with Raman and autofluorescence spectral parameters for more accurate identification of skin tumors. The spectral and patient data of skin tumors were classified by projection on latent structures and discriminant analysis. The importance of patient risk factors was determined using statistical improvement of ROC AUCs when spectral parameters were combined with risk factors. Gender, age and tumor localization were found significant for classification of malignant versus benign neoplasms, resulting in improvement of ROC AUCs from 0.610 to 0.818 (p < 0.05). To distinguish melanoma versus pigmented skin tumors, the same factors significantly improved ROC AUCs from 0.709 to 0.810 (p < 0.05) when analyzed together according to the spectral data, but insignificantly (p > 0.05) when analyzed individually. For classification of melanoma versus seborrheic keratosis, no statistical improvement of ROC AUC was observed when the patient data were added to the spectral data. In all three classification models, additional risk factors such as occupational hazards, family history, sun exposure, size, and personal history did not statistically improve the ROC AUCs. In summary, combined analysis of spectral and patient data can be significant for certain diagnostic tasks: patient data demonstrated the distribution of skin tumor incidence in different demographic groups, whereas tumors within each group were distinguished using the spectral differences.

MobileSkin: Classification of Skin Lesion Images Acquired Using Mobile Phone-Attached Hand-Held Dermoscopes

Dermoscopy is the visual examination of the skin under a polarized or non-polarized light source. By using dermoscopic equipment, many lesion patterns that are invisible under visible light can be clearly distinguished. Thus, more accurate decisions can be made regarding the treatment of skin lesions. The use of images collected from a dermoscope has both increased the performance of human examiners and allowed the development of deep learning models. The availability of large-scale dermoscopic datasets has allowed the development of deep learning models that can classify skin lesions with high accuracy. However, most dermoscopic datasets contain images that were collected from digital dermoscopic devices, as these devices are frequently used for clinical examination. However, dermatologists also often use non-digital hand-held (optomechanical) dermoscopes. This study presents a dataset consisting of dermoscopic images taken using a mobile phone-attached hand-held dermoscope. Four deep learning models based on the MobileNetV1, MobileNetV2, NASNetMobile, and Xception architectures have been developed to classify eight different lesion types using this dataset. The number of images in the dataset was increased with different data augmentation methods. The models were initialized with weights that were pre-trained on the ImageNet dataset, and then they were further fine-tuned using the presented dataset. The most successful models on the unseen test data, MobileNetV2 and Xception, had performances of 89.18% and 89.64%. The results were evaluated with the 5-fold cross-validation method and compared. Our method allows for automated examination of dermoscopic images taken with mobile phone-attached hand-held dermoscopes.

MDFNet: application of multimodal fusion method based on skin image and clinical data to skin cancer classification

PURPOSE

Skin cancer is one of the ten most common cancer types in the world. Early diagnosis and treatment can effectively reduce the mortality of patients. Therefore, it is of great significance to develop an intelligent diagnosis system for skin cancer. According to the survey, at present, most intelligent diagnosis systems of skin cancer only use skin image data, but the multi-modal cross-fusion analysis using image data and patient clinical data is limited. Therefore, to further explore the complementary relationship between image data and patient clinical data, we propose multimode data fusion diagnosis network (MDFNet), a framework for skin cancer based on data fusion strategy.

METHODS

MDFNet establishes an effective mapping among heterogeneous data features, effectively fuses clinical skin images and patient clinical data, and effectively solves the problems of feature paucity and insufficient feature richness that only use single-mode data.

RESULTS

The experimental results present that our proposed smart skin cancer diagnosis model has an accuracy of 80.42%, which is an improvement of about 9% compared with the model accuracy using only medical images, thus effectively confirming the unique fusion advantages exhibited by MDFNet.

CONCLUSIONS

This illustrates that MDFNet can not only be applied as an effective auxiliary diagnostic tool for skin cancer diagnosis, help physicians improve clinical decision-making ability and effectively improve the efficiency of clinical medicine diagnosis, but also its proposed data fusion method fully exerts the advantage of information convergence and has a certain reference value for the intelligent diagnosis of numerous clinical diseases.

Classification of skin cancer using convolutional neural networks analysis of Raman spectra

BACKGROUND AND OBJECTIVE

Skin cancer is the most common malignancy in whites accounting for about one third of all cancers diagnosed per year. Portable Raman spectroscopy setups for skin cancer "optical biopsy" are utilized to detect tumors based on their spectral features caused by the comparative presence of different chemical components. However, low signal-to-noise ratio in such systems may prevent accurate tumors classification. Thus, there is a challenge to develop methods for efficient skin tumors classification.

METHODS

We compare the performance of convolutional neural networks and the projection on latent structures with discriminant analysis for discriminating skin cancer using the analysis of Raman spectra with a high autofluorescence background stimulated by a 785 nm laser. We have registered the spectra of 617 cases of skin neoplasms (615 patients, 70 melanomas, 122 basal cell carcinomas, 12 squamous cell carcinomas and 413 benign tumors) in vivo with a portable Raman setup and created classification models both for convolutional neural networks and projection on latent structures approaches. To check the classification models stability, a 10-fold cross-validation was performed for all created models. To avoid models overfitting, the data was divided into a training set (80% of spectral dataset) and a test set (20% of spectral dataset).

RESULTS

The results for different classification tasks demonstrate that the convolutional neural networks significantly (p<0.01) outperforms the projection on latent structures. For the convolutional neural networks implementation we obtained ROC AUCs of 0.96 (0.94 - 0.97; 95% CI), 0.90 (0.85-0.94; 95% CI), and 0.92 (0.87 - 0.97; 95% CI) for classifying a) malignant vs benign tumors, b) melanomas vs pigmented tumors and c) melanomas vs seborrheic keratosis respectively.

CONCLUSIONS

The performance of the convolutional neural networks classification of skin tumors based on Raman spectra analysis is higher or comparable to the accuracy provided by trained dermatologists. The increased accuracy with the convolutional neural networks implementation is due to a more precise accounting of low intensity Raman bands in the intense autofluorescence background. The achieved high performance of skin tumors classifications with convolutional neural networks analysis opens a possibility for wide implementation of Raman setups in clinical setting.

Skin Lesion Analysis for Melanoma Detection Using the Novel Deep Learning Model Fuzzy GC-SCNN

Melanoma is easily detectable by visual examination since it occurs on the skin’s surface. In melanomas, which are the most severe types of skin cancer, the cells that make melanin are affected. However, the lack of expert opinion increases the processing time and cost of computer-aided skin cancer detection. As such, we aimed to incorporate deep learning algorithms to conduct automatic melanoma detection from dermoscopic images. The fuzzy-based GrabCut-stacked convolutional neural networks (GC-SCNN) model was applied for image training. The image features extraction and lesion classification were performed on different publicly available datasets. The fuzzy GC-SCNN coupled with the support vector machines (SVM) produced 99.75% classification accuracy and 100% sensitivity and specificity, respectively. Additionally, model performance was compared with existing techniques and outcomes suggesting the proposed model could detect and classify the lesion segments with higher accuracy and lower processing time than other techniques.

The Application of Differing Machine Learning Algorithms and Their Related Performance in Detecting Skin Cancers and Melanomas

Skin cancer, and its less common form melanoma, is a disease affecting a wide variety of people. Since it is usually detected initially by visual inspection, it makes for a good candidate for the application of machine learning. With early detection being key to good outcomes, any method that can enhance the diagnostic accuracy of dermatologists and oncologists is of significant interest. When comparing different existing implementations of machine learning against public datasets and several we seek to create, we attempted to create a more accurate model that can be readily adapted to use in clinical settings. We tested combinations of models, including convolutional neural networks (CNNs), and various layers of data manipulation, such as the application of Gaussian functions and trimming of images to improve accuracy. We also created more traditional data models, including support vector classification, K-nearest neighbor, Naïve Bayes, random forest, and gradient boosting algorithms, and compared them to the CNN-based models we had created. Results had indicated that CNN-based algorithms significantly outperformed other data models we had created. Partial results of this work were presented at the CSET Presentations for Research Month at the Minnesota State University, Mankato.

Automatic skin disease diagnosis using deep learning from clinical image and patient information

Background

Skin diseases are the fourth most common cause of human illness which results in enormous non-fatal burden in daily life activities. They are caused by chemical, physical and biological factors. Visual assessment in combination with clinical information is the common diagnostic procedure for diseases. However, these procedures are manual, time-consuming, and require experience and excellent visual perception.

Objectives

In this study, an automated system is proposed for the diagnosis of five common skin diseases by using data from clinical images and patient information using deep learning pre-trained mobilenet-v2 model.

Methods

Clinical images were acquired using different smartphone cameras and patient's information were collected during patient registration. Different data preprocessing and augmentation techniques were applied to boost the performance of the model prior to training.

Results

A multiclass classification accuracy of 97.5%, sensitivity of 97.7% and precision of 97.7% has been achieved using the proposed technique for the common five skin disease. The results demonstrate that, the developed system provides excellent diagnosis performance for the five skin diseases.

Conclusion

The system has been designed as a smartphone application and it has the potential to be used as a decision support system in low resource settings, where both the expert dermatologist and the means are limited.

InSiNet: a deep convolutional approach to skin cancer detection and segmentation

Cancer is among the common causes of death around the world. Skin cancer is one of the most lethal types of cancer. Early diagnosis and treatment are vital in skin cancer. In addition to traditional methods, method such as deep learning is frequently used to diagnose and classify the disease. Expert experience plays a major role in diagnosing skin cancer. Therefore, for more reliable results in the diagnosis of skin lesions, deep learning algorithms can help in the correct diagnosis. In this study, we propose InSiNet, a deep learning-based convolutional neural network to detect benign and malignant lesions. The performance of the method is tested on International Skin Imaging Collaboration HAM10000 images (ISIC 2018), ISIC 2019, and ISIC 2020, under the same conditions. The computation time and accuracy comparison analysis was performed between the proposed algorithm and other machine learning techniques (GoogleNet, DenseNet-201, ResNet152V2, EfficientNetB0, RBF-support vector machine, logistic regression, and random forest). The results show that the developed InSiNet architecture outperforms the other methods achieving an accuracy of 94.59%, 91.89%, and 90.54% in ISIC 2018, 2019, and 2020 datasets, respectively. Since the deep learning algorithms eliminate the human factor during diagnosis, they can give reliable results in addition to traditional methods.

Modified PNN classifier for diagnosing skin cancer severity condition using SMO optimization technique

<abstract> <p>Skin cancer is a pandemic disease now worldwide, and it is responsible for numerous deaths. Early phase detection is pre-eminent for controlling the spread of tumours throughout the body. However, existing algorithms for skin cancer severity detections still have some drawbacks, such as the analysis of skin lesions is not insignificant, slightly worse than that of dermatologists, and costly and time-consuming. Various machine learning algorithms have been used to detect the severity of the disease diagnosis. But it is more complex when detecting the disease. To overcome these issues, a modified Probabilistic Neural Network (MPNN) classifier has been proposed to determine the severity of skin cancer. The proposed method contains two phases such as training and testing the data. The collected features from the data of infected people are used as input to the modified PNN classifier in the current model. The neural network is also trained using Spider Monkey Optimization (SMO) approach. For analyzing the severity level, the classifier predicts four classes. The degree of skin cancer is determined depending on classifications. According to findings, the system achieved a 0.10% False Positive Rate (FPR), 0.03% error and 0.98% accuracy, while previous methods like KNN, NB, RF and SVM have accuracies of 0.90%, 0.70%, 0.803% and 0.86% correspondingly, which is lesser than the proposed approach.</p> </abstract>

Neural Network Based Filtering Method for Cancer Detection

Objective:

The growth of anomalous cells in the human body in an uncontrolled manner is characterized as cancer. The detection of cancer is a multi-stage process in the clinical examination.

Methods:

It is mainly involved with the assistance of radiological imaging. The imaging technique is used to identify the spread of cancer in the human body. This imaging-based detection can be improved by incorporating the Image Processing methodologies. In image processing, the preprocessing is applied at the lower-level abstraction. It removes the unwanted noise pixel present in the image, which also distributes the pixel values based on the specific distribution method.

Results:

Neural Network is a learning and processing engine, which is mainly used to create cognitive intelligence in various domains. In this work, the Neural Network (NN) based filtering approach is developed to improve the preprocessing operation in the cancer detection process.

Conclusion:

The performance of the proposed filtering method is compared with the existing linear and non-linear filters in terms of Mean Squared Error (MSE), Peak Signal to Noise Ratio (PSNR) and Image Enhancement Factor (IEF).

Characteristics of publicly available skin cancer image datasets: a systematic review

Publicly available skin image datasets are increasingly used to develop machine learning algorithms for skin cancer diagnosis. However, the total number of datasets and their respective content is currently unclear. This systematic review aimed to identify and evaluate all publicly available skin image datasets used for skin cancer diagnosis by exploring their characteristics, data access requirements, and associated image metadata. A combined MEDLINE, Google, and Google Dataset search identified 21 open access datasets containing 106 950 skin lesion images, 17 open access atlases, eight regulated access datasets, and three regulated access atlases. Images and accompanying data from open access datasets were evaluated by two independent reviewers. Among the 14 datasets that reported country of origin, most (11 [79%]) originated from Europe, North America, and Oceania exclusively. Most datasets (19 [91%]) contained dermoscopic images or macroscopic photographs only. Clinical information was available regarding age for 81 662 images (76·4%), sex for 82 848 (77·5%), and body site for 79 561 (74·4%). Subject ethnicity data were available for 1415 images (1·3%), and Fitzpatrick skin type data for 2236 (2·1%). There was limited and variable reporting of characteristics and metadata among datasets, with substantial under-representation of darker skin types. This is the first systematic review to characterise publicly available skin image datasets, highlighting limited applicability to real-life clinical settings and restricted population representation, precluding generalisability. Quality standards for characteristics and metadata reporting for skin image datasets are needed.

Decision and feature level fusion of deep features extracted from public COVID-19 data-sets

The Coronavirus disease (COVID-19), which is an infectious pulmonary disorder, has affected millions of people and has been declared as a global pandemic by the WHO. Due to highly contagious nature of COVID-19 and its high possibility of causing severe conditions in the patients, the development of rapid and accurate diagnostic tools have gained importance. The real-time reverse transcription-polymerize chain reaction (RT-PCR) is used to detect the presence of Coronavirus RNA by using the mucus and saliva mixture samples taken by the nasopharyngeal swab technique. But, RT-PCR suffers from having low-sensitivity especially in the early stage. Therefore, the usage of chest radiography has been increasing in the early diagnosis of COVID-19 due to its fast imaging speed, significantly low cost and low dosage exposure of radiation. In our study, a computer-aided diagnosis system for X-ray images based on convolutional neural networks (CNNs) and ensemble learning idea, which can be used by radiologists as a supporting tool in COVID-19 detection, has been proposed. Deep feature sets extracted by using seven CNN architectures were concatenated for feature level fusion and fed to multiple classifiers in terms of decision level fusion idea with the aim of discriminating COVID-19, pneumonia and no-finding classes. In the decision level fusion idea, a majority voting scheme was applied to the resultant decisions of classifiers. The obtained accuracy values and confusion matrix based evaluation criteria were presented for three progressively created data-sets. The aspects of the proposed method that are superior to existing COVID-19 detection studies have been discussed and the fusion performance of proposed approach was validated visually by using Class Activation Mapping technique. The experimental results show that the proposed approach has attained high COVID-19 detection performance that was proven by its comparable accuracy and superior precision/recall values with the existing studies.

Incorporating clinical knowledge with constrained classifier chain into a multimodal deep network for melanoma detection

In recent years, vast developments in Computer-Aided Diagnosis (CAD) for skin diseases have generated much interest from clinicians and other eventual end-users of this technology. Introducing clinical domain knowledge to these machine learning strategies can help dispel the black box nature of these tools, strengthening clinician trust. Clinical domain knowledge also provides new information channels which can improve CAD diagnostic performance. In this paper, we propose a novel framework for malignant melanoma (MM) detection by fusing clinical images and dermoscopic images. The proposed method combines a multi-labeled deep feature extractor and clinically constrained classifier chain (CC). This allows the 7-point checklist, a clinician diagnostic algorithm, to be included in the decision level while maintaining the clinical importance of the major and minor criteria in the checklist. Our proposed framework achieved an average accuracy of 81.3% for detecting all criteria and melanoma when testing on a publicly available 7-point checklist dataset. This is the highest reported results, outperforming state-of-the-art methods in the literature by 6.4% or more. Analyses also show that the proposed system surpasses the single modality system of using either clinical images or dermoscopic images alone and the systems without adopting the approach of multi-label and clinically constrained classifier chain. Our carefully designed system demonstrates a substantial improvement over melanoma detection. By keeping the familiar major and minor criteria of the 7-point checklist and their corresponding weights, the proposed system may be more accepted by physicians as a human-interpretable CAD tool for automated melanoma detection.

Effect of Patient Clinical Variables in Osteoporosis Classification Using Hip X-rays in Deep Learning Analysis

Background and Objectives: A few deep learning studies have reported that combining image features with patient variables enhanced identification accuracy compared with image-only models. However, previous studies have not statistically reported the additional effect of patient variables on the image-only models. This study aimed to statistically evaluate the osteoporosis identification ability of deep learning by combining hip radiographs with patient variables. Materials andMethods: We collected a dataset containing 1699 images from patients who underwent skeletal-bone-mineral density measurements and hip radiography at a general hospital from 2014 to 2021. Osteoporosis was assessed from hip radiographs using convolutional neural network (CNN) models (ResNet18, 34, 50, 101, and 152). We also investigated ensemble models with patient clinical variables added to each CNN. Accuracy, precision, recall, specificity, F1 score, and area under the curve (AUC) were calculated as performance metrics. Furthermore, we statistically compared the accuracy of the image-only model with that of an ensemble model that included images plus patient factors, including effect size for each performance metric. Results: All metrics were improved in the ResNet34 ensemble model compared with the image-only model. The AUC score in the ensemble model was significantly improved compared with the image-only model (difference 0.004; 95% CI 0.002–0.0007; p = 0.0004, effect size: 0.871). Conclusions: This study revealed the additional effect of patient variables in identification of osteoporosis using deep CNNs with hip radiographs. Our results provided evidence that the patient variables had additive synergistic effects on the image in osteoporosis identification.

Integrating Patient Data Into Skin Cancer Classification Using Convolutional Neural Networks: Systematic Review

BACKGROUND

Recent years have been witnessing a substantial improvement in the accuracy of skin cancer classification using convolutional neural networks (CNNs). CNNs perform on par with or better than dermatologists with respect to the classification tasks of single images. However, in clinical practice, dermatologists also use other patient data beyond the visual aspects present in a digitized image, further increasing their diagnostic accuracy. Several pilot studies have recently investigated the effects of integrating different subtypes of patient data into CNN-based skin cancer classifiers.

OBJECTIVE

This systematic review focuses on the current research investigating the impact of merging information from image features and patient data on the performance of CNN-based skin cancer image classification. This study aims to explore the potential in this field of research by evaluating the types of patient data used, the ways in which the nonimage data are encoded and merged with the image features, and the impact of the integration on the classifier performance.

METHODS

Google Scholar, PubMed, MEDLINE, and ScienceDirect were screened for peer-reviewed studies published in English that dealt with the integration of patient data within a CNN-based skin cancer classification. The search terms skin cancer classification, convolutional neural network(s), deep learning, lesions, melanoma, metadata, clinical information, and patient data were combined.

RESULTS

A total of 11 publications fulfilled the inclusion criteria. All of them reported an overall improvement in different skin lesion classification tasks with patient data integration. The most commonly used patient data were age, sex, and lesion location. The patient data were mostly one-hot encoded. There were differences in the complexity that the encoded patient data were processed with regarding deep learning methods before and after fusing them with the image features for a combined classifier.

CONCLUSIONS

This study indicates the potential benefits of integrating patient data into CNN-based diagnostic algorithms. However, how exactly the individual patient data enhance classification performance, especially in the case of multiclass classification problems, is still unclear. Moreover, a substantial fraction of patient data used by dermatologists remains to be analyzed in the context of CNN-based skin cancer classification. Further exploratory analyses in this promising field may optimize patient data integration into CNN-based skin cancer diagnostics for patients' benefits.