Using deep learning for dermatologist-level detection of suspicious pigmented skin lesions from wide-field images

A survey on cell nuclei instance segmentation and classification: Leveraging context and attention

Nuclear-derived morphological features and biomarkers provide relevant insights regarding the tumour microenvironment, while also allowing diagnosis and prognosis in specific cancer types. However, manually annotating nuclei from the gigapixel Haematoxylin and Eosin (H&E)-stained Whole Slide Images (WSIs) is a laborious and costly task, meaning automated algorithms for cell nuclei instance segmentation and classification could alleviate the workload of pathologists and clinical researchers and at the same time facilitate the automatic extraction of clinically interpretable features for artificial intelligence (AI) tools. But due to high intra- and inter-class variability of nuclei morphological and chromatic features, as well as H&E-stains susceptibility to artefacts, state-of-the-art algorithms cannot correctly detect and classify instances with the necessary performance. In this work, we hypothesize context and attention inductive biases in artificial neural networks (ANNs) could increase the performance and generalization of algorithms for cell nuclei instance segmentation and classification. To understand the advantages, use-cases, and limitations of context and attention-based mechanisms in instance segmentation and classification, we start by reviewing works in computer vision and medical imaging. We then conduct a thorough survey on context and attention methods for cell nuclei instance segmentation and classification from H&E-stained microscopy imaging, while providing a comprehensive discussion of the challenges being tackled with context and attention. Besides, we illustrate some limitations of current approaches and present ideas for future research. As a case study, we extend both a general (Mask-RCNN) and a customized (HoVer-Net) instance segmentation and classification methods with context- and attention-based mechanisms and perform a comparative analysis on a multicentre dataset for colon nuclei identification and counting. Although pathologists rely on context at multiple levels while paying attention to specific Regions of Interest (RoIs) when analysing and annotating WSIs, our findings suggest translating that domain knowledge into algorithm design is no trivial task, but to fully exploit these mechanisms in ANNs, the scientific understanding of these methods should first be addressed.

The Role of Digital Dermoscopy and Follow-Up in the Detection of Amelanotic/Hypomelanotic Melanoma in a Group of High-Risk Patients-Is It Useful?

The prognosis, outcome, and overall survival of melanoma patients improve with early diagnosis which has been facilitated in the past few decades with the introduction of dermoscopy. Further advancements in dermoscopic research, coupled with skilled, educated dermatologists in dermoscopy, have contributed to timely diagnoses. However, detecting amelanotic and hypomelanotic melanoma remains a challenge even to the most skilled experts because these melanomas can mimic inflammatory diseases, numerous benign lesions, and non-melanoma skin cancers. The list of the possible differential diagnoses can be long. Melanoma prediction without the pigment relies only on vascular criteria, and all classic dermoscopic algorithms have failed to fulfill our expectations. In fact, the diagnosis of amelanotic and hypomelanotic melanomas is very challenging, which is why every tool in detecting these lesions is of significance. This review aims to explore the current knowledge and the literature on the possibility of detecting amelanotic/hypomelanotic melanomas using sequential monitoring with digital dermoscopy and total body skin photography.

The Role of Artificial Intelligence in the Diagnosis of Melanoma

The incidence of melanoma, the most aggressive form of skin cancer, continues to rise globally, particularly among fair-skinned populations (type I and II). Early detection is crucial for improving patient outcomes, and recent advancements in artificial intelligence (AI) have shown promise in enhancing the accuracy and efficiency of melanoma diagnosis and management. This review examines the role of AI in skin lesion diagnostics, highlighting two main approaches: machine learning, particularly convolutional neural networks (CNNs), and expert systems. AI techniques have demonstrated high accuracy in classifying dermoscopic images, often matching or surpassing dermatologists' performance. Integrating AI into dermatology has improved tasks, such as lesion classification, segmentation, and risk prediction, facilitating earlier and more accurate interventions. Despite these advancements, challenges remain, including biases in training data, interpretability issues, and integration of AI into clinical workflows. Ensuring diverse data representation and maintaining high standards of image quality are essential for reliable AI performance. Future directions involve the development of more sophisticated models, such as vision-language and multimodal models, and federated learning to address data privacy and generalizability concerns. Continuous validation and ethical integration of AI into clinical practice are vital for realizing its full potential for improving melanoma diagnosis and patient care.

Hardware-accelerated integrated optoelectronic platform towards real-time high-resolution hyperspectral video understanding

Recent advancements in artificial intelligence have significantly expanded capabilities in processing language and images. However, the challenge of comprehensively understanding video content still needs to be solved. The main problem is the requirement to process real-time multidimensional video information at data rates exceeding 1 Tb/s, a demand that current hardware technologies cannot meet. This work introduces a hardware-accelerated integrated optoelectronic platform specifically designed for the real-time analysis of multidimensional video. By leveraging optical information processing within artificial intelligence hardware and combining it with advanced machine vision networks, the platform achieves data processing speeds of 1.2 Tb/s. This capability supports the analysis of hundreds of frequency bands with megapixel spatial resolution at video frame rates, significantly outperforming existing technologies in speed by three to four orders of magnitude. The platform demonstrates effectiveness for AI-driven tasks, such as video semantic segmentation and object understanding, across indoor and aerial scenarios. By overcoming the current data processing speed limitations, the platform shows promise in real-time AI video understanding, with potential implications for enhancing human-machine interactions and advancing cognitive processing technologies.

Deep Learning Algorithms for the Detection of Suspicious Pigmented Skin Lesions in Primary Care Settings: A Systematic Review and Meta-Analysis

Early detection of suspicious pigmented skin lesions is crucial for improving the outcomes and survival rates of skin cancers. However, the accuracy of clinical diagnosis by primary care physicians (PCPs) is suboptimal, leading to unnecessary referrals and biopsies. In recent years, deep learning (DL) algorithms have shown promising results in the automated detection and classification of skin lesions. This systematic review and meta-analysis aimed to evaluate the diagnostic performance of DL algorithms for the detection of suspicious pigmented skin lesions in primary care settings. A comprehensive literature search was conducted using electronic databases, including PubMed, Scopus, IEEE Xplore, Cochrane Central Register of Controlled Trials (CENTRAL), and Web of Science. Data from eligible studies were extracted, including study characteristics, sample size, algorithm type, sensitivity, specificity, diagnostic odds ratio (DOR), positive likelihood ratio (PLR), negative likelihood ratio (NLR), and receiver operating characteristic curve analysis. Three studies were included. The results showed that DL algorithms had a high sensitivity (90%, 95% CI: 90-91%) and specificity (85%, 95% CI: 84-86%) for detecting suspicious pigmented skin lesions in primary care settings. Significant heterogeneity was observed in both sensitivity (p = 0.0062, I2 = 80.3%) and specificity (p < 0.001, I2 = 98.8%). The analysis of DOR and PLR further demonstrated the strong diagnostic performance of DL algorithms. The DOR was 26.39, indicating a strong overall diagnostic performance of DL algorithms. The PLR was 4.30, highlighting the ability of these algorithms to influence diagnostic outcomes positively. The NLR was 0.16, indicating that a negative test result decreased the odds of misdiagnosis. The area under the curve of DL algorithms was 0.95, indicating excellent discriminative ability in distinguishing between benign and malignant pigmented skin lesions. DL algorithms have the potential to significantly improve the detection of suspicious pigmented skin lesions in primary care settings. Our analysis showed that DL exhibited promising performance in the early detection of suspicious pigmented skin lesions. However, further studies are needed.

From data to diagnosis: skin cancer image datasets for artificial intelligence

Artificial intelligence (AI) solutions for skin cancer diagnosis continue to gain momentum, edging closer towards broad clinical use. These AI models, particularly deep-learning architectures, require large digital image datasets for development. This review provides an overview of the datasets used to develop AI algorithms and highlights the importance of dataset transparency for the evaluation of algorithm generalizability across varying populations and settings. Current challenges for curation of clinically valuable datasets are detailed, which include dataset shifts arising from demographic variations and differences in data collection methodologies, along with inconsistencies in labelling. These shifts can lead to differential algorithm performance, compromise of clinical utility, and the propagation of discriminatory biases when developed algorithms are implemented in mismatched populations. Limited representation of rare skin cancers and minoritized groups in existing datasets are highlighted, which can further skew algorithm performance. Strategies to address these challenges are presented, which include improving transparency, representation and interoperability. Federated learning and generative methods, which may improve dataset size and diversity without compromising privacy, are also examined. Lastly, we discuss model-level techniques that may address biases entrained through the use of datasets derived from routine clinical care. As the role of AI in skin cancer diagnosis becomes more prominent, ensuring the robustness of underlying datasets is increasingly important.

The Use of Artificial Intelligence for Skin Disease Diagnosis in Primary Care Settings: A Systematic Review

The prevalence of dermatological conditions in primary care, coupled with challenges such as dermatologist shortages and rising consultation costs, highlights the need for innovative solutions. Artificial intelligence (AI) holds promise for improving the diagnostic analysis of skin lesion images, potentially enhancing patient care in primary settings. This systematic review following PRISMA guidelines examined primary studies (2012-2022) assessing AI algorithms' diagnostic accuracy for skin diseases in primary care. Studies were screened for eligibility based on their availability in the English language and exclusion criteria, with risk of bias evaluated using QUADAS-2. PubMed, Scopus, and Web of Science were searched. Fifteen studies (2019-2022), primarily from Europe and the USA, focusing on diagnostic accuracy were included. Sensitivity ranged from 58% to 96.1%, with accuracies varying from 0.41 to 0.93. AI applications encompassed triage and diagnostic support across diverse skin conditions in primary care settings, involving both patients and primary care professionals. While AI demonstrates potential for enhancing the accuracy of skin disease diagnostics in primary care, further research is imperative to address study heterogeneity and ensure algorithm reliability across diverse populations. Future investigations should prioritise robust dataset development and consider representative patient samples. Overall, AI may improve dermatological diagnosis in primary care, but careful consideration of algorithm limitations and implementation strategies is required.

Automatized self-supervised learning for skin lesion screening

Melanoma, the deadliest form of skin cancer, has seen a steady increase in incidence rates worldwide, posing a significant challenge to dermatologists. Early detection is crucial for improving patient survival rates. However, performing total body screening (TBS), i.e., identifying suspicious lesions or ugly ducklings (UDs) by visual inspection, can be challenging and often requires sound expertise in pigmented lesions. To assist users of varying expertise levels, an artificial intelligence (AI) decision support tool was developed. Our solution identifies and characterizes UDs from real-world wide-field patient images. It employs a state-of-the-art object detection algorithm to locate and isolate all skin lesions present in a patient's total body images. These lesions are then sorted based on their level of suspiciousness using a self-supervised AI approach, tailored to the specific context of the patient under examination. A clinical validation study was conducted to evaluate the tool's performance. The results demonstrated an average sensitivity of 95% for the top-10 AI-identified UDs on skin lesions selected by the majority of experts in pigmented skin lesions. The study also found that the tool increased dermatologists' confidence when formulating a diagnosis, and the average majority agreement with the top-10 AI-identified UDs reached 100% when assisted by our tool. With the development of this AI-based decision support tool, we aim to address the shortage of specialists, enable faster consultation times for patients, and demonstrate the impact and usability of AI-assisted screening. Future developments will include expanding the dataset to include histologically confirmed melanoma and validating the tool for additional body regions.

A Narrative Review: Opportunities and Challenges in Artificial Intelligence Skin Image Analyses Using Total Body Photography

Artificial intelligence (AI) algorithms for skin lesion classification have reported accuracy at par with and even outperformance of expert dermatologists in experimental settings. However, the majority of algorithms do not represent real-world clinical approach where skin phenotype and clinical background information are considered. We review the current state of AI for skin lesion classification and present opportunities and challenges when applied to total body photography (TBP). AI in TBP analysis presents opportunities for intrapatient assessment of skin phenotype and holistic risk assessment by incorporating patient-level metadata, although challenges exist for protecting patient privacy in algorithm development and improving explainable AI methods.

Facilitating clinically relevant skin tumor diagnostics with spectroscopy-driven machine learning

In the dawning era of artificial intelligence (AI), health care stands to undergo a significant transformation with the increasing digitalization of patient data. Digital imaging, in particular, will serve as an important platform for AI to aid decision making and diagnostics. A growing number of studies demonstrate the potential of automatic pre-surgical skin tumor delineation, which could have tremendous impact on clinical practice. However, current methods rely on having ground truth images in which tumor borders are already identified, which is not clinically possible. We report a novel approach where hyperspectral images provide spectra from small regions representing healthy tissue and tumor, which are used to generate prediction maps using artificial neural networks (ANNs), after which a segmentation algorithm automatically identifies the tumor borders. This circumvents the need for ground truth images, since an ANN model is trained with data from each individual patient, representing a more clinically relevant approach.

Artificial intelligence in the detection of skin cancer: State of the art

The incidence of melanoma is increasing rapidly. This cancer has a good prognosis if detected early. For this reason, various systems of skin lesion image analysis, which support imaging diagnostics of this neoplasm, are developing very dynamically. To detect and recognize neoplastic lesions, such systems use various artificial intelligence (AI) algorithms. This area of computer science applications has recently undergone dynamic development, abounding in several solutions that are effective tools supporting diagnosticians in many medical specialties. In this contribution, a number of applications of different classes of AI algorithms for the detection of this skin melanoma are presented and evaluated. Both classic systems based on the analysis of dermatoscopic images as well as total body systems, enabling the analysis of the patient's whole body to detect moles and pathologic changes, are discussed. These increasingly popular applications that allow the analysis of lesion images using smartphones are also described. The quantitative evaluation of the discussed systems with particular emphasis on the method of validation of the implemented algorithms is presented. The advantages and limitations of AI in the analysis of lesion images are also discussed, and problems requiring a solution for more effective use of AI in dermatology are identified.

Cost-effectiveness and cost-utility of a digital technology-driven hierarchical healthcare screening pattern in China

Utilization of digital technologies for cataract screening in primary care is a potential solution for addressing the dilemma between the growing aging population and unequally distributed resources. Here, we propose a digital technology-driven hierarchical screening (DH screening) pattern implemented in China to promote the equity and accessibility of healthcare. It consists of home-based mobile artificial intelligence (AI) screening, community-based AI diagnosis, and referral to hospitals. We utilize decision-analytic Markov models to evaluate the cost-effectiveness and cost-utility of different cataract screening strategies (no screening, telescreening, AI screening and DH screening). A simulated cohort of 100,000 individuals from age 50 is built through a total of 30 1-year Markov cycles. The primary outcomes are incremental cost-effectiveness ratio and incremental cost-utility ratio. The results show that DH screening dominates no screening, telescreening and AI screening in urban and rural China. Annual DH screening emerges as the most economically effective strategy with 341 (338 to 344) and 1326 (1312 to 1340) years of blindness avoided compared with telescreening, and 37 (35 to 39) and 140 (131 to 148) years compared with AI screening in urban and rural settings, respectively. The findings remain robust across all sensitivity analyses conducted. Here, we report that DH screening is cost-effective in urban and rural China, and the annual screening proves to be the most cost-effective option, providing an economic rationale for policymakers promoting public eye health in low- and middle-income countries.

Cellular nucleus image-based smarter microscope system for single cell analysis

Cell imaging technology is undoubtedly a powerful tool for studying single-cell heterogeneity due to its non-invasive and visual advantages. It covers microscope hardware, software, and image analysis techniques, which are hindered by low throughput owing to abundant hands-on time and expertise. Herein, a cellular nucleus image-based smarter microscope system for single-cell analysis is reported to achieve high-throughput analysis and high-content detection of cells. By combining the hardware of an automatic fluorescence microscope and multi-object recognition/acquisition software, we have achieved more advanced process automation with the assistance of Robotic Process Automation (RPA), which realizes a high-throughput collection of single-cell images. Automated acquisition of single-cell images has benefits beyond ease and throughout and can lead to uniform standard and higher quality images. We further constructed a single-cell image database-based convolutional neural network (Efficient Convolutional Neural Network, E-CNN) exceeding 20618 single-cell nucleus images. Computational analysis of large and complex data sets enhances the content and efficiency of single-cell analysis with the assistance of Artificial Intelligence (AI), which breaks through the super-resolution microscope's hardware limitation, such as specialized light sources with specific wavelengths, advanced optical components, and high-performance graphics cards. Our system can identify single-cell nucleus images that cannot be artificially distinguished with an accuracy of 95.3%. Overall, we build an ordinary microscope into a high-throughput analysis and high-content smarter microscope system, making it a candidate tool for Imaging cytology.

Artificial intelligence and skin cancer

Artificial intelligence is poised to rapidly reshape many fields, including that of skin cancer screening and diagnosis, both as a disruptive and assistive technology. Together with the collection and availability of large medical data sets, artificial intelligence will become a powerful tool that can be leveraged by physicians in their diagnoses and treatment plans for patients. This comprehensive review focuses on current progress toward AI applications for patients, primary care providers, dermatologists, and dermatopathologists, explores the diverse applications of image and molecular processing for skin cancer, and highlights AI's potential for patient self-screening and improving diagnostic accuracy for non-dermatologists. We additionally delve into the challenges and barriers to clinical implementation, paths forward for implementation and areas of active research.

Convolutional neural network misclassification analysis in oral lesions: an error evaluation criterion by image characteristics

OBJECTIVE

This retrospective study analyzed the errors generated by a convolutional neural network (CNN) when performing automated classification of oral lesions according to their clinical characteristics, seeking to identify patterns in systemic errors in the intermediate layers of the CNN.

STUDY DESIGN

A cross-sectional analysis nested in a previous trial in which automated classification by a CNN model of elementary lesions from clinical images of oral lesions was performed. The resulting CNN classification errors formed the dataset for this study. A total of 116 real outputs were identified that diverged from the estimated outputs, representing 7.6% of the total images analyzed by the CNN.

RESULTS

The discrepancies between the real and estimated outputs were associated with problems relating to image sharpness, resolution, and focus; human errors; and the impact of data augmentation.

CONCLUSIONS

From qualitative analysis of errors in the process of automated classification of clinical images, it was possible to confirm the impact of image quality, as well as identify the strong impact of the data augmentation process. Knowledge of the factors that models evaluate to make decisions can increase confidence in the high classification potential of CNNs.

Integrating artificial intelligence into the modernization of traditional Chinese medicine industry: a review

Traditional Chinese medicine (TCM) is the practical experience and summary of the Chinese nation for thousands of years. It shows great potential in treating various chronic diseases, complex diseases and major infectious diseases, and has gradually attracted the attention of people all over the world. However, due to the complexity of prescription and action mechanism of TCM, the development of TCM industry is still in a relatively conservative stage. With the rise of artificial intelligence technology in various fields, many scholars began to apply artificial intelligence technology to traditional Chinese medicine industry and made remarkable progress. This paper comprehensively summarizes the important role of artificial intelligence in the development of traditional Chinese medicine industry from various aspects, including new drug discovery, data mining, quality standardization and industry technology of traditional Chinese medicine. The limitations of artificial intelligence in these applications are also emphasized, including the lack of pharmacological research, database quality problems and the challenges brought by human-computer interaction. Nevertheless, the development of artificial intelligence has brought new opportunities and innovations to the modernization of traditional Chinese medicine. Integrating artificial intelligence technology into the comprehensive application of Chinese medicine industry is expected to overcome the major problems faced by traditional Chinese medicine industry and further promote the modernization of the whole traditional Chinese medicine industry.

Could the underlying biological basis of prognostic radiomics and deep learning signatures be explored in patients with lung cancer? A systematic review

OBJECTIVES

To summarize the underlying biological correlation of prognostic radiomics and deep learning signatures in patients with lung cancer and evaluate the quality of available studies.

METHODS

This study examined databases including the PubMed, Embase, Web of Science Core Collection, and Cochrane Library, for studies that elaborated on the underlying biological correlation with prognostic radiomics and deep learning signatures based on CT or PET/CT for predicting the prognosis in patients with lung cancer. Information about the patient and radiogenomic analyses was extracted for the included studies. The Radiomics Quality Score (RQS) and the Prediction Model Risk of Bias Assessment Tool were used to assess the quality of these studies.

RESULTS

Twelve studies were included with 7,338 patients from 2014 to 2022. All studies except for one were retrospective. Supervised machine learning was adopted in six studies, and the remaining used unsupervised machine learning methods. Gene sequencing and histopathological data were analyzed by 83.33% and 16.67% of the included studies, respectively. Gene set enrichment analysis and correlation analysis were most used to explore the biological meaning of prognostic signatures. The median RQS for supervised learning articles was 13.5 (range 12-19) and 7.0 (range 5-14) for unsupervised learning articles. The studies included in this report were assessed to have high risk of bias overall.

CONCLUSION

The biological basis for the interpretability of data-driven models mainly focused on genomics and histopathological factors, and it may improve the prognosis of lung cancer with more proper biological interpretation in the future.

Artificial intelligence in the neonatal intensive care unit: the time is now

Artificial intelligence (AI) has the potential to revolutionize the neonatal intensive care unit (NICU) care by leveraging the large-scale, high-dimensional data that are generated by NICU patients. There is an emerging recognition that the confluence of technological progress, commercialization pathways, and rich data sets provides a unique opportunity for AI to make a lasting impact on the NICU. In this perspective article, we discuss four broad categories of AI applications in the NICU: imaging interpretation, prediction modeling of electronic health record data, integration of real-time monitoring data, and documentation and billing. By enhancing decision-making, streamlining processes, and improving patient outcomes, AI holds the potential to transform the quality of care for vulnerable newborns, making the excitement surrounding AI advancements well-founded and the potential for significant positive change stronger than ever before.

Combining State-of-the-Art Pre-Trained Deep Learning Models: A Noble Approach for Skin Cancer Detection Using Max Voting Ensemble

Skin cancer poses a significant healthcare challenge, requiring precise and prompt diagnosis for effective treatment. While recent advances in deep learning have dramatically improved medical image analysis, including skin cancer classification, ensemble methods offer a pathway for further enhancing diagnostic accuracy. This study introduces a cutting-edge approach employing the Max Voting Ensemble Technique for robust skin cancer classification on ISIC 2018: Task 1-2 dataset. We incorporate a range of cutting-edge, pre-trained deep neural networks, including MobileNetV2, AlexNet, VGG16, ResNet50, DenseNet201, DenseNet121, InceptionV3, ResNet50V2, InceptionResNetV2, and Xception. These models have been extensively trained on skin cancer datasets, achieving individual accuracies ranging from 77.20% to 91.90%. Our method leverages the synergistic capabilities of these models by combining their complementary features to elevate classification performance further. In our approach, input images undergo preprocessing for model compatibility. The ensemble integrates the pre-trained models with their architectures and weights preserved. For each skin lesion image under examination, every model produces a prediction. These are subsequently aggregated using the max voting ensemble technique to yield the final classification, with the majority-voted class serving as the conclusive prediction. Through comprehensive testing on a diverse dataset, our ensemble outperformed individual models, attaining an accuracy of 93.18% and an AUC score of 0.9320, thus demonstrating superior diagnostic reliability and accuracy. We evaluated the effectiveness of our proposed method on the HAM10000 dataset to ensure its generalizability. Our ensemble method delivers a robust, reliable, and effective tool for the classification of skin cancer. By utilizing the power of advanced deep neural networks, we aim to assist healthcare professionals in achieving timely and accurate diagnoses, ultimately reducing mortality rates and enhancing patient outcomes.

A clinically applicable AI system for diagnosis of congenital heart diseases based on computed tomography images

Congenital heart disease (CHD) is the most common type of birth defect. Without timely detection and treatment, approximately one-third of children with CHD would die in the infant period. However, due to the complicated heart structures, early diagnosis of CHD and its types is quite challenging, even for experienced radiologists. Here, we present an artificial intelligence (AI) system that achieves a comparable performance of human experts in the critical task of classifying 17 categories of CHD types. We collected the first-large CT dataset from three different CT machines, including more than 3750 CHD patients over 14 years. Experimental results demonstrate that it can achieve diagnosis accuracy (86.03%) comparable with junior cardiovascular radiologists (86.27%) in a World Health Organization appointed research and cooperation center in China on most types of CHD, and obtains a higher sensitivity (82.91%) than junior cardiovascular radiologists (76.18%). The accuracy of the combination of our AI system (97.20%) and senior radiologists achieves comparable results to that of junior radiologists and senior radiologists (97.16%) which is the current clinical routine. Our AI system can further provide 3D visualization of hearts to senior radiologists for interpretation and flexible review, surgeons for precise intuition of heart structures, and clinicians for more precise outcome prediction. We demonstrate the potential of our model to be integrated into current clinic practice to improve the diagnosis of CHD globally, especially in regions where experienced radiologists can be scarce.

Automatic Skin Cancer Detection Using Clinical Images: A Comprehensive Review

Skin cancer has become increasingly common over the past decade, with melanoma being the most aggressive type. Hence, early detection of skin cancer and melanoma is essential in dermatology. Computational methods can be a valuable tool for assisting dermatologists in identifying skin cancer. Most research in machine learning for skin cancer detection has focused on dermoscopy images due to the existence of larger image datasets. However, general practitioners typically do not have access to a dermoscope and must rely on naked-eye examinations or standard clinical images. By using standard, off-the-shelf cameras to detect high-risk moles, machine learning has also proven to be an effective tool. The objective of this paper is to provide a comprehensive review of image-processing techniques for skin cancer detection using clinical images. In this study, we evaluate 51 state-of-the-art articles that have used machine learning methods to detect skin cancer over the past decade, focusing on clinical datasets. Even though several studies have been conducted in this field, there are still few publicly available clinical datasets with sufficient data that can be used as a benchmark, especially when compared to the existing dermoscopy databases. In addition, we observed that the available artifact removal approaches are not quite adequate in some cases and may also have a negative impact on the models. Moreover, the majority of the reviewed articles are working with single-lesion images and do not consider typical mole patterns and temporal changes in the lesions of each patient.

Refactoring and performance analysis of the main CNN architectures: using false negative rate minimization to solve the clinical images melanoma detection problem

BACKGROUND

Melanoma is one of the deadliest tumors in the world. Early detection is critical for first-line therapy in this tumor pathology and it remains challenging due to the need for histological analysis to ensure correctness in diagnosis. Therefore, multiple computer-aided diagnosis (CAD) systems working on melanoma images were proposed to mitigate the need of a biopsy. However, although the high global accuracy is declared in literature results, the CAD systems for the health fields must focus on the lowest false negative rate (FNR) possible to qualify as a diagnosis support system. The final goal must be to avoid classification type 2 errors to prevent life-threatening situations. Another goal could be to create an easy-to-use system for both physicians and patients.

RESULTS

To achieve the minimization of type 2 error, we performed a wide exploratory analysis of the principal convolutional neural network (CNN) architectures published for the multiple image classification problem; we adapted these networks to the melanoma clinical image binary classification problem (MCIBCP). We collected and analyzed performance data to identify the best CNN architecture, in terms of FNR, usable for solving the MCIBCP problem. Then, to provide a starting point for an easy-to-use CAD system, we used a clinical image dataset (MED-NODE) because clinical images are easier to access: they can be taken by a smartphone or other hand-size devices. Despite the lower resolution than dermoscopic images, the results in the literature would suggest that it would be possible to achieve high classification performance by using clinical images. In this work, we used MED-NODE, which consists of 170 clinical images (70 images of melanoma and 100 images of naevi). We optimized the following CNNs for the MCIBCP problem: Alexnet, DenseNet, GoogleNet Inception V3, GoogleNet, MobileNet, ShuffleNet, SqueezeNet, and VGG16.

CONCLUSIONS

The results suggest that a CNN built on the VGG or AlexNet structure can ensure the lowest FNR (0.07) and (0.13), respectively. In both cases, discrete global performance is ensured: 73% (accuracy), 82% (sensitivity) and 59% (specificity) for VGG; 89% (accuracy), 87% (sensitivity) and 90% (specificity) for AlexNet.

DermAI 1.0: A Robust, Generalized, and Novel Attention-Enabled Ensemble-Based Transfer Learning Paradigm for Multiclass Classification of Skin Lesion Images

Skin lesion classification plays a crucial role in dermatology, aiding in the early detection, diagnosis, and management of life-threatening malignant lesions. However, standalone transfer learning (TL) models failed to deliver optimal performance. In this study, we present an attention-enabled ensemble-based deep learning technique, a powerful, novel, and generalized method for extracting features for the classification of skin lesions. This technique holds significant promise in enhancing diagnostic accuracy by using seven pre-trained TL models for classification. Six ensemble-based DL (EBDL) models were created using stacking, softmax voting, and weighted average techniques. Furthermore, we investigated the attention mechanism as an effective paradigm and created seven attention-enabled transfer learning (aeTL) models before branching out to construct three attention-enabled ensemble-based DL (aeEBDL) models to create a reliable, adaptive, and generalized paradigm. The mean accuracy of the TL models is 95.30%, and the use of an ensemble-based paradigm increased it by 4.22%, to 99.52%. The aeTL models' performance was superior to the TL models in accuracy by 3.01%, and aeEBDL models outperformed aeTL models by 1.29%. Statistical tests show significant p-value and Kappa coefficient along with a 99.6% reliability index for the aeEBDL models. The approach is highly effective and generalized for the classification of skin lesions.

A multi-branch network to detect post-operative complications following hip arthroplasty on X-ray images

Background: Postoperative complications following total hip arthroplasty (THA) often require revision surgery. X-rays are usually used to detect such complications, but manually identifying the location of the problem and making an accurate assessment can be subjective and time-consuming. Therefore, in this study, we propose a multi-branch network to automatically detect postoperative complications on X-ray images. Methods: We developed a multi-branch network using ResNet as the backbone and two additional branches with a global feature stream and a channel feature stream for extracting features of interest. Additionally, inspired by our domain knowledge, we designed a multi-coefficient class-specific residual attention block to learn the correlations between different complications to improve the performance of the system. Results: Our proposed method achieved state-of-the-art (SOTA) performance in detecting multiple complications, with mean average precision (mAP) and F1 scores of 0.346 and 0.429, respectively. The network also showed excellent performance at identifying aseptic loosening, with recall and precision rates of 0.929 and 0.897, respectively. Ablation experiments were conducted on detecting multiple complications and single complications, as well as internal and external datasets, demonstrating the effectiveness of our proposed modules. Conclusion: Our deep learning method provides an accurate end-to-end solution for detecting postoperative complications following THA.

Saliency-Enhanced Content-Based Image Retrieval for Diagnosis Support in Dermatology Consultation: Reader Study

BACKGROUND

Previous research studies have demonstrated that medical content image retrieval can play an important role by assisting dermatologists in skin lesion diagnosis. However, current state-of-the-art approaches have not been adopted in routine consultation, partly due to the lack of interpretability limiting trust by clinical users.

OBJECTIVE

This study developed a new image retrieval architecture for polarized or dermoscopic imaging guided by interpretable saliency maps. This approach provides better feature extraction, leading to better quantitative retrieval performance as well as providing interpretability for an eventual real-world implementation.

METHODS

Content-based image retrieval (CBIR) algorithms rely on the comparison of image features embedded by convolutional neural network (CNN) against a labeled data set. Saliency maps are computer vision-interpretable methods that highlight the most relevant regions for the prediction made by a neural network. By introducing a fine-tuning stage that includes saliency maps to guide feature extraction, the accuracy of image retrieval is optimized. We refer to this approach as saliency-enhanced CBIR (SE-CBIR). A reader study was designed at the University Hospital Zurich Dermatology Clinic to evaluate SE-CBIR's retrieval accuracy as well as the impact of the participant's confidence on the diagnosis.

RESULTS

SE-CBIR improved the retrieval accuracy by 7% (77% vs 84%) when doing single-lesion retrieval against traditional CBIR. The reader study showed an overall increase in classification accuracy of 22% (62% vs 84%) when the participant is provided with SE-CBIR retrieved images. In addition, the overall confidence in the lesion's diagnosis increased by 24%. Finally, the use of SE-CBIR as a support tool helped the participants reduce the number of nonmelanoma lesions previously diagnosed as melanoma (overdiagnosis) by 53%.

CONCLUSIONS

SE-CBIR presents better retrieval accuracy compared to traditional CBIR CNN-based approaches. Furthermore, we have shown how these support tools can help dermatologists and residents improve diagnosis accuracy and confidence. Additionally, by introducing interpretable methods, we should expect increased acceptance and use of these tools in routine consultation.

A survey on deep learning for skin lesion segmentation

Skin cancer is a major public health problem that could benefit from computer-aided diagnosis to reduce the burden of this common disease. Skin lesion segmentation from images is an important step toward achieving this goal. However, the presence of natural and artificial artifacts (e.g., hair and air bubbles), intrinsic factors (e.g., lesion shape and contrast), and variations in image acquisition conditions make skin lesion segmentation a challenging task. Recently, various researchers have explored the applicability of deep learning models to skin lesion segmentation. In this survey, we cross-examine 177 research papers that deal with deep learning-based segmentation of skin lesions. We analyze these works along several dimensions, including input data (datasets, preprocessing, and synthetic data generation), model design (architecture, modules, and losses), and evaluation aspects (data annotation requirements and segmentation performance). We discuss these dimensions both from the viewpoint of select seminal works, and from a systematic viewpoint, examining how those choices have influenced current trends, and how their limitations should be addressed. To facilitate comparisons, we summarize all examined works in a comprehensive table as well as an interactive table available online³.

Use of Deep Neural Networks in the Detection and Automated Classification of Lesions Using Clinical Images in Ophthalmology, Dermatology, and Oral Medicine-A Systematic Review

Artificial neural networks (ANN) are artificial intelligence (AI) techniques used in the automated recognition and classification of pathological changes from clinical images in areas such as ophthalmology, dermatology, and oral medicine. The combination of enterprise imaging and AI is gaining notoriety for its potential benefits in healthcare areas such as cardiology, dermatology, ophthalmology, pathology, physiatry, radiation oncology, radiology, and endoscopic. The present study aimed to analyze, through a systematic literature review, the application of performance of ANN and deep learning in the recognition and automated classification of lesions from clinical images, when comparing to the human performance. The PRISMA 2020 approach (Preferred Reporting Items for Systematic Reviews and Meta-analyses) was used by searching four databases of studies that reference the use of IA to define the diagnosis of lesions in ophthalmology, dermatology, and oral medicine areas. A quantitative and qualitative analyses of the articles that met the inclusion criteria were performed. The search yielded the inclusion of 60 studies. It was found that the interest in the topic has increased, especially in the last 3 years. We observed that the performance of IA models is promising, with high accuracy, sensitivity, and specificity, most of them had outcomes equivalent to human comparators. The reproducibility of the performance of models in real-life practice has been reported as a critical point. Study designs and results have been progressively improved. IA resources have the potential to contribute to several areas of health. In the coming years, it is likely to be incorporated into everyday life, contributing to the precision and reducing the time required by the diagnostic process.

Recent Advances in Melanoma Diagnosis and Prognosis Using Machine Learning Methods

PURPOSE OF REVIEW

The purpose was to summarize the current role and state of artificial intelligence and machine learning in the diagnosis and management of melanoma.

RECENT FINDINGS

Deep learning algorithms can identify melanoma from clinical, dermoscopic, and whole slide pathology images with increasing accuracy. Efforts to provide more granular annotation to datasets and to identify new predictors are ongoing. There have been many incremental advances in both melanoma diagnostics and prognostic tools using artificial intelligence and machine learning. Higher quality input data will further improve these models' capabilities.

Diagnostic performance of deep learning in infectious keratitis: a systematic review and meta-analysis protocol

INTRODUCTION

Infectious keratitis (IK) represents the fifth-leading cause of blindness worldwide. A delay in diagnosis is often a major factor in progression to irreversible visual impairment and/or blindness from IK. The diagnostic challenge is further compounded by low microbiological culture yield, long turnaround time, poorly differentiated clinical features and polymicrobial infections. In recent years, deep learning (DL), a subfield of artificial intelligence, has rapidly emerged as a promising tool in assisting automated medical diagnosis, clinical triage and decision-making, and improving workflow efficiency in healthcare services. Recent studies have demonstrated the potential of using DL in assisting the diagnosis of IK, though the accuracy remains to be elucidated. This systematic review and meta-analysis aims to critically examine and compare the performance of various DL models with clinical experts and/or microbiological results (the current 'gold standard') in diagnosing IK, with an aim to inform practice on the clinical applicability and deployment of DL-assisted diagnostic models.

METHODS AND ANALYSIS

This review will consider studies that included application of any DL models to diagnose patients with suspected IK, encompassing bacterial, fungal, protozoal and/or viral origins. We will search various electronic databases, including EMBASE and MEDLINE, and trial registries. There will be no restriction to the language and publication date. Two independent reviewers will assess the titles, abstracts and full-text articles. Extracted data will include details of each primary studies, including title, year of publication, authors, types of DL models used, populations, sample size, decision threshold and diagnostic performance. We will perform meta-analyses for the included primary studies when there are sufficient similarities in outcome reporting.

ETHICS AND DISSEMINATION

No ethical approval is required for this systematic review. We plan to disseminate our findings via presentation/publication in a peer-reviewed journal.

PROSPERO REGISTRATION NUMBER

CRD42022348596.

Determining the clinical applicability of machine learning models through assessment of reporting across skin phototypes and rarer skin cancer types: A systematic review

Machine learning (ML) models for skin cancer recognition may have variable performance across different skin phototypes and skin cancer types. Overall performance metrics alone are insufficient to detect poor subgroup performance. We aimed (1) to assess whether studies of ML models reported results separately for different skin phototypes and rarer skin cancers, and (2) to graphically represent the skin cancer training datasets used by current ML models. In this systematic review, we searched PubMed, Embase and CENTRAL. We included all studies in medical journals assessing an ML technique for skin cancer diagnosis that used clinical or dermoscopic images from 1 January 2012 to 22 September 2021. No language restrictions were applied. We considered rarer skin cancers to be skin cancers other than pigmented melanoma, basal cell carcinoma and squamous cell carcinoma. We identified 114 studies for inclusion. Rarer skin cancers were included by 8/114 studies (7.0%), and results for a rarer skin cancer were reported separately in 1/114 studies (0.9%). Performance was reported across all skin phototypes in 1/114 studies (0.9%), but performance was uncertain in skin phototypes I and VI from minimal representation of the skin phototypes in the test dataset (9/3756 and 1/3756, respectively). For training datasets, although public datasets were most frequently used, with the most widely used being the International Skin Imaging Collaboration (ISIC) archive (65/114 studies, 57.0%), the largest datasets were private. Our review identified that most ML models did not report performance separately for rarer skin cancers and different skin phototypes. A degree of variability in ML model performance across subgroups is expected, but the current lack of transparency is not justifiable and risks models being used inappropriately in populations in whom accuracy is low.

Monitoring of Pigmented Skin Lesions Using 3D Whole Body Imaging

BACKGROUND AND OBJECTIVES

Advanced artificial intelligence and machine learning have great potential to redefine how skin lesions are detected, mapped, tracked and documented. Here, we propose a 3D whole-body imaging system known as 3DSkin-mapper to enable automated detection, evaluation and mapping of skin lesions.

METHODS

A modular camera rig arranged in a cylindrical configuration was designed to automatically capture images of the entire skin surface of a subject synchronously from multiple angles. Based on the images, we developed algorithms for 3D model reconstruction, data processing and skin lesion detection and tracking based on deep convolutional neural networks. We also introduced a customised, user-friendly, and adaptable interface that enables individuals to interactively visualise, manipulate, and annotate the images. The interface includes built-in features such as mapping 2D skin lesions onto the corresponding 3D model.

RESULTS

The proposed system is developed for skin lesion screening, the focus of this paper is to introduce the system instead of clinical study. Using synthetic and real images we demonstrate the effectiveness of the proposed system by providing multiple views of a target skin lesion, enabling further 3D geometry analysis and longitudinal tracking. Skin lesions are identified as outliers which deserve more attention from a skin cancer physician. Our detector leverages expert annotated labels to learn representations of skin lesions, while capturing the effects of anatomical variability. It takes only a few seconds to capture the entire skin surface, and about half an hour to process and analyse the images.

CONCLUSIONS

Our experiments show that the proposed system allows fast and easy whole body 3D imaging. It can be used by dermatological clinics to conduct skin screening, detect and track skin lesions over time, identify suspicious lesions, and document pigmented lesions. The system can potentially save clinicians time and effort significantly. The 3D imaging and analysis has the potential to change the paradigm of whole body photography with many applications in skin diseases, including inflammatory and pigmentary disorders. With reduced time requirements for recording and documenting high-quality skin information, doctors could spend more time providing better-quality treatment based on more detailed and accurate information.

The Two-Stage Ensemble Learning Model Based on Aggregated Facial Features in Screening for Fetal Genetic Diseases

With the advancement of medicine, more and more researchers have turned their attention to the study of fetal genetic diseases in recent years. However, it is still a challenge to detect genetic diseases in the fetus, especially in an area lacking access to healthcare. The existing research primarily focuses on using teenagers' or adults' face information to screen for genetic diseases, but there are no relevant directions on disease detection using fetal facial information. To fill the vacancy, we designed a two-stage ensemble learning model based on sonography, Fgds-EL, to identify genetic diseases with 932 images. Concretely speaking, we use aggregated information of facial regions to detect anomalies, such as the jaw, frontal bone, and nasal bone areas. Our experiments show that our model yields a sensitivity of 0.92 and a specificity of 0.97 in the test set, on par with the senior sonographer, and outperforming other popular deep learning algorithms. Moreover, our model has the potential to be an effective noninvasive screening tool for the early screening of genetic diseases in the fetus.

Intraclass Clustering-Based CNN Approach for Detection of Malignant Melanoma

This paper describes the process of developing a classification model for the effective detection of malignant melanoma, an aggressive type of cancer in skin lesions. Primary focus is given on fine-tuning and improving a state-of-the-art convolutional neural network (CNN) to obtain the optimal ROC-AUC score. The study investigates a variety of artificial intelligence (AI) clustering techniques to train the developed models on a combined dataset of images across data from the 2019 and 2020 IIM-ISIC Melanoma Classification Challenges. The models were evaluated using varying cross-fold validations, with the highest ROC-AUC reaching a score of 99.48%.

Identifying the Transcriptional Drivers of Metastasis Embedded within Localized Melanoma

In melanoma, predicting which tumors will ultimately metastasize guides treatment decisions. Transcriptional signatures of primary tumors have been utilized to predict metastasis, but which among these are driver or passenger events remains unclear. We used data from the adjuvant AVAST-M trial to identify a predictive gene signature in localized tumors that ultimately metastasized. Using a zebrafish model of primary melanoma, we interrogated the top genes from the AVAST-M signature in vivo. This identified GRAMD1B, a cholesterol transfer protein, as a bona fide metastasis suppressor, with a majority of knockout animals rapidly developing metastasis. Mechanistically, excess free cholesterol or its metabolite 27-hydroxycholesterol promotes invasiveness via activation of an AP-1 program, which is associated with increased metastasis in humans. Our data demonstrate that the transcriptional seeds of metastasis are embedded within localized tumors, suggesting that early targeting of these programs can be used to prevent metastatic relapse.

SIGNIFICANCE

We analyzed human melanoma transcriptomics data to identify a gene signature predictive of metastasis. To rapidly test clinical signatures, we built a genetic metastasis platform in adult zebrafish and identified GRAMD1B as a suppressor of melanoma metastasis. GRAMD1B-associated cholesterol overload activates an AP-1 program to promote melanoma invasion. This article is highlighted in the In This Issue feature, p. 1.

Challenges in evaluating the accuracy of AI-containing digital triage systems: A systematic review

INTRODUCTION

Patient-operated digital triage systems with AI components are becoming increasingly common. However, previous reviews have found a limited amount of research on such systems' accuracy. This systematic review of the literature aimed to identify the main challenges in determining the accuracy of patient-operated digital AI-based triage systems.

METHODS

A systematic review was designed and conducted in accordance with PRISMA guidelines in October 2021 using PubMed, Scopus and Web of Science. Articles were included if they assessed the accuracy of a patient-operated digital triage system that had an AI-component and could triage a general primary care population. Limitations and other pertinent data were extracted, synthesized and analysed. Risk of bias was not analysed as this review studied the included articles' limitations (rather than results). Results were synthesized qualitatively using a thematic analysis.

RESULTS

The search generated 76 articles and following exclusion 8 articles (6 primary articles and 2 reviews) were included in the analysis. Articles' limitations were synthesized into three groups: epistemological, ontological and methodological limitations. Limitations varied with regards to intractability and the level to which they can be addressed through methodological choices. Certain methodological limitations related to testing triage systems using vignettes can be addressed through methodological adjustments, whereas epistemological and ontological limitations require that readers of such studies appraise the studies with limitations in mind.

DISCUSSION

The reviewed literature highlights recurring limitations and challenges in studying the accuracy of patient-operated digital triage systems with AI components. Some of these challenges can be addressed through methodology whereas others are intrinsic to the area of inquiry and involve unavoidable trade-offs. Future studies should take these limitations in consideration in order to better address the current knowledge gaps in the literature.

Towards successful implementation of artificial intelligence in skin cancer care: a qualitative study exploring the views of dermatologists and general practitioners

Recent studies show promising potential for artificial intelligence (AI) to assist healthcare providers (HCPs) in skin cancer care. The aim of this study is to explore the views of dermatologists and general practitioners (GPs) regarding the successful implementation of AI when assisting HCPs in skin cancer care. We performed a qualitative focus group study, consisting of six focus groups with 16 dermatologists and 17 GPs, varying in prior knowledge and experience with AI, gender, and age. An in-depth inductive thematic content analysis was deployed. Perceived benefits, barriers, and preconditions were identified as main themes. Dermatologists and GPs perceive substantial benefits of AI, particularly an improved health outcome and care pathway between primary and secondary care. Doubts about accuracy, risk of health inequalities, and fear of replacement were among the most stressed barriers. Essential preconditions included adequate algorithm content, sufficient usability, and accessibility of AI. In conclusion, dermatologists and GPs perceive significant benefits from implementing AI in skin cancer care. However, to successfully implement AI, key barriers need to be addressed. Efforts should focus on ensuring algorithm transparency, validation, accessibility for all skin types, and adequate regulation of algorithms. Simultaneously, improving knowledge about AI could reduce the fear of replacement.

Melanoma 2.0. Skin cancer as a paradigm for emerging diagnostic technologies, computational modelling and artificial intelligence

We live in an unprecedented time in oncology. We have accumulated samples and cases in cohorts larger and more complex than ever before. New technologies are available for quantifying solid or liquid samples at the molecular level. At the same time, we are now equipped with the computational power necessary to handle this enormous amount of quantitative data. Computational models are widely used helping us to substantiate and interpret data. Under the label of systems and precision medicine, we are putting all these developments together to improve and personalize the therapy of cancer. In this review, we use melanoma as a paradigm to present the successful application of these technologies but also to discuss possible future developments in patient care linked to them. Melanoma is a paradigmatic case for disruptive improvements in therapies, with a considerable number of metastatic melanoma patients benefiting from novel therapies. Nevertheless, a large proportion of patients does not respond to therapy or suffers from adverse events. Melanoma is an ideal case study to deploy advanced technologies not only due to the medical need but also to some intrinsic features of melanoma as a disease and the skin as an organ. From the perspective of data acquisition, the skin is the ideal organ due to its accessibility and suitability for many kinds of advanced imaging techniques. We put special emphasis on the necessity of computational strategies to integrate multiple sources of quantitative data describing the tumour at different scales and levels.

Melanoma Detection by Non-Specialists: An Untapped Potential for Triage?

INTRODUCTION

The incidence of melanoma increased considerably in recent decades, representing a significant public health problem. We aimed to evaluate the ability of non-specialists for the preliminary screening of skin lesions to identify melanoma-suspect lesions.

MATERIALS AND METHODS

A medical student and a dermatologist specialist examined the total body scans of 50 patients.

RESULTS

The agreement between the expert and the non-specialist was 87.75% (κ = 0.65) regarding the assessment of clinical significance. The four parameters of the ABCD rule were evaluated on the 129 lesions rated as clinically significant by both observers. Asymmetry was evaluated similarly in 79.9% (κ = 0.59), irregular borders in 74.4% (κ = 0.50), color in 81.4% (κ = 0.57), and diameter in 89.9% (κ = 0.77) of the cases. The concordance of the two groups was 96.9% (κ = 0.83) in the case of the detection of the Ugly Duckling Sign.

CONCLUSIONS

Although the involvement of GPs is part of routine care worldwide, emphasizing the importance of educating medical students and general practitioners is crucial, as many European countries lack structured melanoma screening training programs targeting non-dermatologists.

Use of convolutional neural networks in skin lesion analysis using real world image and non-image data

Background

Understanding performance of convolutional neural networks (CNNs) for binary (benign vs. malignant) lesion classification based on real world images is important for developing a meaningful clinical decision support (CDS) tool.

Methods

We developed a CNN based on real world smartphone images with histopathological ground truth and tested the utility of structured electronic health record (EHR) data on model performance. Model accuracy was compared against three board-certified dermatologists for clinical validity.

Results

At a classification threshold of 0.5, the sensitivity was 79 vs. 77 vs. 72%, and specificity was 64 vs. 65 vs. 57% for image-alone vs. combined image and clinical data vs. clinical data-alone models, respectively. The PPV was 68 vs. 69 vs. 62%, AUC was 0.79 vs. 0.79 vs. 0.69, and AP was 0.78 vs. 0.79 vs. 0.64 for image-alone vs. combined data vs. clinical data-alone models. Older age, male sex, and number of prior dermatology visits were important positive predictors for malignancy in the clinical data-alone model.

Conclusion

Additional clinical data did not significantly improve CNN image model performance. Model accuracy for predicting malignant lesions was comparable to dermatologists (model: 71.31% vs. 3 dermatologists: 77.87, 69.88, and 71.93%), validating clinical utility. Prospective validation of the model in primary care setting will enhance understanding of the model’s clinical utility.

Lesion identification and malignancy prediction from clinical dermatological images

We consider machine-learning-based lesion identification and malignancy prediction from clinical dermatological images, which can be indistinctly acquired via smartphone or dermoscopy capture. Additionally, we do not assume that images contain single lesions, thus the framework supports both focal or wide-field images. Specifically, we propose a two-stage approach in which we first identify all lesions present in the image regardless of sub-type or likelihood of malignancy, then it estimates their likelihood of malignancy, and through aggregation, it also generates an image-level likelihood of malignancy that can be used for high-level screening processes. Further, we consider augmenting the proposed approach with clinical covariates (from electronic health records) and publicly available data (the ISIC dataset). Comprehensive experiments validated on an independent test dataset demonstrate that (1) the proposed approach outperforms alternative model architectures; (2) the model based on images outperforms a pure clinical model by a large margin, and the combination of images and clinical data does not significantly improves over the image-only model; and (3) the proposed framework offers comparable performance in terms of malignancy classification relative to three board certified dermatologists with different levels of experience.

Clinically Applicable Pathological Diagnosis System for Cell Clumps in Endometrial Cancer Screening via Deep Convolutional Neural Networks

OBJECTIVES

The soaring demand for endometrial cancer screening has exposed a huge shortage of cytopathologists worldwide. To address this problem, our study set out to establish an artificial intelligence system that automatically recognizes and diagnoses pathological images of endometrial cell clumps (ECCs).

METHODS

We used Li Brush to acquire endometrial cells from patients. Liquid-based cytology technology was used to provide slides. The slides were scanned and divided into malignant and benign groups. We proposed two (a U-net segmentation and a DenseNet classification) networks to identify images. Another four classification networks were used for comparison tests.

RESULTS

A total of 113 (42 malignant and 71 benign) endometrial samples were collected, and a dataset containing 15,913 images was constructed. A total of 39,000 ECCs patches were obtained by the segmentation network. Then, 26,880 and 11,520 patches were used for training and testing, respectively. On the premise that the training set reached 100%, the testing set gained 93.5% accuracy, 92.2% specificity, and 92.0% sensitivity. The remaining 600 malignant patches were used for verification.

CONCLUSIONS

An artificial intelligence system was successfully built to classify malignant and benign ECCs.

Registration of polarimetric images for in vivo skin diagnostics

SIGNIFICANCE

Mueller matrix (MM) polarimetry is a promising tool for the detection of skin cancer. Polarimetric in vivo measurements often suffer from misalignment of the polarimetric images due to motion, which can lead to false results.

AIM

We aim to provide an easy-to-implement polarimetric image data registration method to ensure proper image alignment.

APPROACH

A feature-based image registration is implemented for an MM polarimeter for phantom and in vivo human skin measurements.

RESULTS

We show that the keypoint-based registration of polarimetric images is necessary for in vivo skin polarimetry to ensure reliable results. Further, we deliver an efficient semiautomated method for the registration of polarimetric images.

CONCLUSIONS

Image registration for in vivo polarimetry of human skin is required for improved diagnostics and can be efficiently enhanced with a keypoint-based approach.

Suspicious Skin Lesion Detection in Wide-Field Body Images using Deep Learning Outlier Detection

During consultation dermatologists have to address hundreds of lesions in a limited amount of time. They will not only evaluate the single lesion of interest but more importantly the context of it. Visually comparing the similarity of the majority of lesions within the same patient provides a strong indication for lesions with significantly differing aspects. Deep learning algorithms are capable to identify such outliers, i.e. images that differ considerably from the expected appearance on a larger cohort, and highlight the main differences in those cases. In the present study we evaluate the use of autoencoders as unsupervised tools to detect suspicious skin lesions based on evaluation of real world data acquired during consultation at the USZ Dermatology Clinic. Clinical Relevance- Deep learning algorithms are showing many promising results in dermatology lesion classification. However the context of the lesion is normally not considered in the analysis which prevents these tools to transition into routine practice. An outlier detector based on real world data would allow a dermatologist or general practitioner to detect the suspicious lesions for further examination. The algorithm would additionally provide useful insights by highlighting the feature differences between the original outlier (malignant lesion) and the lesion reconstructed by the autoencoder.

KLF10 upregulates ACSM3 via the PI3K/Akt signaling pathway to inhibit the malignant progression of melanoma

Malignant melanoma is a type of skin cancer caused by mutations in the DNA of melanocytes. Melanoma is relatively rare compared with other types of skin tumors, but has a highly aggressive biological behavior and consequently, a poorer prognosis. Therefore, the present study aimed to explore the role and mechanism of Kruppel-like factor 10 (KLF10) and acyl-CoA medium-chain synthetase 3 (ACSM3) in melanoma progression. KLF10 expression in melanoma tissues was predicted using Gene Expression Profiling Interactive Analysis (GEPIA). KLF10 expression in healthy and melanoma cells was also detected using reverse transcription-quantitative PCR and western blotting. Cell transfection was performed to overexpress KLF10 or silence ACSM3. Cell viability, proliferation, migration, invasion and apoptosis were detected using Cell Counting Kit-8, colony formation, wound healing, Transwell and TUNEL assays, respectively. The activity of the ACSM3 promoter was detected using a dual-luciferase reporter assay, and the relationship between KLF10 and ACSM3 was detected using the GEPIA database and chromatin immunoprecipitation (ChIP). The results demonstrated that KLF10 expression was significantly downregulated in melanoma cells, especially in A375 cells. Compared with the Ov-NC group, KLF10 overexpression significantly inhibited the proliferation, invasion and migration of melanoma cells and promoted their apoptosis. Similar to KLF10, ACSM3 was also downregulated in A375 cells compared with that in the HEM group, and the GEPIA database analysis and ChIP assay results demonstrated that KLF10 expression was positively associated with ACSM3 expression. Furthermore, silencing ACSM3 significantly reversed the effect of KLF10 overexpression on cell proliferation, invasion and migration, and ACSM3 knockdown increased the levels of phosphorylated (p)-PI3K and p-Akt compared with the levels in the Ov-KLF10 + sh-NC group. Overall, the present study suggested that KLF10 inhibited the proliferation, invasion and migration of melanoma cells by targeting ACSM3 via the PI3K/Akt signaling pathway.

Artificial Intelligence and Melanoma: A Comprehensive Review of Clinical, Dermoscopic, and Histologic Applications

Melanoma detection, prognosis, and treatment represent challenging and complex areas of cutaneous oncology with considerable impact on patient outcomes and healthcare economics. Artificial intelligence (AI) applications in these tasks are rapidly developing. Neural networks with increasing levels of sophistication are being implemented in clinical image, dermoscopic image, and histopathologic specimen classification of pigmented lesions. These efforts hold promise of earlier and highly accurate melanoma detection, as well as reliable prognostication and prediction of therapeutic response. Herein, we provide a brief introduction to AI, discuss contemporary investigational applications of AI in melanoma, and summarize challenges encountered with AI.

The Future of Precision Prevention for Advanced Melanoma

Precision prevention of advanced melanoma is fast becoming a realistic prospect, with personalized, holistic risk stratification allowing patients to be directed to an appropriate level of surveillance, ranging from skin self-examinations to regular total body photography with sequential digital dermoscopic imaging. This approach aims to address both underdiagnosis (a missed or delayed melanoma diagnosis) and overdiagnosis (the diagnosis and treatment of indolent lesions that would not have caused a problem). Holistic risk stratification considers several types of melanoma risk factors: clinical phenotype, comprehensive imaging-based phenotype, familial and polygenic risks. Artificial intelligence computer-aided diagnostics combines these risk factors to produce a personalized risk score, and can also assist in assessing the digital and molecular markers of individual lesions. However, to ensure uptake and efficient use of AI systems, researchers will need to carefully consider how best to incorporate privacy and standardization requirements, and above all address consumer trust concerns.

Integrated Pipeline of Rapid Isolation and Analysis of Human Plasma Exosomes for Cancer Discrimination Based on Deep Learning of MALDI-TOF MS Fingerprints

Plasma exosomes have shown great potential for liquid biopsy in clinical cancer diagnosis. Herein, we present an integrated strategy for isolating and analyzing exosomes from human plasma rapidly and then discriminating different cancers excellently based on deep learning fingerprints of plasma exosomes. Sequential size-exclusion chromatography (SSEC) was developed efficiently for separating exosomes from human plasma. SSEC isolated plasma exosomes, taking as less as 2 h for a single sample with high purity such that the discard rates of high-density lipoproteins and low/very low-density lipoproteins were 93 and 85%, respectively. Benefitting from the rapid and high-purity isolation, the contents encapsulated in exosomes, covered by plasma proteins, were well profiled by matrix-assisted laser desorption/ionization time-of-flight mass spectrometry (MS). We further analyzed 220 clinical samples, including 79 breast cancer patients, 57 pancreatic cancer patients, and 84 healthy controls. After MS data pre-processing and feature selection, the extracted MS feature peaks were utilized as inputs for constructing a multi-classifier artificial neural network (denoted as Exo-ANN) model. The optimized model avoided overfitting and performed well in both training cohorts and test cohorts. For the samples in the independent test cohort, it realized a diagnosed accuracy of 80.0% with an area under the curve of 0.91 for the whole group. These results suggest that our integrated pipeline may become a generic tool for liquid biopsy based on the analysis of plasma exosomes in clinics.

Advancing primary care with Artificial Intelligence and Machine Learning

Primary care is the largest healthcare delivery platform in the US. Facing the Artificial Intelligence and Machine Learning technology (AI/ML) revolution, the primary care community would benefit from a roadmap revealing priority areas and opportunities for developing and integrating AI/ML-driven clinical tools. This article presents a framework that identifies five domains for AI/ML integration in primary care to support care delivery transformation and achieve the Quintuple Aims of the healthcare system. We concluded that primary care plays a critical role in developing, introducing, implementing, and monitoring AI/ML tools in healthcare and must not be overlooked as AI/ML transforms healthcare.