Uncovering and Correcting Shortcut Learning in Machine Learning Models for Skin Cancer Diagnosis

Machine learning in the prediction of massive transfusion in trauma: a retrospective analysis as a proof-of-concept

PURPOSE

Early administration and protocolization of massive hemorrhage protocols (MHP) has been associated with decreases in mortality, multiorgan system failure, and number of blood products used. Various prediction tools have been developed for the initiation of MHP, but no single tool has demonstrated strong prediction with early clinical data. We sought to develop a massive transfusion prediction model using machine learning and early clinical data.

METHODS

Using the National Trauma Data Bank from 2013 to 2018, we included severely injured trauma patients and extracted clinical features available from the pre-hospital and emergency department. We subsequently balanced our dataset and used the Boruta algorithm to determine feature selection. Massive transfusion was defined as five units at 4 h and ten units at 24 h. Six machine learning models were trained on the balanced dataset and tested on the original.

RESULTS

A total of 326,758 patients met our inclusion with 18,871 (5.8%) requiring massive transfusion. Emergency department models demonstrated strong performance characteristics with mean areas under the receiver-operating characteristic curve of 0.83. Extreme gradient boost modeling slightly outperformed and demonstrated adequate predictive performance with pre-hospital data only, as well as 4-h transfusion thresholds.

CONCLUSIONS

We demonstrate the use of machine learning in developing an accurate prediction model for massive transfusion in trauma patients using early clinical data. This research demonstrates the potential utility of artificial intelligence as a clinical decision support tool.

Drop the shortcuts: image augmentation improves fairness and decreases AI detection of race and other demographics from medical images

BACKGROUND

It has been shown that AI models can learn race on medical images, leading to algorithmic bias. Our aim in this study was to enhance the fairness of medical image models by eliminating bias related to race, age, and sex. We hypothesise models may be learning demographics via shortcut learning and combat this using image augmentation.

METHODS

This study included 44,953 patients who identified as Asian, Black, or White (mean age, 60.68 years ±18.21; 23,499 women) for a total of 194,359 chest X-rays (CXRs) from MIMIC-CXR database. The included CheXpert images comprised 45,095 patients (mean age 63.10 years ±18.14; 20,437 women) for a total of 134,300 CXRs were used for external validation. We also collected 1195 3D brain magnetic resonance imaging (MRI) data from the ADNI database, which included 273 participants with an average age of 76.97 years ±14.22, and 142 females. DL models were trained on either non-augmented or augmented images and assessed using disparity metrics. The features learned by the models were analysed using task transfer experiments and model visualisation techniques.

FINDINGS

In the detection of radiological findings, training a model using augmented CXR images was shown to reduce disparities in error rate among racial groups (-5.45%), age groups (-13.94%), and sex (-22.22%). For AD detection, the model trained with augmented MRI images was shown 53.11% and 31.01% reduction of disparities in error rate among age and sex groups, respectively. Image augmentation led to a reduction in the model's ability to identify demographic attributes and resulted in the model trained for clinical purposes incorporating fewer demographic features.

INTERPRETATION

The model trained using the augmented images was less likely to be influenced by demographic information in detecting image labels. These results demonstrate that the proposed augmentation scheme could enhance the fairness of interpretations by DL models when dealing with data from patients with different demographic backgrounds.

FUNDING

National Science and Technology Council (Taiwan), National Institutes of Health.

Dermoscopic dark corner artifacts removal: Friend or foe?

BACKGROUND AND OBJECTIVES

One of the more significant obstacles in classification of skin cancer is the presence of artifacts. This paper investigates the effect of dark corner artifacts, which result from the use of dermoscopes, on the performance of a deep learning binary classification task. Previous research attempted to remove and inpaint dark corner artifacts, with the intention of creating an ideal condition for models. However, such research has been shown to be inconclusive due to a lack of available datasets with corresponding labels for dark corner artifact cases.

METHODS

To address these issues, we label 10,250 skin lesion images from publicly available datasets and introduce a balanced dataset with an equal number of melanoma and non-melanoma cases. The training set comprises 6126 images without artifacts, and the testing set comprises 4124 images with dark corner artifacts. We conduct three experiments to provide new understanding on the effects of dark corner artifacts, including inpainted and synthetically generated examples, on a deep learning method.

RESULTS

Our results suggest that introducing synthetic dark corner artifacts which have been superimposed onto the training set improved model performance, particularly in terms of the true negative rate. This indicates that deep learning learnt to ignore dark corner artifacts, rather than treating it as melanoma, when dark corner artifacts were introduced into the training set. Further, we propose a new approach to quantifying heatmaps indicating network focus using a root mean square measure of the brightness intensity in the different regions of the heatmaps.

CONCLUSIONS

The proposed artifact methods can be used in future experiments to help alleviate possible impacts on model performance. Additionally, the newly proposed heatmap quantification analysis will help to better understand the relationships between heatmap results and other model performance metrics.

Identifying the severity of diabetic retinopathy by visual function measures using both traditional statistical methods and interpretable machine learning: a cross-sectional study

AIMS/HYPOTHESIS

To determine the extent to which diabetic retinopathy severity stage may be classified using machine learning (ML) and commonly used clinical measures of visual function together with age and sex.

METHODS

We measured the visual function of 1901 eyes from 1032 participants in the Northern Ireland Sensory Ageing Study, deriving 12 variables from nine visual function tests. Missing values were imputed using chained equations. Participants were divided into four groups using clinical measures and grading of ophthalmic images: no diabetes mellitus (no DM), diabetes but no diabetic retinopathy (DM no DR), diabetic retinopathy without diabetic macular oedema (DR no DMO) and diabetic retinopathy with DMO (DR with DMO). Ensemble ML models were fitted to classify group membership for three tasks, distinguishing (A) the DM no DR group from the no DM group; (B) the DR no DMO group from the DM no DR group; and (C) the DR with DMO group from the DR no DMO group. More conventional multiple logistic regression models were also fitted for comparison. An interpretable ML technique was used to rank the contribution of visual function variables to predictions and to disentangle associations between diabetic eye disease and visual function from artefacts of the data collection process.

RESULTS

The performance of the ensemble ML models was good across all three classification tasks, with accuracies of 0.92, 1.00 and 0.84, respectively, for tasks A-C, substantially exceeding the accuracies for logistic regression (0.84, 0.61 and 0.80, respectively). Reading index was highly ranked for tasks A and B, whereas near visual acuity and Moorfields chart acuity were important for task C. Microperimetry variables ranked highly for all three tasks, but this was partly due to a data artefact (a large proportion of missing values).

CONCLUSIONS/INTERPRETATION

Ensemble ML models predicted status of diabetic eye disease with high accuracy using just age, sex and measures of visual function. Interpretable ML methods enabled us to identify profiles of visual function associated with different stages of diabetic eye disease, and to disentangle associations from artefacts of the data collection process. Together, these two techniques have great potential for developing prediction models using untidy real-world clinical data.

Eye-Gaze-Guided Vision Transformer for Rectifying Shortcut Learning

Learning harmful shortcuts such as spurious correlations and biases prevents deep neural networks from learning meaningful and useful representations, thus jeopardizing the generalizability and interpretability of the learned representation. The situation becomes even more serious in medical image analysis, where the clinical data are limited and scarce while the reliability, generalizability and transparency of the learned model are highly required. To rectify the harmful shortcuts in medical imaging applications, in this paper, we propose a novel eye-gaze-guided vision transformer (EG-ViT) model which infuses the visual attention from radiologists to proactively guide the vision transformer (ViT) model to focus on regions with potential pathology rather than spurious correlations. To do so, the EG-ViT model takes the masked image patches that are within the radiologists' interest as input while has an additional residual connection to the last encoder layer to maintain the interactions of all patches. The experiments on two medical imaging datasets demonstrate that the proposed EG-ViT model can effectively rectify the harmful shortcut learning and improve the interpretability of the model. Meanwhile, infusing the experts' domain knowledge can also improve the large-scale ViT model's performance over all compared baseline methods with limited samples available. In general, EG-ViT takes the advantages of powerful deep neural networks while rectifies the harmful shortcut learning with human expert's prior knowledge. This work also opens new avenues for advancing current artificial intelligence paradigms by infusing human intelligence.

Privacy-Aware Collaborative Learning for Skin Cancer Prediction

Cancer, including the highly dangerous melanoma, is marked by uncontrolled cell growth and the possibility of spreading to other parts of the body. However, the conventional approach to machine learning relies on centralized training data, posing challenges for data privacy in healthcare systems driven by artificial intelligence. The collection of data from diverse sensors leads to increased computing costs, while privacy restrictions make it challenging to employ traditional machine learning methods. Researchers are currently confronted with the formidable task of developing a skin cancer prediction technique that takes privacy concerns into account while simultaneously improving accuracy. In this work, we aimed to propose a decentralized privacy-aware learning mechanism to accurately predict melanoma skin cancer. In this research we analyzed federated learning from the skin cancer database. The results from the study showed that 92% accuracy was achieved by the proposed method, which was higher than baseline algorithms.

A novel artificial intelligence-based predictive analytics technique to detect skin cancer

One of the leading causes of death among people around the world is skin cancer. It is critical to identify and classify skin cancer early to assist patients in taking the right course of action. Additionally, melanoma, one of the main skin cancer illnesses, is curable when detected and treated at an early stage. More than 75% of fatalities worldwide are related to skin cancer. A novel Artificial Golden Eagle-based Random Forest (AGEbRF) is created in this study to predict skin cancer cells at an early stage. Dermoscopic images are used in this instance as the dataset for the system's training. Additionally, the dermoscopic image information is processed using the established AGEbRF function to identify and segment the skin cancer-affected area. Additionally, this approach is simulated using a Python program, and the current research's parameters are assessed against those of earlier studies. The results demonstrate that, compared to other models, the new research model produces better accuracy for predicting skin cancer by segmentation.

Clinical Application of Artificial Intelligence for Non-melanoma Skin Cancer

OPINION STATEMENT

The development and implementation of artificial intelligence is beginning to impact the care of dermatology patients. Although the clinical application of AI in dermatology to date has largely focused on melanoma, the prevalence of non-melanoma skin cancers, including basal cell and squamous cell cancers, is a critical application for this technology. The need for a timely diagnosis and treatment of skin cancers makes finding more time efficient diagnostic methods a top priority, and AI may help improve dermatologists' performance and facilitate care in the absence of dermatology expertise. Beyond diagnosis, for more severe cases, AI may help in predicting therapeutic response and replacing or reinforcing input from multidisciplinary teams. AI may also help in designing novel therapeutics. Despite this potential, enthusiasm in AI must be tempered by realistic expectations regarding performance. AI can only perform as well as the information that is used to train it, and development and implementation of new guidelines to improve transparency around training and performance of algorithms is key for promoting confidence in new systems. Special emphasis should be placed on the role of dermatologists in curating high-quality datasets that reflect a range of skin tones, diagnoses, and clinical scenarios. For ultimate success, dermatologists must not be wary of AI as a potential replacement for their expertise, but as a new tool to complement their diagnostic acumen and extend patient care.

AI-Powered Diagnosis of Skin Cancer: A Contemporary Review, Open Challenges and Future Research Directions

Skin cancer continues to remain one of the major healthcare issues across the globe. If diagnosed early, skin cancer can be treated successfully. While early diagnosis is paramount for an effective cure for cancer, the current process requires the involvement of skin cancer specialists, which makes it an expensive procedure and not easily available and affordable in developing countries. This dearth of skin cancer specialists has given rise to the need to develop automated diagnosis systems. In this context, Artificial Intelligence (AI)-based methods have been proposed. These systems can assist in the early detection of skin cancer and can consequently lower its morbidity, and, in turn, alleviate the mortality rate associated with it. Machine learning and deep learning are branches of AI that deal with statistical modeling and inference, which progressively learn from data fed into them to predict desired objectives and characteristics. This survey focuses on Machine Learning and Deep Learning techniques deployed in the field of skin cancer diagnosis, while maintaining a balance between both techniques. A comparison is made to widely used datasets and prevalent review papers, discussing automated skin cancer diagnosis. The study also discusses the insights and lessons yielded by the prior works. The survey culminates with future direction and scope, which will subsequently help in addressing the challenges faced within automated skin cancer diagnosis.

Learning Concept Credible Models for Mitigating Shortcuts

During training, models can exploit spurious correlations as shortcuts, resulting in poor generalization performance when shortcuts do not persist. In this work, assuming access to a representation based on domain knowledge (i.e., known concepts) that is invariant to shortcuts, we aim to learn robust and accurate models from biased training data. In contrast to previous work, we do not rely solely on known concepts, but allow the model to also learn unknown concepts. We propose two approaches for mitigating shortcuts that incorporate domain knowledge, while accounting for potentially important yet unknown concepts. The first approach is two-staged. After fitting a model using known concepts, it accounts for the residual using unknown concepts. While flexible, we show that this approach is vulnerable when shortcuts are correlated with the unknown concepts. This limitation is addressed by our second approach that extends a recently proposed regularization penalty. Applied to two real-world datasets, we demonstrate that both approaches can successfully mitigate shortcut learning.

Diagnosis of Inflammatory Bowel Disease and Colorectal Cancer through Multi-View Stacked Generalization Applied on Gut Microbiome Data

Most of the microbiome studies suggest that using ensemble models such as Random Forest results in best predictive power. In this study, we empirically evaluate a more powerful ensemble learning algorithm, multi-view stacked generalization, on pediatric inflammatory bowel disease and adult colorectal cancer patients’ cohorts. We aim to check whether stacking would lead to better results compared to using a single best machine learning algorithm. Stacking achieves the best test set Average Precision (AP) on inflammatory bowel disease dataset reaching AP = 0.69, outperforming both the best base classifier (AP = 0.61) and the baseline meta learner built on top of base classifiers (AP = 0.63). On colorectal cancer dataset, the stacked classifier also outperforms (AP = 0.81) both the best base classifier (AP = 0.79) and the baseline meta learner (AP = 0.75). Stacking achieves best predictive performance on test set outperforming the best classifiers on both patient cohorts. Application of the stacking solves the issue of choosing the most appropriate machine learning algorithm by automating the model selection procedure. Clinical application of such a model is not limited to diagnosis task only, but it also can be extended to biomarker selection thanks to feature selection procedure.

Machine Learning and Deep Learning Algorithms for Skin Cancer Classification from Dermoscopic Images

We carry out a critical assessment of machine learning and deep learning models for the classification of skin tumors. Machine learning (ML) algorithms tested in this work include logistic regression, linear discriminant analysis, k-nearest neighbors classifier, decision tree classifier and Gaussian naive Bayes, while deep learning (DL) models employed are either based on a custom Convolutional Neural Network model, or leverage transfer learning via the use of pre-trained models (VGG16, Xception and ResNet50). We find that DL models, with accuracies up to 0.88, all outperform ML models. ML models exhibit accuracies below 0.72, which can be increased to up to 0.75 with ensemble learning. To further assess the performance of DL models, we test them on a larger and more imbalanced dataset. Metrics, such as the F-score and accuracy, indicate that, after fine-tuning, pre-trained models perform extremely well for skin tumor classification. This is most notably the case for VGG16, which exhibits an F-score of 0.88 and an accuracy of 0.88 on the smaller database, and metrics of 0.70 and 0.88, respectively, on the larger database.