Possible Bias in Supervised Deep Learning Algorithms for CT Lung Nodule Detection and Classification

A survey of recent methods for addressing AI fairness and bias in biomedicine

OBJECTIVES

Artificial intelligence (AI) systems have the potential to revolutionize clinical practices, including improving diagnostic accuracy and surgical decision-making, while also reducing costs and manpower. However, it is important to recognize that these systems may perpetuate social inequities or demonstrate biases, such as those based on race or gender. Such biases can occur before, during, or after the development of AI models, making it critical to understand and address potential biases to enable the accurate and reliable application of AI models in clinical settings. To mitigate bias concerns during model development, we surveyed recent publications on different debiasing methods in the fields of biomedical natural language processing (NLP) or computer vision (CV). Then we discussed the methods, such as data perturbation and adversarial learning, that have been applied in the biomedical domain to address bias.

METHODS

We performed our literature search on PubMed, ACM digital library, and IEEE Xplore of relevant articles published between January 2018 and December 2023 using multiple combinations of keywords. We then filtered the result of 10,041 articles automatically with loose constraints, and manually inspected the abstracts of the remaining 890 articles to identify the 55 articles included in this review. Additional articles in the references are also included in this review. We discuss each method and compare its strengths and weaknesses. Finally, we review other potential methods from the general domain that could be applied to biomedicine to address bias and improve fairness.

RESULTS

The bias of AIs in biomedicine can originate from multiple sources such as insufficient data, sampling bias and the use of health-irrelevant features or race-adjusted algorithms. Existing debiasing methods that focus on algorithms can be categorized into distributional or algorithmic. Distributional methods include data augmentation, data perturbation, data reweighting methods, and federated learning. Algorithmic approaches include unsupervised representation learning, adversarial learning, disentangled representation learning, loss-based methods and causality-based methods.

Effect of emphysema on AI software and human reader performance in lung nodule detection from low-dose chest CT

BACKGROUND

Emphysema influences the appearance of lung tissue in computed tomography (CT). We evaluated whether this affects lung nodule detection by artificial intelligence (AI) and human readers (HR).

METHODS

Individuals were selected from the "Lifelines" cohort who had undergone low-dose chest CT. Nodules in individuals without emphysema were matched to similar-sized nodules in individuals with at least moderate emphysema. AI results for nodular findings of 30-100 mm3 and 101-300 mm3 were compared to those of HR; two expert radiologists blindly reviewed discrepancies. Sensitivity and false positives (FPs)/scan were compared for emphysema and non-emphysema groups.

RESULTS

Thirty-nine participants with and 82 without emphysema were included (n = 121, aged 61 ± 8 years (mean ± standard deviation), 58/121 males (47.9%)). AI and HR detected 196 and 206 nodular findings, respectively, yielding 109 concordant nodules and 184 discrepancies, including 118 true nodules. For AI, sensitivity was 0.68 (95% confidence interval 0.57-0.77) in emphysema versus 0.71 (0.62-0.78) in non-emphysema, with FPs/scan 0.51 and 0.22, respectively (p = 0.028). For HR, sensitivity was 0.76 (0.65-0.84) and 0.80 (0.72-0.86), with FPs/scan of 0.15 and 0.27 (p = 0.230). Overall sensitivity was slightly higher for HR than for AI, but this difference disappeared after the exclusion of benign lymph nodes. FPs/scan were higher for AI in emphysema than in non-emphysema (p = 0.028), while FPs/scan for HR were higher than AI for 30-100 mm3 nodules in non-emphysema (p = 0.009).

CONCLUSIONS

AI resulted in more FPs/scan in emphysema compared to non-emphysema, a difference not observed for HR.

RELEVANCE STATEMENT

In the creation of a benchmark dataset to validate AI software for lung nodule detection, the inclusion of emphysema cases is important due to the additional number of FPs.

KEY POINTS

• The sensitivity of nodule detection by AI was similar in emphysema and non-emphysema. • AI had more FPs/scan in emphysema compared to non-emphysema. • Sensitivity and FPs/scan by the human reader were comparable for emphysema and non-emphysema. • Emphysema and non-emphysema representation in benchmark dataset is important for validating AI.

Deep Learning-Based Detect-Then-Track Pipeline for Treatment Outcome Assessments in Immunotherapy-Treated Liver Cancer

Accurate treatment outcome assessment is crucial in clinical trials. However, due to the image-reading subjectivity, there exist discrepancies among different radiologists. The situation is common in liver cancer due to the complexity of abdominal scans and the heterogeneity of radiological imaging manifestations in liver subtypes. Therefore, we developed a deep learning-based detect-then-track pipeline that can automatically identify liver lesions from 3D CT scans then longitudinally track target lesions, thereby providing the evaluation of RECIST treatment outcomes in liver cancer. We constructed and validated the pipeline on 173 multi-national patients (344 venous-phase CT scans) consisting of a public dataset and two in-house cohorts of 28 centers. The proposed pipeline achieved a mean average precision of 0.806 and 0.726 of lesion detection on the validation and test sets. The model's diameter measurement reliability and consistency are significantly higher than that of clinicians (p = 1.6 × 10-4). The pipeline can make precise lesion tracking with accuracies of 85.7% and 90.8% then finally yield the RECIST accuracies of 82.1% and 81.4% on the validation and test sets. Our proposed pipeline can provide precise and convenient RECIST outcome assessments and has the potential to aid clinicians with more efficient therapeutic decisions.

Enhancing Lung Nodule Classification: A Novel CViEBi-CBGWO Approach with Integrated Image Preprocessing

Cancer detection and accurate classification pose significant challenges for medical professionals, as it is described as a lethal illness. Diagnosing the malignant lung nodules in its initial stage significantly enhances the recovery and survival rates. Therefore, a novel model named convolutional vision Elman bidirectional-based crossover boosted grey wolf optimization (CViEBi-CBGWO) has been proposed to enhance classification accuracy. CT images selected for further preprocessing are obtained from the LUNA16 dataset and LIDC-IDRI dataset. The data undergoes preprocessing phases involving normalization, data augmentation, and filtering to improve the generalization ability as well as image quality. The local features within the preprocessed images are extracted by implementing the convolutional neural network (CNN). For extracting the global features within the preprocessed images, the vision transformer (ViT) model consists of five encoder blocks. The attained local and global features are combined to generate the feature map. The Elman bidirectional long short-term memory (EBiLSTM) model is applied to categorize the generated feature map as benign and malignant. The crossover operation is integrated with the grey wolf optimization (GWO) algorithm, and the combined form of CBGWO fine-tunes the parameters of the CViEBi model, eliminating the problem of local optima. Experimental validation is conducted using various evaluation measures to assess effectiveness. Comparative analysis demonstrates a superior classification accuracy of 98.72% in the proposed method compared to existing methods.

Measuring the performance of computer vision artificial intelligence to interpret images of HIV self-testing results

Introduction

HIV self-testing (HIVST) is highly sensitive and specific, addresses known barriers to HIV testing (such as stigma), and is recommended by the World Health Organization as a testing option for the delivery of HIV pre-exposure prophylaxis (PrEP). Nevertheless, HIVST remains underutilized as a diagnostic tool in community-based, differentiated HIV service delivery models, possibly due to concerns about result misinterpretation, which could lead to inadvertent onward transmission of HIV, delays in antiretroviral therapy (ART) initiation, and incorrect initiation on PrEP. Ensuring that HIVST results are accurately interpreted for correct clinical decisions will be critical to maximizing HIVST's potential. Early evidence from a few small pilot studies suggests that artificial intelligence (AI) computer vision and machine learning could potentially assist with this task. As part of a broader study that task-shifted HIV testing to a new setting and cadre of healthcare provider (pharmaceutical technologists at private pharmacies) in Kenya, we sought to understand how well AI technology performed at interpreting HIVST results.

Methods

At 20 private pharmacies in Kisumu, Kenya, we offered free blood-based HIVST to clients ≥18 years purchasing products indicative of sexual activity (e.g., condoms). Trained pharmacy providers assisted clients with HIVST (as needed), photographed the completed HIVST, and uploaded the photo to a web-based platform. In real time, each self-test was interpreted independently by the (1) client and (2) pharmacy provider, with the HIVST images subsequently interpreted by (3) an AI algorithm (trained on lab-captured images of HIVST results) and (4) an expert panel of three HIVST readers. Using the expert panel's determination as the ground truth, we calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for HIVST result interpretation for the AI algorithm as well as for pharmacy clients and providers, for comparison.

Results

From March to June 2022, we screened 1,691 pharmacy clients and enrolled 1,500 in the study. All clients completed HIVST. Among 854 clients whose HIVST images were of sufficient quality to be interpretable by the AI algorithm, 63% (540/854) were female, median age was 26 years (interquartile range: 22-31), and 39% (335/855) reported casual sexual partners. The expert panel identified 94.9% (808/854) of HIVST images as HIV-negative, 5.1% (44/854) as HIV-positive, and 0.2% (2/854) as indeterminant. The AI algorithm demonstrated perfect sensitivity (100%), perfect NPV (100%), and 98.8% specificity, and 81.5% PPV (81.5%) due to seven false-positive results. By comparison, pharmacy clients and providers demonstrated lower sensitivity (93.2% and 97.7% respectively) and NPV (99.6% and 99.9% respectively) but perfect specificity (100%) and perfect PPV (100%).

Conclusions

AI computer vision technology shows promise as a tool for providing additional quality assurance of HIV testing, particularly for catching Type II error (false-negative test interpretations) committed by human end-users. We discuss possible use cases for this technology to support differentiated HIV service delivery and identify areas for future research that is needed to assess the potential impacts-both positive and negative-of deploying this technology in real-world HIV service delivery settings.

Diagnostic value of artificial intelligence based on computed tomography (CT) density in benign and malignant pulmonary nodules: a retrospective investigation

Objective

To evaluate the diagnostic value of artificial intelligence (AI) in the detection and management of benign and malignant pulmonary nodules (PNs) using computed tomography (CT) density.

Methods

A retrospective analysis was conducted on the clinical data of 130 individuals diagnosed with PNs based on pathological confirmation. The utilization of AI and physicians has been employed in the diagnostic process of distinguishing benign and malignant PNs. The CT images depicting PNs were integrated into AI-based software. The gold standard for evaluating the accuracy of AI diagnosis software and physician interpretation was the pathological diagnosis.

Results

Out of 226 PNs screened from 130 patients diagnosed by AI and physician reading based on CT, 147 were confirmed by pathology. AI had a sensitivity of 94.69% and radiologists had a sensitivity of 85.40% in identifying PNs. The chi-square analysis indicated that the screening capacity of AI was superior to that of physician reading, with statistical significance (p < 0.05). 195 of the 214 PNs suggested by AI were confirmed pathologically as malignant, and 19 were identified as benign; among the 29 PNs suggested by AI as low risk, 13 were confirmed pathologically as malignant, and 16 were identified as benign. From the physician reading, 193 PNs were identified as malignant, 183 were confirmed malignant by pathology, and 10 appeared benign. Physician reading also identified 30 low-risk PNs, 19 of which were pathologically malignant and 11 benign. The physician readings and AI had kappa values of 0.432 and 0.547, respectively. The physician reading and AI area under curves (AUCs) were 0.814 and 0.798, respectively. Both of the diagnostic techniques had worthy diagnostic value, as indicated by their AUCs of >0.7.

Conclusion

It is anticipated that the use of AI-based CT diagnosis in the detection of PNs would increase the precision in early detection of lung carcinoma, as well as yield more precise evidence for clinical management.

Opportunities and challenges for ChatGPT and large language models in biomedicine and health

ChatGPT has drawn considerable attention from both the general public and domain experts with its remarkable text generation capabilities. This has subsequently led to the emergence of diverse applications in the field of biomedicine and health. In this work, we examine the diverse applications of large language models (LLMs), such as ChatGPT, in biomedicine and health. Specifically, we explore the areas of biomedical information retrieval, question answering, medical text summarization, information extraction and medical education and investigate whether LLMs possess the transformative power to revolutionize these tasks or whether the distinct complexities of biomedical domain presents unique challenges. Following an extensive literature survey, we find that significant advances have been made in the field of text generation tasks, surpassing the previous state-of-the-art methods. For other applications, the advances have been modest. Overall, LLMs have not yet revolutionized biomedicine, but recent rapid progress indicates that such methods hold great potential to provide valuable means for accelerating discovery and improving health. We also find that the use of LLMs, like ChatGPT, in the fields of biomedicine and health entails various risks and challenges, including fabricated information in its generated responses, as well as legal and privacy concerns associated with sensitive patient data. We believe this survey can provide a comprehensive and timely overview to biomedical researchers and healthcare practitioners on the opportunities and challenges associated with using ChatGPT and other LLMs for transforming biomedicine and health.

Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health

ChatGPT has drawn considerable attention from both the general public and domain experts with its remarkable text generation capabilities. This has subsequently led to the emergence of diverse applications in the field of biomedicine and health. In this work, we examine the diverse applications of large language models (LLMs), such as ChatGPT, in biomedicine and health. Specifically we explore the areas of biomedical information retrieval, question answering, medical text summarization, information extraction, and medical education, and investigate whether LLMs possess the transformative power to revolutionize these tasks or whether the distinct complexities of biomedical domain presents unique challenges. Following an extensive literature survey, we find that significant advances have been made in the field of text generation tasks, surpassing the previous state-of-the-art methods. For other applications, the advances have been modest. Overall, LLMs have not yet revolutionized biomedicine, but recent rapid progress indicates that such methods hold great potential to provide valuable means for accelerating discovery and improving health. We also find that the use of LLMs, like ChatGPT, in the fields of biomedicine and health entails various risks and challenges, including fabricated information in its generated responses, as well as legal and privacy concerns associated with sensitive patient data. We believe this survey can provide a comprehensive and timely overview to biomedical researchers and healthcare practitioners on the opportunities and challenges associated with using ChatGPT and other LLMs for transforming biomedicine and health.

MSM-ViT: A multi-scale MobileViT for pulmonary nodule classification using CT images

BACKGROUND

Accurate classification of benign and malignant pulmonary nodules using chest computed tomography (CT) images is important for early diagnosis and treatment of lung cancer. In terms of natural image classification, the VIT-based model has greater advantages in extracting global features than the traditional CNN model. However, due to the small image dataset and low image resolution, it is difficult to directly apply the Vit-based model to pulmonary nodule classification.

OBJECTIVE

To propose and test a new Vit-based MSM-ViT model aiming to achieve good performance in classifying pulmonary nodules.

METHODS

In this study, CNN structure was used in the task of classifying pulmonary nodules to compensate for the poor generalization of ViT structure and the difficulty in extracting multi-scale features. First, sub-pixel fusion was designed to improve the ability of the model to extract tiny features. Second, multi-scale local features were extracted by combining dilated convolution with ordinary convolution. Finally, MobileViT module was used to extract global features and predict them at the spatial level.

RESULTS

CT images involving 442 benign nodules and 406 malignant nodules were extracted from LIDC-IDRI data set to verify model performance, which yielded the best accuracy of 94.04% and AUC value of 0.9636 after 10 cross-validations.

CONCLUSION

The proposed new model can effectively extract multi-scale local and global features. The new model performance is also comparable to the most advanced models that use 3D volume data training, but its occupation of video memory (training resources) is less than 1/10 of the conventional 3D models.

Construction of U-Net++ pulmonary nodule intelligent analysis model based on feature weighted aggregation

BACKGROUND

Lung cancer is a malignant tumor originating from the bronchial mucosa or glands of the lung. Early lung cancer patients often have no obvious symptoms, but early detection and treatment have an important clinical significance for prognostic effect. Computed tomography (CT) is one of the important means in the diagnosis of lung cancer. In order to better solve the problem of diagnosis efficiency, and reduce the rate of misdiagnosis and missed diagnosis, computer aided diagnosis are employed in the accurate localization and segmentation of pulmonary nodules through imaging diagnostics, image processing technology, and other clinical means.

OBJECTIVE

This present study was envisaged to establish an intelligent segmentation model of pulmonary nodules to improve the accuracy of early screening for lung cancer patients.

METHODS

Compared with the traditional segmentation model of fully convolutional neural network, the U-Net++ algorithm based on feature-weighted integration (WI-U-Net++) effectively utilized the feature weight information, adopted the adaptive weighted method for weighted integration, and performed an intelligent segmentation of the anatomical structure and image details, which was applied in the auxiliary diagnosis of pulmonary nodules in CT images. Standard chest X-ray phantom was selected as CT scanning objects, and 30 spherical and irregular simulated nodules were built into them, respectively. CT images were collected by setting different tube voltage and noise index, and randomly included into the training set, validation set and test set at a ratio of 8:1:1.

RESULTS

The experimental results showed that the segmentation accuracy of WI-U-Net++ algorithm for spheroid nodules and irregular nodules was 98.75% and 83.47%, respectively, which was better than that of U-Net and U-Net++ algorithm. In the auxiliary diagnosis, the recall rate of the WI-U-Net++ algorithm for spheroid nodules and irregular nodules was 93.47% and 84.52%, respectively. The accuracy of the benign or malignant identification was 80.27%, and the AUC was 0.9342.

CONCLUSION

U-Net++ algorithm based on feature-weighted integration could improve the segmentation effect of pulmonary nodules. Especially in the case of irregular nodules with malignant signs, the accuracy of clinical diagnosis was significantly improved, and the level of differential diagnosis between benign and malignant was improved.

Artificial Intelligence in Lung Cancer Imaging: Unfolding the Future

Lung cancer is one of the malignancies with higher morbidity and mortality. Imaging plays an essential role in each phase of lung cancer management, from detection to assessment of response to treatment. The development of imaging-based artificial intelligence (AI) models has the potential to play a key role in early detection and customized treatment planning. Computer-aided detection of lung nodules in screening programs has revolutionized the early detection of the disease. Moreover, the possibility to use AI approaches to identify patients at risk of developing lung cancer during their life can help a more targeted screening program. The combination of imaging features and clinical and laboratory data through AI models is giving promising results in the prediction of patients’ outcomes, response to specific therapies, and risk for toxic reaction development. In this review, we provide an overview of the main imaging AI-based tools in lung cancer imaging, including automated lesion detection, characterization, segmentation, prediction of outcome, and treatment response to provide radiologists and clinicians with the foundation for these applications in a clinical scenario.