Visual Interpretability in Computer-Assisted Diagnosis of Thyroid Nodules Using Ultrasound Images

Enhancing the fairness of AI prediction models by Quasi-Pareto improvement among heterogeneous thyroid nodule population

Artificial Intelligence (AI) models for medical diagnosis often face challenges of generalizability and fairness. We highlighted the algorithmic unfairness in a large thyroid ultrasound dataset with significant diagnostic performance disparities across subgroups linked causally to sample size imbalances. To address this, we introduced the Quasi-Pareto Improvement (QPI) approach and a deep learning implementation (QP-Net) combining multi-task learning and domain adaptation to improve model performance among disadvantaged subgroups without compromising overall population performance. On the thyroid ultrasound dataset, our method significantly mitigated the area under curve (AUC) disparity for three less-prevalent subgroups by 0.213, 0.112, and 0.173 while maintaining the AUC for dominant subgroups; we also further confirmed the generalizability of our approach on two public datasets: the ISIC2019 skin disease dataset and the CheXpert chest radiograph dataset. Here we show the QPI approach to be widely applicable in promoting AI for equitable healthcare outcomes.

A scoping review of interpretability and explainability concerning artificial intelligence methods in medical imaging

PURPOSE

To review eXplainable Artificial Intelligence/(XAI) methods available for medical imaging/(MI).

METHOD

A scoping review was conducted following the Joanna Briggs Institute's methodology. The search was performed on Pubmed, Embase, Cinhal, Web of Science, BioRxiv, MedRxiv, and Google Scholar. Studies published in French and English after 2017 were included. Keyword combinations and descriptors related to explainability, and MI modalities were employed. Two independent reviewers screened abstracts, titles and full text, resolving differences through discussion.

RESULTS

228 studies met the criteria. XAI publications are increasing, targeting MRI (n = 73), radiography (n = 47), CT (n = 46). Lung (n = 82) and brain (n = 74) pathologies, Covid-19 (n = 48), Alzheimer's disease (n = 25), brain tumors (n = 15) are the main pathologies explained. Explanations are presented visually (n = 186), numerically (n = 67), rule-based (n = 11), textually (n = 11), and example-based (n = 6). Commonly explained tasks include classification (n = 89), prediction (n = 47), diagnosis (n = 39), detection (n = 29), segmentation (n = 13), and image quality improvement (n = 6). The most frequently provided explanations were local (78.1 %), 5.7 % were global, and 16.2 % combined both local and global approaches. Post-hoc approaches were predominantly employed. The used terminology varied, sometimes indistinctively using explainable (n = 207), interpretable (n = 187), understandable (n = 112), transparent (n = 61), reliable (n = 31), and intelligible (n = 3).

CONCLUSION

The number of XAI publications in medical imaging is increasing, primarily focusing on applying XAI techniques to MRI, CT, and radiography for classifying and predicting lung and brain pathologies. Visual and numerical output formats are predominantly used. Terminology standardisation remains a challenge, as terms like "explainable" and "interpretable" are sometimes being used indistinctively. Future XAI development should consider user needs and perspectives.

Diagnostic value of a computer-assisted diagnosis system for the ultrasound features in thyroid nodules

Objective

To explore the diagnostic value of the TUIAS (SW_TH01/II) computer-aided diagnosis (CAD) software system for the ultrasound Thyroid Imaging Reporting and Data System (TI-RADS) features in thyroid nodules.

Materials and methods

This retrospective study enrolled patients with thyroid nodules in Shanghai East Hospital between January 2017 and October 2021. The novel CAD software (SW_TH01/II) and three sonographers performed a qualitative analysis of the ultrasound TI-RADS features in aspect ratio, margin irregularity, margin smoothness, calcification, and echogenicity of the thyroid nodules.

Results

A total of 225 patients were enrolled. The accuracy, sensitivity, and specificity of the CAD software in "aspect ratio" were 95.6%, 96.2%, and 95.4%, in "margin irregularity" were 90.7%, 90.5%, and 90.9%, in "margin smoothness" were 85.8%, 88.5%, and 83.0%, in "calcification" were 83.6%, 81.7%, and 82.0%, in "homogeneity" were 88.9%, 90.6%, and 82.2%, in "major echo" were 85.3%, 88.0%, and 85.4%, and in "contains very hypoechoic echo" were 92.0%, 90.0%, and 92.4%. The analysis time of the CAD software was significantly shorter than for the sonographers (2.7 ± 1.6 vs. 29.7 ± 12.7 s, P < 0.001).

Conclusion

The CAD system achieved high accuracy in describing thyroid nodule features. It might assist in clinical thyroid nodule analysis.

Study on diagnosis of thyroid nodules based on convolutional neural network

OBJECTIVE

An artificial intelligence (AI) algorithm based on convolutional neural networks was used in ultrasound diagnosis in order to evaluate its performance in judging the nature of thyroid nodules and nodule classification.

METHODS

A total of 105 patients with thyroid nodules confirmed by surgery or biopsy were retrospectively analyzed. The properties, characteristics, and classification of thyroid nodules were evaluated by sonographers and by AI to obtain combined diagnoses. Receiver operating characteristic curves were generated to evaluate the performance of AI, the sonographer, and their combined effort in diagnosing the nature of thyroid nodules and classifying their characteristics. In the diagnosis of thyroid nodules with solid components, hypoechoic appearance, indistinct borders, Anteroposterior/transverse diameter ratio > 1(A/T > 1), and calcification performed by sonographers and by AI, the properties exhibited statistically significant differences.

RESULTS

Sonographers had a sensitivity of 80.7%, specificity of 73.7%, accuracy of 79.0%, and area under the curve (AUC) of 0.751 in the diagnosis of benign and malignant thyroid nodules. AI had a sensitivity of 84.5%, specificity of 81.0%, accuracy of 84.7%, and AUC of 0.803. The combined AI and sonographer diagnosis had a sensitivity of 92.1%, specificity of 86.3%, accuracy of 91.7%, and AUC of 0.910.

CONCLUSION

The efficacy of a combined diagnosis for benign and malignant thyroid nodules is higher than that of an AI-based diagnosis alone or a sonographer-based diagnosis alone. The combined diagnosis can reduce unnecessary fine-needle aspiration biopsy procedures and better evaluate the necessity of surgery in clinical practice.

Artificial intelligence technology enhances the performance of shear wave elastography in thyroid nodule diagnosis

OBJECTIVE

To investigate the diagnostic value of Artificial Intelligence (AI) in thyroid nodules diseases.

METHODS

This study included 100 patients (100 nodules - 23 benign; 77 malignant) who underwent shear wave elastography (SWE) and AI imaging of nodules prior to biopsy and/or surgery in Zhangjiakou First Hospital from January 2021 to December 2021. The image diagnostic value of AI was analyzed.

RESULTS

Among the 100 patients, there were 77 malignant nodules (77%) and 23 benign nodules (23%). Papillary thyroid carcinoma accounted for 94.8% (74/77) of the malignant nodules, and nodular goiter accounted for 100% of the benign nodules. The overall detection rate of AI+SWE was higher than that of SWE alone (P < 0.05). The accuracy, sensitivity, specificity, negative predictive value, and positive predictive value of AI+SWE were all higher than those of SWE only (P < 0.05). The ROC curve results showed that the area under the curve of AI+SWE in the diagnosis of thyroid nodules was 0.903. This was higher than that of SWE (P < 0.05).

CONCLUSION

SWE+AI is effective in the diagnosis of thyroid nodules, and its sensitivity and specificity are better than those of SWE only.

Aided diagnosis of thyroid nodules based on an all-optical diffraction neural network

Background

Thyroid cancer is the most common malignancy in the endocrine system, with its early manifestation being the presence of thyroid nodules. With the advantages of convenience, noninvasiveness, and a lack of radiation, ultrasound is currently the first-line screening tool for the clinical diagnosis of thyroid nodules. The use of artificial intelligence to assist diagnosis is an emerging technology. This paper proposes the use optical neural networks for potential application in the auxiliary diagnosis of thyroid nodules.

Methods

Ultrasound images obtained from January 2013 to December 2018 at the Institute and Hospital of Oncology, Tianjin Medical University, were included in a dataset. Patients who consecutively underwent thyroid ultrasound diagnosis and follow-up procedures were included. We developed an all-optical diffraction neural network to assist in the diagnosis of thyroid nodules. The network is composed of 5 diffraction layers and 1 detection plane. The input image is placed 10 mm away from the first diffraction layer. The input of the diffractive neural network is light at a wavelength of 632.8 nm, and the output of this network is determined by the amplitude and light intensity obtained from the detection region.

Results

The all-optical neural network was used to assist in the diagnosis of thyroid nodules. In the classification task of benign and malignant thyroid nodules, the accuracy of classification on the test set was 97.79%, with an area under the curve value of 99.8%. In the task of detecting thyroid nodules, we first trained the model to determine whether any nodules were present and achieved an accuracy of 84.92% on the test set.

Conclusions

Our study demonstrates the potential of all-optical neural networks in the field of medical image processing. The performance of the models based on optical neural networks is comparable to other widely used network models in the field of image classification.

A new xAI framework with feature explainability for tumors decision-making in Ultrasound data: comparing with Grad-CAM

BACKGROUND AND OBJECTIVE

The value of implementing artificial intelligence (AI) on ultrasound screening for thyroid cancer has been acknowledged, with numerous early studies confirming AI might help physicians acquire more accurate diagnoses. However, the black box nature of AI's decision-making process makes it difficult for users to grasp the foundation of AI's predictions. Furthermore, explainability is not only related to AI performance, but also responsibility and risk in medical diagnosis. In this paper, we offer Explainer, an intrinsically explainable framework that can categorize images and create heatmaps highlighting the regions on which its prediction is based.

METHODS

A dataset of 19341 thyroid ultrasound images with pathological results and physician-annotated TI-RADS features is used to train and test the robustness of the proposed framework. Then we conducted a benign-malignant classification study to determine whether physicians perform better with the assistance of an explainer than they do alone or with Gradient-weighted Class Activation Mapping (Grad-CAM).

RESULTS

Reader studies show that the Explainer can achieve a more accurate diagnosis while explaining heatmaps, and that physicians' performances are improved when assisted by the Explainer. Case study results confirm that the Explainer is capable of locating more reasonable and feature-related regions than the Grad-CAM.

CONCLUSIONS

The Explainer offers physicians a tool to understand the basis of AI predictions and evaluate their reliability, which has the potential to unbox the "black box" of medical imaging AI.

Explainability of deep learning models in medical video analysis: a survey

Deep learning methods have proven to be effective for multiple diagnostic tasks in medicine and have been performing significantly better in comparison to other traditional machine learning methods. However, the black-box nature of deep neural networks has restricted their use in real-world applications, especially in healthcare. Therefore, explainability of the machine learning models, which focuses on providing of the comprehensible explanations of model outputs, may affect the possibility of adoption of such models in clinical use. There are various studies reviewing approaches to explainability in multiple domains. This article provides a review of the current approaches and applications of explainable deep learning for a specific area of medical data analysis-medical video processing tasks. The article introduces the field of explainable AI and summarizes the most important requirements for explainability in medical applications. Subsequently, we provide an overview of existing methods, evaluation metrics and focus more on those that can be applied to analytical tasks involving the processing of video data in the medical domain. Finally we identify some of the open research issues in the analysed area.

Radiomic Detection of Malignancy within Thyroid Nodules Using Ultrasonography-A Systematic Review and Meta-Analysis

Background: Despite investigation, 95% of thyroid nodules are ultimately benign. Radiomics is a field that uses radiological features to inform individualized patient care. We aimed to evaluate the diagnostic utility of radiomics in classifying undetermined thyroid nodules into benign and malignant using ultrasonography (US). Methods: A diagnostic test accuracy systematic review and meta-analysis was performed in accordance with PRISMA guidelines. Sensitivity, specificity, and area under curve (AUC) delineating benign and malignant lesions were recorded. Results: Seventy-five studies including 26,373 patients and 46,175 thyroid nodules met inclusion criteria. Males accounted for 24.6% of patients, while 75.4% of patients were female. Radiomics provided a pooled sensitivity of 0.87 (95% CI: 0.86−0.87) and a pooled specificity of 0.84 (95% CI: 0.84−0.85) for characterizing benign and malignant lesions. Using convolutional neural network (CNN) methods, pooled sensitivity was 0.85 (95% CI: 0.84−0.86) and pooled specificity was 0.82 (95% CI: 0.82−0.83); significantly lower than studies using non-CNN: sensitivity 0.90 (95% CI: 0.89−0.90) and specificity 0.88 (95% CI: 0.87−0.89) (p < 0.05). The diagnostic ability of radiologists and radiomics were comparable for both sensitivity (OR 0.98) and specificity (OR 0.95). Conclusions: Radiomic analysis using US provides a reproducible, reliable evaluation of undetermined thyroid nodules when compared to current best practice.

Applications of Explainable Artificial Intelligence in Diagnosis and Surgery

In recent years, artificial intelligence (AI) has shown great promise in medicine. However, explainability issues make AI applications in clinical usages difficult. Some research has been conducted into explainable artificial intelligence (XAI) to overcome the limitation of the black-box nature of AI methods. Compared with AI techniques such as deep learning, XAI can provide both decision-making and explanations of the model. In this review, we conducted a survey of the recent trends in medical diagnosis and surgical applications using XAI. We have searched articles published between 2019 and 2021 from PubMed, IEEE Xplore, Association for Computing Machinery, and Google Scholar. We included articles which met the selection criteria in the review and then extracted and analyzed relevant information from the studies. Additionally, we provide an experimental showcase on breast cancer diagnosis, and illustrate how XAI can be applied in medical XAI applications. Finally, we summarize the XAI methods utilized in the medical XAI applications, the challenges that the researchers have met, and discuss the future research directions. The survey result indicates that medical XAI is a promising research direction, and this study aims to serve as a reference to medical experts and AI scientists when designing medical XAI applications.

Reliability of a computer-aided system in the evaluation of indeterminate ultrasound images of thyroid nodules

INTRODUCTION

Computer-aided diagnostic (CAD) programs for malignancy risk stratification from ultrasound (US) imaging of thyroid nodules are being validated both experimentally and in real-world practice. However, they have not been tested for reliability in analyzing difficult or unclear images.

METHODS

US images with indeterminate characteristics were evaluated by five observers with different experience in US examination and by a commercial CAD program. The nodules, on which the observers widely agreed, were considered concordant and, if there was little agreement, not concordant or difficult to assess. The diagnostic performance of the readers and the CAD program was calculated and compared in both groups of nodule images.

RESULTS

In the group of concordant thyroid nodules (n = 37), the clinicians and the CAD system obtained similar levels of accuracy (77.0% vs 74.2%, respectively; P = 0.7) and no differences were found in sensitivity (SEN) (95.0% vs 87.5%, P = 0.2), specificity (SPE) (45.5 vs 49.4, respectively; P = 0.7), positive predictive value (PPV) (75.2% vs 77.7%, respectively; P = 0.8), nor negative predictive value (NPV) (85.6 vs 77.7, respectively; P = 0.3). When analyzing the non-concordant nodules (n = 43), the CAD system presented a decrease in accuracy of 4.2%, which was significantly lower than that observed by the experts (19.9%, P = 0.02).

CONCLUSIONS

Clinical observers are similar to the CAD system in the US assessment of the risk of thyroid nodules. However, the AI system for thyroid nodules AmCAD-UT® showed more reliability in the analysis of unclear or misleading images.

Accuracy of Ultrasound Diagnosis of Thyroid Nodules Based on Artificial Intelligence-Assisted Diagnostic Technology: A Systematic Review and Meta-Analysis

BACKGROUND

Ultrasonography (US) is the most common method of identifying thyroid nodules, but US images require an experienced surgeon for identification. Many artificial intelligence (AI) techniques such as computer-aided diagnostic systems (CAD), deep learning (DL), and machine learning (ML) have been used to assist in the diagnosis of thyroid nodules, but whether AI techniques can improve the diagnostic accuracy of thyroid nodules still needs to be explored.

OBJECTIVE

To clarify the accuracy of AI-based thyroid nodule US images for differentiating benign and malignant thyroid nodules.

METHODS

A search strategy of "subject terms + key words" was used to search PubMed, Cochrane Library, Embase, Web of Science, China Biology Medicine (CBM), and China National Knowledge Infrastructure (CNKI) for studies on AI-assisted diagnosis of thyroid nodules based on US images. The summarized receiver operating characteristic (SROC) curve and the pooled sensitivity and specificity were used to assess the performance of the diagnostic tests. The quality assessment of diagnostics accuracy studies-2 (QUADAS-2) tool was used to assess the methodological quality of the included studies. The Review Manager 5.3 and Stata 15 were used to process the data. Subgroup analysis was based on the integrity of data collection.

RESULTS

A total of 25 studies with 17,429 US images of thyroid nodules were included. AI-assisted diagnostic techniques had better diagnostic efficacy in the diagnosis of benign and malignant thyroid nodules: sensitivity 0.88 (95% CI: (0.85-0.90)), specificity 0.81 (95% CI: 0.74-0.86), diagnostic odds ratio (DOR) 30 (95% CI: 19-46). The SROC curve indicated that the area under the curve (AUC) was 0.92 (95% CI: 0.89-0.94). Threshold effect analysis showed a Spearman correlation coefficient: 0.17 < 0.5, suggesting no threshold effect for the included studies. After a meta-regression analysis of 4 different subgroups, the results showed a statistically significant effect of mean age ≥50 years on heterogeneity. Compared with studies with an average age of ≥50 years, AI-assisted diagnostic techniques had higher diagnostic performance in studies with an average age of <50 years (0.89 (95% CI: 0.87-0.92) vs. 0.80 (95% CI: 0.73-0.88)), (0.83 (95% CI: 0.77-0.88) vs. 0.73 (95% CI: 0.60-0.87)).

CONCLUSIONS

AI-assisted diagnostic techniques had good diagnostic efficacy for thyroid nodules. For the diagnosis of <50 year olds, AI-assisted diagnostic technology was more effective in diagnosis.

The value of color Doppler ultrasound in the diagnosis of thyroid nodules: a systematic review and meta-analysis

BACKGROUND

This study aimed to analyze the value of color Doppler ultrasound in the diagnosis of thyroid nodules.

METHODS

We searched the PubMed, Web of Science, Embase, and Cochrane Library databases for randomized controlled trials (RCTs) on using color Doppler ultrasound, thyroid nodules, thyroid tumors, and Doppler ultrasound to diagnose the thyroid nodules. The outcome indicators in the articles had to include the numbers of true positives (TP), false positives (FP), false negatives (FN), and true negatives (TN). Subsequently, the Jadad tool was adopted to evaluate the quality of the included articles, and Review Manager 5.3 software was used to conduct a meta-analysis of the experimental data.

RESULTS

A total of eight suitable articles were selected. The results showed that the estimated sensitivity and specificity of color Doppler ultrasound for the diagnostic of thyroid nodules were 0.46-0.89 and 0.00-1.00, respectively. The pooled estimate of sensitivity for the different articles was 0.71 [95% confidence interval (CI): 0.46-0.89], and the pooled estimate of specificity was 0.77 (95% CI: 0.00-1.00). The area under the summary receiver operating characteristic (SROC) curve (AUC) was 0.917, which was larger than 0.9, signifying high diagnostic accuracy. This suggests that color doppler ultrasound can realize the clinical diagnosis of thyroid nodules.

DISCUSSION

In summary, the results of this study could provide a clinical data for the promotion and application of color Doppler ultrasound in the clinical diagnosis of thyroid nodules, as well as further reliable data for follow-up clinical research on the diagnosis and treatment of thyroid nodules.

Artificial Intelligence for Personalized Medicine in Thyroid Cancer: Current Status and Future Perspectives

Thyroid cancers (TC) have increasingly been detected following advances in diagnostic methods. Risk stratification guided by refined information becomes a crucial step toward the goal of personalized medicine. The diagnosis of TC mainly relies on imaging analysis, but visual examination may not reveal much information and not enable comprehensive analysis. Artificial intelligence (AI) is a technology used to extract and quantify key image information by simulating complex human functions. This latent, precise information contributes to stratify TC on the distinct risk and drives tailored management to transit from the surface (population-based) to a point (individual-based). In this review, we started with several challenges regarding personalized care in TC, for example, inconsistent rating ability of ultrasound physicians, uncertainty in cytopathological diagnosis, difficulty in discriminating follicular neoplasms, and inaccurate prognostication. We then analyzed and summarized the advances of AI to extract and analyze morphological, textural, and molecular features to reveal the ground truth of TC. Consequently, their combination with AI technology will make individual medical strategies possible.