A beneficial role of computer-aided diagnosis system for less experienced physicians in the diagnosis of thyroid nodule on ultrasound

Diagnostic Accuracy of Preoperative Radiologic Findings in Papillary Thyroid Microcarcinoma: Discrepancies with the Postoperative Pathologic Diagnosis and Implications for Clinical Outcomes

BACKGRUOUND

The diagnostic accuracy of preoperative radiologic findings in predicting the tumor characteristics and clinical outcomes of papillary thyroid microcarcinoma (PTMC) was evaluated across all risk groups.

METHODS

In total, 939 PTMC patients, comprising both low-risk and non-low-risk groups, who underwent surgery were enrolled. The preoperative tumor size and lymph node metastasis (LNM) were evaluated by ultrasonography within 6 months before surgery and compared with the postoperative pathologic findings. Discrepancies between the preoperative and postoperative tumor sizes were analyzed, and clinical outcomes were assessed.

RESULTS

The agreement rate between radiological and pathological tumor size was approximately 60%. Significant discrepancies were noted, including an increase in tumor size in 24.3% of cases. Notably, in 10.8% of patients, the postoperative tumor size exceeded 1 cm, despite being initially classified as 0.5 to 1.0 cm based on preoperative imaging. A postoperative tumor size >1 cm was associated with aggressive pathologic factors such as multiplicity, microscopic extrathyroidal extension, and LNM, as well as a higher risk of distant metastasis. In 30.1% of patients, LNM was diagnosed after surgery despite not being suspected before the procedure. This group was characterized by smaller metastatic foci and lower risks of distant metastasis or recurrence than patients with LNM detected both before and after surgery.

CONCLUSION

Among all risk groups of PTMCs, a subset showed an increase in tumor size, reaching 1 cm after surgery. These cases require special consideration due to their association with adverse clinical outcomes, including an elevated risk of distant metastasis.

Learnability of Thyroid Nodule Assessment on Ultrasonography: Using a Big Data Set

OBJECTIVE

The aims of the work described here were to evaluate the learnability of thyroid nodule assessment on ultrasonography (US) using a big data set of US images and to evaluate the diagnostic utilities of artificial intelligence computer-aided diagnosis (AI-CAD) used by readers with varying experience to differentiate benign and malignant thyroid nodules.

METHODS

Six college freshmen independently studied the "learning set" composed of images of 13,560 thyroid nodules, and their diagnostic performance was evaluated after their daily learning sessions using the "test set" composed of images of 282 thyroid nodules. The diagnostic performance of two residents and an experienced radiologist was evaluated using the same "test set." After an initial diagnosis, all readers once again evaluated the "test set" with the assistance of AI-CAD.

RESULTS

Diagnostic performance of almost all students increased after the learning program. Although the mean areas under the receiver operating characteristic curves (AUROCs) of residents and the experienced radiologist were significantly higher than those of students, the AUROCs of five of the six students did not differ significantly compared with that of the one resident. With the assistance of AI-CAD, sensitivity significantly increased in three students, specificity in one student, accuracy in four students and AUROC in four students. Diagnostic performance of the two residents and the experienced radiologist was better with the assistance of AI-CAD.

CONCLUSION

A self-learning method using a big data set of US images has potential as an ancillary tool alongside traditional training methods. With the assistance of AI-CAD, the diagnostic performance of readers with varying experience in thyroid imaging could be further improved.

A new xAI framework with feature explainability for tumors decision-making in Ultrasound data: comparing with Grad-CAM

BACKGROUND AND OBJECTIVE

The value of implementing artificial intelligence (AI) on ultrasound screening for thyroid cancer has been acknowledged, with numerous early studies confirming AI might help physicians acquire more accurate diagnoses. However, the black box nature of AI's decision-making process makes it difficult for users to grasp the foundation of AI's predictions. Furthermore, explainability is not only related to AI performance, but also responsibility and risk in medical diagnosis. In this paper, we offer Explainer, an intrinsically explainable framework that can categorize images and create heatmaps highlighting the regions on which its prediction is based.

METHODS

A dataset of 19341 thyroid ultrasound images with pathological results and physician-annotated TI-RADS features is used to train and test the robustness of the proposed framework. Then we conducted a benign-malignant classification study to determine whether physicians perform better with the assistance of an explainer than they do alone or with Gradient-weighted Class Activation Mapping (Grad-CAM).

RESULTS

Reader studies show that the Explainer can achieve a more accurate diagnosis while explaining heatmaps, and that physicians' performances are improved when assisted by the Explainer. Case study results confirm that the Explainer is capable of locating more reasonable and feature-related regions than the Grad-CAM.

CONCLUSIONS

The Explainer offers physicians a tool to understand the basis of AI predictions and evaluate their reliability, which has the potential to unbox the "black box" of medical imaging AI.

Investigating the diagnostic efficiency of a computer-aided diagnosis system for thyroid nodules in the context of Hashimoto's thyroiditis

Objectives

This study aims to investigate the efficacy of a computer-aided diagnosis (CAD) system in distinguishing between benign and malignant thyroid nodules in the context of Hashimoto's thyroiditis (HT) and to evaluate the role of the CAD system in reducing unnecessary biopsies of benign lesions.

Methods

We included a total of 137 nodules from 137 consecutive patients (mean age, 43.5 ± 11.8 years) who were histopathologically diagnosed with HT. The two-dimensional ultrasound images and videos of all thyroid nodules were analyzed by the CAD system and two radiologists with different experiences according to ACR TI-RADS. The diagnostic cutoff values of ACR TI-RADS were divided into two categories (TR4 and TR5), and then the sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) of the CAD system and the junior and senior radiologists were compared in both cases. Moreover, ACR TI-RADS classification was revised according to the results of the CAD system, and the efficacy of recommended fine-needle aspiration (FNA) was evaluated by comparing the unnecessary biopsy rate and the malignant rate of punctured nodules.

Results

The accuracy, sensitivity, specificity, PPV, and NPV of the CAD system were 0.876, 0.905, 0.830, 0.894, and 0.846, respectively. With TR4 as the cutoff value, the AUCs of the CAD system and the junior and senior radiologists were 0.867, 0.628, and 0.722, respectively, and the CAD system had the highest AUC (P < 0.0001). With TR5 as the cutoff value, the AUCs of the CAD system and the junior and senior radiologists were 0.867, 0.654, and 0.812, respectively, and the CAD system had a higher AUC than the junior radiologist (P < 0.0001) but comparable to the senior radiologist (P = 0.0709). With the assistance of the CAD system, the number of TR4 nodules was decreased by both junior and senior radiologists, the malignant rate of punctured nodules increased by 30% and 22%, and the unnecessary biopsies of benign lesions were both reduced by nearly half.

Conclusions

The CAD system based on deep learning can improve the diagnostic performance of radiologists in identifying benign and malignant thyroid nodules in the context of Hashimoto's thyroiditis and can play a role in FNA recommendations to reduce unnecessary biopsy rates.

Diagnostic efficiency of existing guidelines and the AI-SONIC™ artificial intelligence for ultrasound-based risk assessment of thyroid nodules

INTRODUCTION

The thyroid ultrasound guidelines include the American College of Radiology Thyroid Imaging Reporting and Data System, Chinese-Thyroid Imaging Reporting and Data System, Korean Society of Thyroid Radiology, European-Thyroid Imaging Reporting and Data System, American Thyroid Association, and American Association of Clinical Endocrinologists/American College of Endocrinology/Associazione Medici Endocrinologi guidelines. This study aimed to compare the efficiency of the six ultrasound guidelines vs. an artificial intelligence system (AI-SONICTM) in differentiating thyroid nodules, especially medullary thyroid carcinoma.

METHODS

This retrospective study included patients with medullary thyroid carcinoma, papillary thyroid carcinoma, or benign nodules who underwent nodule resection between May 2010 and April 2020 at one hospital. The diagnostic efficacy of the seven diagnostic tools was evaluated using the receiver operator characteristic curves.

RESULTS

Finally, 432 patients with 450 nodules were included for analysis. The American Association of Clinical Endocrinologists/American College of Endocrinology/Associazione Medici Endocrinologi guidelines had the best sensitivity (88.1%) and negative predictive value (78.6%) for differentiating papillary thyroid carcinoma or medullary thyroid carcinoma vs. benign nodules, while the Korean Society of Thyroid Radiology guidelines had the best specificity (85.6%) and positive predictive value (89.6%), and the American Thyroid Association guidelines had the best accuracy (83.7%). When assessing medullary thyroid carcinoma, the American Thyroid Association guidelines had the highest area under the curve (0.78), the American College of Radiology Thyroid Imaging Reporting and Data System guidelines had the best sensitivity (90.2%), and negative predictive value (91.8%), and AI-SONICTM had the best specificity (85.6%) and positive predictive value (67.5%). The Chinese-Thyroid Imaging Reporting and Data System guidelines had the best under the curve (0.86) in diagnosing malignant tumors vs. benign tumors, followed by the American Thyroid Association and Korean Society of Thyroid Radiology guidelines. The best positive likelihood ratios were achieved by the Korean Society of Thyroid Radiology guidelines and AI-SONICTM (both 5.37). The best negative likelihood ratio was achieved by the American Association of Clinical Endocrinologists/American College of Endocrinology/Associazione Medici Endocrinologi guidelines (0.17). The highest diagnostic odds ratio was achieved by the American Thyroid Association guidelines (24.78).

DISCUSSION

All six guidelines and the AI-SONICTM system had satisfactory value in differentiating benign vs. malignant thyroid nodules.

Systematic review and meta-analysis: diagnostic value of different ultrasound for benign and malignant thyroid nodules

BACKGROUND

Conventional ultrasound and contrast-enhanced ultrasound (CEUS) are commonly used in the diagnosis of benign and malignant thyroid nodules. However, the value of the two methods in the diagnosis of benign and malignant thyroid nodules remains controversial.

METHODS

PubMed, Medline, EBSCO, Science Direct, Cochrane Library, China National Knowledge Infrastructure (CNKI) database and manual journal retrieval were searched from January 2000 to January 2022, to include research on conventional ultrasound or CEUS in the diagnosis of benign and malignant thyroid nodule related clinical studies. Meta-analysis was conducted using RevMan5.3 and Stata Corp to analyze the sensitivity and specificity of conventional ultrasound and CEUS in the diagnosis of benign and malignant thyroid nodules with 95% confidence interval (CI) as indicators. Heterogeneity of the results was evaluated by Q test and I2 in RevMan5.3. Deek's method was used to evaluate publication bias.

RESULTS

A total of 1,378 nodules were included in 11 literatures, including 535 malignant thyroid nodules and 843 benign thyroid nodules. Heterogeneity tests conducted for CEUS diagnostic sensitivity of the 6 included literatures indicated that there was no heterogeneity among the study groups [Q=2.05, degree of freedom (df) =5.00, I2=0.00%, P=0.84]. The combined sensitivity was 0.87, with 95% confidence interval (CI): 0.82 to 0.90. Heterogeneity tests on the diagnostic specificity of CEUS of the six included literatures suggested that there was heterogeneity among the different study groups (Q=14.27, df =5.00, I2=64.96%, P=0.01). The combined specificity was 0.84 (95% CI: 0.78 to 0.89). Heterogeneity tests performed on the sensitivity of five conventional ultrasound diagnosis articles revealed that there was heterogeneity among different study groups (Q=13.62, df =4.00, I2=70.64%, P=0.01). The combined sensitivity was 0.86 (95% CI: 0.78 to 0.92). Heterogeneity tests on the specificity of conventional ultrasound diagnosis in five included literatures indicated that there was heterogeneity among different study groups (Q=16.94, df =4.00, I2=76.39%, P=0.00). The combined specificity was 0.84 (95% CI: 0.75 to 0.90). There was no bias in the included literature.

DISCUSSION

The sensitivity of CEUS in the diagnosis of benign and malignant thyroid nodules was slightly higher than that of conventional ultrasound, which provides a reference for the clinical diagnosis of benign and malignant thyroid nodules.

Prediction models constructed for Hashimoto's thyroiditis risk based on clinical and laboratory factors

BACKGROUND

Hashimoto's thyroiditis (HT) frequently occurs among autoimmune diseases and may simultaneously appear with thyroid cancer. However, it is difficult to diagnose HT at an early stage just by clinical symptoms. Thus, it is urgent to integrate multiple clinical and laboratory factors for the early diagnosis and risk prediction of HT.

METHODS

We recruited 1,303 participants, including 866 non-HT controls and 437 diagnosed HT patients. 44 HT patients also had thyroid cancer. Firstly, we compared the difference in thyroid goiter degrees between controls and patients. Secondly, we collected 15 factors and analyzed their significant differences between controls and HT patients, including age, body mass index, gender, history of diabetes, degrees of thyroid goiter, UIC, 25-(OH)D, FT3, FT4, TSH, TAG, TC, FPG, low-density lipoprotein cholesterol, and high-density lipoprotein cholesterol. Thirdly, logistic regression analysis demonstrated the risk factors for HT. For machine learning modeling of HT and thyroid cancer, we conducted the establishment and evaluation of six models in training and test sets.

RESULTS

The degrees of thyroid goiter were significantly different among controls, HT patients without cancer (HT-C), and HT patients with thyroid cancer (HT+C). Most factors had significant differences between controls and patients. Logistic regression analysis confirmed diabetes, UIC, FT3, and TSH as important risk factors for HT. The AUC scores of XGBoost, LR, SVM, and MLP models indicated appropriate predictive power for HT. The features were arranged by their importance, among which, 25-(OH)D, FT4, and TSH were the top three high-ranking factors.

CONCLUSIONS

We firstly analyzed comprehensive factors of HT patients. The proposed machine learning modeling, combined with multiple factors, are efficient for thyroid diagnosis. These discoveries will extensively promote precise diagnosis, personalized therapies, and reduce unnecessary cost for thyroid diseases.