Diagnostic Performance of Six Ultrasound Risk Stratification Systems for Thyroid Nodules: A Systematic Review and Network Meta-Analysis

A real-world comparison of the diagnostic performances of six different TI-RADS guidelines, including ACR-/Kwak-/K-/EU-/ATA-/C-TIRADS

PURPOSE

To compare the diagnostic performance of six different currently available guidelines including the American College of Radiology Thyroid Imaging and Reporting Data System (ACR-TIRADS), Kwak-TIRADS, Korean TIRADS (K-TIRADS), European TIRADS (EU-TIRADS), American Thyroid Association (ATA) and Chinese TIRADS (C-TIRADS), in differentiating malignant from benign thyroid nodules (TN).

MATERIALS AND METHODS

In this single-center study, between January-2007 and September-2023, ultrasound (US) images of TNs that were pathologically proven either by surgery or by fine needle aspiration biopsy (FNAB), were retrospectively evaluated and categorized according to six different currently available guidelines. Area under curve (AUC), sensitivity, specificity, positive and negative predictive values (PPV and NPV, respectively) and miss rates for malignancy (MRM) were calculated for each guideline.

RESULTS

A total of 829 TNs (n = 234 malignant and n = 595 benign) were included. AUC, sensitivity, specificity, PPV, NPV and accuracy for ACR-TIRADS were 0.786, 99.8 %, 27.1 %, 31.92 %, 99.73 % and 54.6 %, respectively; for Kwak-TIRADS 0.839, 97.8 %, 42.1 %, 36.29 %, 98.11 % and 63.1 %, respectively; for K-TIRADS 0.797, 97.6 %, 41.6 %, 36.01 %, 84.85 % and 62.8, respectively, for EU-TIRADS 0.766, 97.8 %, 35.6 %, 33.89 %, 97.92 % and 59.1 %, respectively, for ATA 0.788, 97.5 %, 49.8 %, 32.86 %, 88.16 % and 64.2 %, respectively and for C-TIRADS 0.842, 0 %, 92.8 %, 54.3 %, 39.53 %, 90.42 %, and 68.8 % respectively. MRM regarding ACR-/Kwak-/K-/EU-/ATA-/C-TIRADS were 2.2 %, 0.5 %, 2.9 %, 2.5 %, 3.3 % and 0.1 %, respectively.

CONCLUSION

Six different currently available TIRADS guidelines can provide effective differentiation of malignant TNs from benign ones with similar diagnostic performances. However; C-TIRADS offered the highest AUC and the lowest MRM than the other guidelines, in this series.

Investigation of artificial intelligence-based clinical decision support system's performance in reducing the fine needle aspiration rate of thyroid nodules: A pilot study

Introduction

This pilot study aims to evaluate the clinical impact of artificial intelligence-based decision support, Koios Decision Support™, on the diagnostic performance of ultrasound assessment of thyroid nodules, and as a result to avoid fine needle aspiration.

Methods

This retrospective pilot study was conducted on ultrasound images of thyroid nodules investigated with fine needle aspiration from January 2022 to December 2022. Orthogonal ultrasound images of thyroid nodules, previously investigated with fine needle aspiration, were compared with the Koios Decision Support™ suggestion to perform fine needle aspiration. Surgical histology was used as ground truth.

Results

A total of 29 patients (76% women) with a mean age of 48 ± 16.5 years were evaluated, n = 15 (52%) were histologically proven benign and n = 14 (48%) were malignant. In the benign group, Koios Decision Support™ suggested avoidable fine needle aspiration in n = 8 (53%). In the malignant group, Koios Decision Support™ suggested follow-up or no fine needle aspiration in n = 2 (14%). Sensitivity is 85.7% (n = 12) (p = 0.027), whereas specificity is 53.3% (n = 8) (p = 0.027). The positive predictive value is 63.2% (n = 12), negative predictive value is 80% (n = 8), false-negative value is 20% (n = 2) and false-positive value is 36.8% (n = 7). Based on artificial intelligence decision, one cancer would have been missed.

Conclusion

Artificial intelligence can improve specificity without significantly compromising sensitivity. There was a suggested reduction in the fine needle aspiration rate, in the histologically proven benign nodules, by 53%. This had no statistical significance, likely due to the small population, however, it is thought to be the largest study to date. Further investigation with wider-ranging studies is suggested.

Thyroid nodules: diagnosis and management

Thyroid nodules, with a prevalence of almost 25% in the general population, are a common occurrence. Their prevalence varies considerably depending on demographics such as age and sex as well as the presence of risk factors. This article provides a comprehensive overview of the prevalence, risk stratification and current management strategies for thyroid nodules, with a particular focus on changes in diagnostic and therapeutic protocols that have occurred over the past 10 years. Several sonography-based stratification systems (such as Thyroid Imaging Reporting and Data Systems (TIRADS)) might help to predict the malignancy risk of nodules, potentially eliminating the need for biopsy in many instances. However, large or suspicious nodules necessitate cytological evaluation following fine-needle aspiration biopsy for accurate classification. In the case of cytology yielding indeterminate results, additional tools, such as molecular testing, can assist in guiding the management plan. Surgery is no longer the only treatment for symptomatic or malignant nodules: active surveillance or local ablative treatments might be beneficial for appropriately selected patients. To enhance clinician-patient interactions and discussions about diagnostic options, shared decision-making tools have been developed. A personalized, risk-based protocol promotes high-quality care while minimizing costs and unnecessary testing.

Artificial intelligence-enhanced infrared thermography as a diagnostic tool for thyroid malignancy detection

INTRODUCTION

Thyroid nodules are common, and investigation is crucial for excluding malignancy. Increased intranodular vascularity is frequently observed in malignant tumors, which can be detected through increased skin surface temperatures using noninvasive infrared thermography. We aimed to develop a diagnostic tool for thyroid cancer using infrared thermal images combined with an artificial intelligence (AI) algorithm.

METHODS

We conducted a prospective cross-sectional study involving participants with thyroid nodules undergoing thyroid surgery. Infrared thermal images were collected using a thermal camera on the day prior to surgery. In combination with the final thyroid pathological reports, we utilized a machine learning model based on the pre-trained ResNet50V2 model, a convolutional neural network, to evaluate diagnostic accuracy for malignancy diagnosis.

RESULTS

The study included 98 participants, 58 with malignant thyroid nodules and 40 with benign thyroid nodules, as determined by pathological results. The AI-enhanced infrared thermal image analyses demonstrated good performance in distinguishing between benign and malignant thyroid nodules, achieving an accuracy of 75% and a sensitivity of 78%. These parameters were slightly lower than those of the AI-model predictor that integrated current practice using preoperative thyroid ultrasound findings and cytological results, yielding an accuracy of 81% and a sensitivity of 84%.

CONCLUSIONS

The infrared thermal images, assisted by an AI model, exhibit good performance in distinguishing thyroid malignancy from benign nodules. This imaging modality has great potential to be used as a noninvasive screening tool for adjunct evaluation of thyroid nodules.

Comparative Analysis of AI-SONICTM Thyroid System and Six Thyroid Risk Stratification Guidelines in Papillary Thyroid Cancer: A Retrospective Cohort Study

Aim

The study aimed to compare the diagnostic performance of AI-SONICTM Thyroid System (AI-SONICTM) with six thyroid nodule ultrasound risk stratification systems, as well as the interobserver agreement among different-year ultrasound examiners using the same diagnostic approach.

Methods

This retrospective study included patients who underwent thyroid ultrasound examination and surgery between 2010 and 2022. Three ultrasound examiners with 2, 5, and 10 years of experience, respectively, used AI-SONICTM and six guidelines to risk-stratify the nodules. The diagnostic performance and interobserver agreement were assessed.

Results

A total of 370 thyroid nodules were included, including 195 papillary thyroid carcinomas (PTC) and 175 benign nodules. For physicians of varying seniority from low to high, AI-SONICTM had a moderate sensitivities of 82.56%, 83.08%, 84.62%, respectively, while AACE/ACE/AME had the highest diagnostic sensitivities (96.41%, 95.38%, 96.41%, respectively); And relatively higher specificities were 85.14%, 85.71%, 85.71% for KSThR, while moderate specificities with values of 84.0%, 85.14%, and 85.71%, respectively were found for AI-SONICTM; The accuracy was highest for ATA (excluding non-classifiable nodules), with values of 87.26%, 87.93%, and 88.82%, respectively, while the accuracy for AI-SONICTM were 83.24%, 84.05%, and 85.14%, respectively. The Kendall's tau coefficient indicated strong or moderate interobserver agreement among all examiners using different diagnostic methods (Kendall's tau coefficient >0.6, P<0.001). AI-SONICTM showed the highest interobserver agreement (Kendall's tau coefficient=0.995, P<0.001). A binary probit regression analysis showed that nodules with cystic components had a significantly higher regression coefficient value of 0.983 (P=0.002), indicating that AI-SONICTM may have higher accuracy for nodules with cystic components.

Conclusion

AI-SONICTM and the six thyroid nodule ultrasound risk stratification systems showed high diagnostic performance for papillary thyroid carcinoma. All examiners showed strong or moderate interobserver agreement when using different diagnostic methods. AI-SONICTM may have higher accuracy for nodules with cystic components.

The clinical value of artificial intelligence in assisting junior radiologists in thyroid ultrasound: a multicenter prospective study from real clinical practice

BACKGROUND

This study is to propose a clinically applicable 2-echelon (2e) diagnostic criteria for the analysis of thyroid nodules such that low-risk nodules are screened off while only suspicious or indeterminate ones are further examined by histopathology, and to explore whether artificial intelligence (AI) can provide precise assistance for clinical decision-making in the real-world prospective scenario.

METHODS

In this prospective study, we enrolled 1036 patients with a total of 2296 thyroid nodules from three medical centers. The diagnostic performance of the AI system, radiologists with different levels of experience, and AI-assisted radiologists with different levels of experience in diagnosing thyroid nodules were evaluated against our proposed 2e diagnostic criteria, with the first being an arbitration committee consisting of 3 senior specialists and the second being cyto- or histopathology.

RESULTS

According to the 2e diagnostic criteria, 1543 nodules were classified by the arbitration committee, and the benign and malignant nature of 753 nodules was determined by pathological examinations. Taking pathological results as the evaluation standard, the sensitivity, specificity, accuracy, and area under the receiver operating characteristic curve (AUC) of the AI systems were 0.826, 0.815, 0.821, and 0.821. For those cases where diagnosis by the Arbitration Committee were taken as the evaluation standard, the sensitivity, specificity, accuracy, and AUC of the AI system were 0.946, 0.966, 0.964, and 0.956. Taking the global 2e diagnostic criteria as the gold standard, the sensitivity, specificity, accuracy, and AUC of the AI system were 0.868, 0.934, 0.917, and 0.901, respectively. Under different criteria, AI was comparable to the diagnostic performance of senior radiologists and outperformed junior radiologists (all P < 0.05). Furthermore, AI assistance significantly improved the performance of junior radiologists in the diagnosis of thyroid nodules, and their diagnostic performance was comparable to that of senior radiologists when pathological results were taken as the gold standard (all p > 0.05).

CONCLUSIONS

The proposed 2e diagnostic criteria are consistent with real-world clinical evaluations and affirm the applicability of the AI system. Under the 2e criteria, the diagnostic performance of the AI system is comparable to that of senior radiologists and significantly improves the diagnostic capabilities of junior radiologists. This has the potential to reduce unnecessary invasive diagnostic procedures in real-world clinical practice.

Theranostics of Thyroid Cancer

Molecular imaging is pivotal in evaluating and managing patients with different thyroid cancer histotypes. The existing, pathology-based, risk stratification systems can be usefully refined, by incorporating tumor-specific molecular and molecular imaging biomarkers with theranostic value, allowing patient-specific treatment decisions. Molecular imaging with different radioactive iodine isotopes (ie, I131, I123, I124) is a central component of differentiated carcinoma (DTC)'s risk stratification while [18F]F-fluorodeoxyglucose ([18F]FDG) PET/CT is interrogated about disease aggressiveness and presence of distant metastases. Moreover, it is particularly useful to assess and risk-stratify patients with radioiodine-refractory DTC, poorly differentiated, and anaplastic thyroid cancers. [18F]F-dihydroxyphenylalanine (6-[18F]FDOPA) PET/CT is the most specific and accurate molecular imaging procedure for patients with medullary thyroid cancer (MTC), a neuroendocrine tumor derived from thyroid C-cells. In addition, [18F]FDG PET/CT can be used in patients with more aggressive clinical or biochemical (ie, serum markers levels and kinetics) MTC phenotypes. In addition to conventional radioiodine therapy for DTC, new redifferentiation strategies are now available to restore uptake in radioiodine-refractory DTC. Moreover, peptide receptor theranostics showed promising results in patients with advanced and metastatic radioiodine-refractory DTC and MTC, respectively. The current appropriate role and future perspectives of molecular imaging and theranostics in thyroid cancer are discussed in our present review.

Comparison of ultrasound-guided microwave ablation, laser ablation, and radiofrequency ablation for the treatment of elderly patients with benign thyroid nodules: A meta-analysis

BACKGROUND

A new minimally invasive technique, ultrasound-guided thermal ablation has become one of the treatment methods for benign thyroid nodules. This study aims to evaluate the efficacy and safety of laser ablation (LA), radiofrequency ablation (RFA), and microwave ablation (MWA) in the treatment of elderly patients with benign thyroid nodules.

METHODS

PubMed, Web of Science, and Cochrane Library were searched for qualified randomized controlled studies (RCTs) issued from establishing databases to March 2022. After screening and evaluating the article quality, the data on nodular volume reduction rate (VRR) and the incidence of complications after thermal ablation were extracted and analyzed by RevMan 5.3 and Stata l4.0.

RESULTS

The meta-analysis included seven articles with 3055 participants. We found that LA, RFA, and MWA could markedly reduce the volume of benign thyroid nodules. LA was superior to RFA and MWA in reducing the volume of benign thyroid nodules in 6 months of follow-up (all P < 0.05). LA, RFA, and MWA can be safely implemented in patients with benign thyroid nodules. The incidence of significant complications after the RFA group was enhanced compared with that in the MWA (P < 0.05), and the incidence of secondary complications after RFA was slightly higher than that of LA (P < 0.05).

CONCLUSION

LA, RFA, and MWA can markedly reduce the volume of benign thyroid nodules in elderly patients and can safely treat benign thyroid nodules.

Assessment of the risk of malignancy in Bethesda III thyroid nodules: a comprehensive review

The increasing prevalence of thyroid cancer emphasizes the need for a thorough assessment of risk of malignancy in Bethesda III nodules. Various methods ranging commercial platforms of molecular genetic testing (including Afirma® GEC, Afirma® GSC, ThyroSeq® V3, RosettaGX®, ThyGeNEXT®/ThyraMIR®, ThyroidPRINT®) to radionuclide scans and ultrasonography have been investigated to provide a more nuanced comprehension of risk estimation. The integration of molecular studies and imaging techniques into clinical practice may provide clinicians with improved and personalized risk assessment. This integrated approach we feel may enable clinicians to carefully tailor interventions, thereby minimizing the likelihood of unnecessary thyroid surgeries and overall crafting the optimal treatment. By aligning with the evolving landscape of personalized healthcare, this comprehensive strategy ensures a patient-centric approach to thyroid nodule and thyroid cancer management.

Thyroid Cancer: A Review

Importance

Approximately 43 720 new cases of thyroid carcinoma are expected to be diagnosed in 2023 in the US. Five-year relative survival is approximately 98.5%. This review summarizes current evidence regarding pathophysiology, diagnosis, and management of early-stage and advanced thyroid cancer.

Observations

Papillary thyroid cancer accounts for approximately 84% of all thyroid cancers. Papillary, follicular (≈4%), and oncocytic (≈2%) forms arise from thyroid follicular cells and are termed well-differentiated thyroid cancer. Aggressive forms of follicular cell-derived thyroid cancer are poorly differentiated thyroid cancer (≈5%) and anaplastic thyroid cancer (≈1%). Medullary thyroid cancer (≈4%) arises from parafollicular C cells. Most cases of well-differentiated thyroid cancer are asymptomatic and detected during physical examination or incidentally found on diagnostic imaging studies. For microcarcinomas (≤1 cm), observation without surgical resection can be considered. For tumors larger than 1 cm with or without lymph node metastases, surgery with or without radioactive iodine is curative in most cases. Surgical resection is the preferred approach for patients with recurrent locoregional disease. For metastatic disease, surgical resection or stereotactic body irradiation is favored over systemic therapy (eg, lenvatinib, dabrafenib). Antiangiogenic multikinase inhibitors (eg, sorafenib, lenvatinib, cabozantinib) are approved for thyroid cancer that does not respond to radioactive iodine, with response rates 12% to 65%. Targeted therapies such as dabrafenib and selpercatinib are directed to genetic mutations (BRAF, RET, NTRK, MEK) that give rise to thyroid cancer and are used in patients with advanced thyroid carcinoma.

Conclusions

Approximately 44 000 new cases of thyroid cancer are diagnosed each year in the US, with a 5-year relative survival of 98.5%. Surgery is curative in most cases of well-differentiated thyroid cancer. Radioactive iodine treatment after surgery improves overall survival in patients at high risk of recurrence. Antiangiogenic multikinase inhibitors and targeted therapies to genetic mutations that give rise to thyroid cancer are increasingly used in the treatment of metastatic disease.

Diagnostic performance of ultrasound risk stratification systems on thyroid nodules cytologically classified as indeterminate: a systematic review and meta-analysis

PURPOSE

Ultrasound (US) risk stratification systems (RSSs) are increasingly being utilized for the optimal management of thyroid nodules, including those with indeterminate cytology. The goal of this study was to evaluate the category-based diagnostic performance of US RSSs in identifying malignancy in indeterminate nodules.

METHODS

This systematic review and meta-analysis was registered on PROSPERO (CRD42021266195). PubMed, EMBASE, and Web of Science were searched through December 1, 2022. Original articles reporting data on the performance of US RSSs for indeterminate nodules were included. The numbers of nodules classified as true negative, true positive, false negative, and false positive were extracted.

RESULTS

Thirty-three studies evaluating 7,225 indeterminate thyroid nodules were included. The diagnostic accuracy was quantitatively synthesized using a Bayesian bivariate model based on the integrated nested Laplace approximation in R. For the intermediate- to high-risk category, the sensitivity levels of the American College of Radiology, the American Thyroid Association, the European Thyroid Association, the Korean Thyroid Association/Korean Society of Thyroid Radiology, and Kwak et al. were found to be 0.80, 0.72, 0.76, 0.96, and 0.97, respectively. The corresponding specificity measurements were 0.36, 0.50, 0.49, 0.28, and 0.17. Furthermore, for the high-risk category, the sensitivity values were 0.40, 0.46, 0.55, 0.47, and 0.10, while the specificity levels were 0.91, 0.90, 0.71, 0.91, and 0.99, respectively.

CONCLUSION

The overall diagnostic performance of the US RSSs was moderate in the differentiation of indeterminate nodules.

Diagnostic Performance of ACR TI-RADS and ATA Guidelines in the Prediction of Thyroid Malignancy: A Prospective Single Tertiary Center Study and Literature Review

AIM

This study sought to compare two common risk stratification systems in terms of their diagnostic performance for the evaluation of thyroid malignancy.

METHODS

The American College of Radiology (ACR) Thyroid Imaging, Reporting and Data System (TI-RADS) and the American Thyroid Association (ATA) guidelines were compared among 571 thyroid nodules with definitive fine needle aspiration (FNA) cytology or postoperative histopathology. Ultrasound characteristics such as composition, echogenicity, shape, margin, size, and vascularity were assessed for each thyroid nodule. Diagnostic performance measures were determined and compared through receiver operating characteristic (ROC) curves, and decision curve analysis (DCA).

RESULTS

Of 571 nodules, 65 (11.4%) were malignant. The AUC, sensitivity, specificity, positive predictive value, and negative predictive value were 0.691, 49.2%, 84.9%, 29.6%, and 92.8% for ATA guideline, and 0.776, 72.3%, 79.2%, 30.9%, and 95.7%, for ACR TI-RADS, respectively. ACR TI-RADS was more sensitive (p = 0.003), while the ATA guideline was more specific (p < 0.001). DCA demonstrated that the ACR TI-RADS provided a greater net benefit than the ATA guideline. In addition, the net reduction in unnecessary biopsies is higher for ACR TI-RADS than ATA guidelines. The total number of indicated biopsies and unnecessary FNA rates were lower in ACR TI-RADS compared to ATA guideline (293 vs. 527 and 80.2 vs. 87.8). ACR TI-RADS presented no biopsy indication in seven malignant nodules (all categorized as TR2), whereas ATA guideline missed one. Hypoechogenicity was the most significant predictor of malignancy (OR = 8.34, 95% CI: 3.75-19.45), followed by a taller-than-wide shape (OR = 6.73, 95% CI: 3.07-14.77).

CONCLUSIONS

Our findings suggest that each system has particular advantages in the evaluation of thyroid nodules. ACR TI-RADS reduces unnecessary FNA rates, however, malignant nodules categorized as TR2 might be missed using this system. Further evaluation of this group of nodules using Doppler and other ultrasound modalities is recommended.

Artificial intelligence in the diagnosis of thyroid cancer: Recent advances and future directions

The diagnosis and management of thyroid cancer is fraught with challenges despite the advent of innovative diagnostic, surgical, and chemotherapeutic modalities. Challenges like inaccuracy in prognostication, uncertainty in cytopathological diagnosis, trouble in differentiating follicular neoplasms, intra-observer and inter-observer variability on ultrasound imaging preclude personalised treatment in thyroid cancer. Artificial intelligence (AI) is bringing a paradigm shift to the healthcare, powered by quick advancement of the analytic techniques. Several recent studies have shown remarkable progress in thyroid cancer diagnostics based on AI-assisted algorithms. Application of AI techniques in thyroid ultrasonography and cytopathology have shown remarkable impro-vement in sensitivity and specificity over the traditional diagnostic modalities. AI has also been explored in the development of treatment algorithms for indeterminate nodules and for prognostication in the patients with thyroid cancer. The benefits of high repeatability and straightforward implementation of AI in the management of thyroid cancer suggest that it holds promise for clinical application. Limited clinical experience and lack of prospective validation studies remain the biggest drawbacks. Developing verified and trustworthy algorithms after extensive testing and validation using prospective, multi-centre trials is crucial for the future use of AI in the pipeline of precision medicine in the management of thyroid cancer.

Diagnostic efficiency among Eu-/C-/ACR-TIRADS and S-Detect for thyroid nodules: a systematic review and network meta-analysis

Background

The performance in evaluating thyroid nodules on ultrasound varies across different risk stratification systems, leading to inconsistency and uncertainty regarding diagnostic sensitivity, specificity, and accuracy.

Objective

Comparing diagnostic performance of detecting thyroid cancer among distinct ultrasound risk stratification systems proposed in the last five years.

Evidence acquisition

Systematic search was conducted on PubMed, EMBASE, and Web of Science databases to find relevant research up to December 8, 2022, whose study contents contained elucidation of diagnostic performance of any one of the above ultrasound risk stratification systems (European Thyroid Imaging Reporting and Data System[Eu-TIRADS]; American College of Radiology TIRADS [ACR TIRADS]; Chinese version of TIRADS [C-TIRADS]; Computer-aided diagnosis system based on deep learning [S-Detect]). Based on golden diagnostic standard in histopathology and cytology, single meta-analysis was performed to obtain the optimal cut-off value for each system, and then network meta-analysis was conducted on the best risk stratification category in each system.

Evidence synthesis

This network meta-analysis included 88 studies with a total of 59,304 nodules. The most accurate risk category thresholds were TR5 for Eu-TIRADS, TR5 for ACR TIRADS, TR4b and above for C-TIRADS, and possible malignancy for S-Detect. At the best thresholds, sensitivity of these systems ranged from 68% to 82% and specificity ranged from 71% to 81%. It identified the highest sensitivity for C-TIRADS TR4b and the highest specificity for ACR TIRADS TR5. However, sensitivity for ACR TIRADS TR5 was the lowest. The diagnostic odds ratio (DOR) and area under curve (AUC) were ranked first in C-TIRADS.

Conclusion

Among four ultrasound risk stratification options, this systemic review preliminarily proved that C-TIRADS possessed favorable diagnostic performance for thyroid nodules.

Systematic review registration

https://www.crd.york.ac.uk/prospero, CRD42022382818.