Diagnostic performance rates of the ACR-TIRADS and EU-TIRADS based on histopathological evidence

Effects of Implementation of European Thyroid Imaging Reporting and Data System Risk Stratification in a Thyroid Cancer Program in Western Sweden: A Retrospective Cohort Study

OBJECTIVE

The European Thyroid Imaging Reporting and Data System (EU-TIRADS) allows for selective fine needle aspiration cytology (FNAC). In 2017, EU-TIRADS was implemented as part of a nationwide standardized care bundle for thyroid cancer in Western Sweden with a population of approximately 1.7 million. The objective of this study was to investigate the clinical value of EU-TIRADS attempting to reduce the number of unnecessary FNACs in referred patients with thyroid nodules.

METHODS

The study cohort consisted of all patients referred to Sahlgrenska University Hospital due to a palpable, newly detected or growing thyroid nodules or a positron emission tomography-positive finding for examination with thyroid ultrasound and selective cytology between 2018 and 2022. Medical records on EU-TIRADS classification, corresponding FNAC results, and histopathologic diagnosis were retrospectively collected. Adherence to the EU-TIRADS guidelines, use of selective FNAC, and rate of malignancy in patients who underwent surgery were assessed.

RESULTS

In total, 1246 thyroid nodules in 990 patients were evaluated. The distributions of EU-TIRADS 2 to 5 (number [percentage]) for all examined nodules were 63 (5%), 462 (37%), 443 (36%), and 278 (22%), respectively. FNAC was omitted in 7% of the investigated patients. FNAC was performed in 124 nodules (10%) despite not fulfilling the EU-TIRADS criteria or absence of positron emission tomography-positive findings. The rate of malignancy was 33% and 1/50 in patients who underwent "unnecessary" FNAC.

CONCLUSION

Implementation of EU-TIRADS in routine management of thyroid nodules led to the selective use of FNAC; however, the clinical impact was limited. This study provides real-world data on the value and magnitude of diagnostic improvement by implementing EU-TIRADS in clinical practice.

Blood leukocytes as a non-invasive diagnostic tool for thyroid nodules: a prospective cohort study

BACKGROUND

Thyroid nodule (TN) patients in China are subject to overdiagnosis and overtreatment. The implementation of existing technologies such as thyroid ultrasonography has indeed contributed to the improved diagnostic accuracy of TNs. However, a significant issue persists, where many patients undergo unnecessary biopsies, and patients with malignant thyroid nodules (MTNs) are advised to undergo surgery therapy.

METHODS

This study included a total of 293 patients diagnosed with TNs. Differential methylation haplotype blocks (MHBs) in blood leukocytes between MTNs and benign thyroid nodules (BTNs) were detected using reduced representation bisulfite sequencing (RRBS). Subsequently, an artificial intelligence blood leukocyte DNA methylation (BLDM) model was designed to optimize the management and treatment of patients with TNs for more effective outcomes.

RESULTS

The DNA methylation profiles of peripheral blood leukocytes exhibited distinctions between MTNs and BTNs. The BLDM model we developed for diagnosing TNs achieved an area under the curve (AUC) of 0.858 in the validation cohort and 0.863 in the independent test cohort. Its specificity reached 90.91% and 88.68% in the validation and independent test cohorts, respectively, outperforming the specificity of ultrasonography (43.64% in the validation cohort and 47.17% in the independent test cohort), albeit with a slightly lower sensitivity (83.33% in the validation cohort and 82.86% in the independent test cohort) compared to ultrasonography (97.62% in the validation cohort and 100.00% in the independent test cohort). The BLDM model could correctly identify 89.83% patients whose nodules were suspected malignant by ultrasonography but finally histological benign. In micronodules, the model displayed higher specificity (93.33% in the validation cohort and 92.00% in the independent test cohort) and accuracy (88.24% in the validation cohort and 87.50% in the independent test cohort) for diagnosing TNs. This performance surpassed the specificity and accuracy observed with ultrasonography. A TN diagnostic and treatment framework that prioritizes patients is provided, with fine-needle aspiration (FNA) biopsy performed only on patients with indications of MTNs in both BLDM and ultrasonography results, thus avoiding unnecessary biopsies.

CONCLUSIONS

This is the first study to demonstrate the potential of non-invasive blood leukocytes in diagnosing TNs, thereby making TN diagnosis and treatment more efficient in China.

Real-Time Ultrasonography and the Evaluation of Static Images Yield Different Results in the Assessment of EU-TIRADS Categories

The studies on the effectiveness of various TIRADS in the diagnostics of thyroid nodules differ in the method of ultrasound image assessment: real time (rtUS) vs. static ultrasonography (stUS). The aim of the study was to evaluate the impact of those two methods on the categorization of nodules in EU-TIRADS. Three experienced raters assessed 842 nodules in routine rtUS and reassessed with the use of sUS. Reproducibility of the assessment of malignancy risk features and categorization of nodules with EU-TIRADS was estimated with Krippendorff's alpha coefficient (Kα). The reproducibility of EU-TIRADS categories on sUS in relation to rtUS was in range 70.9-76.5% for all raters (Kα: 0.60-0.68) with the highest reproducibility for category 3 (80.0-86.5%) and the lowest for category 5 (48.7-77.8%). There was a total disagreement of the identification of microcalcifications on sUS in relation to rtUS, a strongly variable reproducibility of marked hypoechogenicity (12.5-84.6%, Kα: 0.14-0.48) and a tendency toward more frequent identification of the non-oval shape on sUS. The percentage of agreement for each pair of raters in assigning the EU-TIRADS category on sUS was in the range 71.6-72.3% (Kα: 0.60-0.62). The method of sonographic image evaluation influences the nodule's feature assessment and, eventually, the categorization within EU-TIRADS.

Diagnostic efficiency among Eu-/C-/ACR-TIRADS and S-Detect for thyroid nodules: a systematic review and network meta-analysis

Background

The performance in evaluating thyroid nodules on ultrasound varies across different risk stratification systems, leading to inconsistency and uncertainty regarding diagnostic sensitivity, specificity, and accuracy.

Objective

Comparing diagnostic performance of detecting thyroid cancer among distinct ultrasound risk stratification systems proposed in the last five years.

Evidence acquisition

Systematic search was conducted on PubMed, EMBASE, and Web of Science databases to find relevant research up to December 8, 2022, whose study contents contained elucidation of diagnostic performance of any one of the above ultrasound risk stratification systems (European Thyroid Imaging Reporting and Data System[Eu-TIRADS]; American College of Radiology TIRADS [ACR TIRADS]; Chinese version of TIRADS [C-TIRADS]; Computer-aided diagnosis system based on deep learning [S-Detect]). Based on golden diagnostic standard in histopathology and cytology, single meta-analysis was performed to obtain the optimal cut-off value for each system, and then network meta-analysis was conducted on the best risk stratification category in each system.

Evidence synthesis

This network meta-analysis included 88 studies with a total of 59,304 nodules. The most accurate risk category thresholds were TR5 for Eu-TIRADS, TR5 for ACR TIRADS, TR4b and above for C-TIRADS, and possible malignancy for S-Detect. At the best thresholds, sensitivity of these systems ranged from 68% to 82% and specificity ranged from 71% to 81%. It identified the highest sensitivity for C-TIRADS TR4b and the highest specificity for ACR TIRADS TR5. However, sensitivity for ACR TIRADS TR5 was the lowest. The diagnostic odds ratio (DOR) and area under curve (AUC) were ranked first in C-TIRADS.

Conclusion

Among four ultrasound risk stratification options, this systemic review preliminarily proved that C-TIRADS possessed favorable diagnostic performance for thyroid nodules.

Systematic review registration

https://www.crd.york.ac.uk/prospero, CRD42022382818.

Diagnostic Performance of Six Ultrasound Risk Stratification Systems for Thyroid Nodules: A Systematic Review and Network Meta-Analysis

BACKGROUND. Risk stratification systems for evaluating thyroid nodules on ultrasound use varying approaches to classify levels of suspicion for malignancy, leading to variable performance. OBJECTIVE. The purpose of this study was to perform a network meta-analysis comparing six risk stratification systems used to evaluate thyroid nodules on ultrasound in terms of their diagnostic performance for the detection of thyroid cancer. EVIDENCE ACQUISITION. Five bibliometric databases were searched for studies published through August 31, 2022, that compared at least two of six ultrasound risk stratification systems (the American Association of Clinical Endocrinologists, American College of Endocrinology, and Associazione Medici Endocrinologi [AACE/ACE/AME] system; American College of Radiology Thyroid Imaging Reporting and Data System [ACR TI-RADS]; the American Thyroid Association [ATA] risk stratification system; European Thyroid Association Thyroid Imaging Reporting and Data System [EU-TIRADS]; the Korean Thyroid Imaging Reporting and Data System [K-TIRADS] endorsed by the Korean Thyroid Association and the Korean Society of Thyroid Radiology; and the Thyroid Imaging Reporting and Data System developed by Kwak et al. [Kwak TIRADS]) in terms of their diagnostic performance for the detection of thyroid cancer, with cytologic or histologic evaluation used as a reference standard. The studies' risk of bias was evaluated using the Newcastle-Ottawa Scale. A meta-analysis of each system was performed to identify the risk category threshold that had the highest accuracy as well as the highest sensitivity and specificity at this threshold. Network meta-analysis was used to perform hierarchic ranking and identify the systems having the highest sensitivities and specificities at each system's most accurate threshold. EVIDENCE SYNTHESIS. The analysis included 39 studies with 49,661 patients. All studies were of fair (n = 17) or good (n = 22) quality. The most accurate risk category thresholds were class 3 (high risk) for the AACE/ACE/AME system, TR5 (highly suspicious) for ACR TI-RADS, EU-TIRADS 5 (high risk) for EU-TIRADS, 4c (moderate concern but not classic for malignancy) for Kwak TIRADS, K-TIRADS 5 (high suspicion) for K-TIRADS, and high suspicion for the ATA system. At these thresholds, the systems had sensitivity of 64-77% and specificity of 82-90%. Network meta-analysis identified the highest sensitivity and highest specificity for ACR TI-RADS, followed by K-TIRADS. CONCLUSION. Of six risk stratification systems, ACR TI-RADS had the highest diagnostic performance for the detection of thyroid nodules on ultrasound. CLINICAL IMPACT. This network meta-analysis can inform decisions regarding implementation of the risk stratification systems and can aid future system updates.

Effect of the categorization method on the diagnostic performance of ultrasound risk stratification systems for thyroid nodules

Objective

To evaluate whether the categorization methods of risk stratification systems (RSSs) is a decisive factor that influenced the diagnostic performances and unnecessary FNA rates in order to choose optimal RSS for the management of thyroid nodules.

Methods

From July 2013 to January 2019, 2667 patients with 3944 thyroid nodules had undergone pathological diagnosis after thyroidectomy and/or US-guided FNA. US categories were assigned according to the six RSSs. The diagnostic performances and unnecessary FNA rates were calculated and compared according to the US-based final assessment categories and the unified size thresholds for biopsy proposed by ACR-TIRADS, respectively.

Results

A total of 1781 (45.2%) thyroid nodules were diagnosed as malignant after thyroidectomy or biopsy. Significantly lowest specificity and accuracy, along with the highest unnecessary FNA rates were seen in EU-TIRADS for both US categories (47.9%, 70.2%, and 39.4%, respectively, all P < 0.05) and indications for FNA (54.2%, 50.0%, and 55.4%, respectively, all P < 0.05). Diagnostic performances for US-based final assessment categories exhibited similar accuracy for AI-TIRADS, Kwak-TIRADS, C-TIRADS, and ATA guidelines (78.0%, 77.8%, 77.9%, and 76.3%, respectively, all P > 0.05), while the lowest unnecessary FNA rate was seen in C-TIRADS (30.9%) and without significant differences to that of AI-TIRADS, Kwak-TIRADS, and ATA guideline (31.5%, 31.7%, and 33.6%, respectively, all P > 0.05). Diagnostic performance for US-FNA indications showed similar accuracy for ACR-TIRADS, Kwak-TIRADS, C-TIRADS and ATA guidelines (58.0%, 59.7%, 58.7%, and 57.1%, respectively, all P > 0.05). The highest accuracy and lowest unnecessary FNA rate were seen in AI-TIRADS (61.9%, 38.6%) and without significant differences to that of Kwak-TIRADS(59.7%, 42.9%) and C-TIRADS 58.7%, 43.9%, all P > 0.05).

Conclusion

The different US categorization methods used by each RSS were not determinant influential factors in diagnostic performance and unnecessary FNA rate. For daily clinical practice, the score-based counting RSS was an optimal choice.

Inter-Rater Reliability of Thyroid Ultrasound Risk Criteria: A Systematic Review and Meta-Analysis

OBJECTIVE

The most commonly employed diagnostic criteria for identifying thyroid nodules include Thyroid Imaging and Reporting Data System (TI-RADS) and American Thyroid Association (ATA) guidelines. The purpose of this systematic review and meta-analysis is to determine the inter-rater reliability of thyroid ultrasound criteria.

METHODS

We performed a library search of MEDLINE (Ovid), EMBASE (Ovid), and Web of Science for full-text articles published from January 2005 to June 2022. We included full-text primary research articles that used TI-RADS and/or ATA guidelines to evaluate thyroid nodules in adults. These included studies must have calculated inter-rater reliability using any validated metric. The Quality Appraisal for Reliability Studies (QAREL) was used to assess study quality. We planned for a random-effects meta-analysis, in addition to covariate and publication bias analyses. This study was performed in accordance with Preferred Reporting Items for a Systematic Review and Meta-analysis guidelines and registered prior to conduction (International prospective register of systematic reviews-PROSPERO: CRD42021275072).

RESULTS

Of the 951 articles identified via the database search, 35 met eligibility criteria. All studies were observational. The most commonly utilized criteria were ACR Thyroid Imaging and Reporting Data System (TI-RADS) and/or ATA criteria, while the majority of studies employed Κ statistics. For ACR TI-RADS, the pooled Κ was 0.51 (95% confidence interval [CI]: 0.42, 0.57; n = 7) while for ATA, the pooled Κ was 0.52 (95% CI: 0.37, 0.67; n = 3). Due to the small number of studies, covariate or publication bias analyses were not performed.

CONCLUSION

Ultrasound criteria demonstrate moderate inter-rater reliability, but these findings are impacted by poor study quality and a lack of standardization. Laryngoscope, 133:485-493, 2023.

Diagnostic Performance of Various Ultrasound Risk Stratification Systems for Benign and Malignant Thyroid Nodules: A Meta-Analysis

BACKGROUND

To evaluate the diagnostic performance of ultrasound risk-stratification systems for the discrimination of benign and malignant thyroid nodules and to determine the optimal cutoff values of individual risk-stratification systems.

METHODS

PubMed, Embase, SCOPUS, Web of Science, and Cochrane library databases were searched up to August 2022. Sensitivity and specificity data were collected along with the characteristics of each study related to ultrasound risk stratification systems.

RESULTS

Sixty-seven studies involving 76,512 thyroid nodules were included in this research. The sensitivity, specificity, diagnostic odds ratios, and area under the curves by K-TIRADS (4), ACR-TIRADS (TR5), ATA (high suspicion), EU-TIRADS (5), and Kwak-TIRADS (4b) for malignancy risk stratification of thyroid nodules were 92.5%, 63.5%, 69.8%, 70.6%, and 95.8%, respectively; 62.8%, 89.6%, 87.2%, 83.9%, and 63.8%, respectively; 20.7111, 16.8442, 15.7398, 12.2986, and 38.0578, respectively; and 0.792, 0.882, 0.859, 0.843, and 0.929, respectively.

CONCLUSION

All ultrasound-based risk-stratification systems had good diagnostic performance. Although this study determined the best cutoff values in individual risk-stratification systems based on statistical assessment, clinicians could adjust or alter cutoff values based on the clinical purpose of the ultrasound and the reciprocal changes in sensitivity and specificity.

Diagnostic Performance of ACR and Kwak TI-RADS for Benign and Malignant Thyroid Nodules: An Update Systematic Review and Meta-Analysis

(1) Background: To determine the optimal cut-off values of two risk stratification systems to discriminate malignant thyroid nodules and to compare the diagnostic performance; (2) Methods: True and false positive and negative data were collected, and methodological quality was assessed for forty-six studies involving 39,085 patients; (3) Results: The highest area under the receiver operating characteristic (ROC) curve (AUC) of ACR and Kwak TI-RADS were 0.875 and 0.884. Based on the optimal sensitivity and specificity, the highest accuracy values of ROC curves or diagnostic odds ratios (DOR) were taken as the cut-off values for TR4 (moderate suspicious) and 4B. The sensitivity, specificity, DOR, and AUC by ACR (TR4) and Kwak TI-RADS (4B) for malignancy risk stratification of thyroid nodules were 94.3% and 96.4%; 52.2% and 53.7%; 17.5185 and 31.8051; 0.786 and 0.884, respectively. There were no significant differences in diagnostic accuracy in any of the direction comparisons of the two systems; (4) Conclusions: ACR and Kwak TI-RADS had good diagnostic performances (AUCs > 85%). Although we determined the best cut-off values in individual risk stratification systems based on statistical assessment, clinicians can adjust the optimal cut-off value according to the clinical purpose of the ultrasonography because raising or lowering cut-points leads to reciprocal changes in sensitivity and specificity.

Evaluation of Ultrasound Elastography Combined With Chi-Square Automatic Interactive Detector in Reducing Unnecessary Fine-Needle Aspiration on TIRADS 4 Thyroid Nodules

Background

Conventional ultrasound diagnosis of thyroid nodules (TNs) had a high false-positive rate, resulting in many unnecessary fine-needle aspirations (FNAs).

Objective

This study aimed to establish a simple algorithm to reduce unnecessary FNA on TIRADS 4 TNs using different quantitative parameters of ultrasonic elasticity and chi-square automatic interactive detector (CHAID) method.

Methods

From January 2020 to May 2021, 432 TNs were included in the study, which were confirmed by FNA or surgical pathology. Each TN was examined using conventional ultrasound, sound touch elastography, and Shell measurement function. The quantitative parameters E and E_shell were recorded, and the E_shell/E values were calculated for each TN. The diagnostic performance of the quantitative parameters was evaluated using the receiver operating characteristic curves. The CHAID was used to classify and analyze the quantitative parameters, and the prediction model was established.

Results

A total of 226 TNs were malignant and 206 were benign. E_shell and E_shell/E ratio were included in the classification algorithm, which showed a depth of two ramifications (E_shell/E ≤ 0.988 or 0.988–1.043 or >1.043; if E_shell/E ≤ 0.988, then E_shell ≤ 64.0 or 64.0–74.0 or >74.0; if E_shell/E = 0.988–1.043, then E_shell ≤ 66.0 or > 66.0; if E_shell/E >1.043, then E_shell ≤ 69.0 or >69.0). The unnecessary FNAs could have been avoided in 57.3% of the cases using this algorithm.

Conclusion

The prediction model using quantitative parameters had high diagnostic performance; it could quickly distinguish benign lesions and avoid subjective influence to some extent.

Evaluation of the efficacy of EU-TIRADS and ACR-TIRADS in risk stratification of pediatric patients with thyroid nodules

BACKGROUND

Pediatric thyroid nodules have a lower prevalence but a higher rate of malignancy (ROM) than those in adults. Ultrasound features suspected of malignancy lead to fine needle aspiration biopsy (FNAB) and subsequent cytological determination, upon which management is decided. Based on the characteristics of ultrasound, to standardize clinician decisions and avoid unnecessary FNAB, the European Thyroid Association and the American Radiology College have established guidelines for Thyroid Imaging, Reporting and Data System (EU-TIRADS and ACR-TIRADS) for ROM stratification of thyroid nodules. The aim of this study is to evaluate the diagnostic performance of ACR-TIRADS and EU-TIRADS in pediatric age.

MATERIALS AND METHODS

Subjects younger than 18 years of age with thyroid nodules greater than 0.5 cm observed in the 2000-2020 period were included.

RESULTS

Data from 200 subjects were collected. The overall ROM was 13%, rising to 26% if nodules with a diameter >1 cm were considered. Patients with a malignant nodule were more likely to have a higher EU-TIRADS score (p=0.03). Missed cancer diagnoses were 26.9%. Using the EU-TIRADS system, 40% of FNABs could have been avoided, while this scoring system would have resulted in FNAB being performed in 12% of cases where the assessment of ultrasound features would not recommend FNAB. Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were 73.1%, 57.1%, 73.1%, and 50%, respectively. Even considering the ACR-TIRADS, a higher score correlated with a higher ROM (p<0.001). This system missed 6 diagnoses of cancer (23.1%). Using the ACR-TIRADS system, 45.3% of FNABs could have been avoided, while FNAB should have been performed in 12% of cases where it was not recommended by ultrasound characteristics. Sensitivity, specificity, PPV and NPV were 76.9%, 50%, 76.9%, and 42.9%, respectively.

CONCLUSION

The present study confirms the correspondence of the EU-TIRADS and ACR-TIRADS categories with respect to malignancy but indicates not entirely satisfactory performance compared to FNAB alone. However, the use of the two TIRADS systems should be encouraged in multicentre studies to increase their performance and establish paediatric-specific points in the scoring criteria.