Contrast-Enhanced Ultrasound Improves the Accuracy of the ACR TI-RADS in the Diagnosis of Thyroid Nodules Located in the Isthmus

The clinical value of artificial intelligence in assisting junior radiologists in thyroid ultrasound: a multicenter prospective study from real clinical practice

BACKGROUND

This study is to propose a clinically applicable 2-echelon (2e) diagnostic criteria for the analysis of thyroid nodules such that low-risk nodules are screened off while only suspicious or indeterminate ones are further examined by histopathology, and to explore whether artificial intelligence (AI) can provide precise assistance for clinical decision-making in the real-world prospective scenario.

METHODS

In this prospective study, we enrolled 1036 patients with a total of 2296 thyroid nodules from three medical centers. The diagnostic performance of the AI system, radiologists with different levels of experience, and AI-assisted radiologists with different levels of experience in diagnosing thyroid nodules were evaluated against our proposed 2e diagnostic criteria, with the first being an arbitration committee consisting of 3 senior specialists and the second being cyto- or histopathology.

RESULTS

According to the 2e diagnostic criteria, 1543 nodules were classified by the arbitration committee, and the benign and malignant nature of 753 nodules was determined by pathological examinations. Taking pathological results as the evaluation standard, the sensitivity, specificity, accuracy, and area under the receiver operating characteristic curve (AUC) of the AI systems were 0.826, 0.815, 0.821, and 0.821. For those cases where diagnosis by the Arbitration Committee were taken as the evaluation standard, the sensitivity, specificity, accuracy, and AUC of the AI system were 0.946, 0.966, 0.964, and 0.956. Taking the global 2e diagnostic criteria as the gold standard, the sensitivity, specificity, accuracy, and AUC of the AI system were 0.868, 0.934, 0.917, and 0.901, respectively. Under different criteria, AI was comparable to the diagnostic performance of senior radiologists and outperformed junior radiologists (all P < 0.05). Furthermore, AI assistance significantly improved the performance of junior radiologists in the diagnosis of thyroid nodules, and their diagnostic performance was comparable to that of senior radiologists when pathological results were taken as the gold standard (all p > 0.05).

CONCLUSIONS

The proposed 2e diagnostic criteria are consistent with real-world clinical evaluations and affirm the applicability of the AI system. Under the 2e criteria, the diagnostic performance of the AI system is comparable to that of senior radiologists and significantly improves the diagnostic capabilities of junior radiologists. This has the potential to reduce unnecessary invasive diagnostic procedures in real-world clinical practice.

Thyroid imaging reporting and data system with MRI morphological features for thyroid nodules: diagnostic performance and unnecessary biopsy rate

BACKGROUND

To assess MRI-based morphological features in improving the American College of Radiology Thyroid Imaging Reporting and Data System (ACR-TIRADS) for categorizing thyroid nodules.

METHODS

A retrospective analysis was performed on 728 thyroid nodules (453 benign and 275 malignant) that postoperative pathology confirmed. Univariate and multivariate logistic regression analyses were used to find independent predictors of MRI morphological features in benign and malignant thyroid nodules. The improved method involved increasing the ACR-TIRADS level by one when there are independent predictors of MRI-based morphological features, whether individually or in combination, and conversely decreasing it by one. The study compared the performance of conventional ACR-TIRADS and different improved versions.

RESULTS

Among the various MRI morphological features analyzed, restricted diffusion and reversed halo sign were determined to be significant independent risk factors for malignant thyroid nodules (OR = 45.1, 95% CI = 23.2-87.5, P < 0.001; OR = 38.0, 95% CI = 20.4-70.7, P < 0.001) and were subsequently included in the final assessment of performance. The areas under the receiver operating characteristic curves (AUCs) for both the conventional and four improved ACR-TIRADSs were 0.887 (95% CI: 0.861-0.909), 0.945 (95% CI: 0.926-0.961), 0.947 (95% CI: 0.928-0.962), 0.945 (95% CI: 0.926-0.961) and 0.951 (95% CI: 0.932-0.965), respectively. The unnecessary biopsy rates for the conventional and four improved ACR-TIRADSs were 62.8%, 30.0%, 27.1%, 26.8% and 29.1%, respectively, while the malignant missed diagnosis rates were 1.1%, 2.8%, 3.7%, 5.4% and 1.2%.

CONCLUSIONS

MRI morphological features with ACR-TIRADS has improved diagnostic performance and reduce unnecessary biopsy rate while maintaining a low malignant missed diagnosis rate.

Incorporation of CEUS and SWE parameters into a multivariate logistic regression model for the differential diagnosis of benign and malignant TI-RADS 4 thyroid nodules

PURPOSE

To investigate the diagnostic value of contrast-enhanced ultrasound (CEUS) quantitative analysis parameters combined with shear wave elastography (SWE) quantitative parameters in the differentiation of benign and malignant ACR TI-RADS category 4 thyroid nodules and to provide a more effective reference for clinical work.

METHODS

We analyzed 187 category 4 nodules, including 132 nodules in the development cohort and 55 nodules in the validation cohort, divided the development cohort into benign and malignant groups, and analyzed the differences in all CEUS and SWE quantitative parameters between the two groups. We selected the highest AUC of the two parameters, performed binary logistic regression analysis with the ACR TI-RADS score and constructed a diagnostic model. ROC curves were applied to evaluate their diagnostic efficacy.

RESULTS

1) The diagnostic model had an AUC of 0.926, sensitivity of 87.5%, specificity of 86.8%, diagnostic threshold of 3, accuracy of 87.12%, positive predictive value of 86.15%, and negative predictive value of 88.06%. 2) The diagnostic model had an AUC of 0.890 in the validation cohort, sensitivity of 81.5%, specificity of 79.6%, and accuracy of 80.00%.

CONCLUSION

The combined multiparameter construction of the nodule diagnostic model can effectively improve the diagnostic efficacy of 4 types of thyroid nodules and provide a new reference index for clinical diagnostic work.

Diagnostic efficiency among Eu-/C-/ACR-TIRADS and S-Detect for thyroid nodules: a systematic review and network meta-analysis

Background

The performance in evaluating thyroid nodules on ultrasound varies across different risk stratification systems, leading to inconsistency and uncertainty regarding diagnostic sensitivity, specificity, and accuracy.

Objective

Comparing diagnostic performance of detecting thyroid cancer among distinct ultrasound risk stratification systems proposed in the last five years.

Evidence acquisition

Systematic search was conducted on PubMed, EMBASE, and Web of Science databases to find relevant research up to December 8, 2022, whose study contents contained elucidation of diagnostic performance of any one of the above ultrasound risk stratification systems (European Thyroid Imaging Reporting and Data System[Eu-TIRADS]; American College of Radiology TIRADS [ACR TIRADS]; Chinese version of TIRADS [C-TIRADS]; Computer-aided diagnosis system based on deep learning [S-Detect]). Based on golden diagnostic standard in histopathology and cytology, single meta-analysis was performed to obtain the optimal cut-off value for each system, and then network meta-analysis was conducted on the best risk stratification category in each system.

Evidence synthesis

This network meta-analysis included 88 studies with a total of 59,304 nodules. The most accurate risk category thresholds were TR5 for Eu-TIRADS, TR5 for ACR TIRADS, TR4b and above for C-TIRADS, and possible malignancy for S-Detect. At the best thresholds, sensitivity of these systems ranged from 68% to 82% and specificity ranged from 71% to 81%. It identified the highest sensitivity for C-TIRADS TR4b and the highest specificity for ACR TIRADS TR5. However, sensitivity for ACR TIRADS TR5 was the lowest. The diagnostic odds ratio (DOR) and area under curve (AUC) were ranked first in C-TIRADS.

Conclusion

Among four ultrasound risk stratification options, this systemic review preliminarily proved that C-TIRADS possessed favorable diagnostic performance for thyroid nodules.

Systematic review registration

https://www.crd.york.ac.uk/prospero, CRD42022382818.

The use of modified TI-RADS using contrast-enhanced ultrasound features for classification purposes in the differential diagnosis of benign and malignant thyroid nodules: A prospective and multi-center study

OBJECTIVES

To evaluate the diagnostic efficacy of a modified thyroid imaging reporting and data system (TI-RADS) in combination with contrast-enhanced ultrasound (CEUS) for differentiating between benign and malignant thyroid nodules and to assess inter-observer concordance between different observers.

METHODS

This study included 3353 patients who underwent thyroid ultrasound (US) and CEUS in ten multi-centers between September 2018 and March 2020. Based on a modified TI-RADS classification using the CEUS enhancement pattern of thyroid lesions, ten radiologists analyzed all US and CEUS examinations independently and assigned a TI-RADS category to each thyroid nodule. Pathology was the reference standard for determining the diagnostic performance (accuracy (ACC), sensitivity (SEN), specificity (SPN), positive predictive value (PPV), and negative predictive value (NPV)) of the modified TI-RADS for predicting malignant thyroid nodules. The risk of malignancy was stratified for each TI-RADS category-based on the total number of benign and malignant lesions in that category. ROC curve was used to determine the cut-off value and the area under the curve (AUC). Cohen's Kappa statistic was applied to assess the inter-observer agreement of each sonological feature and TI-RADS category for thyroid nodules.

RESULTS

The calculated malignancy risk in the modified TI-RADS categories 5, 4b, 4a, 3 and 2 nodules was 95.4%, 86.0%, 12.0%, 4.1% and 0%, respectively. The malignancy risk for the five categories was in agreement with the suggested malignancy risk. The ROC curve showed that the AUC under the ROC curve was 0.936, and the cutoff value of the modified TI-RADS classification was >TI-RADS 4a, whose SEN, ACC, PPV, NPV and SPN were 93.6%, 91.9%, 90.4%, 93.7% and 88.5% respectively. The Kappa value for taller than wide, microcalcification, marked hypoechoic, solid composition, irregular margins and enhancement pattern of CEUS was 0.94, 0.93, 0.75, 0.89, 0.86 and 0.81, respectively. There was also good agreement between the observers with regards to the modified TI-RADS classification, the Kappa value was 0.80.

CONCLUSIONS

The actual risk of malignancy according to the modified TI-RADS concurred with the suggested risk of malignancy. Inter-observer agreement for the modified TI-RADS category was good, thus suggesting that this classification was very suitable for clinical application.

Multiparametric Sonographic Imaging of Thyroid Lesions: Chances of B-Mode, Elastography and CEUS in Relation to Preoperative Histopathology

Simple Summary

As the incidence of thyroid lesions in Europe is rising, more and more people affected by thyroid pathologies seek treatment in a clinic. Every suspicious thyroid nodule needs to be confirmed as benign or malignant in order to be treated correctly. Unnecessary invasive diagnostics and thyroid surgery should be avoided. The aim of this retrospective study was to improve the distinction between benign and malignant nodules by using new high-performance multiparametric ultrasound examination techniques. By analyzing 122 thyroid nodules we created a score-based system combining B-mode, shear-wave elastography and contrast-enhanced ultrasound malignancy criteria. This system allows for a quite accurate detection of thyroid carcinomas with a sensitivity of 95% and specificity of 75.49%. Shear-wave elastography and contrast-enhanced ultrasound can detect unique malignancy features, which cannot be found in B-mode. Therefore, these criteria would present a relevant addition to the B-mode TI-RADS classification.

Abstract

Background: The aim was to improve preoperative diagnostics of solid non-cystic thyroid lesions by using new high-performance multiparametric ultrasound examination techniques. Methods: Multiparametric ultrasound consists of B-mode, shear-wave elastography and contrast enhanced ultrasound (CEUS) including Time-Intensity-Curve (TIC) analysis. A bolus of 1–2.4 mL Sulfur Hexafluorid microbubbles was injected for CEUS. Postoperative histopathology was the diagnostic gold standard. Results: 116 patients were included in this study. 102 benign thyroid nodules were diagnosed as well as 20 carcinomas. Suspicious B-mode findings like microcalcifications, a blurry edge and no homogeneous sonomorphological structure were detected in 60, 75 and 80% of all carcinomas but only in 13.7, 36.3 and 46.1% of all benign lesions. The average shear-wave elastography measurements of malignant lesions (4.6 m/s or 69.8 kPa centrally and 4.2 m/s or 60.1 kPa marginally) exceed the values of benign nodules. Suspicious CEUS findings like a not-homogeneous wash-in and a wash-out were detected almost twice as often in carcinomas. Conclusion: Multiparametric ultrasound offers new possibilities for the preoperative distinction between benign and malignant thyroid nodules. A score based system of B-mode, shear-wave and CEUS malignancy criteria shows promising results in the detection of thyroid carcinomas. It reaches a sensitivity of 95% and specificity of 75.49%.

The diagnostic value of a nomogram based on multimodal ultrasonography for thyroid-nodule differentiation: A multicenter study

Objective

To establish and verify a nomogram based on multimodal ultrasonography (US) for the assessment of the malignancy risk of thyroid nodules and to explore its value in distinguishing benign from malignant thyroid nodules.

Methods

From September 2020 to December 2021, the data of 447 individuals with thyroid nodules were retrieved from the multicenter database of medical images of the National Health Commission’s Capacity Building and Continuing Education Center, which includes data from more than 20 hospitals. All patients underwent contrast-enhanced US (CEUS) and elastography before surgery or fine needle aspiration. The training set consisted of three hundred datasets from the multicenter database (excluding Zhejiang Cancer Hospital), and the external validation set consisted of 147 datasets from Zhejiang Cancer Hospital. As per the pathological results, the training set was separated into benign and malignant groups. The characteristics of the lesions in the two groups were analyzed and compared using conventional US, CEUS, and elastography score. Using multivariate logistic regression to screen independent predictive risk indicators, then a nomogram for risk assessment of malignant thyroid nodules was created. The diagnostic performance of the nomogram was assessed utilizing calibration curves and receiver operating characteristic (ROC) from the training and validation cohorts. The nomogram and The American College of Radiology Thyroid Imaging, Reporting and Data System were assessed clinically using decision curve analysis (DCA).

Results

Multivariate regression showed that irregular shape, elastography score (≥ 3), lack of ring enhancement, and unclear margin after enhancement were independent predictors of malignancy. During the training (area under the ROC [AUC]: 0.936; 95% confidence interval [CI]: 0.902–0.961) and validation (AUC: 0.902; 95% CI: 0.842–0.945) sets, the multimodal US nomogram with these four variables demonstrated good calibration and discrimination. The DCA results confirmed the good clinical applicability of the multimodal US nomogram for predicting thyroid cancer.

Conclusions

As a preoperative prediction tool, our multimodal US-based nomogram showed good ability to distinguish benign from malignant thyroid nodules.

Systematic Review and Meta-Analysis of American College of Radiology TI-RADS Inter-Reader Reliability for Risk Stratification of Thyroid Nodules

Purpose

To investigate the inter-reader agreement of using the American College of Radiology (ACR) Thyroid Imaging Reporting and Data System (TI-RADS) for risk stratification of thyroid nodules.

Methods

A literature search of Web of Science, PubMed, Cochrane Library, EMBASE, and Google Scholar was performed to identify eligible articles published from inception until October 31, 2021. We included studies reporting inter-reader agreement of different radiologists who applied ACR TI-RADS for the classification of thyroid nodules. Quality assessment of the included studies was performed with the Quality Assessment of Diagnostic Accuracy Studies-2 tool and Guidelines for Reporting Reliability and Agreement Studies. The summary estimates of the inter-reader agreement were pooled with the random-effects model, and multiple subgroup analyses and meta-regression were performed to investigate various clinical settings.

Results

A total of 13 studies comprising 5,238 nodules were included in the current meta-analysis and systematic review. The pooled inter-reader agreement for overall ACR TI-RADS classification was moderate (κ = 0.51, 95% CI 0.42–0.59). Substantial heterogeneity was presented throughout the studies, and meta-regression analyses suggested that the malignant rate was the significant factor. Regarding the ultrasound (US) features, the best inter-reader agreement was composition (κ = 0.58, 95% CI 0.53–0.63), followed by shape (κ = 0.57, 95% CI 0.41–0.72), echogenicity (κ = 0.50, 95% CI 0.40–0.60), echogenic foci (κ = 0.44, 95% CI 0.36–0.53), and margin (κ = 0.34, 95% CI 0.24–0.44).

Conclusions

The ACR TI-RADS demonstrated moderate inter-reader agreement between radiologists for the overall classification. However, the US feature of margin only showed fair inter-reader reliability among different observers.