Learnability and reproducibility of ACR Thyroid Imaging, Reporting and Data System (TI-RADS) in postgraduate freshmen

Development and Validation of a Nomogram Based on Multimodality Ultrasonography Images for Differentiating Malignant from Benign American College of Radiology Thyroid Imaging, Reporting and Data System (TI-RADS) 3-5 Thyroid Nodules

OBJECTIVE

The aim of the work described here was to develop and validate a predictive nomogram based on combined image features of gray-scale ultrasonography (US), elastosonography (ES) and contrast-enhanced US (CEUS) to differentiate malignant from benign American College of Radiology Thyroid Imaging, Reporting and Data System (ACR TI-RADS) 3-5 thyroid nodules.

METHODS

Among 2767 thyroid nodules scanned by CEUS in Xijing Hospital between April 2014 and November 2018, 669 nodules classified as ACR TI-RADS 3-5 were included, with confirmed diagnosis and ES examination. Four hundred fifty-five nodules were set as a training cohort and 214 as a validation cohort. Images were categorized as gray-scale US ACR TI-RADS 3, TI-RADS 4 and TI-RADS 5; ES patterns of ES-1 and ES-2; and CEUS patterns of either heterogeneous hypo-enhancement, concentric hypo-enhancement, homogeneous hyper-/iso-enhancement, no perfusion, hypo-enhancement with sharp margin, island-like enhancement or ring-like enhancement. On the basis of multivariate logistic regression analysis, a predictive nomogram model was developed and validated by receiver operating characteristic curve analysis.

RESULTS

In the training cohort, ACR TI-RADS 4 and 5, ES-2, heterogeneous hypo-enhancement, concentric hypo-enhancement and homogeneous hyper-/iso-enhancement were selected as predictors of malignancy by univariate logistic regression analysis. A predictive nomogram (combining indices of ACR TI-RADS, ES and CEUS) indicated excellent predictive ability for differentiating malignant from benign lesions in the training cohort: area under the receiver operating characteristic curve (AUC) = 0.93, 95% confidence interval (CI): 0.90-0.95. The prediction nomogram model was determined to have a sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of 0.84, 0.88, 0.91 and 0.81. In the validation cohort, the AUC of the prediction nomogram model was significantly higher than those of the single modalities (p < 0.005) . The AUCs of the validation cohort were 0.93 (95% CI: 0.89-0.96) and 0.93 (95% CI: 0.89-0.97), respectively, for senior and junior radiologists. The prediction nomogram model has a sensitivity, specificity, PPV and NPV of 0.86, 0.87, 0.87 and 0.86.

CONCLUSION

A predictive nomogram model combining ACR TI-RADS, ES and CEUS exhibited potential clinical utility in differentiating malignant from benign ACR TI-RADS 3-5 thyroid nodules.

Learnability of Thyroid Nodule Assessment on Ultrasonography: Using a Big Data Set

OBJECTIVE

The aims of the work described here were to evaluate the learnability of thyroid nodule assessment on ultrasonography (US) using a big data set of US images and to evaluate the diagnostic utilities of artificial intelligence computer-aided diagnosis (AI-CAD) used by readers with varying experience to differentiate benign and malignant thyroid nodules.

METHODS

Six college freshmen independently studied the "learning set" composed of images of 13,560 thyroid nodules, and their diagnostic performance was evaluated after their daily learning sessions using the "test set" composed of images of 282 thyroid nodules. The diagnostic performance of two residents and an experienced radiologist was evaluated using the same "test set." After an initial diagnosis, all readers once again evaluated the "test set" with the assistance of AI-CAD.

RESULTS

Diagnostic performance of almost all students increased after the learning program. Although the mean areas under the receiver operating characteristic curves (AUROCs) of residents and the experienced radiologist were significantly higher than those of students, the AUROCs of five of the six students did not differ significantly compared with that of the one resident. With the assistance of AI-CAD, sensitivity significantly increased in three students, specificity in one student, accuracy in four students and AUROC in four students. Diagnostic performance of the two residents and the experienced radiologist was better with the assistance of AI-CAD.

CONCLUSION

A self-learning method using a big data set of US images has potential as an ancillary tool alongside traditional training methods. With the assistance of AI-CAD, the diagnostic performance of readers with varying experience in thyroid imaging could be further improved.

Prediction of cytology-histology discrepancy when Bethesda cytology reports benign results for thyroid nodules in women: with special emphasis on pregnancy

Objectives: Benign category of Bethesda classification is generally well known to carry a false-negative rate of 0-3%. The current study was designed to investigate the rate of false-negative cytology in patients who underwent thyroidectomy for presumably benign thyroid diseases. Predictive risk factors for false results and malignancy were evaluated along with cytology-histology discrepant cases.Materials and methods: Females who underwent thyroidectomy between May 2014 and December 2022 were included. Demographics, ultrasound (US) features, fine-needle aspiration (FNA) diagnosis, surgical indications and outcomes, final histology reports, risk factors, and malignancy rate were recorded. Cytology-histology discrepant cases were further evaluated for interpretation errors and risk factors. Statistical analyses were performed using Fisher's exact and Mann-Whitney U tests.Results: Of 581 women with a benign thyroid disease who underwent thyroidectomy, 91 was diagnosed as incidental carcinoma (15.6%) and most was T1a (4.9 ± 2.7 mm, 95.6%). Final histology reports revealed mostly papillary carcinoma (93.4%). Predictors of malignancy such as age, family history, previous radiation exposure, and iodine-deficient diet did not help in risk stratification (p > 0.05, for each). However, FNA taken during pregnancy was determined as a risk factor (n = 7, 7.6%, p < 0.05) since it may cause a delay in diagnosis. Cytology-histology discrepant cases were seen to be mostly due to sampling errors (45%, p < 0.05), followed by misinterpretations (37.3%, p < 0.05). There was no reason for discrepancy in 17.5%, and this was linked to inherent nature of thyroid nodule with overlapping cytologic features. Best identifiable risk factor for misinterpretation was pregnancy as well (n = 5, 14.7%, p < 0.05).Conclusions: Risk of malignancy in a presumably benign thyroid disease should not be ignored. Radiology-cytology correlation by an experienced dedicated team may help in decreasing sampling errors. Physiologic changes caused by pregnancy may shade malignant transformation in thyrocytes, and it would be appropriate to be cautious about benign FNA taken during this period.

The use of modified TI-RADS using contrast-enhanced ultrasound features for classification purposes in the differential diagnosis of benign and malignant thyroid nodules: A prospective and multi-center study

OBJECTIVES

To evaluate the diagnostic efficacy of a modified thyroid imaging reporting and data system (TI-RADS) in combination with contrast-enhanced ultrasound (CEUS) for differentiating between benign and malignant thyroid nodules and to assess inter-observer concordance between different observers.

METHODS

This study included 3353 patients who underwent thyroid ultrasound (US) and CEUS in ten multi-centers between September 2018 and March 2020. Based on a modified TI-RADS classification using the CEUS enhancement pattern of thyroid lesions, ten radiologists analyzed all US and CEUS examinations independently and assigned a TI-RADS category to each thyroid nodule. Pathology was the reference standard for determining the diagnostic performance (accuracy (ACC), sensitivity (SEN), specificity (SPN), positive predictive value (PPV), and negative predictive value (NPV)) of the modified TI-RADS for predicting malignant thyroid nodules. The risk of malignancy was stratified for each TI-RADS category-based on the total number of benign and malignant lesions in that category. ROC curve was used to determine the cut-off value and the area under the curve (AUC). Cohen's Kappa statistic was applied to assess the inter-observer agreement of each sonological feature and TI-RADS category for thyroid nodules.

RESULTS

The calculated malignancy risk in the modified TI-RADS categories 5, 4b, 4a, 3 and 2 nodules was 95.4%, 86.0%, 12.0%, 4.1% and 0%, respectively. The malignancy risk for the five categories was in agreement with the suggested malignancy risk. The ROC curve showed that the AUC under the ROC curve was 0.936, and the cutoff value of the modified TI-RADS classification was >TI-RADS 4a, whose SEN, ACC, PPV, NPV and SPN were 93.6%, 91.9%, 90.4%, 93.7% and 88.5% respectively. The Kappa value for taller than wide, microcalcification, marked hypoechoic, solid composition, irregular margins and enhancement pattern of CEUS was 0.94, 0.93, 0.75, 0.89, 0.86 and 0.81, respectively. There was also good agreement between the observers with regards to the modified TI-RADS classification, the Kappa value was 0.80.

CONCLUSIONS

The actual risk of malignancy according to the modified TI-RADS concurred with the suggested risk of malignancy. Inter-observer agreement for the modified TI-RADS category was good, thus suggesting that this classification was very suitable for clinical application.

Multimodal ultrasound imaging: A method to improve the accuracy of diagnosing thyroid TI-RADS 4 nodules

Thyroid nodule is a common and frequently occurring disease in the neck in recent years, and ultrasound has become the preferred imaging diagnosis method for thyroid nodule due to its advantages of noninvasive, nonradiation, real-time, and repeatable. The thyroid imaging, reporting and data system (TI-RADS) classification standard scores suspicious nodules that are difficult to determine benign and malignant as grade 4, and further pathological puncture is recommended clinically, which may lead to a large number of unnecessary biopsies and operations. Including conventional ultrasound, ACR TI-RADS, shear wave elastography, super microvascular imaging, contrast enhanced ultrasound, "firefly," artificial intelligence, and multimodal ultrasound imaging used in combination. In order to identify the most clinically significant malignant tumors when reducing invasive operations. This article reviews the application and research progress of multimodal ultrasound imaging in the diagnosis of TI-RADS 4 thyroid nodules.

Effect of training on resident inter-reader agreement with American College of Radiology Thyroid Imaging Reporting and Data System

BACKGROUND

The American College of Radiology Thyroid Imaging Reporting and Data System (ACR TI-RADS) was introduced to standardize the ultrasound characterization of thyroid nodules. Studies have shown that ACR-TIRADS reduces unnecessary biopsies and improves consistency of imaging recommendations. Despite its widespread adoption, there are few studies to date assessing the inter-reader agreement amongst radiology trainees with limited ultrasound experience. We hypothesize that in PGY-4 radiology residents with no prior exposure to ACR TI-RADS, a statistically significant improvement in inter-reader reliability can be achieved with a one hour training session.

AIM

To evaluate the inter-reader agreement of radiology residents in using ACR TI-RADS before and after training.

METHODS

A single center retrospective cohort study evaluating 50 thyroid nodules in 40 patients of varying TI-RADS levels was performed. Reference standard TI-RADS scores were established through a consensus panel of three fellowship-trained staff radiologists with between 1 and 14 years of clinical experience each. Three PGY-4 radiology residents (trainees) were selected as blinded readers for this study. Each trainee had between 4 to 5 mo of designated ultrasound training. No trainee had received specialized TI-RADS training prior to this study. Each of the readers independently reviewed the 50 testing cases and assigned a TI-RADS score to each case before and after TI-RADS training performed 6 wk apart. Fleiss kappa was used to measure the pooled inter-reader agreement. The relative diagnostic performance of readers, pre- and post-training, when compared against the reference standard.

RESULTS

There were 33 females and 7 males with a mean age of 56.6 ± 13.6 years. The mean nodule size was 19 ± 14 mm (range from 5 to 63 mm). A statistically significant superior inter-reader agreement was found on the post-training assessment compared to the pre-training assessment for the following variables: 1. “Shape” (k of 0.09 [slight] pre-training vs 0.67 [substantial] post-training, P < 0.001), 2. “Echogenic foci” (k of 0.28 [fair] pre-training vs 0.45 [moderate] post-training, P = 0.004), 3. ‘TI-RADS level’ (k of 0.14 [slight] pre-training vs 0.36 [fair] post-training, P < 0.001) and 4. ‘Recommendations’ (k of 0.36 [fair] pre-training vs 0.50 [moderate] post-training, P = 0.02). No significant differences between the pre- and post-training assessments were found for the variables 'composition', 'echogenicity' and 'margins'. There was a general trend towards improved pooled sensitivity with TI-RADS levels 1 to 4 for the post-training assessment while the pooled specificity was relatively high (76.6%-96.8%) for all TI-RADS level.

CONCLUSION

Statistically significant improvement in inter-reader agreement in the assigning TI-RADS level and recommendations after training is observed. Our study supports the use of dedicated ACR TI-RADS training in radiology residents.