Ultrasound-based clinical prediction rule model for detecting papillary thyroid cancer in cervical lymph nodes: A pilot study

Incorporation of a machine learning pathological diagnosis algorithm into the thyroid ultrasound imaging data improves the diagnosis risk of malignant thyroid nodules

Objective

This study aimed at establishing a new model to predict malignant thyroid nodules using machine learning algorithms.

Methods

A retrospective study was performed on 274 patients with thyroid nodules who underwent fine-needle aspiration (FNA) cytology or surgery from October 2018 to 2020 in Xianyang Central Hospital. The least absolute shrinkage and selection operator (lasso) regression analysis and logistic analysis were applied to screen and identified variables. Six machine learning algorithms, including Decision Tree (DT), Extreme Gradient Boosting (XGBoost), Gradient Boosting Machine (GBM), Naive Bayes Classifier (NBC), Random Forest (RF), and Logistic Regression (LR), were employed and compared in constructing the predictive model, coupled with preoperative clinical characteristics and ultrasound features. Internal validation was performed by using 10-fold cross-validation. The performance of the model was measured by the area under the receiver operating characteristic curve (AUC), accuracy, precision, recall, F1 score, Shapley additive explanations (SHAP) plot, feature importance, and correlation of features. The best cutoff value for risk stratification was identified by probability density function (PDF) and clinical utility curve (CUC).

Results

The malignant rate of thyroid nodules in the study cohort was 53.2%. The predictive models are constructed by age, margin, shape, echogenic foci, echogenicity, and lymph nodes. The XGBoost model was significantly superior to any one of the machine learning models, with an AUC value of 0.829. According to the PDF and CUC, we recommended that 51% probability be used as a threshold for determining the risk stratification of malignant nodules, where about 85.6% of patients with malignant nodules could be detected. Meanwhile, approximately 89.8% of unnecessary biopsy procedures would be saved. Finally, an online web risk calculator has been built to estimate the personal likelihood of malignant thyroid nodules based on the best-performing ML-ed model of XGBoost.

Conclusions

Combining clinical characteristics and features of ultrasound images, ML algorithms can achieve reliable prediction of malignant thyroid nodules. The online web risk calculator based on the XGBoost model can easily identify in real-time the probability of malignant thyroid nodules, which can assist clinicians to formulate individualized management strategies for patients.

Radiologist versus Non-Radiologist Detection of Lymph Node Metastasis in Papillary Thyroid Carcinoma by Ultrasound: A Meta-Analysis

Papillary thyroid carcinoma (PTC) is the most common thyroid cancer worldwide and is known to spread to adjacent neck lymphatics. Lymph node metastasis (LNM) is a known predictor of disease recurrence and is an indicator for aggressive resection. Our study aims to determine if ultrasound sonographers’ degree of training influences overall LNM detection. PubMed, Embase, and Scopus articles were searched and screened for relevant articles. Two investigators independently screened and extracted the data. Diagnostic test parameters were determined for all studies, studies reported by radiologists, and studies reported by non-radiologists. The total sample size amounted to 5768 patients and 10,030 lymph nodes. Radiologists performed ultrasounds in 18 studies, while non-radiologists performed ultrasounds in seven studies, corresponding to 4442 and 1326 patients, respectively. The overall sensitivity of LNM detection by US was 59% (95%CI = 58–60%), and the overall specificity was 85% (95%CI = 84–86%). The sensitivity and specificity of US performed by radiologists were 58% and 86%, respectively. The sensitivity and specificity of US performed by non-radiologists were 62% and 78%, respectively. Summary receiver operating curve (sROC) found radiologists and non-radiologists to detect LNM on US with similar accuracy (p = 0.517). Our work suggests that both radiologists and non-radiologists alike detect overall LNM with high accuracy on US.

Ultrasonographic features of cervical lymph node metastases from medullary thyroid cancer: a retrospective study

Background

To investigate sonographic features of cervical lymph node metastases from medullary thyroid cancer (LNM-MTC), as compared with lymph node metastases from papillary thyroid cancer (LNM-PTC).

Methods

A total of 42 MTC patients with 52 metastatic LNs and 222 PTC patients with 234 metastatic LNs who were confirmed by fine needle aspiration and post-operative pathology, were enrolled in this study. The clinical characteristics and sonographic features of LNs were compared between the two groups. Univariate analysis and multivariate logistic regression analysis were performed on the sonographic features of LNs, including short and long-axis diameter, long-axis diameter/short-axis, shape, border, hilum, echogenicity, calcifications, cystic change and vascularity pattern. The discriminating performance was assessed with the area under the receiver operating characteristic curve (AUC).

Results

The mean age of patients with LNM-MTC was older than that of patients with LNM-PTC (46.81 ± 13.05 vs 39.09 ± 12.05, P < 0.001). No differences were observed in gender, location, long-axis diameter/short-axis, shape, border, echogenicity, cystic change and vascularity pattern between LNM-MTC and LNM-PTC groups (P > 0.05, for all). However, long-axis and short-axis diameter, hilum and calcifications were statistically different between these two groups (P < 0.05, for all). The AUC of discriminate value between LNM-MTC and LNM-PTC was 0.808 (95% confidence interval 0.739–0.877).

Conclusion

Compared with LNM-PTC, LNM-MTC tended to have the sonographic characteristics of larger size, absence of hilum, and less calcifications, and awareness of these features might be helpful to in the diagnosis of LNM-MTC.

Diagnostic Test Accuracy of Ultrasonography vs Computed Tomography for Papillary Thyroid Cancer Cervical Lymph Node Metastasis: A Systematic Review and Meta-analysis

Importance

The use of ultrasonography (US) vs cross-sectional imaging for preoperative evaluation of papillary thyroid cancer is debated.

Objective

To compare thyroid US and computed tomography (CT) in the preoperative evaluation of papillary thyroid cancer for cervical lymph node metastasis (CLNM), as well as extrathyroidal disease extension.

Data Sources

MEDLINE and Embase were searched from January 1, 2000, to July 18, 2020.

Study Selection

Studies reporting on the diagnostic accuracy of US and/or CT in individuals with treatment-naive papillary thyroid cancer for CLNM and/or extrathyroidal disease extension were included. The reference standard was defined as histopathology/cytology or imaging follow-up. Independent title and abstract review (2515 studies) followed by full-text review (145 studies) was completed by multiple investigators.

Data Extraction and Synthesis

PRISMA guidelines were followed. Methodologic and diagnostic accuracy data were abstracted independently by multiple investigators. Risk of bias assessment was conducted using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool independently and in duplicate. Bivariate random-effects model meta-analysis and multivariable meta-regression modeling was used.

Main Outcomes and Measures

Diagnostic test accuracy of US and CT of the neck for lateral and central compartment CLNM, as well as for extrathyroidal disease extension, determined prior to study commencement.

Results

A total of 47 studies encompassing 31 942 observations for thyroid cancer (12 771 with CLNM; 1747 with extrathyroidal thyroid extension) were included; 21 and 26 studies were at low and high risk for bias, respectively. Based on comparative design studies, US and CT demonstrated no significant difference in sensitivity (73% [95% CI, 64%-80%] and 77% [95% CI, 67%-85%], respectively; P = .11) or specificity (89% [95% CI, 80%-94%] and 88% [95% CI, 79%-94%], respectively; P = .79) for lateral compartment CLNM. For central compartment metastasis, sensitivity was higher in CT (39% [95% CI, 27%-52%]) vs US (28% [95% CI, 21%-36%]; P = .004), while specificity was higher in US (95% [95% CI, 92%-98%]) vs CT (87% [95% CI, 77%-93%]; P < .001). Ultrasonography demonstrated a sensitivity of 91% (95% CI, 81%-96%) and specificity of 47% (95% CI, 35%-60%) for extrathyroidal extension.

Conclusions and Relevance

The findings of this systematic review and meta-analysis suggest that further study is warranted of the role of CT for papillary thyroid cancer staging, possibly as an adjunct to US.

Predictors and a Prediction Model for Central Cervical Lymph Node Metastasis in Papillary Thyroid Carcinoma (cN0)

OBJECTIVES

To screen out the predictors of central cervical lymph node metastasis (CLNM) for papillary thyroid carcinoma (PTC) and establish a prediction model to guide the operation of PTC patients (cN0).

METHODS

Data from 296 PTC patients (cN0) who underwent thyroid operation at the Second Affiliated Hospital of Chongqing Medical University were collected and retrospectively analyzed. They were divided into two groups in accordance with central CLNM or not. Their information, including ultrasound (US) features, BRAF^V600E status, and other characteristics of the two groups, was analyzed and compared using univariate and multivariate logistic regression analyses, and the independent predictors were selected to construct a nomogram. The calibration plot, C-index, and decision curve analysis were used to assess the prediction model's calibration, discrimination, and clinical usefulness.

RESULTS

A total of 37.8% (112/296) of PTC patients had central CLNM, and 62.2% (184/296) did not. The two groups were compared using a univariate logistic regression analysis, and there were no significant differences between the two groups in sex, aspect ratio, boundary, morphology, hypoechoic nodule, thyroid peroxidase antibody, or tumor location (P>0.05), and there were significant differences between age, tumor size, capsule contact, microcalcifications, blood flow signal, thyroglobulin antibodies (TgAb), and BRAF gene status (P<0.05). A multivariate logistic regression analysis was performed to further clarify the correlation of these indices. However, only tumor size (OR=2.814, 95% Cl=1.634~4.848, P<0.001), microcalcifications (OR=2.839, 95% Cl=1,684~4.787, P<0.001) and TgAb (OR=1.964, 95% Cl=1.039~3,711, P=0.038) were independent predictors of central CLNM and were incorporated and used to construct the prediction nomogram. The model had good discrimination with a C-index of 0.715. An ROC curve analysis was performed to evaluate the accuracy of this model. The decision curve analysis showed that the model was clinically useful when intervention was decided in the threshold range of 16% to 80%.

CONCLUSION

In conclusion, three independent predictors of central CLNM, including tumor size (> 1.0 cm), US features (microcalcifications), and TgAb (positive), were screened out. A visualized nomogram model was established based on the three predictors in this study, which could be used as a basis of central cervical lymph node dissection (CLND) for PTC patients (cN0).

An Ultrasound Radiomics Nomogram for Preoperative Prediction of Central Neck Lymph Node Metastasis in Papillary Thyroid Carcinoma

Purpose: This study aimed to establish and validate an ultrasound radiomics nomogram for the preoperative prediction of central lymph node (LN) metastasis in patients with papillary thyroid carcinoma (PTC).

Patients and Methods: The prediction model was developed in 609 patients with clinicopathologically confirmed unifocal PTC who received ultrasonography between Jan 2018 and June 2018. Radiomic features were extracted after the ultrasonography of PTC. Lasso regression model was used for data dimensionality reduction, feature selection, and radiomics signature building. The predicting model was established based on the multivariable logistic regression analysis in which the radiomics signature, ultrasonography-reported LN status, and independent clinicopathologic risk factors were incorporated, and finally a radiomics nomogram was established. The performance of the nomogram was assessed with respect to the discrimination and consistence. An independent validation was performed in 326 consecutive patients from July 2018 to Sep 2018.

Results: The radiomics signature consisted of 23 selected features and was significantly associated with LN status in both primary and validation cohorts. The independent predictors in the radiomics nomogram included the radiomics signature, age, TG level, TPOAB level, and ultrasonography-reported LN status. The model showed good discrimination and consistence in both cohorts: C-index of 0.816 (95% CI, 0.808–0.824) in the primary cohort and 0.858 (95% CI, 0.849–0.867) in the validation cohort. The area under receiver operating curve was 0.858. In the validation cohort, the accuracy, sensitivity, specificity and AUC of this model were 0.812, 0.816, 0.810, and 0.858 (95% CI, 0.785–0.930), respectively. Decision curve analysis indicated the radiomics nomogram was clinically useful.

Conclusion: This study presents a convenient, clinically useful ultrasound radiomics nomogram that can be used for the pre-operative individualized prediction of central LN metastasis in patients with PTC.

A new ultrasound nomogram for differentiating benign and malignant thyroid nodules

OBJECTIVE

The Thyroid Imaging Reporting and Data System (TI-RADS) is commonly used for risk stratification of thyroid nodules. However, this system has a poor sensitivity and specificity. The aim of this study was to build a new model based on TI-RADS for evaluating ultrasound image patterns that offer improved efficacy for differentiating benign and malignant thyroid nodules.

DESIGN AND PATIENTS

The study population consisted of 1092 participants with thyroid nodules.

MEASUREMENTS

The nodules were analysed by the TI-RADS and the new model. The prediction properties and decision curve analysis of the nomogram were compared between the two models.

RESULTS

The proportions of thyroid cancer and benign disease were 36.17% and 63.83%. The new model showed good agreement between the prediction and observation of thyroid cancer. The nomogram indicated excellent prediction properties with an area under the curve (AUC) of 0.946, sensitivity of 0.884 and specificity of 0.917 for training data as well as a high sensitivity, specificity, negative predictive value and positive predictive value for the validation data also. The optimum cut-off for the nomogram was 0.469 for predicting cancer. The decision curve analysis results corroborated the good clinical applicability of the nomogram and the TI-RADS for predicting thyroid cancer with wide and practical ranges for threshold probabilities.

CONCLUSIONS

Based on the TI-RADS, we built a new model using a combination of ultrasound patterns including margin, shape, echogenic foci, echogenicity and nodule halo sign with age to differentiate benign and malignant thyroid nodules, which had high sensitivity and specificity.

Clinical Validation of a Predictive Model for the Presence of Cervical Lymph Node Metastasis in Papillary Thyroid Cancer

BACKGROUND AND PURPOSE

Ultrasound is a standard technique to detect lymph node metastasis in papillary thyroid cancer. Cystic changes and microcalcifications are the most specific features of metastasis, but with low sensitivity. This prospective study compared the diagnostic accuracy of a predictive model for sonographic evaluation of lymph nodes relative to the radiologist's standard assessment in detecting papillary thyroid cancer metastasis in patients after thyroidectomy.

MATERIALS AND METHODS

Cervical lymph node sonographic images were reported by a radiologist (R method) per standard practice. The same images were independently evaluated by another radiologist using a sonographic predictive model (M method). A test was considered positive for metastasis if the R or M method suggested lymph node biopsy. The result of lymph node biopsy or surgical pathology was used as the reference standard. We estimated relative true-positive fraction and relative false-positive fraction using log-linear models for correlated binary data for the M method compared with the R method.

RESULTS

A total of 237 lymph nodes in 103 patients were evaluated. Our analysis of relative true-positive fraction and relative false-positive fraction included 54 nodes with pathologic results in which at least 1 method (R or M) was positive. The M method had a higher relative true-positive fraction of 1.46 (95% CI, 1.12-1.91; P = .006) and a lower relative false-positive fraction of 0.58 (95% CI, 0.36-0.92; P = .02) compared with the R method.

CONCLUSIONS

The sonographic predictive model outperformed the standard assessment to detect lymph node metastasis in patients with papillary thyroid cancer and may reduce unnecessary biopsies.

ACR Thyroid Imaging, Reporting and Data System (TI-RADS): White Paper of the ACR TI-RADS Committee

classification that is widely used in breast imaging, their authors chose to apply the acronym TI-RADS, for Thyroid Imaging, Reporting and Data System. In 2012, the ACR convened committees to (1) provide recommendations for reporting incidental thyroid nodules, (2) develop a set of standard terms (lexicon) for ultrasound reporting, and (3) propose a TI-RADS on the basis of the lexicon. The committees published the results of the first two efforts in 2015. In this article, the authors present the ACR TI-RADS Committee's recommendations, which provide guidance regarding management of thyroid nodules on the basis of their ultrasound appearance. The authors also describe the committee's future directions.