Observer variation in ultrasound assessment of the thyroid gland

Considerable interobserver variation calls for unambiguous definitions of thyroid nodule ultrasound characteristics

Objective

Thyroid nodule ultrasound characteristics are used as an indication for fine-needle aspiration cytology, usually as the basis for Thyroid Imaging Reporting and Data System (TIRADS) score calculation. Few studies on interobserver variation are available, all of which are based on analysis of preselected still ultrasound images and often lack surgical confirmation.

Methods

After the blinded online evaluation of video recordings of the ultrasound examinations of 47 consecutive malignant and 76 consecutive benign thyroid lesions, 7 experts from 7 thyroid centers answered 17 TIRADS-related questions. Surgical histology was the reference standard. Interobserver variations of each ultrasound characteristic were compared using Gwet's AC1 inter-rater coefficients; higher values mean better concordance, the maximum being 1.0.

Results

On a scale from 0.0 to 1.0, the Gwet's AC1 values were 0.34, 0.53, 0.72, and 0.79 for the four most important features in decision-making, i.e. irregular margins, microcalcifications, echogenicity, and extrathyroidal extension, respectively. The concordance in the discrimination between mildly/moderately and very hypoechogenic nodules was 0.17. The smaller the nodule size the better the agreement in echogenicity, and the larger the nodule size the better the agreement on the presence of microcalcifications. Extrathyroidal extension was correctly identified in just 45.8% of the cases.

Conclusions

Examination of video recordings, closely simulating the real-world situation, revealed substantial interobserver variation in the interpretation of each of the four most important ultrasound characteristics. In view of the importance for the management of thyroid nodules, unambiguous and widely accepted definitions of each nodule characteristic are warranted, although it remains to be investigated whether this diminishes observer variation.

Differences in the ultrasonographic appearance of thyroid nodules after radiofrequency ablation

CONTEXT

Radiofrequency ablation (RFA) is a well-tolerated approach to treating benign thyroid nodules (TNs), but no index can predict its success. Other than size decrease, little is known about TN appearance on ultrasonography (US) after RFA.

OBJECTIVE

This study aimed to (a) assess the effectiveness of single-session RFA treatment, (b) determine whether pre-ablation US characteristics correlate with its effectiveness, and (c) demonstrate TN characteristics on baseline and follow-up US.

DESIGN

Retrospective cohort study among the patients who underwent single-session RFA for the treatment of benign TNs at a referral medical center between January 2018 and April 2019.

PATIENTS

A total of 116 patients (137 nodules) were included in the study.

MEASUREMENTS

Characteristics were quantified using commercial software. TNs were classified into 2015 American Thyroid Association (ATA) sonographic patterns and American College of Radiology Thyroid Imaging Reporting and Data System (ACR-TI-RADS) categories.

RESULTS

The average volume reduction ratio (VRR) was 74.51% in 1 year (95% confidence interval, 70.63%-78.39%). The only pre-ablation US feature significantly different between nodules with VRR <50% and VRR >50% was the cyst composition (0.05 vs. 0.02, p-value = .02). The VRR and margin change in the first 3 months after ablation were found to be leading indicators significantly correlated to the VRR in 6 months with correlation coefficients (r) = .72 and -.28 (p-value < .0001 and = .0008) and VRR in 1 year with r = .65 and -.17 (p-value < .0001 and = .046), respectively. After RFA, more TNs became ATA high suspicion (2.9% vs. 19.7%, p < .0001) and more appeared to be the non-ATA patterns (12.4% vs. 23.4%, p < .0001). Also, a greater number of post-RFA TNs were classified as ACR-TI-RADS categories 4 and 5 (40.1% vs. 70.1%, p < .0001).

CONCLUSIONS

Radiofrequency ablation therapy is effective for treating TNs. Pre-ablation cyst components, 3-month post-ablation volume reduction and margin change of TNs were related to the 6-month and 1-year response. Clinicians should consider that TNs would appear peculiar on US after RFA, mistakenly suggesting malignant potential.

Software-Based Analysis of the Taller-Than-Wide Feature of High-Risk Thyroid Nodules

BACKGROUND

Shape is one of the most important features in the diagnosis of malignant thyroid nodules. This characteristic has been described qualitatively, but only shapes that appear markedly different can be easily differentiated at first interpretation. This study sought to clarify whether software-based shape indexes are useful for the detection of thyroid cancers.

METHODS

In the final analysis, 200 participants with 231 pathologically proven nodules participated in the study. Ultrasound features were assessed by clinicians. The tumor contour was auto-defined, and shape indexes were calculated using commercial software.

RESULTS

Of the 231 nodules, 134 were benign and 97 were malignant. The presence of taller-than-wide (TTW) dimensions differed significantly between the benign and malignant thyroid tumors. Designation of TTW assessed by the software had a higher kappa value and proportional agreement than TTW assessed by clinicians. Disagreement between the clinician and software in designating nodules as TTW occurred for 28 nodules. The presence of other ultrasonic characteristics and small differences in the height and width measurements were causes for the incorrect interpretation of the TTW feature.

CONCLUSION

The proposed software-based quantitative analysis of tumor shape seems to be promising as an important advance compared with conventional TTW features evaluated by operators because it allows for a more reliable and consistent distinction and is less influenced by other ultrasonic features.

Risk Stratification in Patients With Follicular Neoplasm on Cytology: Use of Quantitative Characteristics and Sonographic Patterns

OBJECTIVES

Differentiating thyroid nodules with a cytological diagnosis of follicular neoplasm remains an issue. The goal of this study was to determine whether ultrasonographic (US) findings obtained preoperatively from the computer-aided detection (CAD) system are sufficient to further stratify the risk of malignancy for this diagnostic cytological category.

METHODS

From September 2016 to September 2018 in our hospital, patients diagnosed with Bethesda category IV (follicular neoplasm or suspicion of follicular neoplasm) thyroid nodules and underwent surgical excisions were include in the study. Quantification and analysis of tumor features were performed using CAD software. The US findings of the region of interest, including index of composition, margin, echogenicity, texture, echogenic dots indicative of calcifications, tall and wide orientation, and margin were calculated into computerized values. The nodules were further classified into American Thyroid Association (ATA) and American College of Radiology Thyroid Imaging Reporting & Data System (TI-RADS) categories.

RESULTS

92 (10.1%) of 913 patients were diagnosed with Bethesda category IV thyroid nodules. In 65 patients, the histological type of the nodule was identified. The quantitative features between patients with benign and malignant conditions differed significantly. The presence of heterogeneous echotexture, blurred margins, or irregular margins was shown to have the highest diagnostic value. The risks of malignancy for nodules classified as having very low to intermediate suspicion ATA, non-ATA, and high suspicion ATA patterns were 9%, 35.7%, and 51.7%, respectively. Meanwhile, the risks of malignancy were 12.5%, 26.1%, and 53.8% for nodules classified as TIRADS 3, 4, and 5, respectively. When compared to human observers, among whom poor agreement was noticeable, the CAD software has shown a higher average accuracy.

CONCLUSIONS

For patients with nodules diagnosed as Bethesda category IV, the software-based characterizations of US features, along with the associated ATA patterns and TIRADS system, were shown helpful in the risk stratification of malignancy.

Cancer Measurement at Ultrasound: State of the Art

Reliable and reproducible tumor measurement is fundamental in the oncologic decision making. In this article, we first highlight the importance of a precise tumor measurement, reviewing the correct modality of measuring tumor lesions at ultrasound. Then we analyze the measurement discrepancies between ultrasound and pathology as well as the discrepancies reported between ultrasound and other imaging modalities. Thereafter, basing on the existent literature and on our experience, we discuss the factors influencing the tumor size measurements at ultrasound. Finally, we illustrate the current strategies to improve the effectiveness of cancer lesions measurement.

Discrepancies between the ultrasonographic and gross pathological size of papillary thyroid carcinomas

Purpose:

The goal of this study was to investigate the level of agreement between tumor sizes measured on ultrasonography (US) and in pathological specimens of papillary thyroid carcinomas (PTCs) and to identify the US characteristics contributing to discrepancies in these measurements.

Methods:

We retrospectively reviewed the US findings and pathological reports of 490 tumors in 431 patients who underwent surgery for PTC. Agreement was defined as a difference of <20% between the US and pathological tumor size measurements. Tumors were divided by size into groups of 0.5-1 cm, 1-2 cm, 2-3 cm, and ≥3 cm. We compared tumors in which the US and pathological tumor size measurements agreed and those in which they disagreed with regard to the following parameters: taller-than-wide shape, infiltrative margin, echogenicity, microcalcifications, cystic changes in tumors, and the US diagnosis.

Results:

The rate of agreement between US and the pathological tumor size measurements was 64.1% (314/490). Statistical analysis indicated that the US and pathological measurements significantly differed in tumors <1.0 cm in size (P=0.033), with US significantly overestimating the tumor size by 0.2 cm in such tumors (P<0.001). Cystic changes were significantly more frequent in the tumors where US and pathological tumor size measurements disagreed (P<0.001).

Conclusion:

Thyroid US may overestimate the size of PTCs, particularly for tumors <1.0 cm in size. This information may be helpful in guiding decision making regarding surgical extent.

Epidemiology of nodular goitre. Influence of iodine intake

More than one tenth of the world population is to some degree affected by goitre and most of these harbour nodules. The large differences in thyroid disease prevalence between populations may be caused by genetic and environmental factors. Among the latter, iodine deficiency seems by far to be the most important risk factor. Thus, nodular goitre is a condition predominantly seen in iodine deficient areas of the world. In the present review, we evaluated in detail autopsy and ultrasound studies of the thyroid gland. In autopsy studies, large thyroid volumes and high frequencies of goitres have been reported in countries affected by iodine deficiency. Many cross-sectional studies using thyroid ultrasound investigations have been performed world-wide and reported high thyroid volumes and goitre prevalences, and to some extent also high prevalences of thyroid nodules in iodine-deficient countries. Most of these goitres were classified as nodular goitres. On the other hand, few studies have shown that abundant iodine intake may lead to development of diffuse goitres, but world-wide this has been a minor problem compared with development of nodular goitres. In the past century we have observed a trend towards smaller thyroid glands, and hopefully less than 10% of the world population will experience goitre within a few decades.

Diagnostic use of molecular markers in the evaluation of thyroid nodules

OBJECTIVE

To describe the molecular markers thus far evaluated for use in the care of patients with clinically relevant thyroid nodules.

METHODS

We review the currently available molecular tests that have been applied to patients with thyroid nodules.

RESULTS

In the United States, approximately 450 000 diagnostic fine-needle aspirates will be performed on patients with thyroid nodules this year in an effort to identify thyroid cancer. Unfortunately, this test is imprecise and, at times, inaccurate. Because of this, novel diagnostic testing modalities have been pursued, the most promising of which involve molecular analysis of thyroid tissue. Immunohistochemical staining, analysis for mutations and gene rearrangements, and microarray analysis have all been investigated with regard to their performance characteristics in targeted patient populations.

CONCLUSIONS

Molecular tests to evaluate thyroid nodules demonstrate variable performance characteristics. Further evaluation of available and emerging molecular tests will necessarily rely on prospective real-world test validation in the clinical setting.

Advances in ultrasound for the diagnosis and management of thyroid cancer

BACKGROUND

Ultrasonography (US) is increasingly being employed by endocrinologists and surgeons in the diagnosis and management of patients with thyroid nodules and thyroid cancer. Recent consensus guidelines from the American Thyroid Association highlight the importance of this modality by recommending that patients with thyroid nodules should undergo further evaluation with cervical US to stratify the risk of malignancy. Likewise, ultrasound is advocated for the preoperative and postoperative diagnosis of cervical lymph node metastases.

SUMMARY

This article will summarize the US characteristics that impart a suspicious appearance on thyroid nodules and cervical lymph nodes as well as those findings that are reassuring. Likewise, the indications for thyroid nodule and lymph node fine-needle aspiration will be reviewed. Finally, this article will briefly discuss adjunctive tools in US such as elastography, percutaneous ethanol ablation, and radiofrequency ablation.

CONCLUSIONS

US may be used to help stratify the risk of malignancy in thyroid nodules and cervical lymph nodes. This tool further aids in the diagnosis of malignancy when used in conjunction with fine-needle aspiration. US plays an important role as both a diagnostic and therapeutic tool in the evaluation of patients with neck masses.

Diagnostic Use of Ultrasonography in Patients with Nodular Thyroid Disease

OBJECTIVE

To review the published reports pertaining to the diagnostic utility of ultrasonography for evaluation of thyroid nodules.

METHODS

Various roles for diagnostic thyroid ultrasonography and screening ultrasound studies of the thyroid are discussed, and ultrasound characteristics of thyroid nodules and their association with malignant potential are described.

RESULTS

In two studies that correlated ultrasound findings with physical examination findings in patients with a solitary thyroid nodule detected by palpation, 16% of such patients had no corresponding nodule evident on ultrasonography, and 45% of such patients had an additional nodule detected by ultrasonography. Similarly, approximately 18% of patients with a palpable multinodular thyroid had no nodules larger than 1 cm in diameter on ultrasound studies. Thyroid nodules larger than 1 cm have been found by ultrasonography to be present in from 2 to almost 5% of the population with normal findings on examination of the thyroid. Use of screening ultrasound study of the thyroid has been suggested for patients with a history of childhood irradiation to the head and neck or a family history of thyroid cancer. Numerous investigations that have evaluated ultrasound features of thyroid nodules have suggested five characteristics as suggestive of malignant potential--hypoechogenicity, microcalcifications, irregular or microlobulated border, absent or irregular thick halo, and increased intranodular vascularity.

CONCLUSION

Correlation of ultrasound and palpation findings will provide a comprehensive evaluation of nodular thyroid disease. Moreover, real-time ultrasonography facilitates characterization of features associated with an increased risk of a malignant lesion. High-resolution thyroid ultrasonography is a dynamic tool for endocrinologists.

Ultrasound thyroid nodule measurements: the "gold standard" and its limitations in clinical decision making

OBJECTIVE

To highlight the limitations associated with use of ultrasonography for the evaluation of thyroid nodule or gland volume for clinical decision making.

METHODS

We review the equipment (scanners and probes) used for ultrasonography and the appropriate measurements for assessment of thyroid nodules. The limitations of ultrasound thyroid measurements are summarized, particularly in reference to repeated measurements over time.

RESULTS

Thyroid ultrasonography is the recognized "gold standard" for an accurate and reliable assessment of gland volume and thyroid nodules. Many endocrinologists refer patients for surgical treatment because of detection of growth of thyroid nodules. In daily practice, they often make this decision by comparing ultrasound thyroid measurements determined over time. Although reliable, evaluation of thyroid nodule volume by ultrasonography has technologic, biologic, and examination technique limitations. These are particularly important in routine clinical practice, where ultrasound measurements are performed in less standardized settings than in experimental trials.

CONCLUSION

In daily medical office applications, ultrasound measurements of thyroid nodules should be used with caution in decision making.

Validation of ultrasonography of the thyroid gland for epidemiological purposes

Ultrasonography of the thyroid is often used in epidemiological surveys, thus thorough characterization of the interobserver variation of the different parameters obtained is important. Various methods have been used for measuring thyroid volume, and different formulas have been used for calculation of thyroid volume from the measured dimensions. In this article, two principles of thyroid volume measurement are described in detail: the wellknown method based on the three axes of each lobe and a new principle based on planimetry in two planes. The interobserver variation of the examination and the measuring procedure in itself were tested on 25 participants in a population study. A comparison of postmortem ultrasonography of the thyroid and results of an autopsy was performed. Good correlation and agreement between observers was found for thyroid volume (r = 0.98) and prevalence of thyroid nodules (kappa = 0.72), whereas echogenecity and echopattern showed little agreement. The correlation of thyroid volume by ultrasonography to autopsy results was satisfactory (r = 0.93), but the volume tended to be slightly underestimated even when using the formula pi/6(= 0.52)*length*width*depth. No major differences were found between the performance of the two principles of volume calculation. We conclude that when the measuring procedure is well defined, results of ultrasonography are comparable between observers for thyroid volume and prevalence of thyroid nodules, but not for echogenecity or echopattern. The formula of length*depth*width*pi/6 is suitable for thyroid volume measurement.

Protocols for the preoperative selection of palpable thyroid nodules: review and progress

Modern protocols for the management of patients with palpable thyroid nodules agree that fine-needle aspiration is the first examination to be performed. However, they differ very much in the role attributed to scintigraphy and ultrasound examinations. In some protocols, these two techniques are not considered, whereas in others they are recommended at the end of the diagnostic workup to select for surgery those nodules with nondiagnostic or suspect fine-needle aspiration biopsy results. We report original data and literature showing that such use of scintigraphy and ultrasonography is not cost effective. Furthermore, we report original data showing that large-needle aspiration biopsy can be used to select for surgery those nodules with nondiagnostic or suspect results after fine-needle aspiration. Consequently, we suggest a new protocol for the preoperative selection of palpable thyroid nodules.

Is there an improvement in performance when radiographs are re-reported at 24 hours?

This study was designed to assess the accuracy and intraobserver variability when plain radiographs are re-reported after 24 h, in order to determine if there was a change in observer performance. Five consultant radiologists reported 50 plain radiographs on patients in whom the diagnosis had been established. The radiographs were reviewed by each radiologist after an interval of 24 h. The same clinical information was available on both occasions. 32 of the radiographs showed an abnormality and 18 were normal. The accuracy of the report in relation to the established diagnosis was assessed, intraobserver agreement being calculated using kappa (kappa) statistics. The kappa-value for intraobserver agreement was in the range 0.33-0.88. Accuracy showed a range of 78-90% on the first viewing and 80-92% on the subsequent viewing. ROC curves were produced for each viewing for the group as a whole. The areas under the curves were 0.88 and 0.9 respectively. For the group, the accuracy increased on the second occasion, but this did not reach statistical significance.

Observer variation in the clinical and laboratory evaluation of patients with thyroid dysfunction and goiter

Three endocrinologists assessed thyroid function (hypothyroid, possibly hypothyroid, euthyroid, possibly hyperthyroid, or hyperthyroid), thyroid size (small, medium, or large), thyroid type (diffuse, nodular, or solitary nodule), and diagnosis and treatment options in 55 patients (47 women and 8 men) with a median age of 43 years (range 19 to 74) suspected of thyroid disease. The observers were presented stepwise for the (1) patient, clinical examination, and patient history; (2) blood tests; (3) 99mTc-pertechnetate scintigraphy; and (4) ultrasonography. The reproducibility was assessed by means of the K coefficient. Compared with evaluation of the patient alone, agreement on thyroid dysfunction was almost perfect when the results of the blood tests were known. The K values for pairs of observers rose significantly from 0.55 to 0.65 to 0.88 to 0.93. All three observers altered their opinion as to thyroid dysfunction in one third of the patients when the blood tests were known. Compared with evaluation of the patient alone, agreement on the morphology of the thyroid gland did not improve significantly in spite of access to thyroid scintigraphy; with the addition of thyroid ultrasound, agreement improved significantly for some pairs of observers. The three observers agreed on the rough estimate of thyroid size in only 36% of the patients. When all information was available, the three observers agreed on diagnosis and treatment category in 60% of the patients. Doctors should bear in mind the considerable observer variation when they evaluate patients with suspected thyroid disease.

Thyroid nodules in the follow-up of irradiated individuals: comparison of thyroid ultrasound with scanning and palpation

In 1974 we began a prospective study of a cohort of 4296 individuals exposed to therapeutic head and neck irradiation during childhood for benign conditions. To define the role of thyroid ultrasonography in following irradiated individuals, we studied a subgroup of 54 individuals. They all had been screened between 1974-1976 and had normal thyroid scans and no palpable nodules at that time. Thyroid ultrasonography, thyroid scanning, physical examination, and serum thyroglobulin measurements were performed. One or more discrete ultrasound-detected nodules were present in 47 of 54 (87%) subjects. There were a total of 157 nodules, 40 of which were 1.0 cm or larger in largest dimension. These 40 nodules occurred in 28 (52%) of the subjects. Thirty (75%) of these 1.0-cm or larger nodules matched discrete areas of diminished uptake on corresponding thyroid scans. The 10 that did not match (false negative scans for > or = 1.0-cm nodules) were the only nodules of this size in 7 subjects. Of 11 nodules 1.5 cm or larger, only 5 were palpable. Serum thyroglobulin correlated to the number (P = 0.04; r2 = 0.10), but not the volume of the thyroid nodules (P = 0.07; r2 = 0.08). We conclude that thyroid nodules are continuing to occur and are exceedingly common in this irradiated cohort of individuals. The results confirm that thyroid ultrasonography is more sensitive than physical examination and scanning. However, thyroid ultrasound is so sensitive and nodules so prevalent that great caution is needed in interpreting the results.

Thyroid ultrasonography with fine-needle aspiration cytology for the diagnosis of thyroid cancer

This series retrospectively reviewed 3657 patients who received thyroid ultrasonography with fine-needle aspiration cytological (FNAC) examinations at Chang Gung Memorial Hospital during the period from January 1, 1993 to December 31, 1993. Thyroid ultrasonography studies were performed using a real-time ultrasonographic machine with a 10 MHz transducer. The aspirates were air dried and stained by the Romanowsky-based Liu method. Three hundred seventy-eight cases (10.3%) received surgical treatment after the ultrasonographic and FNAC examinations. Benign lesions were diagnosed in 269 patients. Thyroid malignancy was confirmed histopathologically in 109 cases including 76 papillary thyroid carcinomas, 17 follicular carcinomas, 5 medullary thyroid carcinomas, 3 anaplastic carcinomas, 3 Hürthle cell carcinomas, and 3 lymphomas. Another 2 cases were metastatic cancer to thyroid. The results demonstrated that 28.8% of the surgically treated patients had histopathologically proven malignancies. The incidence of thyroid malignancy was 2.98% in this study. The sensitivity of the cytological diagnosis was 79.80% and the specificity was 98.66%. The positive predictive value was 96.34%. Negative predictive value was 91.70%. The false negative index was 20.20%. The diagnostic accuracy was 92.89%. Thyroid ultrasonography with the FNAC can provide high specificity and sensitivity in differentiating malignant lesions from benign.