Multi-Reader Multi-Case Study for Performance Evaluation of High-Risk Thyroid Ultrasound with Computer-Aided Detection

Assessing Detection Accuracy of Computerized Sonographic Features and Computer-Assisted Reading Performance in Differentiating Thyroid Cancers

For ultrasound imaging of thyroid nodules, medical guidelines are all based on findings of sonographic features to provide clinicians management recommendations. Due to the recent development of artificial intelligence and machine learning (AI/ML) technologies, there have been computer-assisted detection (CAD) software devices available for clinical use to detect and quantify the sonographic features of thyroid nodules. This study is to validate the accuracy of the computerized sonographic features (CSF) by a CAD software device, namely, AmCAD-UT, and then to assess how the reading performance of clinicians (readers) can be improved providing the computerized features. The feature detection accuracy is tested against the ground truth established by a panel of thyroid specialists and a multiple-reader multiple-case (MRMC) study is performed to assess the sequential reading performance with the assistance of the CSF. Five computerized features, including anechoic area, hyperechoic foci, hypoechoic pattern, heterogeneous texture, and indistinct margin, were tested, with AUCs ranging from 0.888~0.946, 0.825~0.913, 0.812~0.847, 0.627~0.77, and 0.676~0.766, respectively. With the five CSFs, the sequential reading performance of 18 clinicians is found significantly improved, with the AUC increasing from 0.720 without CSF to 0.776 with CSF. Our studies show that the computerized features are consistent with the clinicians’ findings and provide additional value in assisting sonographic diagnosis.

Artificial Intelligence (AI) Tools for Thyroid Nodules on Ultrasound, From the AJR Special Series on AI Applications

Artificial intelligence (AI) methods for evaluating thyroid nodules on ultrasound have been widely described in the literature, with reported performance of AI tools matching or in some instances surpassing radiologists. As these data have accumulated, products for classification and risk stratification of thyroid nodules on ultrasound have become commercially available. This article reviews FDA-approved products currently on the market, with a focus on product features, reported performance, and considerations for implementation. The products perform risk stratification primarily using the Thyroid Imaging Reporting and Data System (TI-RADS), though may provide additional prediction tools independent of TI-RADS. Key issues in implementation include integration with radiologist interpretation, impact on workflow and efficiency, and performance monitoring. AI applications beyond nodule classification, including report construction and incidental findings follow-up, are also described. Anticipated future directions of research and development in AI tools for thyroid nodules are highlighted.

Judgment of benign and early malignant colorectal tumors from ultrasound images with deep multi-View fusion

BACKGROUND AND OBJECTIVE

Colorectal cancer (CRC) is currently one of the main cancers world-wide, with a high incidence in the elderly. In the diagnosis of CRC, endorectal ultrasound plays an important role in judging benign and early malignant tumors. However, malignant tumors in the early-stage are not easy to identify visually and experts usually seek help from multi-view images, which increases the workload and also exists a certain probability of misdiagnosis. In recent years, with the widespread use of deep learning methods in the analysis of medical images, it becomes necessary to design an effective computer-aided diagnosis (CAD) system of CRC based on multi-view endorectal ultrasound images.

METHOD

In this study, we proposed a CAD system for judging benign and early malignant colorectal tumors, and constructed the first multi-view ultrasound image dataset of CRC to validate our algorithm. Our system is an end-to-end model based on a deep neural network (DNN) which includes a feature extraction module based on dense blocks, a multi-view fusion module, and a Multi-Layer Perception-based classifier. A center loss was used for the first time in CAD tasks, to optimize our model.

RESULT

On the constructed dataset, the proposed system surpasses expert diagnosis in accuracy, sensitivity, specificity, and F1-score. Compared with the popular deep classification networks and other CAD methods, the algorithm has reached the best performance. Comparative experiments using different feature extraction methods, different view fusion strategies, and different classifiers verify the effectiveness of each part of the algorithm.

CONCLUSION

We propose a CAD system for judging benign and early malignant colorectal tumors based on DNN, which combines information of ultrasound images from different views for comprehension. On the first CRC multi-view ultrasound image dataset which we constructed, our method outperforms expert diagnosis results and all other methods, and the effectiveness of each part of the system has been verified. Our system has application value in future medical practice on early diagnosis of CRC.

Differences in the ultrasonographic appearance of thyroid nodules after radiofrequency ablation

CONTEXT

Radiofrequency ablation (RFA) is a well-tolerated approach to treating benign thyroid nodules (TNs), but no index can predict its success. Other than size decrease, little is known about TN appearance on ultrasonography (US) after RFA.

OBJECTIVE

This study aimed to (a) assess the effectiveness of single-session RFA treatment, (b) determine whether pre-ablation US characteristics correlate with its effectiveness, and (c) demonstrate TN characteristics on baseline and follow-up US.

DESIGN

Retrospective cohort study among the patients who underwent single-session RFA for the treatment of benign TNs at a referral medical center between January 2018 and April 2019.

PATIENTS

A total of 116 patients (137 nodules) were included in the study.

MEASUREMENTS

Characteristics were quantified using commercial software. TNs were classified into 2015 American Thyroid Association (ATA) sonographic patterns and American College of Radiology Thyroid Imaging Reporting and Data System (ACR-TI-RADS) categories.

RESULTS

The average volume reduction ratio (VRR) was 74.51% in 1 year (95% confidence interval, 70.63%-78.39%). The only pre-ablation US feature significantly different between nodules with VRR <50% and VRR >50% was the cyst composition (0.05 vs. 0.02, p-value = .02). The VRR and margin change in the first 3 months after ablation were found to be leading indicators significantly correlated to the VRR in 6 months with correlation coefficients (r) = .72 and -.28 (p-value < .0001 and = .0008) and VRR in 1 year with r = .65 and -.17 (p-value < .0001 and = .046), respectively. After RFA, more TNs became ATA high suspicion (2.9% vs. 19.7%, p < .0001) and more appeared to be the non-ATA patterns (12.4% vs. 23.4%, p < .0001). Also, a greater number of post-RFA TNs were classified as ACR-TI-RADS categories 4 and 5 (40.1% vs. 70.1%, p < .0001).

CONCLUSIONS

Radiofrequency ablation therapy is effective for treating TNs. Pre-ablation cyst components, 3-month post-ablation volume reduction and margin change of TNs were related to the 6-month and 1-year response. Clinicians should consider that TNs would appear peculiar on US after RFA, mistakenly suggesting malignant potential.

COVID-19 Detection Empowered with Machine Learning and Deep Learning Techniques: A Systematic Review

COVID-19 has infected 223 countries and caused 2.8 million deaths worldwide (at the time of writing this article), and the death rate is increasing continuously. Early diagnosis of COVID patients is a critical challenge for medical practitioners, governments, organizations, and countries to overcome the rapid spread of the deadly virus in any geographical area. In this situation, the previous epidemic evidence on Machine Learning (ML) and Deep Learning (DL) techniques encouraged the researchers to play a significant role in detecting COVID-19. Similarly, the rising scope of ML/DL methodologies in the medical domain also advocates its significant role in COVID-19 detection. This systematic review presents ML and DL techniques practiced in this era to predict, diagnose, classify, and detect the coronavirus. In this study, the data was retrieved from three prevalent full-text archives, i.e., Science Direct, Web of Science, and PubMed, using the search code strategy on 16 March 2021. Using professional assessment, among 961 articles retrieved by an initial query, only 40 articles focusing on ML/DL-based COVID-19 detection schemes were selected. Findings have been presented as a country-wise distribution of publications, article frequency, various data collection, analyzed datasets, sample sizes, and applied ML/DL techniques. Precisely, this study reveals that ML/DL technique accuracy lay between 80% to 100% when detecting COVID-19. The RT-PCR-based model with Support Vector Machine (SVM) exhibited the lowest accuracy (80%), whereas the X-ray-based model achieved the highest accuracy (99.7%) using a deep convolutional neural network. However, current studies have shown that an anal swab test is super accurate to detect the virus. Moreover, this review addresses the limitations of COVID-19 detection along with the detailed discussion of the prevailing challenges and future research directions, which eventually highlight outstanding issues.

Diagnosis of thyroid cancer using a TI-RADS-based computer-aided diagnosis system: a multicenter retrospective study

OBJECTIVE

The purpose of this study was to use a computer-aided diagnosis (CAD) system based on the Thyroid Imaging, Reporting, and Data System (TI-RADS) to improve the diagnostic performance of thyroid cancer by analyzing clinical ultrasound imaging data.

METHODS

A retrospective diagnostic study of ultrasound image sets was conducted at five hospitals in China. A CAD system based on TI-RADS was applied in this study, and the diagnostic performance of CAD system was tested through multi-center data. The performance of the CAD system was compared with the consensus of three experienced radiologists. The interobserver agreement for cancer diagnosis was calculated between the CAD system and the consensus of the three experienced radiologists.

RESULTS

The CAD system performed well in the diagnosis of thyroid cancer, with an area under the curve (AUC) value of 0.902 (95% CI: 0.884-0.918), and obtained results similar to those of the three experienced radiologists. The CAD system performed better in the internal test set than in the external test set (AUC: 0.930 vs 0.877, respectively). The performance of the CAD system in the diagnosis of thyroid cancer for nodules of different sizes (<1 cm, 1-2 cm and ≥2 cm) was basically similar (accuracy: 84.6% vs 85% vs 84.2%). The CAD system can recognize 15 ultrasound features of thyroid nodules, most of which reached the level of 3 experienced radiologists (12/15, 85%).

CONCLUSION

The CAD system achieved an improved AUC and similar sensitivity and specificity in the diagnosis of thyroid cancer compared with the consensus of experienced radiologists.

Diagnostic performance evaluation of different TI-RADS using ultrasound computer-aided diagnosis of thyroid nodules: An experience with adjusted settings

BACKGROUND

Thyroid cancer diagnosis has evolved to include computer-aided diagnosis (CAD) approaches to overcome the limitations of human ultrasound feature assessment. This study aimed to evaluate the diagnostic performance of a CAD system in thyroid nodule differentiation using varied settings.

METHODS

Ultrasound images of 205 thyroid nodules from 198 patients were analysed in this retrospective study. AmCAD-UT software was used at default settings and 3 adjusted settings to diagnose the nodules. Six risk-stratification systems in the software were used to classify the thyroid nodules: The American Thyroid Association (ATA), American College of Radiology Thyroid Imaging, Reporting, and Data System (ACR-TIRADS), British Thyroid Association (BTA), European Union (EU-TIRADS), Kwak (2011) and the Korean Society of Thyroid Radiology (KSThR). The diagnostic performance of CAD was determined relative to the histopathology and/or cytology diagnosis of each nodule.

RESULTS

At the default setting, EU-TIRADS yielded the highest sensitivity, 82.6% and lowest specificity, 42.1% while the ATA-TIRADS yielded the highest specificity, 66.4%. Kwak had the highest AUROC (0.74) which was comparable to that of ACR, ATA, and KSThR TIRADS (0.72, 0.73, and 0.70 respectively). At a hyperechoic foci setting of 3.5 with other settings at median values; ATA had the best-balanced sensitivity, specificity and good AUROC (70.4%; 67.3% and 0.71 respectively).

CONCLUSION

The default setting achieved the best diagnostic performance with all TIRADS and was best for maximizing the sensitivity of EU-TIRADS. Adjusting the settings by only reducing the sensitivity to echogenic foci may be most helpful for improving specificity with minimal change in sensitivity.

Software-Based Analysis of the Taller-Than-Wide Feature of High-Risk Thyroid Nodules

BACKGROUND

Shape is one of the most important features in the diagnosis of malignant thyroid nodules. This characteristic has been described qualitatively, but only shapes that appear markedly different can be easily differentiated at first interpretation. This study sought to clarify whether software-based shape indexes are useful for the detection of thyroid cancers.

METHODS

In the final analysis, 200 participants with 231 pathologically proven nodules participated in the study. Ultrasound features were assessed by clinicians. The tumor contour was auto-defined, and shape indexes were calculated using commercial software.

RESULTS

Of the 231 nodules, 134 were benign and 97 were malignant. The presence of taller-than-wide (TTW) dimensions differed significantly between the benign and malignant thyroid tumors. Designation of TTW assessed by the software had a higher kappa value and proportional agreement than TTW assessed by clinicians. Disagreement between the clinician and software in designating nodules as TTW occurred for 28 nodules. The presence of other ultrasonic characteristics and small differences in the height and width measurements were causes for the incorrect interpretation of the TTW feature.

CONCLUSION

The proposed software-based quantitative analysis of tumor shape seems to be promising as an important advance compared with conventional TTW features evaluated by operators because it allows for a more reliable and consistent distinction and is less influenced by other ultrasonic features.

Risk Stratification in Patients With Follicular Neoplasm on Cytology: Use of Quantitative Characteristics and Sonographic Patterns

OBJECTIVES

Differentiating thyroid nodules with a cytological diagnosis of follicular neoplasm remains an issue. The goal of this study was to determine whether ultrasonographic (US) findings obtained preoperatively from the computer-aided detection (CAD) system are sufficient to further stratify the risk of malignancy for this diagnostic cytological category.

METHODS

From September 2016 to September 2018 in our hospital, patients diagnosed with Bethesda category IV (follicular neoplasm or suspicion of follicular neoplasm) thyroid nodules and underwent surgical excisions were include in the study. Quantification and analysis of tumor features were performed using CAD software. The US findings of the region of interest, including index of composition, margin, echogenicity, texture, echogenic dots indicative of calcifications, tall and wide orientation, and margin were calculated into computerized values. The nodules were further classified into American Thyroid Association (ATA) and American College of Radiology Thyroid Imaging Reporting & Data System (TI-RADS) categories.

RESULTS

92 (10.1%) of 913 patients were diagnosed with Bethesda category IV thyroid nodules. In 65 patients, the histological type of the nodule was identified. The quantitative features between patients with benign and malignant conditions differed significantly. The presence of heterogeneous echotexture, blurred margins, or irregular margins was shown to have the highest diagnostic value. The risks of malignancy for nodules classified as having very low to intermediate suspicion ATA, non-ATA, and high suspicion ATA patterns were 9%, 35.7%, and 51.7%, respectively. Meanwhile, the risks of malignancy were 12.5%, 26.1%, and 53.8% for nodules classified as TIRADS 3, 4, and 5, respectively. When compared to human observers, among whom poor agreement was noticeable, the CAD software has shown a higher average accuracy.

CONCLUSIONS

For patients with nodules diagnosed as Bethesda category IV, the software-based characterizations of US features, along with the associated ATA patterns and TIRADS system, were shown helpful in the risk stratification of malignancy.

Use of artificial intelligence and machine learning for estimating malignancy risk of thyroid nodules

PURPOSE OF REVIEW

Current methods for thyroid nodule risk stratification are subjective, and artificial intelligence algorithms have been used to overcome this shortcoming. In this review, we summarize recent developments in the application of artificial intelligence algorithms for estimating the risks of malignancy in a thyroid nodule.

RECENT FINDINGS

Artificial intelligence have been used to predict malignancy in thyroid nodules using ultrasound images, cytopathology images, and molecular markers. Recent clinical trials have shown that artificial intelligence model's performance matched that of experienced radiologists and pathologists. Explainable artificial intelligence models are being developed to avoid the black box problem. Risk stratification algorithms using artificial intelligence for thyroid nodules are now commercially available in many countries.

SUMMARY

Artificial intelligence models could become a useful tool in a thyroidolgist's armamentarium as a decision support tool. Increased adoption of this emerging technology will depend upon increased awareness of the potential benefits and pitfalls in using artificial intelligence.