Thyroid nodules risk stratification through deep learning based on ultrasound images

From Bench-to-Bedside: How Artificial Intelligence is Changing Thyroid Nodule Diagnostics, a Systematic Review

CONTEXT

Use of artificial intelligence (AI) to predict clinical outcomes in thyroid nodule diagnostics has grown exponentially over the past decade. The greatest challenge is in understanding the best model to apply to one's own patient population, and how to operationalize such a model in practice.

EVIDENCE ACQUISITION

A literature search of PubMed and IEEE Xplore was conducted for English-language publications between January 1, 2015 and January 1, 2023, studying diagnostic tests on suspected thyroid nodules that used AI. We excluded articles without prospective or external validation, nonprimary literature, duplicates, focused on nonnodular thyroid conditions, not using AI, and those incidentally using AI in support of an experimental diagnostic outside standard clinical practice. Quality was graded by Oxford level of evidence.

EVIDENCE SYNTHESIS

A total of 61 studies were identified; all performed external validation, 16 studies were prospective, and 33 compared a model to physician prediction of ground truth. Statistical validation was reported in 50 papers. A diagnostic pipeline was abstracted, yielding 5 high-level outcomes: (1) nodule localization, (2) ultrasound (US) risk score, (3) molecular status, (4) malignancy, and (5) long-term prognosis. Seven prospective studies validated a single commercial AI; strengths included automating nodule feature assessment from US and assisting the physician in predicting malignancy risk, while weaknesses included automated margin prediction and interobserver variability.

CONCLUSION

Models predominantly used US images to predict malignancy. Of 4 Food and Drug Administration-approved products, only S-Detect was extensively validated. Implementing an AI model locally requires data sanitization and revalidation to ensure appropriate clinical performance.

Localization and Risk Stratification of Thyroid Nodules in Ultrasound Images Through Deep Learning

OBJECTIVE

Deep learning algorithms have commonly been used for the differential diagnosis between benign and malignant thyroid nodules. The aim of the study described here was to develop an integrated system that combines a deep learning model and a clinical standard Thyroid Imaging Reporting and Data System (TI-RADS) for the simultaneous segmentation and risk stratification of thyroid nodules.

METHODS

Three hundred four ultrasound images from two independent sites with TI-RADS 4 thyroid nodules were collected. The edge connection and Criminisi algorithm were used to remove manually induced markers in ultrasound images. An integrated system based on TI-RADS and a mask region-based convolution neural network (Mask R-CNN) was proposed to stratify subclasses of TI-RADS 4 thyroid nodules and to segment thyroid nodules in the ultrasound images. Accuracy and the precision-recall curve were used to evaluate stratification performance, and the Dice similarity coefficient (DSC) between the segmentation of Mask R-CNN and the radiologist's contour was used to evaluate the segmentation performance of the model.

RESULTS

The combined approach could significantly enhance the performance of the proposed integrated system. Overall stratification accuracy of TI-RADS 4 thyroid nodules, mean average precision and mean DSC of the proposed model in the independent test set was 90.79%, 0.8579 and 0.83, respectively. Specifically, stratification accuracy values for TI-RADS 4a, 4b and 4c thyroid nodules were 95.83%, 84.21% and 77.78%, respectively.

CONCLUSION

An integrated system combining TI-RADS and a deep learning model was developed. The system can provide clinicians with not only diagnostic assistance from TI-RADS but also accurate segmentation of thyroid nodules, which improves the applicability of the system in clinical practice.

Deep learning driven diagnosis of malignant soft tissue tumors based on dual-modal ultrasound images and clinical indexes

Background

Soft tissue tumors (STTs) are benign or malignant superficial neoplasms arising from soft tissues throughout the body with versatile pathological types. Although Ultrasonography (US) is one of the most common imaging tools to diagnose malignant STTs, it still has several drawbacks in STT diagnosis that need improving.

Objectives

The study aims to establish this deep learning (DL) driven Artificial intelligence (AI) system for predicting malignant STTs based on US images and clinical indexes of the patients.

Methods

We retrospectively enrolled 271 malignant and 462 benign masses to build the AI system using 5-fold validation. A prospective dataset of 44 malignant masses and 101 benign masses was used to validate the accuracy of system. A multi-data fusion convolutional neural network, named ultrasound clinical soft tissue tumor net (UC-STTNet), was developed to combine gray scale and color Doppler US images and clinic features for malignant STTs diagnosis. Six radiologists (R1-R6) with three experience levels were invited for reader study.

Results

The AI system achieved an area under receiver operating curve (AUC) value of 0.89 in the retrospective dataset. The diagnostic performance of the AI system was higher than that of one of the senior radiologists (AUC of AI vs R2: 0.89 vs. 0.84, p=0.022) and all of the intermediate and junior radiologists (AUC of AI vs R3, R4, R5, R6: 0.89 vs 0.75, 0.81, 0.80, 0.63; p <0.01). The AI system also achieved an AUC of 0.85 in the prospective dataset. With the assistance of the system, the diagnostic performances and inter-observer agreement of the radiologists was improved (AUC of R3, R5, R6: 0.75 to 0.83, 0.80 to 0.85, 0.63 to 0.69; p<0.01).

Conclusion

The AI system could be a useful tool in diagnosing malignant STTs, and could also help radiologists improve diagnostic performance.

Applying machine-learning models to differentiate benign and malignant thyroid nodules classified as C-TIRADS 4 based on 2D-ultrasound combined with five contrast-enhanced ultrasound key frames

Objectives

To apply machine learning to extract radiomics features from thyroid two-dimensional ultrasound (2D-US) combined with contrast-enhanced ultrasound (CEUS) images to classify and predict benign and malignant thyroid nodules, classified according to the Chinese version of the thyroid imaging reporting and data system (C-TIRADS) as category 4.

Materials and methods

This retrospective study included 313 pathologically diagnosed thyroid nodules (203 malignant and 110 benign). Two 2D-US images and five CEUS key frames ("2nd second after the arrival time" frame, "time to peak" frame, "2nd second after peak" frame, "first-flash" frame, and "second-flash" frame) were selected to manually label the region of interest using the "Labelme" tool. A total of 7 images of each nodule and their annotates were imported into the Darwin Research Platform for radiomics analysis. The datasets were randomly split into training and test cohorts in a 9:1 ratio. Six classifiers, namely, support vector machine, logistic regression, decision tree, random forest (RF), gradient boosting decision tree and extreme gradient boosting, were used to construct and test the models. Performance was evaluated using a receiver operating characteristic curve analysis. The area under the curve (AUC), sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy (ACC), and F1-score were calculated. One junior radiologist and one senior radiologist reviewed the 2D-US image and CEUS videos of each nodule and made a diagnosis. We then compared their AUC and ACC with those of our best model.

Results

The AUC of the diagnosis of US, CEUS and US combined CEUS by junior radiologist and senior radiologist were 0.755, 0.750, 0.784, 0.800, 0.873, 0.890, respectively. The RF classifier performed better than the other five, with an AUC of 1 for the training cohort and 0.94 (95% confidence interval 0.88-1) for the test cohort. The sensitivity, specificity, accuracy, PPV, NPV, and F1-score of the RF model in the test cohort were 0.82, 0.93, 0.90, 0.85, 0.92, and 0.84, respectively. The RF model with 2D-US combined with CEUS key frames achieved equivalent performance as the senior radiologist (AUC: 0.94 vs. 0.92, P = 0.798; ACC: 0.90 vs. 0.92) and outperformed the junior radiologist (AUC: 0.94 vs. 0.80, P = 0.039, ACC: 0.90 vs. 0.81) in the test cohort.

Conclusions

Our model, based on 2D-US and CEUS key frames radiomics features, had good diagnostic efficacy for thyroid nodules, which are classified as C-TIRADS 4. It shows promising potential in assisting less experienced junior radiologists.

A multi-task model for reliable classification of thyroid nodules in ultrasound images

Thyroid nodules are common, and patients with potential malignant lesions are usually diagnosed using ultrasound imaging to determine further treatment options. This study aims to propose a computer-aided diagnosis method for benign and malignant classification of thyroid nodules in ultrasound images. We propose a novel multi-task framework that combines the advantages of dense connectivity, Squeeze-and-Excitation (SE) connectivity, and Atrous Spatial Pyramid Pooling (ASPP) layer to enhance feature extraction. The Dense connectivity is used to optimize feature reuse, the SE connectivity to optimize feature weights, the ASPP layer to fuse feature information, and a multi-task learning framework to adjust the attention of the network. We evaluate our model using a 10-fold cross-validation approach based on our established Thyroid dataset. We assess the performance of our method using six average metrics: accuracy, sensitivity, specificity, positive predictive value, negative predictive value, and AUC, which are 93.49, 95.54, 91.52, 91.63, 95.47, and 96.84%, respectively. Our proposed method outperforms other classification networks in all metrics, achieving optimal performance. We propose a multi-task model, DSMA-Net, for distinguishing thyroid nodules in ultrasound images. This method can further enhance the diagnostic ability of doctors for suspected cancer patients and holds promise for clinical applications.

The performance of deep learning on thyroid nodule imaging predicts thyroid cancer: A systematic review and meta-analysis of epidemiological studies with independent external test sets

BACKGROUND AND AIMS

It is still controversial whether deep learning (DL) systems add accuracy to thyroid nodule imaging classification based on the recent available evidence. We conducted this study to analyze the current evidence of DL in thyroid nodule imaging diagnosis in both internal and external test sets.

METHODS

Until the end of December 2022, PubMed, IEEE, Embase, Web of Science, and the Cochrane Library were searched. We included primary epidemiological studies using externally validated DL techniques in image-based thyroid nodule appraisal. This systematic review was registered on PROSPERO (CRD42022362892).

RESULTS

We evaluated evidence from 17 primary epidemiological studies using externally validated DL techniques in image-based thyroid nodule appraisal. Fourteen studies were deemed eligible for meta-analysis. The pooled sensitivity, specificity, and area under the curve (AUC) of these DL algorithms were 0.89 (95% confidence interval 0.87-0.90), 0.84 (0.82-0.86), and 0.93 (0.91-0.95), respectively. For the internal validation set, the pooled sensitivity, specificity, and AUC were 0.91 (0.89-0.93), 0.88 (0.85-0.91), and 0.96 (0.93-0.97), respectively. In the external validation set, the pooled sensitivity, specificity, and AUC were 0.87 (0.85-0.89), 0.81 (0.77-0.83), and 0.91 (0.88-0.93), respectively. Notably, in subgroup analyses, DL algorithms still demonstrated exceptional diagnostic validity.

CONCLUSIONS

Current evidence suggests DL-based imaging shows diagnostic performances comparable to clinicians for differentiating thyroid nodules in both the internal and external test sets.

Thyroid Nodule Detection and Region Estimation in Ultrasound Images: A Comparison between Physicians and an Automated Decision Support System Approach

BACKGROUND

Thyroid nodules are very common. In most cases, they are benign, but they can be malignant in a low percentage of cases. The accurate assessment of these nodules is critical to choosing the next diagnostic steps and potential treatment. Ultrasound (US) imaging, the primary modality for assessing these nodules, can lack objectivity due to varying expertise among physicians. This leads to observer variability, potentially affecting patient outcomes.

PURPOSE

This study aims to assess the potential of a Decision Support System (DSS) in reducing these variabilities for thyroid nodule detection and region estimation using US images, particularly in lesser experienced physicians.

METHODS

Three physicians with varying levels of experience evaluated thyroid nodules on US images, focusing on nodule detection and estimating cystic and solid regions. The outcomes were compared to those obtained from a DSS for comparison. Metrics such as classification match percentage and variance percentage were used to quantify differences.

RESULTS

Notable disparities exist between physician evaluations and the DSS assessments: the overall classification match percentage was just 19.2%. Individually, Physicians 1, 2, and 3 had match percentages of 57.6%, 42.3%, and 46.1% with the DSS, respectively. Variances in assessments highlight the subjectivity and observer variability based on physician experience levels.

CONCLUSIONS

The evident variability among physician evaluations underscores the need for supplementary decision-making tools. Given its consistency, the CAD offers potential as a reliable "second opinion" tool, minimizing human-induced variabilities in the critical diagnostic process of thyroid nodules using US images. Future integration of such systems could bolster diagnostic precision and improve patient outcomes.

Artificial Intelligence in Thyroidology: A Narrative Review of the Current Applications, Associated Challenges, and Future Directions

Background: The use of artificial intelligence (AI) in health care has grown exponentially with the promise of facilitating biomedical research and enhancing diagnosis, treatment, monitoring, disease prevention, and health care delivery. We aim to examine the current state, limitations, and future directions of AI in thyroidology. Summary: AI has been explored in thyroidology since the 1990s, and currently, there is an increasing interest in applying AI to improve the care of patients with thyroid nodules (TNODs), thyroid cancer, and functional or autoimmune thyroid disease. These applications aim to automate processes, improve the accuracy and consistency of diagnosis, personalize treatment, decrease the burden for health care professionals, improve access to specialized care in areas lacking expertise, deepen the understanding of subtle pathophysiologic patterns, and accelerate the learning curve of less experienced clinicians. There are promising results for many of these applications. Yet, most are in the validation or early clinical evaluation stages. Only a few are currently adopted for risk stratification of TNODs by ultrasound and determination of the malignant nature of indeterminate TNODs by molecular testing. Challenges of the currently available AI applications include the lack of prospective and multicenter validations and utility studies, small and low diversity of training data sets, differences in data sources, lack of explainability, unclear clinical impact, inadequate stakeholder engagement, and inability to use outside of the research setting, which might limit the value of their future adoption. Conclusions: AI has the potential to improve many aspects of thyroidology; however, addressing the limitations affecting the suitability of AI interventions in thyroidology is a prerequisite to ensure that AI provides added value for patients with thyroid disease.

Clinical knowledge embedded method based on multi-task learning for thyroid nodule classification with ultrasound images

Objective. Thyroid nodules are common glandular abnormality that need to be diagnosed as benign or malignant to determine further treatments. Clinically, ultrasonography is the main diagnostic method, but it is highly subjective with severe variability. Recently, many deep-learning-based methods have been proposed to alleviate subjectivity and achieve good results yet, these methods often neglect important guidance from clinical knowledge. Our objective is to utilize such guidance for accurate and reliable thyroid nodule classification.Approach. In this study, a multi-task learning model embedded with clinical knowledge of ACR Thyroid Imaging, Reporting and Data System guideline is proposed. The clinical features defined in the guideline have strong correlations with malignancy and they were modeled as tasks alongside the pathological type. Multi-task learning was utilized to exploit the correlations to improve diagnostic performance. To alleviate the impact of noisy labels on clinical features, a loss-weighting strategy was proposed. Five-fold cross-validation was applied to an internal training set of size 4989, and an external test set of size 243 was used for evaluation.Main results. The proposed multi-task learning model achieved an average AUC of 0.901 and an ensemble AUC of 0.917 on the test set, which significantly outperformed the single-task baseline models.Significance. The results indicated that multi-task learning of clinical features can effectively classify thyroid nodules and reveal the possibility of using clinical indicators as auxiliary tasks to improve performance when diagnosing other diseases.

Investigation of optimal convolutional neural network conditions for thyroid ultrasound image analysis

Neural network models have been used to analyze thyroid ultrasound (US) images and stratify malignancy risk of the thyroid nodules. We investigated the optimal neural network condition for thyroid US image analysis. We compared scratch and transfer learning models, performed stress tests in 10% increments, and compared the performance of three threshold values. All validation results indicated superiority of the transfer learning model over the scratch model. Stress test indicated that training the algorithm using 3902 images (70%) resulted in a performance which was similar to the full dataset (5575). Threshold 0.3 yielded high sensitivity (1% false negative) and low specificity (72% false positive), while 0.7 gave low sensitivity (22% false negative) and high specificity (23% false positive). Here we showed that transfer learning was more effective than scratch learning in terms of area under curve, sensitivity, specificity and negative/positive predictive value, that about 3900 images were minimally required to demonstrate an acceptable performance, and that algorithm performance can be customized according to the population characteristics by adjusting threshold value.

Ultrasound imaging in thyroid nodule diagnosis, therapy, and follow-up: Current status and future trends

Ultrasound, the primary imaging modality in thyroid nodule management, suffers from drawbacks including: high inter- and intra-observer variability, limited field-of-view and limited functional imaging. Developments in ultrasound technologies are taking place to overcome these limitations, including three-dimensional-Doppler, -elastography, -nodule characteristics-extraction, and novel machine-learning algorithms. For thyroid ablative treatments and biopsies, perioperative use of three-dimensional ultrasound opens a new field of research. This review provides an overview of the current and future applications of ultrasound, and discusses the potential of new developments and trends that may improve the diagnosis, therapy, and follow-up of thyroid nodules.

Automatic classification of thyroid nodules in ultrasound images using a multi-task attention network guided by clinical knowledge

Thyroid cancer has been the most prevalent cancer in the recent three decades. Ultrasonography is one of the mainly used methods for diagnosing thyroid nodules. Several computer-aided diagnostic methods were proposed to aid radiologists in analyzing ultrasound images of the thyroid gland. Most methods, however, only determine the benignity or malignancy of the thyroid nodule and do not explain the decision-making process of them, which cannot gain the trustworthiness of clinicians because they are not consistent with the physician's diagnostic process. In our work, we design a multi-task branching attention network in which each of the descriptors of the ACR TI-RADS lexicon is first classified. All respective scores are calculated to get the risk stratification of the nodule. Ultimately, based on the risk stratification, the benignity or malignancy of the nodule is determined. This work provides an automated method that incorporates the ACR TI-RADS characterization of thyroid nodules for detecting the level of risk and the benignity or malignancy of thyroid nodules. Thus the work establishes the trustworthiness of clinicians in deep learning models and improves physician efficiency and diagnostic rates to some extent compared to previous studies. For the diagnosis of thyroid nodules, evaluation indices including accuracy, sensitivity, and specificity were 93.55%, 93.86%, and 93.14%, respectively. The experiments show that our approach obtains comparable performance to most advanced methods in diagnosing ultrasound images of the thyroid nodules and is supported by explanations in clinical terms using the ACR TI-RADS lexicon.

Multimodal ultrasound imaging: A method to improve the accuracy of diagnosing thyroid TI-RADS 4 nodules

Thyroid nodule is a common and frequently occurring disease in the neck in recent years, and ultrasound has become the preferred imaging diagnosis method for thyroid nodule due to its advantages of noninvasive, nonradiation, real-time, and repeatable. The thyroid imaging, reporting and data system (TI-RADS) classification standard scores suspicious nodules that are difficult to determine benign and malignant as grade 4, and further pathological puncture is recommended clinically, which may lead to a large number of unnecessary biopsies and operations. Including conventional ultrasound, ACR TI-RADS, shear wave elastography, super microvascular imaging, contrast enhanced ultrasound, "firefly," artificial intelligence, and multimodal ultrasound imaging used in combination. In order to identify the most clinically significant malignant tumors when reducing invasive operations. This article reviews the application and research progress of multimodal ultrasound imaging in the diagnosis of TI-RADS 4 thyroid nodules.

Artificial Intelligence for Thyroid Nodule Characterization: Where Are We Standing?

Simple Summary

In the present review, an up-to-date summary of the state of the art of artificial intelligence (AI) implementation for thyroid nodule characterization and cancer is provided. The opinion on the real effectiveness of AI systems remains controversial. Taking into consideration the largest and most scientifically valid studies, it is possible to state that AI provides results that are comparable or inferior to expert ultrasound specialists and radiologists. Promising data approve AI as a support tool and simultaneously highlight the need for a radiologist supervisory framework for AI provided results. Therefore, current solutions might be more suitable for educational purposes.

Abstract

Machine learning (ML) is an interdisciplinary sector in the subset of artificial intelligence (AI) that creates systems to set up logical connections using algorithms, and thus offers predictions for complex data analysis. In the present review, an up-to-date summary of the current state of the art regarding ML and AI implementation for thyroid nodule ultrasound characterization and cancer is provided, highlighting controversies over AI application as well as possible benefits of ML, such as, for example, training purposes. There is evidence that AI increases diagnostic accuracy and significantly limits inter-observer variability by using standardized mathematical algorithms. It could also be of aid in practice settings with limited sub-specialty expertise, offering a second opinion by means of radiomics and computer-assisted diagnosis. The introduction of AI represents a revolutionary event in thyroid nodule evaluation, but key issues for further implementation include integration with radiologist expertise, impact on workflow and efficiency, and performance monitoring.

Artificial Intelligence for Personalized Medicine in Thyroid Cancer: Current Status and Future Perspectives

Thyroid cancers (TC) have increasingly been detected following advances in diagnostic methods. Risk stratification guided by refined information becomes a crucial step toward the goal of personalized medicine. The diagnosis of TC mainly relies on imaging analysis, but visual examination may not reveal much information and not enable comprehensive analysis. Artificial intelligence (AI) is a technology used to extract and quantify key image information by simulating complex human functions. This latent, precise information contributes to stratify TC on the distinct risk and drives tailored management to transit from the surface (population-based) to a point (individual-based). In this review, we started with several challenges regarding personalized care in TC, for example, inconsistent rating ability of ultrasound physicians, uncertainty in cytopathological diagnosis, difficulty in discriminating follicular neoplasms, and inaccurate prognostication. We then analyzed and summarized the advances of AI to extract and analyze morphological, textural, and molecular features to reveal the ground truth of TC. Consequently, their combination with AI technology will make individual medical strategies possible.