Machine Learning-Assisted System for Thyroid Nodule Diagnosis

Localization and Risk Stratification of Thyroid Nodules in Ultrasound Images Through Deep Learning

OBJECTIVE

Deep learning algorithms have commonly been used for the differential diagnosis between benign and malignant thyroid nodules. The aim of the study described here was to develop an integrated system that combines a deep learning model and a clinical standard Thyroid Imaging Reporting and Data System (TI-RADS) for the simultaneous segmentation and risk stratification of thyroid nodules.

METHODS

Three hundred four ultrasound images from two independent sites with TI-RADS 4 thyroid nodules were collected. The edge connection and Criminisi algorithm were used to remove manually induced markers in ultrasound images. An integrated system based on TI-RADS and a mask region-based convolution neural network (Mask R-CNN) was proposed to stratify subclasses of TI-RADS 4 thyroid nodules and to segment thyroid nodules in the ultrasound images. Accuracy and the precision-recall curve were used to evaluate stratification performance, and the Dice similarity coefficient (DSC) between the segmentation of Mask R-CNN and the radiologist's contour was used to evaluate the segmentation performance of the model.

RESULTS

The combined approach could significantly enhance the performance of the proposed integrated system. Overall stratification accuracy of TI-RADS 4 thyroid nodules, mean average precision and mean DSC of the proposed model in the independent test set was 90.79%, 0.8579 and 0.83, respectively. Specifically, stratification accuracy values for TI-RADS 4a, 4b and 4c thyroid nodules were 95.83%, 84.21% and 77.78%, respectively.

CONCLUSION

An integrated system combining TI-RADS and a deep learning model was developed. The system can provide clinicians with not only diagnostic assistance from TI-RADS but also accurate segmentation of thyroid nodules, which improves the applicability of the system in clinical practice.

Fully-Automatic Detection and Diagnosis System for Thyroid Nodules Based on Ultrasound Video Sequences by Artificial Intelligence

Background

Interpretation of ultrasound findings of thyroid nodules is subjective and labor-intensive for radiologists. Artificial intelligence (AI) is a relatively objective and efficient technology. We aimed to establish a fully automatic detection and diagnosis system for thyroid nodules based on AI technology by analyzing ultrasound video sequences.

Patients and Methods

We prospectively acquired dynamic ultrasound videos of 1067 thyroid nodules (804 for training and 263 for validation) from December 2018 to January 2021. All the patients underwent hemithyroidectomy or total thyroidectomy. Dynamic ultrasound videos were used to develop an AI system consisting of two deep learning models that could automatically detect and diagnose thyroid nodules. Average precision (AP) was used to estimate the performance of the detection model. The area under the receiver operating characteristic curve (AUC) was used to measure the performance of the diagnostic model.

Results

Location and shape were accurately detected with a high AP of 0.914 in the validation cohort. The AUC of the diagnostic model was 0.953 in the validation cohort. The sensitivity and specificity of junior and senior radiologists were 76.9% vs 78.3% and 68.4% vs 81.1%, respectively. The diagnostic performance of the AI diagnostic model was superior to that of junior radiologists (P = 0.016) and was not significantly different from that of senior radiologists (P = 0.281).

Conclusion

We established a fully automatic detection and diagnosis system for thyroid nodules based on ultrasound video using an AI approach that can be conveniently applied to optimize the management of patients with thyroid nodules.

Exploring the diagnostic value of ultrasound radiomics for neonatal respiratory distress syndrome

BACKGROUND

Neonatal respiratory distress syndrome (NRDS) is a prevalent cause of respiratory failure and death among newborns, and prompt diagnosis is imperative. Historically, diagnosis of NRDS relied mostly on typical clinical manifestations, chest X-rays, and CT scans. However, recently, ultrasound has emerged as a valuable and preferred tool for aiding NRDS diagnosis. Nevertheless, evaluating lung ultrasound imagery necessitates rigorous training and may be subject to operator-dependent bias, limiting its widespread use. As a result, it is essential to investigate a new, reliable, and operator-independent diagnostic approach that does not require subjective factors or operator expertise. This article aims to explore the diagnostic potential of ultrasound-based radiomics in differentiating NRDS from other non-NRDS lung disease.

METHODS

A total of 150 neonatal lung disease cases were consecutively collected from the department of neonatal intensive care unit of the Quanzhou Maternity and Children's Hospital, Fujian Province, from September 2021 to October 2022. Of these patients, 60 were diagnosed with NRDS, whereas 30 were diagnosed with neonatal pneumonia, meconium aspiration syndrome (MAS), and transient tachypnea (TTN). Two ultrasound images with characteristic manifestations of each lung disease were acquired and divided into training (n = 120) and validation cohorts (n = 30) based on the examination date using an 8:2 ratio. The imaging texture features were extracted using PyRadiomics and, after the screening, machine learning models such as random forest (RF), logistic regression (LR), K-nearest neighbors (KNN), support vector machine (SVM), and multilayer perceptron (MLP) were developed to construct an imaging-based diagnostic model. The diagnostic efficacy of each model was analyzed. Lastly, we randomly selected 282 lung ultrasound images and evaluated the diagnostic efficacy disparities between the optimal model and doctors across differing levels of expertise.

RESULTS

Twenty-two imaging-based features with the highest weights were selected to construct a predictive model for neonatal respiratory distress syndrome. All models exhibited favorable diagnostic performances. Analysis of the Youden index demonstrated that the RF model had the highest score in both the training (0.99) and validation (0.90) cohorts. Additionally, the calibration curve indicated that the RF model had the best calibration (P = 0.98). When compared to the diagnostic performance of experienced and junior physicians, the RF model had an area under the curve (AUC) of 0.99; however, the values for experienced and junior physicians were 0.98 and 0.85, respectively. The difference in diagnostic efficacy between the RF model and experienced physicians was not statistically significant (P = 0.24), whereas that between the RF model and junior physicians was statistically significant (P < 0.0001).

CONCLUSION

The RF model exhibited excellent diagnostic performance in the analysis of texture features based on ultrasound radiomics for diagnosing NRDS.

Comparative analysis of machine learning-based ultrasound radiomics in predicting malignancy of partially cystic thyroid nodules

OBJECTIVE

To investigate the application of machine learning (ML) model-based thyroid ultrasound radiomics in the evaluation of malignancy in partially cystic thyroid nodules (PCTNs).

METHODS

One hundred and ninety-two patients with 197 nodules PCTNs from January 2020 to December 2020 were retrospectively analyzed. Radiomics features were extracted based on hand-crafted features from the ultrasound images, and machine learning methods were used to build a classification model by radiomics features. The least absolute shrinkage and selection operator regression was applied to select the features of nonzero coefficients from radiomics features. The prediction performance of the established model was mainly evaluated by the area under the curve (AUC) and accuracy, sensitivity, and specificity.

RESULTS

Nineteen radiomics features were extracted from the original images for each nodule. Eight ML classifiers were able to differentiate malignancy in PCTNs. The AUC, accuracy, sensitivity, and specificity of k-Nearest Neighbor (KNN) model were 0.909, 82.95%, 83.33%, and 89.90%, respectively, on the test cohort. The comparative result showed statistically equivalent performance for thyroid nodule diagnosis based on image fusion and single image. In addition, the ML-Based ultrasound radiomics system showed a better AUC as compared with ACR TI-RADS model and the ultrasound features model.

CONCLUSION

The novel ultrasonic-based ML model has an important clinical value for predicting malignancy in PCTNs. It can provide clinicians with a preoperative non-invasive primary screening method for PCTN diagnosis to avoid unnecessary medical investment and improve treatment outcomes.

A knowledge-interpretable multi-task learning framework for automated thyroid nodule diagnosis in ultrasound videos

Ultrasound has become the most widely used modality for thyroid nodule diagnosis, due to its portability, real-time feedback, lack of toxicity, and low cost. Recently, the computer-aided diagnosis (CAD) of thyroid nodules has attracted significant attention. However, most existing techniques can only be applied to either static images with prominent features (manually selected from scanning videos) or rely on 'black boxes' that cannot provide interpretable results. In this study, we develop a user-friendly framework for the automated diagnosis of thyroid nodules in ultrasound videos, by simulating the typical diagnostic workflow used by radiologists. This process consists of two orderly part-to-whole tasks. The first interprets the characteristics of each image using prior knowledge, to obtain corresponding frame-wise TI-RADS scores. Associated embedded representations not only provide diagnostic information for radiologists but also reduce computational costs. The second task models temporal contextual information in an embedding vector sequence and selectively enhances important information to distinguish benign and malignant thyroid nodules, thereby improving the efficiency and generalizability of the proposed framework. Experimental results demonstrated this approach outperformed other state-of-the-art video classification methods. In addition to assisting radiologists in understanding model predictions, these CAD results could further ease diagnostic workloads and improve patient care.

A novel approach to quantify calcifications of thyroid nodules in US images based on deep learning: predicting the risk of cervical lymph node metastasis in papillary thyroid cancer patients

OBJECTIVE

Based on ultrasound (US) images, this study aimed to detect and quantify calcifications of thyroid nodules, which are regarded as one of the most important features in US diagnosis of thyroid cancer, and to further investigate the value of US calcifications in predicting the risk of lymph node metastasis (LNM) in papillary thyroid cancer (PTC).

METHODS

Based on the DeepLabv3+ networks, 2992 thyroid nodules in US images were used to train a model to detect thyroid nodules, of which 998 were used to train a model to detect and quantify calcifications. A total of 225 and 146 thyroid nodules obtained from two centers, respectively, were used to test the performance of these models. A logistic regression method was used to construct the predictive models for LNM in PTCs.

RESULTS

Calcifications detected by the network model and experienced radiologists had an agreement degree of above 90%. The novel quantitative parameters of US calcification defined in this study showed a significant difference between PTC patients with and without cervical LNM (p < 0.05). The calcification parameters were beneficial to predicting the LNM risk in PTC patients. The LNM prediction model using these calcification parameters combined with patient age and other US nodular features showed a higher specificity and accuracy than the calcification parameters alone.

CONCLUSIONS

Our models not only detect the calcifications automatically, but also have value in predicting cervical LNM risk of PTC patients, thereby making it possible to investigate the relationship between calcifications and highly invasive PTC in detail.

CLINICAL RELEVANCE STATEMENT

Due to the high association of US microcalcifications with thyroid cancers, our model will contribute to the differential diagnosis of thyroid nodules in daily practice.

KEY POINTS

• We developed an ML-based network model for automatically detecting and quantifying calcifications within thyroid nodules in US images. • Three novel parameters for quantifying US calcifications were defined and verified. • These US calcification parameters showed value in predicting the risk of cervical LNM in PTC patients.

Thyroidkeeper: a healthcare management system for patients with thyroid diseases

Thyroid diseases, especially thyroid tumors, have a huge population in China. The postoperative patients, under China's incomplete tertiary diagnosis and treatment system, will frequently go to tertiary hospitals for follow-up and medication adjustment, resulting in heavy burdens on both specialists and patients. To help postoperative patients recover better against the above adverse conditions, a novel mobile application ThyroidKeeper is proposed as a collaborative AI-based platform that benefits both patients and doctors. In addition to routine health records and management functions, ThyroidKeeper has achieved several innovative points. First, it can automatically adjust medication dosage for patients during their rehabilitation based on their medical history, laboratory indicators, physical health status, and current medication. Second, it can comprehensively predict the possible complications based on the patient's health status and the health status of similar groups utilizing graph neural networks. Finally, the employing of graph neural network models can improve the efficiency of online communication between doctors and patients, help doctors obtain medical information for patients more quickly and precisely, and make more accurate diagnoses. The preliminary evaluation in both laboratory and real-world environments shows the advantages of the proposed ThyroidKeeper system.

Establishment of a differential diagnosis method and an online prediction platform for AOSD and sepsis based on gradient boosting decision trees algorithm

OBJECTIVE

The differential diagnosis between adult-onset Still's disease (AOSD) and sepsis has always been a challenge. In this study, a machine learning model for differential diagnosis of AOSD and sepsis was developed and an online platform was developed to facilitate the clinical application of the model.

METHODS

All data were collected from 42 AOSD patients and 50 sepsis patients admitted to Affiliated Hospital of Xuzhou Medical University from December 2018 to December 2021. In addition, 5 AOSD patients and 10 sepsis patients diagnosed in our hospital after March 2022 were collected for external validation. All models were built using the scikit-learn library (version 1.0.2) in Python (version 3.9.7), and feature selection was performed using the SHAP (Shapley Additive exPlanation) package developed in Python.

RESULTS

The results showed that the gradient boosting decision tree(GBDT) optimization model based on arthralgia, ferritin × lymphocyte count, white blood cell count, ferritin × platelet count, and α1-acid glycoprotein/creatine kinase could well identify AOSD and sepsis. The training set interaction test (AUC: 0.9916, ACC: 0.9457, Sens: 0.9556, Spec: 0.9578) and the external validation also achieved satisfactory results (AUC: 0.9800, ACC: 0.9333, Sens: 0.8000, Spec: 1.000). We named this discrimination method AIADSS (AI-assisted discrimination of Still's disease and Sepsis) and created an online service platform for practical operation, the website is http://cppdd.cn/STILL1/ .

CONCLUSION

We created a method for the identification of AOSD and sepsis based on machine learning. This method can provide a reference for clinicians to formulate the next diagnosis and treatment plan.

Study on diagnosis of thyroid nodules based on convolutional neural network

OBJECTIVE

An artificial intelligence (AI) algorithm based on convolutional neural networks was used in ultrasound diagnosis in order to evaluate its performance in judging the nature of thyroid nodules and nodule classification.

METHODS

A total of 105 patients with thyroid nodules confirmed by surgery or biopsy were retrospectively analyzed. The properties, characteristics, and classification of thyroid nodules were evaluated by sonographers and by AI to obtain combined diagnoses. Receiver operating characteristic curves were generated to evaluate the performance of AI, the sonographer, and their combined effort in diagnosing the nature of thyroid nodules and classifying their characteristics. In the diagnosis of thyroid nodules with solid components, hypoechoic appearance, indistinct borders, Anteroposterior/transverse diameter ratio > 1(A/T > 1), and calcification performed by sonographers and by AI, the properties exhibited statistically significant differences.

RESULTS

Sonographers had a sensitivity of 80.7%, specificity of 73.7%, accuracy of 79.0%, and area under the curve (AUC) of 0.751 in the diagnosis of benign and malignant thyroid nodules. AI had a sensitivity of 84.5%, specificity of 81.0%, accuracy of 84.7%, and AUC of 0.803. The combined AI and sonographer diagnosis had a sensitivity of 92.1%, specificity of 86.3%, accuracy of 91.7%, and AUC of 0.910.

CONCLUSION

The efficacy of a combined diagnosis for benign and malignant thyroid nodules is higher than that of an AI-based diagnosis alone or a sonographer-based diagnosis alone. The combined diagnosis can reduce unnecessary fine-needle aspiration biopsy procedures and better evaluate the necessity of surgery in clinical practice.

Application and prospects of AI-based radiomics in ultrasound diagnosis

Artificial intelligence (AI)-based radiomics has attracted considerable research attention in the field of medical imaging, including ultrasound diagnosis. Ultrasound imaging has unique advantages such as high temporal resolution, low cost, and no radiation exposure. This renders it a preferred imaging modality for several clinical scenarios. This review includes a detailed introduction to imaging modalities, including Brightness-mode ultrasound, color Doppler flow imaging, ultrasound elastography, contrast-enhanced ultrasound, and multi-modal fusion analysis. It provides an overview of the current status and prospects of AI-based radiomics in ultrasound diagnosis, highlighting the application of AI-based radiomics to static ultrasound images, dynamic ultrasound videos, and multi-modal ultrasound fusion analysis.

An update on the management of thyroid nodules: rationalising the guidelines

BACKGROUND

Guidance for the management of thyroid nodules has evolved over time, from initial evaluation based predominantly on clinical grounds to now including the established role of ultrasound and fine needle aspiration cytology in their assessment. There is, however, significant variation in the management of thyroid nodules depending on which national guidelines are followed. In addition, there are certain clinical situations such as pregnancy and paediatric thyroid nodules that have differing evaluation priorities.

OBJECTIVES

This review aimed to provide an overview of currently accepted practices for the initial investigation and subsequent management of patients with thyroid nodules for the non-specialist. The review also addresses areas of variance between the systems in common clinical use, as well as newer, evolving technologies, including molecular testing in the evaluation of malignancy in thyroid nodules.

Predicting BRAFV600E mutations in papillary thyroid carcinoma using six machine learning algorithms based on ultrasound elastography

The most common BRAF mutation is thymine (T) to adenine (A) missense mutation in nucleotide 1796 (T1796A, V600E). The BRAFV600E gene encodes a protein-dependent kinase (PDK), which is a key component of the mitogen-activated protein kinase pathway and essential for controlling cell proliferation, differentiation, and death. The BRAFV600E mutation causes PDK to be activated improperly and continuously, resulting in abnormal proliferation and differentiation in PTC. Based on elastography ultrasound (US) radiomic features, this study seeks to create and validate six distinct machine learning algorithms to predict BRAFV6OOE mutation in PTC patients prior to surgery. This study employed routine US strain elastography image data from 138 PTC patients. The patients were separated into two groups: those who did not have the BRAFV600E mutation (n = 75) and those who did have the mutation (n = 63). The patients were randomly assigned to one of two data sets: training (70%), or validation (30%). From strain elastography US images, a total of 479 radiomic features were retrieved. Pearson's Correlation Coefficient (PCC) and Recursive Feature Elimination (RFE) with stratified tenfold cross-validation were used to decrease the features. Based on selected radiomic features, six machine learning algorithms including support vector machine with the linear kernel (SVM_L), support vector machine with radial basis function kernel (SVM_RBF), logistic regression (LR), Naïve Bayes (NB), K-nearest neighbors (KNN), and linear discriminant analysis (LDA) were compared to predict the possibility of BRAFV600E. The accuracy (ACC), the area under the curve (AUC), sensitivity (SEN), specificity (SPEC), positive predictive value (PPV), negative predictive value (NPV), decision curve analysis (DCA), and calibration curves of the machine learning algorithms were used to evaluate their performance. ① The machine learning algorithms' diagnostic performance depended on 27 radiomic features. ② AUCs for NB, KNN, LDA, LR, SVM_L, and SVM_RBF were 0.80 (95% confidence interval [CI]: 0.65-0.91), 0.87 (95% CI 0.73-0.95), 0.91(95% CI 0.79-0.98), 0.92 (95% CI 0.80-0.98), 0.93 (95% CI 0.80-0.98), and 0.98 (95% CI 0.88-1.00), respectively. ③ There was a significant difference in echogenicity,vertical and horizontal diameter ratios, and elasticity between PTC patients with BRAFV600E and PTC patients without BRAFV600E. Machine learning algorithms based on US elastography radiomic features are capable of predicting the likelihood of BRAFV600E in PTC patients, which can assist physicians in identifying the risk of BRAFV600E in PTC patients. Among the six machine learning algorithms, the support vector machine with radial basis function (SVM_RBF) achieved the best ACC (0.93), AUC (0.98), SEN (0.95), SPEC (0.90), PPV (0.91), and NPV (0.95).

Leveraging deep learning to identify calcification and colloid in thyroid nodules

Background

Both calcification and colloid in thyroid nodules are reflected as echogenic foci in ultrasound images. However, calcification and colloid have significantly different probabilities of malignancy. We explored the performance of a deep learning (DL) model in distinguishing the echogenic foci of thyroid nodules as calcification or colloid.

Methods

We conducted a retrospective study using ultrasound image sets. The DL model was trained and tested on 30,388 images of 1127 nodules. All nodules were pathologically confirmed. The area under the receiver-operator characteristic curve (AUC) was employed as the primary evaluation index.

Results

The YoloV5 (You Only Look Once Version 5) transfer learning model for thyroid nodules based on DL detection showed that the average sensitivity, specificity, and accuracy of distinguishing echogenic foci in the test 1 group (n = 192) was 78.41%, 91.36%, and 77.81%, respectively. The average sensitivity, specificity, and accuracy of the three radiologists were 51.14%, 82.58%, and 61.29%, respectively. The average sensitivity, specificity, and accuracy of distinguishing small echogenic foci in the test 2 group (n = 58) was 70.17%, 77.14%, and 73.33%, respectively. Correspondingly, the average sensitivity, specificity, and accuracy of the radiologists were 57.69%, 63.29%, and 59.38%.

Conclusions

The study demonstrated that DL performed far better than radiologists in distinguishing echogenic foci of thyroid nodules as calcifications or colloid.

Artificial Intelligence in Thyroidology: A Narrative Review of the Current Applications, Associated Challenges, and Future Directions

Background: The use of artificial intelligence (AI) in health care has grown exponentially with the promise of facilitating biomedical research and enhancing diagnosis, treatment, monitoring, disease prevention, and health care delivery. We aim to examine the current state, limitations, and future directions of AI in thyroidology. Summary: AI has been explored in thyroidology since the 1990s, and currently, there is an increasing interest in applying AI to improve the care of patients with thyroid nodules (TNODs), thyroid cancer, and functional or autoimmune thyroid disease. These applications aim to automate processes, improve the accuracy and consistency of diagnosis, personalize treatment, decrease the burden for health care professionals, improve access to specialized care in areas lacking expertise, deepen the understanding of subtle pathophysiologic patterns, and accelerate the learning curve of less experienced clinicians. There are promising results for many of these applications. Yet, most are in the validation or early clinical evaluation stages. Only a few are currently adopted for risk stratification of TNODs by ultrasound and determination of the malignant nature of indeterminate TNODs by molecular testing. Challenges of the currently available AI applications include the lack of prospective and multicenter validations and utility studies, small and low diversity of training data sets, differences in data sources, lack of explainability, unclear clinical impact, inadequate stakeholder engagement, and inability to use outside of the research setting, which might limit the value of their future adoption. Conclusions: AI has the potential to improve many aspects of thyroidology; however, addressing the limitations affecting the suitability of AI interventions in thyroidology is a prerequisite to ensure that AI provides added value for patients with thyroid disease.

Assessment of Renal Fibrosis in Patients With Chronic Kidney Disease Using Shear Wave Elastography and Clinical Features: A Random Forest Approach

OBJECTIVE

Renal fibrosis is the common pathological hallmark of chronic kidney disease (CKD) progression. In this study, a random forest (RF) classifier based on 2-D shear wave elastography (SWE) and clinical features for the differential severity of renal fibrosis in patients with CKD is proposed.

METHODS

A total of 162 patients diagnosed with CKD who underwent 2-D SWE and renal biopsy were prospectively enrolled from April 2019 to December 2021 and then randomized into training (n = 114) and validation (n = 48) cohorts at a ratio of 7:3. The least absolute shrinkage and selection operator (LASSO) regression and recursive feature elimination for support vector machines (SVM-RFE) algorithm were employed to select renal fibrosis-related features from clinical information and elastosonographic findings. An RF model was subsequently constructed using the aforementioned informative parameters in the training cohort and evaluated in terms of discrimination, calibration and clinical utility in both cohorts.

RESULTS

The LASSO and SVM-RFE analyses revealed that age, sex, blood urea nitrogen, renal resistive index, hypertension and the 2D-SWE value were independent risk variables associated with renal fibrosis severity. The established RF model incorporating these six variables exhibited fine discrimination in both the derivation (area under the curve [AUC]: 0.84, 95% confidence interval [CI]: 0.76-0.91) and validation (AUC: 0.88, 95% CI: 0.77-0.98) cohorts. Moreover, the calibration curve revealed satisfactory predictive accuracy, and the decision curve analysis revealed a significant clinical net benefit.

CONCLUSION

The developed RF model, via a combination of the 2-D SWE value and clinical information, indicated satisfactory diagnostic performance and clinical practicality toward differentiating moderate-severe from mild renal fibrosis, which may provide critical insight into risk stratification for patients with CKD.

Diagnostic Value of Artificial Intelligence-Assistant Diagnostic System Combined With Contrast-Enhanced Ultrasound in Thyroid TI-RADS 4 Nodules

OBJECTIVES

This study evaluated the diagnostic value of artificial intelligence-assistant diagnostic system combined with contrast-enhanced ultrasound in The American College of Radiology Thyroid Imaging, Reporting and Data System (ACR TI-RADS) 4 category thyroid nodules.

METHODS

Thyroid nodules that were evaluated as ACR TI-RADS 4 by conventional ultrasound were selected, all of which had pathological or fine needle aspiration (FNA) results. All nodules were examined by contrast-enhanced ultrasound (CEUS) and artificial intelligence (AI) analysis. The sensitivity, specificity, accuracy, positive predictive value (PPV) and negative predictive value (NPV) of AI, CEUS and their combined diagnosis were compared; Analyzed and compared the diagnostic efficiency of AI, CEUS and their combined diagnosis.

RESULTS

A total of 148 thyroid nodules were included in 140 patients, including 58 malignant nodules and 89 benign nodules. The sensitivity of combined diagnosis was significantly higher than that of AI or CEUS alone (P < .05). The NPV of AI, CEUS and combined diagnosis were statistically significant (P < .05). There was no significant difference in the diagnostic efficacy between AI and CEUS (P > .05), but there was a significant difference in NPV between AI and combined diagnosis (P < .05). The AUC of the combined diagnosis was 0.859, which was higher than that of AI, CEUS alone.

CONCLUSIONS

AI has a high diagnostic efficiency, which was helpful for radiologists to make rapid assessment. AI combined CEUS can significantly improve the diagnostic sensitivity and NPV, which was beneficial for the early detection of malignant nodules.

Artificial intelligence - based ultrasound elastography for disease evaluation - a narrative review

Ultrasound elastography (USE) provides complementary information of tissue stiffness and elasticity to conventional ultrasound imaging. It is noninvasive and free of radiation, and has become a valuable tool to improve diagnostic performance with conventional ultrasound imaging. However, the diagnostic accuracy will be reduced due to high operator-dependence and intra- and inter-observer variability in visual observations of radiologists. Artificial intelligence (AI) has great potential to perform automatic medical image analysis tasks to provide a more objective, accurate and intelligent diagnosis. More recently, the enhanced diagnostic performance of AI applied to USE have been demonstrated for various disease evaluations. This review provides an overview of the basic concepts of USE and AI techniques for clinical radiologists and then introduces the applications of AI in USE imaging that focus on the following anatomical sites: liver, breast, thyroid and other organs for lesion detection and segmentation, machine learning (ML) - assisted classification and prognosis prediction. In addition, the existing challenges and future trends of AI in USE are also discussed.

Artificial intelligence in thyroid ultrasound

Artificial intelligence (AI), particularly deep learning (DL) algorithms, has demonstrated remarkable progress in image-recognition tasks, enabling the automatic quantitative assessment of complex medical images with increased accuracy and efficiency. AI is widely used and is becoming increasingly popular in the field of ultrasound. The rising incidence of thyroid cancer and the workload of physicians have driven the need to utilize AI to efficiently process thyroid ultrasound images. Therefore, leveraging AI in thyroid cancer ultrasound screening and diagnosis cannot only help radiologists achieve more accurate and efficient imaging diagnosis but also reduce their workload. In this paper, we aim to present a comprehensive overview of the technical knowledge of AI with a focus on traditional machine learning (ML) algorithms and DL algorithms. We will also discuss their clinical applications in the ultrasound imaging of thyroid diseases, particularly in differentiating between benign and malignant nodules and predicting cervical lymph node metastasis in thyroid cancer. Finally, we will conclude that AI technology holds great promise for improving the accuracy of thyroid disease ultrasound diagnosis and discuss the potential prospects of AI in this field.

Automatic detection and diagnosis of thyroid ultrasound images based on attention mechanism

Incidents of thyroid cancer have dramatically increased in recent years; however, early ultrasound diagnosis can reduce morbidity and mortality. The work in clinical situations relies heavily on the subjective experience of the sonographer. Numerous computer-aided diagnostic techniques exist, but most consider how good the results are, ignoring the pre-image collecting and its usefulness in post-clinical practise. To address these issues, this study proposes a computer-aided diagnosis method based on an attentional mechanism. Due to its lightweight properties, the model can rapidly identify nodules and distinguish between benign and malignant ones without using much hardware. The model uses a bounding box to locate the thyroid nodule and determines whether it is benign or cancerous, and outputs the diagnostic result of the thyroid nodule ultrasound images. The latest attention mechanisms are used to get better results at a fraction of the cost. Additionally, ultrasound images with different features of benign and malignant thyroid nodules were collected following the Thyroid Imaging Reporting and Data System standards. The experimental results showed that the approach identifies and classifies thyroid nodules rapidly and effectively; the mAP value of the results reached 0.89, and the mAP value of malignant nodules reached 0.94, with detection rate of single image reached 7 ms. Young physicians and small hospitals with limited resources can benefit from using this method to assist with thyroid ultrasound examination diagnosis.

Endocrine Tumor Classification via Machine-Learning-Based Elastography: A Systematic Scoping Review

Elastography complements traditional medical imaging modalities by mapping tissue stiffness to identify tumors in the endocrine system, and machine learning models can further improve diagnostic accuracy and reliability. Our objective in this review was to summarize the applications and performance of machine-learning-based elastography on the classification of endocrine tumors. Two authors independently searched electronic databases, including PubMed, Scopus, Web of Science, IEEEXpress, CINAHL, and EMBASE. Eleven (n = 11) articles were eligible for the review, of which eight (n = 8) focused on thyroid tumors and three (n = 3) considered pancreatic tumors. In all thyroid studies, the researchers used shear-wave ultrasound elastography, whereas the pancreas researchers applied strain elastography with endoscopy. Traditional machine learning approaches or the deep feature extractors were used to extract the predetermined features, followed by classifiers. The applied deep learning approaches included the convolutional neural network (CNN) and multilayer perceptron (MLP). Some researchers considered the mixed or sequential training of B-mode and elastographic ultrasound data or fusing data from different image segmentation techniques in machine learning models. All reviewed methods achieved an accuracy of ≥80%, but only three were ≥90% accurate. The most accurate thyroid classification (94.70%) was achieved by applying sequential training CNN; the most accurate pancreas classification (98.26%) was achieved using a CNN-long short-term memory (LSTM) model integrating elastography with B-mode and Doppler images.

Investigating the diagnostic efficiency of a computer-aided diagnosis system for thyroid nodules in the context of Hashimoto's thyroiditis

Objectives

This study aims to investigate the efficacy of a computer-aided diagnosis (CAD) system in distinguishing between benign and malignant thyroid nodules in the context of Hashimoto's thyroiditis (HT) and to evaluate the role of the CAD system in reducing unnecessary biopsies of benign lesions.

Methods

We included a total of 137 nodules from 137 consecutive patients (mean age, 43.5 ± 11.8 years) who were histopathologically diagnosed with HT. The two-dimensional ultrasound images and videos of all thyroid nodules were analyzed by the CAD system and two radiologists with different experiences according to ACR TI-RADS. The diagnostic cutoff values of ACR TI-RADS were divided into two categories (TR4 and TR5), and then the sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) of the CAD system and the junior and senior radiologists were compared in both cases. Moreover, ACR TI-RADS classification was revised according to the results of the CAD system, and the efficacy of recommended fine-needle aspiration (FNA) was evaluated by comparing the unnecessary biopsy rate and the malignant rate of punctured nodules.

Results

The accuracy, sensitivity, specificity, PPV, and NPV of the CAD system were 0.876, 0.905, 0.830, 0.894, and 0.846, respectively. With TR4 as the cutoff value, the AUCs of the CAD system and the junior and senior radiologists were 0.867, 0.628, and 0.722, respectively, and the CAD system had the highest AUC (P < 0.0001). With TR5 as the cutoff value, the AUCs of the CAD system and the junior and senior radiologists were 0.867, 0.654, and 0.812, respectively, and the CAD system had a higher AUC than the junior radiologist (P < 0.0001) but comparable to the senior radiologist (P = 0.0709). With the assistance of the CAD system, the number of TR4 nodules was decreased by both junior and senior radiologists, the malignant rate of punctured nodules increased by 30% and 22%, and the unnecessary biopsies of benign lesions were both reduced by nearly half.

Conclusions

The CAD system based on deep learning can improve the diagnostic performance of radiologists in identifying benign and malignant thyroid nodules in the context of Hashimoto's thyroiditis and can play a role in FNA recommendations to reduce unnecessary biopsy rates.

Diagnostic performance of artificial intelligence-based computer-aided diagnosis system in longitudinal and transverse ultrasonic views for differentiating thyroid nodules

OBJECTIVE

To evaluate the diagnostic performance of different ultrasound sections of thyroid nodule (TN) using computer-aided diagnosis system based on artificial intelligence (AI-CADS) in predicting thyroid malignancy.

MATERIALS AND METHODS

This is a retrospective study. From January 2019 to July 2019, patients with preoperative thyroid ultrasound data and postoperative pathological results were enrolled, which were divided into two groups: lower risk group (ACR TI-RADS 1, 2 and 3) and higher risk group (ACR TI-RADS 4 and 5). The malignant risk scores (MRS) of TNs were obtained from longitudinal and transverse sections using AI-CADS. The diagnostic performance of AI-CADS and the consistency of each US characteristic were evaluated between these sections. The receiver operating characteristic (ROC) curve and the Cohen κ-statistic were performed.

RESULTS

A total of 203 patients (45.61 ± 11.59 years, 163 female) with 221 TNs were enrolled. The area under the ROC curve (AUC) of criterion 3 [0.86 (95%CI: 0.80~0.91)] was lower than criterion 1 [0.94 (95%CI: 0.90~ 0.99)], 2 [0.93 (95%CI: 0.89~0.97)] and 4 [0.94 (95%CI: 0.90, 0.99)] significantly (P<0.001, P=0.01, P<0.001, respectively). In the higher risk group, the MRS of transverse section was higher than longitudinal section (P<0.001), and the agreement of extrathyroidal extension and shape was moderate and fair (κ =0.48, 0.31 respectively). The diagnostic agreement of other ultrasonic features was substantial or almost perfect (κ >0.60).

CONCLUSION

The diagnostic performance of computer-aided diagnosis system based on artificial intelligence (AI-CADS) in longitudinal and transverse ultrasonic views for differentiating thyroid nodules (TN) was different, which was higher in the transverse section. It was more dependent on the section for the AI-CADS diagnosis of suspected malignant TNs.

Specification of Neck Muscle Dysfunction through Digital Image Analysis Using Machine Learning

Everyone has or will have experienced some degree of neck pain. Typically, neck pain is associated with the sensation of tense, tight, or stiff neck muscles. However, it is unclear whether the neck muscles are objectively stiffer with neck pain. This study used 1099 ultrasound elastography images (elastograms) obtained from 38 adult women, 20 with chronic neck pain and 18 asymptomatic. For training machine learning algorithms, 28 numerical characteristics were extracted from both the original and transformed shear wave velocity color-coded images as well as from respective image segments. Overall, a total number of 323 distinct features were generated from the data. A supervised binary classification was performed, using six machine-learning algorithms. The random forest algorithm produced the most accurate model to distinguish the elastograms of women with chronic neck pain from asymptomatic women with an AUC of 0.898. When evaluating features that can be used as biomarkers for muscle dysfunction in neck pain, the region of the deepest neck muscles (M. multifidus) provided the most features to support the correct classification of elastograms. By constructing summary images and associated Hotelling's T² maps, we enabled the visualization of group differences and their statistical confirmation.

Incorporation of a machine learning pathological diagnosis algorithm into the thyroid ultrasound imaging data improves the diagnosis risk of malignant thyroid nodules

Objective

This study aimed at establishing a new model to predict malignant thyroid nodules using machine learning algorithms.

Methods

A retrospective study was performed on 274 patients with thyroid nodules who underwent fine-needle aspiration (FNA) cytology or surgery from October 2018 to 2020 in Xianyang Central Hospital. The least absolute shrinkage and selection operator (lasso) regression analysis and logistic analysis were applied to screen and identified variables. Six machine learning algorithms, including Decision Tree (DT), Extreme Gradient Boosting (XGBoost), Gradient Boosting Machine (GBM), Naive Bayes Classifier (NBC), Random Forest (RF), and Logistic Regression (LR), were employed and compared in constructing the predictive model, coupled with preoperative clinical characteristics and ultrasound features. Internal validation was performed by using 10-fold cross-validation. The performance of the model was measured by the area under the receiver operating characteristic curve (AUC), accuracy, precision, recall, F1 score, Shapley additive explanations (SHAP) plot, feature importance, and correlation of features. The best cutoff value for risk stratification was identified by probability density function (PDF) and clinical utility curve (CUC).

Results

The malignant rate of thyroid nodules in the study cohort was 53.2%. The predictive models are constructed by age, margin, shape, echogenic foci, echogenicity, and lymph nodes. The XGBoost model was significantly superior to any one of the machine learning models, with an AUC value of 0.829. According to the PDF and CUC, we recommended that 51% probability be used as a threshold for determining the risk stratification of malignant nodules, where about 85.6% of patients with malignant nodules could be detected. Meanwhile, approximately 89.8% of unnecessary biopsy procedures would be saved. Finally, an online web risk calculator has been built to estimate the personal likelihood of malignant thyroid nodules based on the best-performing ML-ed model of XGBoost.

Conclusions

Combining clinical characteristics and features of ultrasound images, ML algorithms can achieve reliable prediction of malignant thyroid nodules. The online web risk calculator based on the XGBoost model can easily identify in real-time the probability of malignant thyroid nodules, which can assist clinicians to formulate individualized management strategies for patients.

Self-supervised multi-modal fusion network for multi-modal thyroid ultrasound image diagnosis

Ultrasound is a typical non-invasive diagnostic method often used to detect thyroid cancer lesions. However, due to the limitations of the information provided by ultrasound images, shear wave elastography (SWE) and color doppler ultrasound (CDUS) are also used clinically to assist in diagnosis, which makes the diagnosis time-consuming, labor-intensive, and highly subjective process. Therefore, automatic diagnosis of benign and malignant thyroid nodules is beneficial for the clinical diagnosis of the thyroid. To this end, based on three modalities of gray-scale ultrasound images(US), SWE, and CDUS, we propose a deep learning-based multi-modal feature fusion network for the automatic diagnosis of thyroid disease based on the ultrasound images. First, three ResNet18s initialized by self-supervised learning are used as branches to extract the image information of each modality, respectively. Then, a multi-modal multi-head attention branch is used to remove the common information of three modalities, and the knowledge of each modal is combined for thyroid diagnosis. At the same time, to better integrate the features between modalities, a multi-modal feature guidance module is also proposed to guide the feature extraction of each branch and reduce the difference between each-modal feature. We verify the multi-modal thyroid ultrasound image diagnosis method on the self-collected dataset, and the results prove that this method could provide fast and accurate assistance for sonographers in diagnosing thyroid nodules.

Medical Data Classification Assisted by Machine Learning Strategy

With the development of science and technology, data plays an increasingly important role in our daily life. Therefore, much attention has been paid to the field of data mining. Data classification is the premise of data mining, and how well the data is classified directly affects the performance of subsequent models. In particular, in the medical field, data classification can help accurately determine the location of patients' lesions and reduce the workload of doctors in the treatment process. However, medical data has the characteristics of high noise, strong correlation, and high data dimension, which brings great challenges to the traditional classification model. Therefore, it is very important to design an advanced model to improve the effect of medical data classification. In this context, this paper first introduces the structure and characteristics of the convolutional neural network (CNN) model and then demonstrates its unique advantages in medical data processing, especially in data classification. Secondly, we design a new kind of medical data classification model based on the CNN model. Finally, the simulation results show that the proposed method achieves higher classification accuracy with faster model convergence speed and the lower training error when compared with conventional machine leaning methods, which has demonstrated the effectiveness of the new method in respect to medical data classification.

Machine Learning-Assisted Diagnostic System for Indeterminate Thyroid Nodules

To develop an ultrasound-based machine learning classifier to diagnose benignity within indeterminate thyroid nodules (ITNs) by fine-needle aspiration, 180 patients with 194 ITNs (Bethesda classes III, IV and V) undergoing surgery over a 5-y study period were analyzed. The data set was randomly divided into training and testing data sets with 155 and 39 ITNs, respectively. All nodules were evaluated by ultrasound using the American College of Radiology Thyroid Imaging Reporting and Data System by manually scoring composition, echogenicity, shape, margin and echogenic foci. Nodule size, participant age and patient sex were recorded. A support vector machine (SVM) model with a cost-sensitive approach was developed using the aforementioned eight parameters with surgical histopathology as the reference standard. Surgical pathology determined 90 (46.4%) ITNs were malignant and 104 (53.6%) were benign. The SVM model classified 14 nodules as benign in the testing data set, of which 13 were correct (sensitivity = 93.8%, specificity = 56.5%). Considering malignancy prevalence by Bethesda group, the negative predictive values of this model for Bethesda III and IV categories were 93.9% and 93. 8%, respectively. The high negative predictive value of the SVM ultrasound-based model suggests a pathway by which surgical excision of Bethesda III and IV ITNs classified as benign may be avoided.

A machine learning model based on ultrasound image features to assess the risk of sentinel lymph node metastasis in breast cancer patients: Applications of scikit-learn and SHAP

Background

This study aimed to determine an optimal machine learning (ML) model for evaluating the preoperative diagnostic value of ultrasound signs of breast cancer lesions for sentinel lymph node (SLN) status.

Method

This study retrospectively analyzed the ultrasound images and postoperative pathological findings of lesions in 952 breast cancer patients. Firstly, the univariate analysis of the relationship between the ultrasonographic features of breast cancer morphological features and SLN metastasis. Then, based on the ultrasound signs of breast cancer lesions, we screened ten ML models: support vector machine (SVM), extreme gradient boosting (XGBoost), random forest (RF), linear discriminant analysis (LDA), logistic regression (LR), naive bayesian model (NB), k-nearest neighbors (KNN), multilayer perceptron (MLP), long short-term memory (LSTM), and convolutional neural network (CNN). The diagnostic performance of the model was evaluated using the area under the receiver operating characteristic (ROC) curve (AUC), Kappa value, accuracy, F1-score, sensitivity, and specificity. Then we constructed a clinical prediction model which was based on the ML algorithm with the best diagnostic performance. Finally, we used SHapley Additive exPlanation (SHAP) to visualize and analyze the diagnostic process of the ML model.

Results

Of 952 patients with breast cancer, 394 (41.4%) had SLN metastasis, and 558 (58.6%) had no metastasis. Univariate analysis found that the shape, orientation, margin, posterior features, calculations, architectural distortion, duct changes and suspicious lymph node of breast cancer lesions in ultrasound signs were associated with SLN metastasis. Among the 10 ML algorithms, XGBoost had the best comprehensive diagnostic performance for SLN metastasis, with Average-AUC of 0.952, Average-Kappa of 0.763, and Average-Accuracy of 0.891. The AUC of the XGBoost model in the validation cohort was 0.916, the accuracy was 0.846, the sensitivity was 0.870, the specificity was 0.862, and the F1-score was 0.826. The diagnostic performance of the XGBoost model was significantly higher than that of experienced radiologists in some cases (P<0.001). Using SHAP to visualize the interpretation of the ML model screen, it was found that the ultrasonic detection of suspicious lymph nodes, microcalcifications in the primary tumor, burrs on the edge of the primary tumor, and distortion of the tissue structure around the lesion contributed greatly to the diagnostic performance of the XGBoost model.

Conclusions

The XGBoost model based on the ultrasound signs of the primary breast tumor and its surrounding tissues and lymph nodes has a high diagnostic performance for predicting SLN metastasis. Visual explanation using SHAP made it an effective tool for guiding clinical courses preoperatively.

The application value of CDFI and SMI combined with serological markers in distinguishing benign and malignant thyroid nodules

PURPOSE

The purpose of this study is to explore the application value of CDFI and SMI combined with serological markers in distinguishing benign and malignant thyroid nodules.

METHOD

A total of 192 patients with thyroid nodules admitted to our hospital from July 2019 to December 2020 were selected as subjects. Color Doppler blood flow imaging (CDFI) and supermicro blood flow imaging (SMI) methods are used to detect the blood flow of patients and the levels of serum thyroglobulin antibody (TgAb), thyroid peroxidase antibody (TPOAb), and thyroid stimulating hormone (TSH). The receiver operating characteristic curve (ROC curve) was used to observe the sensitivity and specificity of serological markers for distinguishing benign and malignant thyroid nodules, and combined with CDFI and SMI to observe the sensitivity and specificity for distinguishing benign and malignant thyroid nodules.

RESULTS

The levels of TgAb, TPOAb and TSH in benign thyroid nodules were lower than those of the malignant group, and the difference was statistically significant (P < 0.01). There was no statistically significant difference between benign and malignant thyroid nodules in the presence or absence of the capsule and the presence or absence of vocal halo (P > 0.05), while the differences in the nodule morphology, boundary, internal echo and internal calcification were statistically significant (P < 0.01).

CONCLUSION

CDFI and SMI combined with serological index detection have higher value in the differential diagnosis of thyroid cancer, which can significantly improve the sensitivity and specificity of differential diagnosis.

Differential diagnosis of thyroid nodules by the Demetics ultrasound-assisted diagnosis system and contrast-enhanced ultrasound combined with thyroid image reporting and data systems

BACKGROUND

More and more new ultrasound techniques with their own characteristics are applied in the differential diagnosis of thyroid nodules. This study amied to assess and compare the diagnostic value of the Demetics ultrasound-assisted diagnosis system and contrast-enhanced ultrasound (CEUS) combined with the Thyroid Image Reporting and Data Systems (TI-RADS) for thyroid nodules.

DESIGN AND PATIENTS

A total of 600 thyroid nodules with pathological findings were retrospectively analysed. Demetics and CEUS were performed for all nodules. The diagnostic efficacy of Demetics and CEUS for nodules of different sizes was evaluated and compared in terms of sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (LR+) and negative likelihood ratio (LR-). The characteristics of nodules diagnosed and misdiagnosed by Demetics were compared to analyse the factors affecting the diagnostic accuracy of Demetics. The necessity of CEUS for nodules that are prone to misdiagnosis in Demetics was assessed.

RESULTS

Both Demetics and CEUS can be used for the differential diagnosis of benign and malignant thyroid nodules of different sizes. The diagnostic agreement between Demetics and CEUS for thyroid nodules of different sizes was moderate, substantial and fair, respectively. The sensitivity and NPV of Demetics were higher than those of CEUS, and the specificity, PPV and LR+ of CEUS were higher than that of Demetics. The LR- of Demetics was lower than those of CEUS. There were significant differences in age, calcification and margin in analysing the factors affecting Demetics. CEUS correctly diagnosed 50 of the 101 nodules misdiagnosed by Demetics.

CONCLUSIONS

Demetics showed high sensitivity in diagnosing thyroid nodules, while CEUS showed high specificity. In clinical practice, CEUS can further improve the diagnostic accuracy for nodules that are easily misdiagnosed by Demetics.

A comparison of artificial intelligence versus radiologists in the diagnosis of thyroid nodules using ultrasonography: a systematic review and meta-analysis

BACKGROUND

Thyroid nodules are common. Ultrasonography (US) is the first investigation for thyroid nodules. Artificial Intelligence (AI) is widely integrated into medical diagnosis to provide additional information. The primary objective of this study was to accumulate the pooled sensitivity and specificity between all available AI and radiologists using thyroid US imaging. The secondary objective was to compare AI's diagnostic performance to that of radiologists.

MATERIALS AND METHODS

A systematic review meta-analysis. PubMed, Scopus, Web of Science, and Cochrane Library data were searched for studies from inception until June 11, 2020.

RESULTS

Twenty five studies were included in this meta-analysis. The pooled sensitivity and specificity of AI were 0.86 (95% CI 0.81-0.91) and 0.78 (95% CI 0.73-0.83), respectively. The pooled sensitivity and specificity of radiologists were 0.85 (95% CI 0.80-0.89) and 0.82 (95% CI 0.77-0.86), respectively. The accuracy of AI and radiologists is equivalent in terms of AUC [AI 0.89 (95% CI 0.86-0.92), radiologist 0.91 (95% CI 0.88-0.93)]. The diagnostic odd ratio (DOR) between AI 23.10 (95% CI 14.20-37.58) and radiologists 27.12 (95% CI 17.45-42.16) had no statistically significant difference (P = 0.56). Meta-regression analysis revealed that Deep Learning AI had significantly greater sensitivity and specificity than classic machine learning AI (P < 0.001).

CONCLUSION

AI demonstrated comparable performance to radiologists in diagnosing benign and malignant thyroid nodules using ultrasonography. Additional research to establish its equivalency should be conducted.

Differential Diagnosis of Thyrotoxicosis by Machine Learning Models with Laboratory Findings

Differential diagnosis of thyrotoxicosis is essential because therapeutic approaches differ based on disease etiology. We aimed to perform differential diagnosis of thyrotoxicosis using machine learning algorithms with initial laboratory findings. This is a retrospective study through medical records. Patients who visited a single hospital for thyrotoxicosis from June 2016 to December 2021 were enrolled. In total, 230 subjects were analyzed: 124 (52.6%) patients had Graves’ disease, 65 (28.3%) suffered from painless thyroiditis, and 41 (17.8%) were diagnosed with subacute thyroiditis. In consideration that results for the thyroid autoantibody test cannot be immediately confirmed, two different models were devised: Model 1 included triiodothyronine (T3), free thyroxine (FT4), T3 to FT4 ratio, erythrocyte sediment rate, and C-reactive protein (CRP); and Model 2 included all Model 1 variables as well as thyroid autoantibody test results, including thyrotropin binding inhibitory immunoglobulin (TBII), thyroid-stimulating immunoglobulin, anti-thyroid peroxidase antibody, and anti-thyroglobulin antibody (TgAb). Differential diagnosis accuracy was calculated using seven machine learning algorithms. In the initial blood test, Graves’ disease was characterized by increased thyroid hormone levels and subacute thyroiditis showing elevated inflammatory markers. The diagnostic accuracy of Model 1 was 65–70%, and Model 2 accuracy was 78–90%. The random forest model had the highest classification accuracy. The significant variables were CRP and T3 in Model 1 and TBII, CRP, and TgAb in Model 2. We suggest monitoring the initial T3 and CRP levels with subsequent confirmation of TBII and TgAb in the differential diagnosis of thyrotoxicosis.

The Future of Thyroid Nodule Risk Stratification

Clinical evidence supports the association of ultrasound features with benign or malignant thyroid nodules and serves as the basis for sonographic stratification of thyroid nodules, according to an estimated thyroid cancer risk. Contemporary guidelines recommend management strategies according to thyroid cancer risk, thyroid nodule size, and the clinical scenario. Yet, reproducible and accurate thyroid nodule risk stratification requires expertise, time, and understanding of the weight different ultrasound features have on thyroid cancer risk. The application of artificial intelligence to overcome these limitations is promising and has the potential to improve the care of patients with thyroid nodules.

Radiomic Detection of Malignancy within Thyroid Nodules Using Ultrasonography-A Systematic Review and Meta-Analysis

Background: Despite investigation, 95% of thyroid nodules are ultimately benign. Radiomics is a field that uses radiological features to inform individualized patient care. We aimed to evaluate the diagnostic utility of radiomics in classifying undetermined thyroid nodules into benign and malignant using ultrasonography (US). Methods: A diagnostic test accuracy systematic review and meta-analysis was performed in accordance with PRISMA guidelines. Sensitivity, specificity, and area under curve (AUC) delineating benign and malignant lesions were recorded. Results: Seventy-five studies including 26,373 patients and 46,175 thyroid nodules met inclusion criteria. Males accounted for 24.6% of patients, while 75.4% of patients were female. Radiomics provided a pooled sensitivity of 0.87 (95% CI: 0.86−0.87) and a pooled specificity of 0.84 (95% CI: 0.84−0.85) for characterizing benign and malignant lesions. Using convolutional neural network (CNN) methods, pooled sensitivity was 0.85 (95% CI: 0.84−0.86) and pooled specificity was 0.82 (95% CI: 0.82−0.83); significantly lower than studies using non-CNN: sensitivity 0.90 (95% CI: 0.89−0.90) and specificity 0.88 (95% CI: 0.87−0.89) (p < 0.05). The diagnostic ability of radiologists and radiomics were comparable for both sensitivity (OR 0.98) and specificity (OR 0.95). Conclusions: Radiomic analysis using US provides a reproducible, reliable evaluation of undetermined thyroid nodules when compared to current best practice.

Contrast-Enhanced CT-Based Radiomics for the Differentiation of Nodular Goiter from Papillary Thyroid Carcinoma in Thyroid Nodules

Background

Papillary thyroid carcinoma (PTC) and nodular goiter (NG) represent the most commonly malignant and benign diseases of thyroid nodules and are often confused in diagnosis. CT examination has a certain diagnostic value for the diagnosis of suspected malignant thyroid nodules. The application of machine learning to radiomics features provides a new diagnostic approach, which has been widely used in ultrasound examination of the thyroid, but there are few literatures on CT examination.

Purpose

To explore the efficacy of a diagnostic model aided by machine learning for preoperative differentiation of nodular goiter and papillary thyroid carcinoma thyroid nodules on the basis of 3D arterial-phase contrast-enhanced computed tomography (CECT) features.

Materials and Methods

We collected the data of 193 NG and 214 PTC thyroid nodules from 407 patients in CT examinations. Together with the pathologist findings and radiology diagnosis, we built a radiomics model using the 1218 features extracted from the arterial phase of CECT images. By comparing the diagnostic performance of the radiomics model with that of the clinical diagnosis, we assessed the performance of the radiomics model.

Results

The radiomics model was developed based on multivariable logistic regression with the optimal 12 radiomics features after feature dimension reduction. The radiomics model performed well on the classification accuracy of the PTC and NG thyroid nodules in the training group and validation group.

Conclusion

The radiomics model based on the 3D arterial phase of CECT features performed better than the group of experienced radiologists in differentiating NG and PTC thyroid nodules.

Diagnosis of Thyroid Nodules Based on Image Enhancement and Deep Neural Networks

The diagnosis of thyroid nodules at an early stage is a challenging task. Manual diagnosis of thyroid nodules is labor-intensive and time-consuming. Meanwhile, due to the difference of instruments and technical personnel, the original thyroid nodule ultrasound images collected are very different. In order to make better use of ultrasound image information of thyroid nodules, some image processing methods are indispensable. In this paper, we developed a method for automatic thyroid nodule classification based on image enhancement and deep neural networks. The selected image enhancement method is histogram equalization, and the neural networks have four-layer network nodes in our experiments. The dataset in this paper consists of thyroid nodule images of 508 patients. The data are divided into 80% training and 20% validation sets. A comparison result demonstrates that our method can achieve a better performance than other normal machine learning methods. The experimental results show that our method has achieved 0.901961 accuracy, 0.894737 precision, 1 recall, and 0.944444 F1-score. At the same time, we also considered the influence of network structure, activation function of network nodes, number of training iterations, and other factors on the classification results. The experimental results show that the optimal network structure is 2500-40-2-1, the optimal activation function is logistic function, and the best number of training iterations is 500.

Machine intelligence in non-invasive endocrine cancer diagnostics

Artificial intelligence (AI) has illuminated a clear path towards an evolving health-care system replete with enhanced precision and computing capabilities. Medical imaging analysis can be strengthened by machine learning as the multidimensional data generated by imaging naturally lends itself to hierarchical classification. In this Review, we describe the role of machine intelligence in image-based endocrine cancer diagnostics. We first provide a brief overview of AI and consider its intuitive incorporation into the clinical workflow. We then discuss how AI can be applied for the characterization of adrenal, pancreatic, pituitary and thyroid masses in order to support clinicians in their diagnostic interpretations. This Review also puts forth a number of key evaluation criteria for machine learning in medicine that physicians can use in their appraisals of these algorithms. We identify mitigation strategies to address ongoing challenges around data availability and model interpretability in the context of endocrine cancer diagnosis. Finally, we delve into frontiers in systems integration for AI, discussing automated pipelines and evolving computing platforms that leverage distributed, decentralized and quantum techniques.

Accuracy of Ultrasound Diagnosis of Thyroid Nodules Based on Artificial Intelligence-Assisted Diagnostic Technology: A Systematic Review and Meta-Analysis

BACKGROUND

Ultrasonography (US) is the most common method of identifying thyroid nodules, but US images require an experienced surgeon for identification. Many artificial intelligence (AI) techniques such as computer-aided diagnostic systems (CAD), deep learning (DL), and machine learning (ML) have been used to assist in the diagnosis of thyroid nodules, but whether AI techniques can improve the diagnostic accuracy of thyroid nodules still needs to be explored.

OBJECTIVE

To clarify the accuracy of AI-based thyroid nodule US images for differentiating benign and malignant thyroid nodules.

METHODS

A search strategy of "subject terms + key words" was used to search PubMed, Cochrane Library, Embase, Web of Science, China Biology Medicine (CBM), and China National Knowledge Infrastructure (CNKI) for studies on AI-assisted diagnosis of thyroid nodules based on US images. The summarized receiver operating characteristic (SROC) curve and the pooled sensitivity and specificity were used to assess the performance of the diagnostic tests. The quality assessment of diagnostics accuracy studies-2 (QUADAS-2) tool was used to assess the methodological quality of the included studies. The Review Manager 5.3 and Stata 15 were used to process the data. Subgroup analysis was based on the integrity of data collection.

RESULTS

A total of 25 studies with 17,429 US images of thyroid nodules were included. AI-assisted diagnostic techniques had better diagnostic efficacy in the diagnosis of benign and malignant thyroid nodules: sensitivity 0.88 (95% CI: (0.85-0.90)), specificity 0.81 (95% CI: 0.74-0.86), diagnostic odds ratio (DOR) 30 (95% CI: 19-46). The SROC curve indicated that the area under the curve (AUC) was 0.92 (95% CI: 0.89-0.94). Threshold effect analysis showed a Spearman correlation coefficient: 0.17 < 0.5, suggesting no threshold effect for the included studies. After a meta-regression analysis of 4 different subgroups, the results showed a statistically significant effect of mean age ≥50 years on heterogeneity. Compared with studies with an average age of ≥50 years, AI-assisted diagnostic techniques had higher diagnostic performance in studies with an average age of <50 years (0.89 (95% CI: 0.87-0.92) vs. 0.80 (95% CI: 0.73-0.88)), (0.83 (95% CI: 0.77-0.88) vs. 0.73 (95% CI: 0.60-0.87)).

CONCLUSIONS

AI-assisted diagnostic techniques had good diagnostic efficacy for thyroid nodules. For the diagnosis of <50 year olds, AI-assisted diagnostic technology was more effective in diagnosis.

Classification of malignant tumors by a non-sequential recurrent ensemble of deep neural network model

Many significant efforts have so far been made to classify malignant tumors by using various machine learning methods. Most of the studies have considered a particular tumor genre categorized according to its originating organ. This has enriched the domain-specific knowledge of malignant tumor prediction, we are devoid of an efficient model that may predict the stages of tumors irrespective of their origin. Thus, there is ample opportunity to study if a heterogeneous collection of tumor images can be classified according to their respective stages. The present research work has prepared a heterogeneous tumor dataset comprising eight different datasets from The Cancer Imaging Archives and classified them according to their respective stages, as suggested by the American Joint Committee on Cancer. The proposed model has been used for classifying 717 subjects comprising different imaging modalities and varied Tumor-Node-Metastasis stages. A new non-sequential deep hybrid model ensemble has been developed by exploiting branched and re-injected layers, followed by bidirectional recurrent layers to classify tumor images. Results have been compared with standard sequential deep learning models and notable recent studies. The training and validation accuracy along with the ROC-AUC scores have been found satisfactory over the existing models. No model or method in the literature could ever classify such a diversified mix of tumor images with such high accuracy. The proposed model may help radiologists by acting as an auxiliary decision support system and speed up the tumor diagnosis process.

Predicting Malignancy in Pediatric Thyroid Nodules: Early Experience With Machine Learning for Clinical Decision Support

OBJECTIVE

To develop a machine learning tool to integrate clinical data for the prediction of non-benign thyroid cytology and histology.

CONTEXT

Papillary thyroid carcinoma is the most common endocrine malignancy. Since most nodules are benign, the challenge for the clinician is to identify those most likely to harbor malignancy while limiting exposure to surgical risks among those with benign nodules.

METHODS

Random forests (augmented to select features based on our clinical measure of interest), in conjunction with interpretable rule sets, were used on demographic, ultrasound, and biopsy data of thyroid nodules from children younger than 18 years at a tertiary pediatric hospital. Accuracy, false-positive rate (FPR), false-negative rate (FNR), and area under the receiver operator curve (AUROC) are reported.

RESULTS

Our models predict nonbenign cytology and malignant histology better than historical outcomes. Specifically, we expect a 68.04% improvement in the FPR, 11.90% increase in accuracy, and 24.85% increase in AUROC for biopsy predictions in 67 patients (28 with benign and 39 with nonbenign histology). We expect a 23.22% decrease in FPR, 32.19% increase in accuracy, and 3.84% decrease in AUROC for surgery prediction in 53 patients (42 with benign and 11 with nonbenign histology). This improvement comes at the expense of the FNR, for which we expect 10.27% with malignancy would be discouraged from performing biopsy, and 11.67% from surgery. Given the small number of patients, these improvements are estimates and are not tested on an independent test set.

CONCLUSION

This work presents a first attempt at developing an interpretable machine learning based clinical tool to aid clinicians. Future work will involve sourcing more data and developing probabilistic estimates for predictions.

A Computer-Aided Diagnosis System and Thyroid Imaging Reporting and Data System for Dual Validation of Ultrasound-Guided Fine-Needle Aspiration of Indeterminate Thyroid Nodules

Purpose

The fully automatic AI-Sonic computer-aided design (CAD) system was employed for the detection and diagnosis of benign and malignant thyroid nodules. The aim of this study was to investigate the efficiency of the AI-Sonic CAD system with the use of a deep learning algorithm to improve the diagnostic accuracy of ultrasound-guided fine-needle aspiration (FNA).

Methods

A total of 138 thyroid nodules were collected from 124 patients and diagnosed by an expert, a novice, and the Thyroid Imaging Reporting and Data System (TI-RADS). Diagnostic efficiency and feasibility were compared among the expert, novice, and CAD system. The application of the CAD system to enhance the diagnostic efficiency of novices was assessed. Moreover, with the experience of the expert as the gold standard, the values of features detected by the CAD system were also analyzed. The efficiency of FNA was compared among the expert, novice, and CAD system to determine whether the CAD system is helpful for the management of FNA.

Result

In total, 56 malignant and 82 benign thyroid nodules were collected from the 124 patients (mean age, 46.4 ± 12.1 years; range, 12–70 years). The diagnostic area under the curve of the CAD system, expert, and novice were 0.919, 0.891, and 0.877, respectively (p < 0.05). In regard to feature detection, there was no significant differences in the margin and composition between the benign and malignant nodules (p > 0.05), while echogenicity and the existence of echogenic foci were of great significance (p < 0.05). For the recommendation of FNA, the results showed that the CAD system had better performance than the expert and novice (p < 0.05).

Conclusions

Precise diagnosis and recommendation of FNA are continuing hot topics for thyroid nodules. The CAD system based on deep learning had better accuracy and feasibility for the diagnosis of thyroid nodules, and was useful to avoid unnecessary FNA. The CAD system is potentially an effective auxiliary approach for diagnosis and asymptomatic screening, especially in developing areas.

Artificial Intelligence in Thyroid Field-A Comprehensive Review

Simple Summary

The incidence of thyroid pathologies has been increasing worldwide. Historically, the detection of thyroid neoplasms relies on medical imaging analysis, depending mainly on the experience of clinicians. The advent of artificial intelligence (AI) techniques led to a remarkable progress in image-recognition tasks. AI represents a powerful tool that may facilitate understanding of thyroid pathologies, but actually, the diagnostic accuracy is uncertain. This article aims to provide an overview of the basic aspects, limitations and open issues of the AI methods applied to thyroid images. Medical experts should be familiar with the workflow of AI techniques in order to avoid misleading outcomes.

Abstract

Artificial intelligence (AI) uses mathematical algorithms to perform tasks that require human cognitive abilities. AI-based methodologies, e.g., machine learning and deep learning, as well as the recently developed research field of radiomics have noticeable potential to transform medical diagnostics. AI-based techniques applied to medical imaging allow to detect biological abnormalities, to diagnostic neoplasms or to predict the response to treatment. Nonetheless, the diagnostic accuracy of these methods is still a matter of debate. In this article, we first illustrate the key concepts and workflow characteristics of machine learning, deep learning and radiomics. We outline considerations regarding data input requirements, differences among these methodologies and their limitations. Subsequently, a concise overview is presented regarding the application of AI methods to the evaluation of thyroid images. We developed a critical discussion concerning limits and open challenges that should be addressed before the translation of AI techniques to the broad clinical use. Clarification of the pitfalls of AI-based techniques results crucial in order to ensure the optimal application for each patient.

False Negative Rates in Benign Thyroid Nodule Diagnosis: Machine Learning for Detecting Malignancy

BACKGROUND

Thyroid nodules are common; up to 67% of adults will show nodules on high-quality ultrasound, and 95% of these nodules are benign. FNA cytology is a crucial step in determining the risk of malignancy, and a false negative diagnosis at this stage delays cancer treatment. The purpose of this study is to develop a predictive model using machine learning which can identify false negative FNA results based on less-invasive clinical data.

MATERIALS AND METHODS

We conducted a retrospective medical record review at one academic and one community center. Inclusion criteria were thyroid nodules evaluated by ultrasound and FNA with a Bethesda II (benign) result or malignancy detected on pathology or FNA. Linear, non-linear, and ensemble models were generated with scikit-learn using 10-fold cross validation with repetition and compared with AUROC. The classification task was the prediction of malignancy using information acquired from less-invasive ultrasound and FNA.

RESULTS

A total of 604 subjects met inclusion criteria; 38 were diagnosed with malignancy. Of all algorithms tested, a Random Forest method achieved the best AUROC (0.64) in separating benign and malignant nodules, though the improvement over other tested algorithms was not statistically significant.

CONCLUSIONS

A Random Forest model performed better than random chance using readily available data obtained via standard evaluation of thyroid nodules. The diagnostic probability threshold of this model can be varied to minimize false positives at the cost of increasing the number of false negatives. Future studies will prospectively evaluate the model's performance.

A Comparative Analysis of Six Machine Learning Models Based on Ultrasound to Distinguish the Possibility of Central Cervical Lymph Node Metastasis in Patients With Papillary Thyroid Carcinoma

Current approaches to predict central cervical lymph node metastasis (CLNM) in patients with papillary thyroid carcinoma (PTC) have failed to identify patients who would benefit from preventive treatment. Machine learning has offered the opportunity to improve accuracy by comparing the different algorithms. We assessed which machine learning algorithm can best improve CLNM prediction. This retrospective study used routine ultrasound data of 1,364 PTC patients. Six machine learning algorithms were compared to predict the possibility of CLNM. Predictive accuracy was assessed by sensitivity, specificity, positive predictive value, negative predictive value, and the area under the curve (AUC). The patients were randomly split into the training (70%), validation (15%), and test (15%) data sets. Random forest (RF) led to the best diagnostic model in the test cohort (AUC 0.731 ± 0.036, 95% confidence interval: 0.664–0.791). The diagnostic performance of the RF algorithm was most dependent on the following five top-rank features: extrathyroidal extension (27.597), age (17.275), T stage (15.058), shape (13.474), and multifocality (12.929). In conclusion, this study demonstrated promise for integrating machine learning methods into clinical decision-making processes, though these would need to be tested prospectively.

Combining real-time elastography with fine-needle aspiration biopsy to identify malignant thyroid nodules

Objectives

To evaluate the diagnostic performance of real-time elastography (RTE) combined with fine-needle aspiration (FNA) biopsy in identifying malignant thyroid nodules.

Methods

This was a single-centre, retrospective study and involved patients who had underogone partial or total thyroidectomy from 01 January 2014 to 31 December 2018 at our centre. Eligible patients were at least18 years of age, had reliable grayscale ultrasound imaging results, a RTE evaluation and had undergone a FNA biopsy.

Results

Data were available from 437 patients. A high RTE score was a significant independent risk factors for malignancy. RTE plus FNA biopsy increased diagnostic accuracy compared with either method alone and the sensitivity and specificity of the combined model were 86% and 78%, respectively.

Conclusions

The combination of RTE imaging with FNA biopsy improves the diagnostic performance in differentiating benign and malignant thyroid nodules.

Application of Machine Learning Algorithms to Predict Central Lymph Node Metastasis in T1-T2, Non-invasive, and Clinically Node Negative Papillary Thyroid Carcinoma

Purpose: While there are no clear indications of whether central lymph node dissection is necessary in patients with T1-T2, non-invasive, clinically uninvolved central neck lymph nodes papillary thyroid carcinoma (PTC), this study seeks to develop and validate models for predicting the risk of central lymph node metastasis (CLNM) in these patients based on machine learning algorithms.

Methods: This is a retrospective study comprising 1,271 patients with T1-T2 stage, non-invasive, and clinically node negative (cN0) PTC who underwent surgery at the Department of Endocrine and Breast Surgery of The First Affiliated Hospital of Chongqing Medical University from February 1, 2016, to December 31, 2018. We applied six machine learning (ML) algorithms, including Logistic Regression (LR), Gradient Boosting Machine (GBM), Extreme Gradient Boosting (XGBoost), Random Forest (RF), Decision Tree (DT), and Neural Network (NNET), coupled with preoperative clinical characteristics and intraoperative information to develop prediction models for CLNM. Among all the samples, 70% were randomly selected to train the models while the remaining 30% were used for validation. Indices like the area under the receiver operating characteristic (AUROC), sensitivity, specificity, and accuracy were calculated to test the models' performance.

Results: The results showed that ~51.3% (652 out of 1,271) of the patients had pN1 disease. In multivariate logistic regression analyses, gender, tumor size and location, multifocality, age, and Delphian lymph node status were all independent predictors of CLNM. In predicting CLNM, six ML algorithms posted AUROC of 0.70–0.75, with the extreme gradient boosting (XGBoost) model standing out, registering 0.75. Thus, we employed the best-performing ML algorithm model and uploaded the results to a self-made online risk calculator to estimate an individual's probability of CLNM (https://jin63.shinyapps.io/ML_CLNM/).

Conclusions: With the incorporation of preoperative and intraoperative risk factors, ML algorithms can achieve acceptable prediction of CLNM with Xgboost model performing the best. Our online risk calculator based on ML algorithm may help determine the optimal extent of initial surgical treatment for patients with T1-T2 stage, non-invasive, and clinically node negative PTC.

A Comparative Analysis of Two Machine Learning-Based Diagnostic Patterns with Thyroid Imaging Reporting and Data System for Thyroid Nodules: Diagnostic Performance and Unnecessary Biopsy Rate

Background: The risk stratification system of the American College of Radiology Thyroid Imaging Reporting and Data System (ACR TI-RADS) for thyroid nodules is affected by low diagnostic specificity. Machine learning (ML) methods can optimize the diagnostic performance in medical image analysis. However, it is unknown which ML-based diagnostic pattern is more effective in improving diagnostic performance for thyroid nodules and reducing nodule biopsies. Therefore, we compared ML-assisted visual approaches and radiomics approaches with ACR TI-RADS in diagnostic performance and unnecessary fine-needle aspiration biopsy (FNAB) rate for thyroid nodules. Methods: This retrospective study evaluated a data set of ultrasound (US) and shear wave elastography (SWE) images in patients with biopsy-proven thyroid nodules (≥1 cm) from the Shanghai Tenth People's Hospital (743 nodules in 720 patients from September 2017 to January 2019) and an independent test data set from the Ma'anshan People's Hospital (106 nodules in 102 patients from February 2019 to April 2019). Six US features and five SWE parameters from the radiologists' interpretation were used for building the ML-assisted visual approaches. The radiomics features extracted from the US and SWE images were used with ML methods for developing the radiomics approaches. The diagnostic performance for differentiating thyroid nodules and the unnecessary FNAB rate of the ML-assisted visual approaches and the radiomics approaches were compared with ACR TI-RADS. Results: The ML-assisted US visual approach had the best diagnostic performance than the US radiomics approach and ACR TI-RADS (area under the curve [AUC]: 0.900 vs. 0.789 vs. 0.689 for the validation data set, 0.917 vs. 0.770 vs. 0.681 for the test data set). After adding SWE, the ML-assisted visual approach had a better diagnostic performance than US alone (AUC: 0.951 vs. 0.900 for the validation data set, 0.953 vs. 0.917 for the test data set). When applying the ML-assisted US+SWE visual approach, the unnecessary FNAB rate decreased from 30.0% to 4.5% in the validation data set and from 37.7% to 4.7% in the test data set in comparison to ACR TI-RADS. Conclusions: The ML-assisted dual modalities visual approach can assist radiologists to diagnose thyroid nodules more effectively and considerably reduce the unnecessary FNAB rate in the clinical management of thyroid nodules.

Diagnosis of thyroid cancer using a TI-RADS-based computer-aided diagnosis system: a multicenter retrospective study

OBJECTIVE

The purpose of this study was to use a computer-aided diagnosis (CAD) system based on the Thyroid Imaging, Reporting, and Data System (TI-RADS) to improve the diagnostic performance of thyroid cancer by analyzing clinical ultrasound imaging data.

METHODS

A retrospective diagnostic study of ultrasound image sets was conducted at five hospitals in China. A CAD system based on TI-RADS was applied in this study, and the diagnostic performance of CAD system was tested through multi-center data. The performance of the CAD system was compared with the consensus of three experienced radiologists. The interobserver agreement for cancer diagnosis was calculated between the CAD system and the consensus of the three experienced radiologists.

RESULTS

The CAD system performed well in the diagnosis of thyroid cancer, with an area under the curve (AUC) value of 0.902 (95% CI: 0.884-0.918), and obtained results similar to those of the three experienced radiologists. The CAD system performed better in the internal test set than in the external test set (AUC: 0.930 vs 0.877, respectively). The performance of the CAD system in the diagnosis of thyroid cancer for nodules of different sizes (<1 cm, 1-2 cm and ≥2 cm) was basically similar (accuracy: 84.6% vs 85% vs 84.2%). The CAD system can recognize 15 ultrasound features of thyroid nodules, most of which reached the level of 3 experienced radiologists (12/15, 85%).

CONCLUSION

The CAD system achieved an improved AUC and similar sensitivity and specificity in the diagnosis of thyroid cancer compared with the consensus of experienced radiologists.