Computer-aided diagnosis for the differentiation of malignant from benign thyroid nodules on ultrasonography

Investigation of optimal convolutional neural network conditions for thyroid ultrasound image analysis

Neural network models have been used to analyze thyroid ultrasound (US) images and stratify malignancy risk of the thyroid nodules. We investigated the optimal neural network condition for thyroid US image analysis. We compared scratch and transfer learning models, performed stress tests in 10% increments, and compared the performance of three threshold values. All validation results indicated superiority of the transfer learning model over the scratch model. Stress test indicated that training the algorithm using 3902 images (70%) resulted in a performance which was similar to the full dataset (5575). Threshold 0.3 yielded high sensitivity (1% false negative) and low specificity (72% false positive), while 0.7 gave low sensitivity (22% false negative) and high specificity (23% false positive). Here we showed that transfer learning was more effective than scratch learning in terms of area under curve, sensitivity, specificity and negative/positive predictive value, that about 3900 images were minimally required to demonstrate an acceptable performance, and that algorithm performance can be customized according to the population characteristics by adjusting threshold value.

A multi-channel deep convolutional neural network for multi-classifying thyroid diseases

BACKGROUND AND OBJECTIVE

Thyroid disease instances have been continuously increasing since the 1990s, and thyroid cancer has become the most rapidly rising disease among all the malignancies in recent years. Most existing studies focused on applying deep convolutional neural networks for detecting thyroid cancer. Despite their satisfactory performance on binary classification tasks, limited studies have explored multi-class classification of thyroid disease types; much less is known of the diagnosis of co-existence situation for different types of thyroid diseases.

METHOD

This study proposed a novel multi-channel convolutional neural network (CNN) architecture to address the multi-class classification task of thyroid disease. The multi-channel CNN merits from computed tomography characteristics to drive a comprehensive diagnostic decision for the overall thyroid gland, emphasizing the disease co-existence circumstance. Moreover, this study also examined alternative strategies to enhance the diagnostic accuracy of CNN models through concatenation of different scales of feature maps.

RESULTS

Benchmarking experiments demonstrate the improved performance of the proposed multi-channel CNN architecture compared with the standard single-channel CNN architecture. More specifically, the multi-channel CNN achieved an accuracy of 0.909±0.048, precision of 0.944±0.062, recall of 0.896±0.047, specificity of 0.994±0.001, and F1 of 0.917±0.057, in contrast to the single-channel CNN, which obtained 0.902±0.004, 0.892±0.005, 0.909±0.002, 0.993±0.001, 0.898±0.003, respectively. In addition, the proposed model was evaluated in different gender groups; it reached a diagnostic accuracy of 0.908 for the female group and 0.901 for the male group.

CONCLUSION

Collectively, the results highlight that the proposed multi-channel CNN has excellent generalization and has the potential to be deployed to provide computational decision support in clinical settings.

The Value of Thyroid Ultrasound Computer-Aided Diagnosis System in the Evaluation of Thyroid Nodules With Concurrent Hashimoto's Thyroiditis

OBJECTIVE

To investigate the value of computer-aided diagnosis (CAD) system in assessing thyroid nodules concurrent with Hashimoto's thyroiditis (HT).

METHODS

Totally 148 patients with 193 thyroid nodules were enrolled. A radiologist assessed the nodules using a thyroid ultrasound CAD system. Additionally, the nodules were evaluated by one experienced radiologist alone, and one training radiologist without and with CAD assistance. The diagnostic performance was compared between the CAD system and the experienced radiologist, and the training radiologist without and with CAD assistance.

RESULTS

The CAD system demonstrated a similar sensitivity to that of the experienced radiologist in diagnosing thyroid cancers (89.8% versus 92.4%, P > .05). The specificity and accuracy of the CAD system were lower than that of the experienced radiologist in assessing the nodules with diffusedly altered glands (specificity, 60.0% versus 81.7%, P = .007; accuracy, 77.5% versus 88.1%, P = .011). With CAD assistance, the training radiologist had improved sensitivity and accuracy that increased to 87.9% and 86.8% in classifying nodules with sonographically evident HT (both P = .012).

CONCLUSION

The CAD system has comparable sensitivity, but lower specificity compared with the experienced radiologist in diagnosing thyroid malignancies concurrent with HT. For a radiologist with less experience, the CAD system can help improve the diagnostic performance by increasing sensitivity and accuracy in assessing thyroid nodules with diffusely altered parenchyma.

Risk stratification of indeterminate thyroid nodules using ultrasound and machine learning algorithms

BACKGROUND

Indeterminate thyroid nodules (Bethesda III) are challenging to characterize without diagnostic surgery. Auxiliary strategies including molecular analysis, machine learning models, and ultrasound grading with Thyroid Imaging, Reporting and Data System (TI-RADS) can help to triage accordingly, but further refinement is needed to prevent unnecessary surgeries and increase positive predictive values.

DESIGN

Retrospective review of 88 patients with Bethesda III nodules who had diagnostic surgery with final pathological diagnosis.

MEASUREMENTS

Each nodule was retrospectively scored through TI-RADS. Two deep learning models were tested, one previously developed and trained on another data set, mainly containing determinate cases and then validated on our data set while the other one trained and tested on our data set (indeterminate cases).

RESULTS

The mean TI-RADS score was 3 for benign and 4 for malignant nodules (p = .0022). Radiological high risk (TI-RADS 4,5) and low risk (TI-RADS 2,3) categories were established. The PPV for the high radiological risk category in those with >10 mm nodules was 85% (CI: 70%-93%). The NPV for low radiological risk in patients >60 years (mean age was 100% (CI: 83%-100%). The area under the curve (AUC) value of our novel classifier was 0.75 (CI: 0.62-0.84) and differed significantly from the chance-level (p < .00001).

CONCLUSIONS

Novel radiomic and radiologic strategies can be employed to assist with preoperative diagnosis of indeterminate thyroid nodules.

Machine intelligence in non-invasive endocrine cancer diagnostics

Artificial intelligence (AI) has illuminated a clear path towards an evolving health-care system replete with enhanced precision and computing capabilities. Medical imaging analysis can be strengthened by machine learning as the multidimensional data generated by imaging naturally lends itself to hierarchical classification. In this Review, we describe the role of machine intelligence in image-based endocrine cancer diagnostics. We first provide a brief overview of AI and consider its intuitive incorporation into the clinical workflow. We then discuss how AI can be applied for the characterization of adrenal, pancreatic, pituitary and thyroid masses in order to support clinicians in their diagnostic interpretations. This Review also puts forth a number of key evaluation criteria for machine learning in medicine that physicians can use in their appraisals of these algorithms. We identify mitigation strategies to address ongoing challenges around data availability and model interpretability in the context of endocrine cancer diagnosis. Finally, we delve into frontiers in systems integration for AI, discussing automated pipelines and evolving computing platforms that leverage distributed, decentralized and quantum techniques.

Accuracy of Ultrasound Diagnosis of Thyroid Nodules Based on Artificial Intelligence-Assisted Diagnostic Technology: A Systematic Review and Meta-Analysis

BACKGROUND

Ultrasonography (US) is the most common method of identifying thyroid nodules, but US images require an experienced surgeon for identification. Many artificial intelligence (AI) techniques such as computer-aided diagnostic systems (CAD), deep learning (DL), and machine learning (ML) have been used to assist in the diagnosis of thyroid nodules, but whether AI techniques can improve the diagnostic accuracy of thyroid nodules still needs to be explored.

OBJECTIVE

To clarify the accuracy of AI-based thyroid nodule US images for differentiating benign and malignant thyroid nodules.

METHODS

A search strategy of "subject terms + key words" was used to search PubMed, Cochrane Library, Embase, Web of Science, China Biology Medicine (CBM), and China National Knowledge Infrastructure (CNKI) for studies on AI-assisted diagnosis of thyroid nodules based on US images. The summarized receiver operating characteristic (SROC) curve and the pooled sensitivity and specificity were used to assess the performance of the diagnostic tests. The quality assessment of diagnostics accuracy studies-2 (QUADAS-2) tool was used to assess the methodological quality of the included studies. The Review Manager 5.3 and Stata 15 were used to process the data. Subgroup analysis was based on the integrity of data collection.

RESULTS

A total of 25 studies with 17,429 US images of thyroid nodules were included. AI-assisted diagnostic techniques had better diagnostic efficacy in the diagnosis of benign and malignant thyroid nodules: sensitivity 0.88 (95% CI: (0.85-0.90)), specificity 0.81 (95% CI: 0.74-0.86), diagnostic odds ratio (DOR) 30 (95% CI: 19-46). The SROC curve indicated that the area under the curve (AUC) was 0.92 (95% CI: 0.89-0.94). Threshold effect analysis showed a Spearman correlation coefficient: 0.17 < 0.5, suggesting no threshold effect for the included studies. After a meta-regression analysis of 4 different subgroups, the results showed a statistically significant effect of mean age ≥50 years on heterogeneity. Compared with studies with an average age of ≥50 years, AI-assisted diagnostic techniques had higher diagnostic performance in studies with an average age of <50 years (0.89 (95% CI: 0.87-0.92) vs. 0.80 (95% CI: 0.73-0.88)), (0.83 (95% CI: 0.77-0.88) vs. 0.73 (95% CI: 0.60-0.87)).

CONCLUSIONS

AI-assisted diagnostic techniques had good diagnostic efficacy for thyroid nodules. For the diagnosis of <50 year olds, AI-assisted diagnostic technology was more effective in diagnosis.

Ultrasound Image Classification of Thyroid Nodules Using Machine Learning Techniques

Background and Objectives: Thyroid nodules are lumps of solid or liquid-filled tumors that form inside the thyroid gland, which can be malignant or benign. Our aim was to test whether the described features of the Thyroid Imaging Reporting and Data System (TI-RADS) could improve radiologists' decision making when integrated into a computer system. In this study, we developed a computer-aided diagnosis system integrated into multiple-instance learning (MIL) that would focus on benign-malignant classification. Data were available from the Universidad Nacional de Colombia. Materials and Methods: There were 99 cases (33 Benign and 66 malignant). In this study, the median filter and image binarization were used for image pre-processing and segmentation. The grey level co-occurrence matrix (GLCM) was used to extract seven ultrasound image features. These data were divided into 87% training and 13% validation sets. We compared the support vector machine (SVM) and artificial neural network (ANN) classification algorithms based on their accuracy score, sensitivity, and specificity. The outcome measure was whether the thyroid nodule was benign or malignant. We also developed a graphic user interface (GUI) to display the image features that would help radiologists with decision making. Results: ANN and SVM achieved an accuracy of 75% and 96% respectively. SVM outperformed all the other models on all performance metrics, achieving higher accuracy, sensitivity, and specificity score. Conclusions: Our study suggests promising results from MIL in thyroid cancer detection. Further testing with external data is required before our classification model can be employed in practice.

Using Deep Neural Network to Diagnose Thyroid Nodules on Ultrasound in Patients With Hashimoto's Thyroiditis

Objective

The aim of this study is to develop a model using Deep Neural Network (DNN) to diagnose thyroid nodules in patients with Hashimoto’s Thyroiditis.

Methods

In this retrospective study, we included 2,932 patients with thyroid nodules who underwent thyroid ultrasonogram in our hospital from January 2017 to August 2019. 80% of them were included as training set and 20% as test set. Nodules suspected for malignancy underwent FNA or surgery for pathological results. Two DNN models were trained to diagnose thyroid nodules, and we chose the one with better performance. The features of nodules as well as parenchyma around nodules will be learned by the model to achieve better performance under diffused parenchyma. 10-fold cross-validation and an independent test set were used to evaluate the performance of the algorithm. The performance of the model was compared with that of the three groups of radiologists with clinical experience of <5 years, 5–10 years, >10 years respectively.

Results

In total, 9,127 images were collected from 2,932 patients with 7,301 images for the training set and 1,806 for the test set. 56% of the patients enrolled had Hashimoto’s Thyroiditis. The model achieved an AUC of 0.924 for distinguishing malignant and benign nodules in the test set. It showed similar performance under diffused thyroid parenchyma and normal parenchyma with sensitivity of 0.881 versus 0.871 (p = 0.938) and specificity of 0.846 versus 0.822 (p = 0.178). In patients with HT, the model achieved an AUC of 0.924 to differentiate malignant and benign nodules which was significantly higher than that of the three groups of radiologists (AUC = 0.824, 0.857, 0.863 respectively, p < 0.05).

Conclusion

The model showed high performance in diagnosing thyroid nodules under both normal and diffused parenchyma. In patients with Hashimoto’s Thyroiditis, the model showed a better performance compared to radiologists with various years of experience.

Artificial Intelligence for Personalized Medicine in Thyroid Cancer: Current Status and Future Perspectives

Thyroid cancers (TC) have increasingly been detected following advances in diagnostic methods. Risk stratification guided by refined information becomes a crucial step toward the goal of personalized medicine. The diagnosis of TC mainly relies on imaging analysis, but visual examination may not reveal much information and not enable comprehensive analysis. Artificial intelligence (AI) is a technology used to extract and quantify key image information by simulating complex human functions. This latent, precise information contributes to stratify TC on the distinct risk and drives tailored management to transit from the surface (population-based) to a point (individual-based). In this review, we started with several challenges regarding personalized care in TC, for example, inconsistent rating ability of ultrasound physicians, uncertainty in cytopathological diagnosis, difficulty in discriminating follicular neoplasms, and inaccurate prognostication. We then analyzed and summarized the advances of AI to extract and analyze morphological, textural, and molecular features to reveal the ground truth of TC. Consequently, their combination with AI technology will make individual medical strategies possible.

Artificial intelligence to predict the BRAFV600E mutation in patients with thyroid cancer

Purpose

To investigate whether a computer-aided diagnosis (CAD) program developed using the deep learning convolutional neural network (CNN) on neck US images can predict the BRAF^V600E mutation in thyroid cancer.

Methods

469 thyroid cancers in 469 patients were included in this retrospective study. A CAD program recently developed using the deep CNN provided risks of malignancy (0–100%) as well as binary results (cancer or not). Using the CAD program, we calculated the risk of malignancy based on a US image of each thyroid nodule (CAD value). Univariate and multivariate logistic regression analyses were performed including patient demographics, the American College of Radiology (ACR) Thyroid Imaging, Reporting and Data System (TIRADS) categories and risks of malignancy calculated through CAD to identify independent predictive factors for the BRAF^V600E mutation in thyroid cancer. The predictive power of the CAD value and final multivariable model for the BRAF^V600E mutation in thyroid cancer were measured using the area under the receiver operating characteristic (ROC) curves.

Results

In this study, 380 (81%) patients were positive and 89 (19%) patients were negative for the BRAF^V600E mutation. On multivariate analysis, older age (OR = 1.025, p = 0.018), smaller size (OR = 0.963, p = 0.006), and higher CAD value (OR = 1.016, p = 0.004) were significantly associated with the BRAF^V600E mutation. The CAD value yielded an AUC of 0.646 (95% CI: 0.576, 0.716) for predicting the BRAF^V600E mutation, while the multivariable model yielded an AUC of 0.706 (95% CI: 0.576, 0.716). The multivariable model showed significantly better performance than the CAD value alone (p = 0.004).

Conclusion

Deep learning-based CAD for thyroid US can help us predict the BRAF^V600E mutation in thyroid cancer. More multi-center studies with more cases are needed to further validate our study results.

Computer-Aided Diagnosis System for the Evaluation of Thyroid Nodules on Ultrasonography: Prospective Non-Inferiority Study according to the Experience Level of Radiologists

Objective

To determine whether a computer-aided diagnosis (CAD) system for the evaluation of thyroid nodules is non-inferior to radiologists with different levels of experience.

Materials and Methods

Patients with thyroid nodules with a decisive diagnosis of benign or malignant nodule were consecutively enrolled from November 2017 to September 2018. Three radiologists with different levels of experience (1 month, 4 years, and 7 years) in thyroid ultrasound (US) reviewed the thyroid US with and without using the CAD system. Statistical analyses included non-inferiority testing of the diagnostic accuracy for malignant thyroid nodules between the CAD system and the three radiologists with a non-inferiority margin of 10%, comparison of the diagnostic performance, and the added value of the CAD system to the radiologists.

Results

Altogether, 197 patients were included in the study cohort. The diagnostic accuracy of the CAD system (88.48%, 95% confidence interval [CI] = 82.65–92.53) was non-inferior to that of the radiologists with less experience (1 month and 4 year) of thyroid US (83.03%, 95% CI = 76.52–88.02; p < 0.001), whereas it was inferior to that of the experienced radiologist (7 years) (95.76%, 95% CI = 91.37–97.96; p = 0.138). The sensitivity and negative predictive value of the CAD system were significantly higher than those of the less-experienced radiologists were, whereas no significant difference was found with those of the experienced radiologist. A combination of US and the CAD system significantly improved sensitivity and negative predictive value, although the specificity and positive predictive value deteriorated for the less-experienced radiologists.

Conclusion

The CAD system may offer support for decision-making in the diagnosis of malignant thyroid nodules for operators who have less experience with thyroid US.

A Clinical Assessment of an Ultrasound Computer-Aided Diagnosis System in Differentiating Thyroid Nodules With Radiologists of Different Diagnostic Experience

Introduction

This study aimed to assess the diagnostic performance and the added value to radiologists of different levels of a computer-aided diagnosis (CAD) system for the detection of thyroid cancers.

Methods

303 patients who underwent thyroidectomy from October 2018 to July 2019 were retrospectively reviewed. The diagnostic performance of the senior radiologist, the junior radiologist, and the CAD system were compared. The added value of the CAD system was assessed and subgroup analyses were performed according to the size of thyroid nodules.

Results

In total, 186 malignant thyroid nodules, and 179 benign thyroid nodules were included; 168 were papillary thyroid carcinoma (PTC), 7 were medullary thyroid carcinoma (MTC), 11 were follicular carcinoma (FTC), 127 were follicular adenoma (FA) and 52 were nodular goiters. The CAD system showed a comparable specificity as the senior radiologist (86.0% vs. 86.0%, p > 0.99), but a lower sensitivity and a lower area under the receiver operating characteristic (AUROC) curve (sensitivity: 71.5% vs. 95.2%, p < 0.001; AUROC: 0.788 vs. 0.906, p < 0.001). The CAD system improved the diagnostic sensitivities of both the senior and the junior radiologists (97.8% vs. 95.2%, p = 0.063; 88.2% vs. 75.3%, p < 0.001).

Conclusion

The use of the CAD system using artificial intelligence is a potential tool to distinguish malignant thyroid nodules and is preferable to serve as a second opinion for less experienced radiologists to improve their diagnosis performance.

Visual Interpretability in Computer-Assisted Diagnosis of Thyroid Nodules Using Ultrasound Images

BACKGROUND The number of studies on deep learning in artificial intelligence (AI)-assisted diagnosis of thyroid nodules is increasing. However, it is difficult to explain what the models actually learn in artificial intelligence-assisted medical research. Our aim is to investigate the visual interpretability of the computer-assisted diagnosis of malignant and benign thyroid nodules using ultrasound images. MATERIAL AND METHODS We designed and implemented 2 experiments to test whether our proposed model learned to interpret the ultrasound features used by ultrasound experts to diagnose thyroid nodules. First, in an anteroposterior/transverse (A/T) ratio experiment, multiple models were trained by changing the A/T ratio of the original nodules, and their classification, accuracy, sensitivity, and specificity were tested. Second, in a visualization experiment, class activation mapping used global average pooling and a fully connected layer to visualize the neural network to show the most important features. We also examined the importance of data preprocessing. RESULTS The A/T ratio experiment showed that after changing the A/T ratio of the nodules, the accuracy of the neural network model was reduced by 9.24-30.45%, indicating that our neural network model learned the A/T ratio information of the nodules. The visual experiment results showed that the nodule margins had a strong influence on the prediction of the neural network. CONCLUSIONS This study was an active exploration of interpretability in the deep learning classification of thyroid nodules. It demonstrated the neural network-visualized model focused on irregular nodule margins and the A/T ratio to classify thyroid nodules.

Computer-Aided Diagnosis Systems in Diagnosing Malignant Thyroid Nodules on Ultrasonography: A Systematic Review and Meta-Analysis

BACKGROUND

Computer-aided diagnosis (CAD) systems are being applied to the ultrasonographic diagnosis of malignant thyroid nodules, but it remains controversial whether the systems add any accuracy for radiologists.

OBJECTIVE

To determine the accuracy of CAD systems in diagnosing malignant thyroid nodules.

METHODS

PubMed, EMBASE, and the Cochrane Library were searched for studies on the diagnostic performance of CAD systems. The diagnostic performance was assessed by pooled sensitivity and specificity, and their accuracy was compared with that of radiologists. The present systematic review was registered in PROSPERO (CRD42019134460).

RESULTS

Nineteen studies with 4,781 thyroid nodules were included. Both the classic machine learning- and the deep learning-based CAD system had good performance in diagnosing malignant thyroid nodules (classic machine learning: sensitivity 0.86 [95% CI 0.79-0.92], specificity 0.85 [95% CI 0.77-0.91], diagnostic odds ratio (DOR) 37.41 [95% CI 24.91-56.20]; deep learning: sensitivity 0.89 [95% CI 0.81-0.93], specificity 0.84 [95% CI 0.75-0.90], DOR 40.87 [95% CI 18.13-92.13]). The diagnostic performance of the deep learning-based CAD system was comparable to that of the radiologists (sensitivity 0.87 [95% CI 0.78-0.93] vs. 0.87 [95% CI 0.85-0.89], specificity 0.85 [95% CI 0.76-0.91] vs. 0.87 [95% CI 0.81-0.91], DOR 40.12 [95% CI 15.58-103.33] vs. DOR 44.88 [95% CI 30.71-65.57]).

CONCLUSIONS

The CAD systems demonstrated good performance in diagnosing malignant thyroid nodules. However, experienced radiologists may still have an advantage over CAD systems during real-time diagnosis.

Ensemble Deep Learning Model for Multicenter Classification of Thyroid Nodules on Ultrasound Images

Background

Thyroid nodules are extremely common and typically diagnosed with ultrasound whether benign or malignant. Imaging diagnosis assisted by Artificial Intelligence has attracted much attention in recent years. The aim of our study was to build an ensemble deep learning classification model to accurately differentiate benign and malignant thyroid nodules.

Material/Methods

Based on current advanced methods of image segmentation and classification algorithms, we proposed an ensemble deep learning classification model for thyroid nodules (EDLC-TN) after precise localization. We compared diagnostic performance with four other state-of-the-art deep learning algorithms and three ultrasound radiologists according to ACR TI-RADS criteria. Finally, we demonstrated the general applicability of EDLC-TN for diagnosing thyroid cancer using ultrasound images from multi medical centers.

Results

The method proposed in this paper has been trained and tested on a thyroid ultrasound image dataset containing 26 541 images and the accuracy of this method could reach 98.51%. EDLC-TN demonstrated the highest value for area under the curve, sensitivity, specificity, and accuracy among five state-of-the-art algorithms. Combining EDLC-TN with models and radiologists could improve diagnostic accuracy. EDLC-TN achieved excellent diagnostic performance when applied to ultrasound images from another independent hospital.

Conclusions

Based on ensemble deep learning, the proposed approach in this paper is superior to other similar existing methods of thyroid classification, as well as ultrasound radiologists. Moreover, our network represents a generalized platform that potentially can be applied to medical images from multiple medical centers.

A Machine Learning-based Diagnosis of Thyroid Cancer Using Thyroid Nodules Ultrasound Images

Background::

Ultrasound test is one of the routine tests for the diagnosis of thyroid cancer. The diagnosis accuracy depends largely on the correct interpretation of ultrasound images of thyroid nodules. However, human eye-based image recognition is usually subjective and sometimes error-prone especially for less experienced doctors, which presents a need for computeraided diagnostic systems.

Objective: :

To our best knowledge, there is no well-maintained ultrasound image database for the Chinese population. In addition, though there are several computational methods for image-based thyroid cancer detection, a comparison among them is missing. Finally, the effects of features like the choice of distance measures have not been assessed. The study aims to give the improvement of these limitations and proposes a highly accurate image-based thyroid cancer diagnosis system, which can better assist doctors in the diagnosis of thyroid cancer.

Methods::

We first establish a novel thyroid nodule ultrasound image database consisting of 508 images collected from the Third Hospital of Hebei Medical University in China. The clinical information for the patients is also collected from the hospital, where 415 patients are diagnosed to be benign and 93 are malignant by doctors following a standard diagnosis procedure. We develop and apply five machine learning methods to the dataset including deep neural network, support vector machine, the center clustering method, k-nearest neighbor, and logistic regression.

Results::

Experimental results show that deep neural network outperforms other diagnosis methods with an average cross-validation accuracy of 0.87 in 10 runs. Meanwhile, we also explore the performance of four image distance measures including the Euclidean distance, the Manhattan distance, the Chebyshev distance, and the Minkowski distance, among which the Chebyshev distance is the best. The resource can be directly used to aid doctors in thyroid cancer diagnosis and treatment.

Conclusions: :

The paper establishes a novel thyroid nodule ultrasound image database and develops a high accurate image-based thyroid cancer diagnosis system which can better assist doctors in the diagnosis of thyroid cancer.

False-Positive Malignant Diagnosis of Nodule Mimicking Lesions by Computer-Aided Thyroid Nodule Analysis in Clinical Ultrasonography Practice

This study aims to test computer-aided diagnosis (CAD) for thyroid nodules in clinical ultrasonography (US) practice with a focus towards identifying thyroid entities associated with CAD system misdiagnoses. Two-hundred patients referred to thyroid US were prospectively enrolled. An experienced radiologist evaluated the thyroid nodules and saved axial images for further offline blinded analysis using a commercially available CAD system. To represent clinical practice, not only true nodules, but mimicking lesions were also included. Fine needle aspiration biopsy (FNAB) was performed according to present guidelines. US features and thyroid entities significantly associated with CAD system misdiagnosis were identified along with the diagnostic accuracy of the radiologist and the CAD system. Diagnostic specificity regarding the radiologist was significantly (p < 0.05) higher than when compared with the CAD system (88.1% vs. 40.5%) while no significant difference was found in the sensitivity (88.6% vs. 80%). Focal inhomogeneities and true nodules in thyroiditis, nodules with coarse calcification and inspissated colloid cystic nodules were significantly (p < 0.05) associated with CAD system misdiagnosis as false-positives. The commercially available CAD system is promising when used to exclude thyroid malignancies, however, it currently may not be able to reduce unnecessary FNABs, mainly due to the false-positive diagnoses of nodule mimicking lesions.

Application of machine learning to ultrasound images to differentiate follicular neoplasms of the thyroid gland

Purpose

This study was conducted to evaluate the diagnostic performance of machine learning in differentiating follicular adenoma from carcinoma using preoperative ultrasonography (US).

Methods

In this retrospective study, preoperative US images of 348 nodules from 340 patients were collected from two tertiary referral hospitals. Two experienced radiologists independently reviewed each image and categorized the nodules according to the 2015 American Thyroid Association guideline. Categorization of a nodule as highly suspicious was considered a positive diagnosis for malignancy. The nodules were manually segmented, and 96 radiomic features were extracted from each region of interest. Ten significant features were selected and used as final input variables in our in-house developed classifier models based on an artificial neural network (ANN) and support vector machine (SVM). The diagnostic performance of radiologists and both classifier models was calculated and compared.

Results

In total, 252 nodules from 245 patients were confirmed as follicular adenoma and 96 nodules from 95 patients were diagnosed as follicular carcinoma. As measures of diagnostic performance, the average sensitivity, specificity, and accuracy of the two experienced radiologists in discriminating follicular adenoma from carcinoma on preoperative US images were 24.0%, 84.0%, and 64.8%, respectively. The sensitivity, specificity, and accuracy of the ANN and SVM-based models were 32.3%, 90.1%, and 74.1% and 41.7%, 79.4%, and 69.0%, respectively. The kappa value of the two radiologists was 0.076, corresponding to slight agreement.

Conclusion

Machine learning-based classifier models may aid in discriminating follicular adenoma from carcinoma using preoperative US.

Differentiation of thyroid nodules on US using features learned and extracted from various convolutional neural networks

Thyroid nodules are a common clinical problem. Ultrasonography (US) is the main tool used to sensitively diagnose thyroid cancer. Although US is non-invasive and can accurately differentiate benign and malignant thyroid nodules, it is subjective and its results inevitably lack reproducibility. Therefore, to provide objective and reliable information for US assessment, we developed a CADx system that utilizes convolutional neural networks and the machine learning technique. The diagnostic performances of 6 radiologists and 3 representative results obtained from the proposed CADx system were compared and analyzed.

The Diagnostic Efficiency of Ultrasound Computer-Aided Diagnosis in Differentiating Thyroid Nodules: A Systematic Review and Narrative Synthesis

Computer-aided diagnosis (CAD) techniques have emerged to complement qualitative assessment in the diagnosis of benign and malignant thyroid nodules. The aim of this review was to summarize the current evidence on the diagnostic performance of various ultrasound CAD in characterizing thyroid nodules. PUBMED, EMBASE and Cochrane databases were searched for studies published until August 2019. The Quality Assessment of Studies of Diagnostic Accuracy included in Systematic Review 2 (QUADAS-2) tool was used to assess the methodological quality of the studies. Reported diagnostic performance data were analyzed and discussed. Fourteen studies with 2232 patients and 2675 thyroid nodules met the inclusion criteria. The study quality based on QUADAS-2 assessment was moderate. At best performance, grey scale CAD had a sensitivity of 96.7% while Doppler CAD was 90%. Combined techniques of qualitative grey scale features and Doppler CAD assessment resulted in overall increased sensitivity (92%) and optimal specificity (85.1%). The experience of the CAD user, nodule size and the thyroid malignancy risk stratification system used for interpretation were the main potential factors affecting diagnostic performance outcomes. The diagnostic performance of CAD of thyroid ultrasound is comparable to that of qualitative visual assessment; however, combined techniques have the potential for better optimized diagnostic accuracy.

Prediction of suspicious thyroid nodule using artificial neural network based on radiofrequency ultrasound and conventional ultrasound: A preliminary study

This study explored the use of backscattered radiofrequency ultrasound signals combined with artificial neural network (ANN) technology to differentiate benign and malignant thyroid nodules, in comparison with conventional ultrasound techniques. The proposed method uses the gray level co-occurrence matrix algorithm and principal component analysis to identify principal characteristics for use as inputs in the ANN. The dataset consisted of 131 ultrasound images, of which 59 were benign and 72 were malignant, as determined by subsequent surgeries. The nodules were divided randomly into training, validation, and testing groups. Receiver operating characteristic curves (ROC) were drawn to compare the diagnostic efficiency of the ANN when applied to radiofrequency and conventional ultrasound images. The sensitivity, specificity, and accuracy of the ANN in predicting malignancy from the radiofrequency ultrasound images were 100, 91.5, and 96.2%, respectively; from conventional ultrasound, the corresponding values were 94.4, 93.2, and 93.9%, respectively. The area under the receiver operating characteristic curve (AUC) was also higher for radiofrequency than conventional ultrasound (AUC = 0.945 vs. 0.917, 95% confidence interval = 0.901-0.998 vs. 0.854-0.979, using a P-value of 0.26). We then classified each nodule into new risk categories according to the output of each sample generated by the proposed method. The malignancy risks in the proposed Categories 3, 4, and 5 were 0, 18.8, and 94.5%, respectively, compared with 0, 55.1, and 88.2% using the American College of Radiology's Thyroid Imaging Reporting and Data System. Thus, this preliminary study initially indicated that the proposed method of using radiofrequency ultrasound and the ANN was more accurate at predicting malignancy and stratifying thyroid nodules than conventional ultrasound methods, thus offering significant potential to reduce the number of unnecessary thyroid biopsies.

Effectiveness evaluation of computer-aided diagnosis system for the diagnosis of thyroid nodules on ultrasound: A systematic review and meta-analysis

BACKGROUND

More and more automated efficient ultrasound image analysis techniques, such as ultrasound-based computer-aided diagnosis system (CAD), were developed to obtain accurate, reproducible, and more objective diagnosis results for thyroid nodules. So far, whether the diagnostic performance of existing CAD systems can reach the diagnostic level of experienced radiologists is still controversial. The aim of the meta-analysis was to evaluate the accuracy of CAD for thyroid nodules' diagnosis by reviewing current literatures and summarizing the research status.

METHODS

A detailed literature search on PubMed, Embase, and Cochrane Libraries for articles published until December 2018 was carried out. The diagnostic performances of CAD systems vs radiologist were evaluated by meta-analysis. We determined the sensitivity and the specificity across studies, calculated positive and negative likelihood ratios and constructed summary receiver-operating characteristic (SROC) curves. Meta-analysis of studies was performed using a mixed-effect, hierarchical logistic regression model.

RESULTS

Five studies with 536 patients and 723 thyroid nodules were included in this meta-analysis. The pooled sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and diagnostic odds ratio (DOR) for CAD system were 0.87 (95% confidence interval [CI], 0.73-0.94), 0.79 (95% CI 0.63-0.89), 4.1 (95% CI 2.5-6.9), 0.17 (95% CI 0.09-0.32), and 25 (95% CI 15-42), respectively. The SROC curve indicated that the area under the curve was 0.90 (95% CI 0.87-0.92). The pooled sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and DOR for experienced radiologists were 0.82 (95% CI 0.69-0.91), 0.83 (95% CI 0.76-0.89), 4.9 (95% CI 3.4-7.0), 0.22 (95% CI 0.12-0.38), and 23 (95% CI 11-46), respectively. The SROC curve indicated that the area under the curve was 0.96 (95% CI 0.94-0.97).

CONCLUSION

The sensitivity of the CAD system in the diagnosis of thyroid nodules was similar to that of experienced radiologists. However, the CAD system had lower specificity and DOR than experienced radiologists. The CAD system may play the potential role as a decision-making assistant alongside radiologists in the thyroid nodules' diagnosis. Future technical improvements would be helpful to increase the accuracy as well as diagnostic efficiency.

Automatic thyroid nodule recognition and diagnosis in ultrasound imaging with the YOLOv2 neural network

BACKGROUND

In this study, images of 2450 benign thyroid nodules and 2557 malignant thyroid nodules were collected and labeled, and an automatic image recognition and diagnosis system was established by deep learning using the YOLOv2 neural network. The performance of the system in the diagnosis of thyroid nodules was evaluated, and the application value of artificial intelligence in clinical practice was investigated.

METHODS

The ultrasound images of 276 patients were retrospectively selected. The diagnoses of the radiologists were determined according to the Thyroid Imaging Reporting and Data System; the images were automatically recognized and diagnosed by the established artificial intelligence system. Pathological diagnosis was the gold standard for the final diagnosis. The performances of the established system and the radiologists in diagnosing the benign and malignant thyroid nodules were compared.

RESULTS

The artificial intelligence diagnosis system correctly identified the lesion area, with an area under the receiver operating characteristic (ROC) curve of 0.902, which is higher than that of the radiologists (0.859). This finding indicates a higher diagnostic accuracy (p = 0.0434). The sensitivity, positive predictive value, negative predictive value, and accuracy of the artificial intelligence diagnosis system for the diagnosis of malignant thyroid nodules were 90.5%, 95.22%, 80.99%, and 90.31%, respectively, and the performance did not significantly differ from that of the radiologists (p > 0.05). The artificial intelligence diagnosis system had a higher specificity (89.91% vs 77.98%, p = 0.026).

CONCLUSIONS

Compared with the performance of experienced radiologists, the artificial intelligence system has comparable sensitivity and accuracy for the diagnosis of malignant thyroid nodules and better diagnostic ability for benign thyroid nodules. As an auxiliary tool, this artificial intelligence diagnosis system can provide radiologists with sufficient assistance in the diagnosis of benign and malignant thyroid nodules.

Computer-Aided Diagnosis of Thyroid Nodules via Ultrasonography: Initial Clinical Experience

Objective

To prospectively evaluate the diagnostic performance of computer-aided diagnosis (CAD) for detection of thyroid cancers via ultrasonography (US).

Materials and Methods

This study included 50 consecutive patients with 117 thyroid nodules on US during the period between June 2016 and July 2016. A radiologist performed US examinations using real-time CAD integrated into a US scanner. We compared the diagnostic performance of radiologist, the CAD system, and the CAD-assisted radiologist for the detection of thyroid cancers.

Results

The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy of the CAD system were 80.0, 88.1, 83.3, 85.5, and 84.6%, respectively, and were not significantly different from those of the radiologist (p > 0.05). The CAD-assisted radiologist showed improved diagnostic sensitivity compared with the radiologist alone (92.0% vs. 84.0%, p = 0.037), while the specificity and PPV were reduced (85.1% vs. 95.5%, p = 0.005 and 82.1% vs. 93.3%, p = 0.008). The radiologist assisted by the CAD system exhibited better diagnostic sensitivity and NPV than the CAD system alone (92.0% vs. 80.0%, p = 0.009 and 93.4% vs. 88.9%, p = 0.013), while the specificities and PPVs were not significantly different (88.1% vs. 85.1%, p = 0.151 and 83.3% vs. 82.1%, p = 0.613, respectively).

Conclusion

The CAD system may be an adjunct to radiological intervention in the diagnosis of thyroid cancer.

Thyroid Nodule Classification in Ultrasound Images by Fine-Tuning Deep Convolutional Neural Network

With many thyroid nodules being incidentally detected, it is important to identify as many malignant nodules as possible while excluding those that are highly likely to be benign from fine needle aspiration (FNA) biopsies or surgeries. This paper presents a computer-aided diagnosis (CAD) system for classifying thyroid nodules in ultrasound images. We use deep learning approach to extract features from thyroid ultrasound images. Ultrasound images are pre-processed to calibrate their scale and remove the artifacts. A pre-trained GoogLeNet model is then fine-tuned using the pre-processed image samples which leads to superior feature extraction. The extracted features of the thyroid ultrasound images are sent to a Cost-sensitive Random Forest classifier to classify the images into “malignant” and “benign” cases. The experimental results show the proposed fine-tuned GoogLeNet model achieves excellent classification performance, attaining 98.29% classification accuracy, 99.10% sensitivity and 93.90% specificity for the images in an open access database (Pedraza et al. 16), while 96.34% classification accuracy, 86% sensitivity and 99% specificity for the images in our local health region database.

Investigation of thyroid nodules: A practical algorithm and review of guidelines

BACKGROUND

High resolution ultrasound has led to early detection of subclinical tumors and drastic increase in incidence of thyroid malignancy. To achieve a balance in appropriate investigation without perpetuating an overdiagnosis phenomenon, a concise set of evidence-based recommendations to stratify risk is required.

METHODS

We sought to assemble an evidence-based diagnostic algorithm and accompanying pictorial review for workup of thyroid nodules that summarizes the most recent guidelines. In addition, we conducted a literature search and analysis of our imaging databases.

RESULTS

Although many imaging features of benign and malignant nodules can be nonspecific, others, such as microcalcifications, lymphadenopathy, and peripheral invasion, are highly suggestive of malignancy. The predictive values of salient imaging characteristics are presented.

CONCLUSION

Evidence-based guidelines are available such that a cost-effective algorithm for thyroid nodule workup can be devised. Conservative management with a focus on periodic monitoring is the working clinical consensus on the approach to thyroid nodules.

Differentiation of the Follicular Neoplasm on the Gray-Scale US by Image Selection Subsampling along with the Marginal Outline Using Convolutional Neural Network

We conducted differentiations between thyroid follicular adenoma and carcinoma for 8-bit bitmap ultrasonography (US) images utilizing a deep-learning approach. For the data sets, we gathered small-boxed selected images adjacent to the marginal outline of nodules and applied a convolutional neural network (CNN) to have differentiation, based on a statistical aggregation, that is, a decision by majority. From the implementation of the method, introducing a newly devised, scalable, parameterized normalization treatment, we observed meaningful aspects in various experiments, collecting evidence regarding the existence of features retained on the margin of thyroid nodules, such as 89.51% of the overall differentiation accuracy for the test data, with 93.19% of accuracy for benign adenoma and 71.05% for carcinoma, from 230 benign adenoma and 77 carcinoma US images, where we used only 39 benign adenomas and 39 carcinomas to train the CNN model, and, with these extremely small training data sets and their model, we tested 191 benign adenomas and 38 carcinomas. We present numerical results including area under receiver operating characteristic (AUROC).

Computer-aided diagnosis of malignant or benign thyroid nodes based on ultrasound images

The objective of this study is to evaluate the diagnostic value of combination of artificial neural networks (ANN) and support vector machine (SVM)-based CAD systems in differentiating malignant from benign thyroid nodes with gray-scale ultrasound images. Two morphological and 65 texture features extracted from regions of interest in 610 2D-ultrasound thyroid node images from 543 patients (207 malignant, 403 benign) were used to develop the ANN and SVM models. Tenfold cross validation evaluated their performance; the best models showed accuracy of 99% for ANN and 100% for SVM. From 50 thyroid node ultrasound images from 45 prospectively enrolled patients, the ANN model showed sensitivity, specificity, positive and negative predictive values, Youden index, and accuracy of 88.24, 90.91, 83.33, 93.75, 79.14, and 90.00%, respectively, the SVM model 76.47, 90.91, 81.25, 88.24, 67.38, and 86.00%, respectively, and in combination 100.00, 87.88, 80.95, 100.00, 87.88, and 92.00%, respectively. Both ANN and SVM had high value in classifying thyroid nodes. In combination, the sensitivity increased but specificity decreased. This combination might provide a second opinion for radiologists dealing with difficult to diagnose thyroid node ultrasound images.

GAUGING THE EXTENT OF THYROIDECTOMY FOR INDETERMINATE THYROID NODULES: AN ONCOLOGIC PERSPECTIVE

OBJECTIVE

Increasing emphasis is being placed on appropriateness of care and avoidance of over- and under-treatment. Indeterminate thyroid nodules (ITNs) present a particular risk for this problem because cancer found via diagnostic lobectomy (DL) often requires a completion thyroidectomy (CT). However, initial total thyroidectomy (TT) for benign ITN results in lifelong thyroid hormone replacement. We sought to measure the accuracy and factors associated with the extent of initial thyroidectomy for ITN.

METHODS

We queried a single institution thyroid surgery database for all adult patients undergoing an initial operation for ITN. Multivariate logistic regression identified factors associated with either oncologic under- or overtreatment at initial operation.

RESULTS

There were 639 patients with ITN. The median age was 52 (range, 18 to 93) years, 78.4% were female, and final pathology revealed a cancer >1 cm in 24.7%. The most common cytology was follicular neoplasm (45.1%) followed by Hürthle cell neoplasm (20.2%). CT or initial oncologic undertreatment was required in 58 patients (9.3%). Excluding those with goiters, 19.0% were treated with TT for benign final pathology. Multivariate analysis failed to identify any factor that independently predicted the need for CT. Female gender was associated with TT in benign disease (odds ratio [OR], 2.1; 95% confidence interval [CI], 1.0 to 4.5; P = .05). Age >45 years predicted correct initial use of DL (OR, 2.6; 95% CI, 1.2 to 5.7; P = .02). Suspicious for papillary thyroid carcinoma (OR, 5.7; 95% CI, 2.1 to 15.3; P<.01) and frozen section (OR, 9.7; 95% CI, 2.5 to 38.6; P<.01) were associated with oncologically appropriate initial TT. The highest frequency of CT occurred in patients with follicular lesion of undetermined significance (11.6%). TT for benign final pathology occurred most frequently in patients with a Hürthle cell neoplasm (24.8%).

CONCLUSION

In patients with ITN, nearly 30% received an inappropriate extent of initial thyroidectomy from an oncologic standpoint. Tools to pre-operatively identify both benign and malignant disease can assist in the complex decision making to gauge the proper extent of initial surgery for ITN.

ABBREVIATIONS

ATA = American Thyroid Association AUS = atypia of undetermined significance CI = confidence interval CT = completion thyroidectomy FLUS = follicular lesion of undetermined significance ITN = indeterminate thyroid nodule OR = odds ratio PTC = papillary thyroid carcinoma TT = total thyroidectomy.

Computerized analysis of calcification of thyroid nodules as visualized by ultrasonography

OBJECTIVE

The purpose of this study is to quantify computerized calcification features from ultrasonography (US) images of thyroid nodules in order to determine the ability to differentiate between malignant and benign thyroid nodules.

METHODS

We designed and implemented a computerized analysis scheme to quantitatively analyze the US features of the calcified thyroid nodules from 99 pathologically determined calcified thyroid nodules. Univariate analysis was used to identify features that were significantly associated with tumor malignancy, and neural-network analysis was performed to classify tumors as benign or malignant. The diagnostic performance of the neural network was evaluated using receiver operating characteristic (ROC) analysis, where in the area under the ROC curve (Az) summarized the diagnostic performance of specific calcification features.

RESULTS

The performance values for each calcification feature were as follows: ratio of calcification distance=0.80, number of calcifications=0.68, skewness=0.82, and maximum intensity=0.75. The combined value of the four features was 0.84.With a threshold of 0.64, the Az value of calcification features was 0.83 with a sensitivity of 83.0%, specificity of 82.4%, and accuracy of 82.8%.

CONCLUSIONS

These results support the clinical feasibility of using computerized analysis of calcification features from thyroid US for differentiating between malignant and benign nodules.