1
|
Sun D, Li H, Wang Y, Li D, Xu D, Zhang Z. Artificial intelligence-based pathological application to predict regional lymph node metastasis in Papillary Thyroid Cancer. Curr Probl Cancer 2024; 53:101150. [PMID: 39342815 DOI: 10.1016/j.currproblcancer.2024.101150] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2024] [Revised: 08/27/2024] [Accepted: 09/19/2024] [Indexed: 10/01/2024]
Abstract
In this study, a model for predicting lymph node metastasis in papillary thyroid cancer was trained using pathology images from the TCGA(The Cancer Genome Atlas) public dataset of papillary thyroid cancer, and a front-end inference model was trained using our center's dataset based on the concept of probabilistic propagation of nodes in graph neural networks. Effectively predicting whether a tumor will spread to regional lymph nodes using a single pathological image is the capacity of the model described above. This study demonstrates that regional lymph nodes in papillary thyroid cancer are a common and predictable occurrence, providing valuable ideas for future research. Now we publish the above research process and code for further study by other researchers, and we also make the above inference algorithm public at the URL: http:// thyroid-diseases-research.com/, with the hope that other researchers will validate it and provide us with ideas or datasets for further study.
Collapse
Affiliation(s)
- Dawei Sun
- The Affiliated Hospital of Qingdao University, PR China
| | - Huichao Li
- The Affiliated Hospital of Qingdao University, PR China
| | - Yaozong Wang
- Ningbo Huamei Hospital University of Chinese Academy of Sciences(Ningbo No.2 Hospital), PR China
| | - Dayuan Li
- Ningbo Institute of Material Technology and Engineering University of Chinese Academy of Sciences, PR China
| | - Di Xu
- Ningbo Institute of Material Technology and Engineering University of Chinese Academy of Sciences, PR China
| | - Zhoujing Zhang
- The Affiliated Hospital of Qingdao University, PR China; Ningbo Institute of Material Technology and Engineering University of Chinese Academy of Sciences, PR China; Ningbo Huamei Hospital University of Chinese Academy of Sciences(Ningbo No.2 Hospital), PR China.
| |
Collapse
|
2
|
Sant VR, Radhachandran A, Ivezic V, Lee DT, Livhits MJ, Wu JX, Masamed R, Arnold CW, Yeh MW, Speier W. From Bench-to-Bedside: How Artificial Intelligence is Changing Thyroid Nodule Diagnostics, a Systematic Review. J Clin Endocrinol Metab 2024; 109:1684-1693. [PMID: 38679750 PMCID: PMC11180510 DOI: 10.1210/clinem/dgae277] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/30/2024] [Revised: 04/04/2024] [Accepted: 04/16/2024] [Indexed: 05/01/2024]
Abstract
CONTEXT Use of artificial intelligence (AI) to predict clinical outcomes in thyroid nodule diagnostics has grown exponentially over the past decade. The greatest challenge is in understanding the best model to apply to one's own patient population, and how to operationalize such a model in practice. EVIDENCE ACQUISITION A literature search of PubMed and IEEE Xplore was conducted for English-language publications between January 1, 2015 and January 1, 2023, studying diagnostic tests on suspected thyroid nodules that used AI. We excluded articles without prospective or external validation, nonprimary literature, duplicates, focused on nonnodular thyroid conditions, not using AI, and those incidentally using AI in support of an experimental diagnostic outside standard clinical practice. Quality was graded by Oxford level of evidence. EVIDENCE SYNTHESIS A total of 61 studies were identified; all performed external validation, 16 studies were prospective, and 33 compared a model to physician prediction of ground truth. Statistical validation was reported in 50 papers. A diagnostic pipeline was abstracted, yielding 5 high-level outcomes: (1) nodule localization, (2) ultrasound (US) risk score, (3) molecular status, (4) malignancy, and (5) long-term prognosis. Seven prospective studies validated a single commercial AI; strengths included automating nodule feature assessment from US and assisting the physician in predicting malignancy risk, while weaknesses included automated margin prediction and interobserver variability. CONCLUSION Models predominantly used US images to predict malignancy. Of 4 Food and Drug Administration-approved products, only S-Detect was extensively validated. Implementing an AI model locally requires data sanitization and revalidation to ensure appropriate clinical performance.
Collapse
Affiliation(s)
- Vivek R Sant
- Division of Endocrine Surgery, UT Southwestern Medical Center, Dallas, TX 75390, USA
| | - Ashwath Radhachandran
- Biomedical Artificial Intelligence Research Lab, UCLA Department of Bioengineering, Los Angeles, CA 90024, USA
| | - Vedrana Ivezic
- Biomedical Artificial Intelligence Research Lab, UCLA Department of Bioengineering, Los Angeles, CA 90024, USA
| | - Denise T Lee
- Department of Surgery, Icahn School of Medicine at Mount Sinai Hospital, New York, NY 10029, USA
| | - Masha J Livhits
- Section of Endocrine Surgery, UCLA David Geffen School of Medicine, Los Angeles, CA 90095, USA
| | - James X Wu
- Section of Endocrine Surgery, UCLA David Geffen School of Medicine, Los Angeles, CA 90095, USA
| | - Rinat Masamed
- Department of Radiology, University of California, Los Angeles, Los Angeles, CA 90095, USA
| | - Corey W Arnold
- Biomedical Artificial Intelligence Research Lab, UCLA Department of Bioengineering, Los Angeles, CA 90024, USA
| | - Michael W Yeh
- Section of Endocrine Surgery, UCLA David Geffen School of Medicine, Los Angeles, CA 90095, USA
| | - William Speier
- Biomedical Artificial Intelligence Research Lab, UCLA Department of Bioengineering, Los Angeles, CA 90024, USA
| |
Collapse
|
3
|
Chen C, Jiang Y, Yao J, Lai M, Liu Y, Jiang X, Ou D, Feng B, Zhou L, Xu J, Wu L, Zhou Y, Yue W, Dong F, Xu D. Deep learning to assist composition classification and thyroid solid nodule diagnosis: a multicenter diagnostic study. Eur Radiol 2024; 34:2323-2333. [PMID: 37819276 DOI: 10.1007/s00330-023-10269-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2023] [Revised: 08/01/2023] [Accepted: 08/07/2023] [Indexed: 10/13/2023]
Abstract
OBJECTIVES This study aimed to propose a deep learning (DL)-based framework for identifying the composition of thyroid nodules and assessing their malignancy risk. METHODS We conducted a retrospective multicenter study using ultrasound images from four hospitals. Convolutional neural network (CNN) models were constructed to classify ultrasound images of thyroid nodules into solid and non-solid, as well as benign and malignant. A total of 11,201 images of 6784 nodules were used for training, validation, and testing. The area under the receiver-operating characteristic curve (AUC) was employed as the primary evaluation index. RESULTS The models had AUCs higher than 0.91 in the benign and malignant grading of solid thyroid nodules, with the Inception-ResNet AUC being the highest at 0.94. In the test set, the best algorithm for identifying benign and malignant thyroid nodules had a sensitivity of 0.88, and a specificity of 0.86. In the human vs. DL test set, the best algorithm had a sensitivity of 0.93, and a specificity of 0.86. The Inception-ResNet model performed better than the senior physicians (p < 0.001). The sensitivity and specificity of the optimal model based on the external test set were 0.90 and 0.75, respectively. CONCLUSIONS This research demonstrates that CNNs can assist thyroid nodule diagnosis and reduce the rate of unnecessary fine-needle aspiration (FNA). CLINICAL RELEVANCE STATEMENT High-resolution ultrasound has led to increased detection of thyroid nodules. This results in unnecessary fine-needle aspiration and anxiety for patients whose nodules are benign. Deep learning can solve these problems to some extent. KEY POINTS • Thyroid solid nodules have a high probability of malignancy. • Our models can improve the differentiation between benign and malignant solid thyroid nodules. • The differential performance of one model was superior to that of senior radiologists. Applying this could reduce the rate of unnecessary fine-needle aspiration of solid thyroid nodules.
Collapse
Affiliation(s)
- Chen Chen
- Department of Diagnostic Ultrasound Imaging & Interventional Therapy, Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, China
- Wenling Big Data and Artificial Intelligence Institute in Medicine, Taizhou, 317502, China
- Taizhou Key Laboratory of Minimally Invasive Interventional Therapy & Artificial Intelligence, Taizhou Campus of Zhejiang Cancer Hospital (Taizhou Cancer Hospital), Taizhou, 317502, China
| | - Yitao Jiang
- Illuminate, LLC, Shenzhen, Guangdong, 518000, China
| | - Jincao Yao
- Department of Diagnostic Ultrasound Imaging & Interventional Therapy, Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, China
- Key Laboratory of Head & Neck Cancer, Translational Research of Zhejiang Province, Hangzhou, 310022, China
- Zhejiang Provincial Research Center for Cancer Intelligent Diagnosis and Molecular Technology, Hangzhou, 310022, China
| | - Min Lai
- The Second Clinical Medical College, Zhejiang Chinese Medical University, Hangzhou, 310053, China
| | - Yuanzhen Liu
- Department of Diagnostic Ultrasound Imaging & Interventional Therapy, Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, China
- Wenling Big Data and Artificial Intelligence Institute in Medicine, Taizhou, 317502, China
- Taizhou Key Laboratory of Minimally Invasive Interventional Therapy & Artificial Intelligence, Taizhou Campus of Zhejiang Cancer Hospital (Taizhou Cancer Hospital), Taizhou, 317502, China
| | - Xianping Jiang
- Department of Ultrasound, Shengzhou People's Hospital (the First Affiliated Hospital of Zhejiang University Shengzhou Branch), Shengzhou, 312400, China
| | - Di Ou
- Department of Diagnostic Ultrasound Imaging & Interventional Therapy, Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, China
- Key Laboratory of Head & Neck Cancer, Translational Research of Zhejiang Province, Hangzhou, 310022, China
- Zhejiang Provincial Research Center for Cancer Intelligent Diagnosis and Molecular Technology, Hangzhou, 310022, China
| | - Bojian Feng
- Department of Diagnostic Ultrasound Imaging & Interventional Therapy, Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, China
- Wenling Big Data and Artificial Intelligence Institute in Medicine, Taizhou, 317502, China
- Taizhou Key Laboratory of Minimally Invasive Interventional Therapy & Artificial Intelligence, Taizhou Campus of Zhejiang Cancer Hospital (Taizhou Cancer Hospital), Taizhou, 317502, China
| | - Lingyan Zhou
- Department of Diagnostic Ultrasound Imaging & Interventional Therapy, Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, China
- Key Laboratory of Head & Neck Cancer, Translational Research of Zhejiang Province, Hangzhou, 310022, China
- Zhejiang Provincial Research Center for Cancer Intelligent Diagnosis and Molecular Technology, Hangzhou, 310022, China
| | - Jinfeng Xu
- Department of Ultrasound, The Second Clinical Medical College, Jinan University, The First Affiliated Hospital, Southern University of Science and Technology, Shenzhen People's Hospital, Shenzhen, 518020, China
| | - Linghu Wu
- Department of Ultrasound, The Second Clinical Medical College, Jinan University, The First Affiliated Hospital, Southern University of Science and Technology, Shenzhen People's Hospital, Shenzhen, 518020, China
| | - Yuli Zhou
- Department of Ultrasound, The Second Clinical Medical College, Jinan University, The First Affiliated Hospital, Southern University of Science and Technology, Shenzhen People's Hospital, Shenzhen, 518020, China
| | - Wenwen Yue
- Center of Minimally Invasive Treatment for Tumor, Department of Medical Ultrasound, Shanghai Tenth People's Hospital, School of Medicine, Tongji University, Shanghai, 200072, China.
| | - Fajin Dong
- Department of Ultrasound, The Second Clinical Medical College, Jinan University, The First Affiliated Hospital, Southern University of Science and Technology, Shenzhen People's Hospital, Shenzhen, 518020, China.
| | - Dong Xu
- Department of Diagnostic Ultrasound Imaging & Interventional Therapy, Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, 310022, China.
- Wenling Big Data and Artificial Intelligence Institute in Medicine, Taizhou, 317502, China.
- Taizhou Key Laboratory of Minimally Invasive Interventional Therapy & Artificial Intelligence, Taizhou Campus of Zhejiang Cancer Hospital (Taizhou Cancer Hospital), Taizhou, 317502, China.
| |
Collapse
|
4
|
Ajilisa OA, Jagathy Raj VP, Sabu MK. A Deep Learning Framework for the Characterization of Thyroid Nodules from Ultrasound Images Using Improved Inception Network and Multi-Level Transfer Learning. Diagnostics (Basel) 2023; 13:2463. [PMID: 37510206 PMCID: PMC10378664 DOI: 10.3390/diagnostics13142463] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2023] [Revised: 07/07/2023] [Accepted: 07/13/2023] [Indexed: 07/30/2023] Open
Abstract
In the past few years, deep learning has gained increasingly widespread attention and has been applied to diagnosing benign and malignant thyroid nodules. It is difficult to acquire sufficient medical images, resulting in insufficient data, which hinders the development of an efficient deep-learning model. In this paper, we developed a deep-learning-based characterization framework to differentiate malignant and benign nodules from the thyroid ultrasound images. This approach improves the recognition accuracy of the inception network by combining squeeze and excitation networks with the inception modules. We have also integrated the concept of multi-level transfer learning using breast ultrasound images as a bridge dataset. This transfer learning approach addresses the issues regarding domain differences between natural images and ultrasound images during transfer learning. This paper aimed to investigate how the entire framework could help radiologists improve diagnostic performance and avoid unnecessary fine-needle aspiration. The proposed approach based on multi-level transfer learning and improved inception blocks achieved higher precision (0.9057 for the benign class and 0.9667 for the malignant class), recall (0.9796 for the benign class and 0.8529 for malignant), and F1-score (0.9412 for benign class and 0.9062 for malignant class). It also obtained an AUC value of 0.9537, which is higher than that of the single-level transfer learning method. The experimental results show that this model can achieve satisfactory classification accuracy comparable to experienced radiologists. Using this model, we can save time and effort as well as deliver potential clinical application value.
Collapse
Affiliation(s)
- O A Ajilisa
- Department of Computer Applications, Cochin University of Science and Technology, South Kalamassery, Kochi 682022, Kerala, India
| | - V P Jagathy Raj
- School of Management Studies, Cochin University of Science and Technology, South Kalamassery, Kochi 682022, Kerala, India
| | - M K Sabu
- Department of Computer Applications, Cochin University of Science and Technology, South Kalamassery, Kochi 682022, Kerala, India
| |
Collapse
|
5
|
Cao CL, Li QL, Tong J, Shi LN, Li WX, Xu Y, Cheng J, Du TT, Li J, Cui XW. Artificial intelligence in thyroid ultrasound. Front Oncol 2023; 13:1060702. [PMID: 37251934 PMCID: PMC10213248 DOI: 10.3389/fonc.2023.1060702] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2022] [Accepted: 04/07/2023] [Indexed: 05/31/2023] Open
Abstract
Artificial intelligence (AI), particularly deep learning (DL) algorithms, has demonstrated remarkable progress in image-recognition tasks, enabling the automatic quantitative assessment of complex medical images with increased accuracy and efficiency. AI is widely used and is becoming increasingly popular in the field of ultrasound. The rising incidence of thyroid cancer and the workload of physicians have driven the need to utilize AI to efficiently process thyroid ultrasound images. Therefore, leveraging AI in thyroid cancer ultrasound screening and diagnosis cannot only help radiologists achieve more accurate and efficient imaging diagnosis but also reduce their workload. In this paper, we aim to present a comprehensive overview of the technical knowledge of AI with a focus on traditional machine learning (ML) algorithms and DL algorithms. We will also discuss their clinical applications in the ultrasound imaging of thyroid diseases, particularly in differentiating between benign and malignant nodules and predicting cervical lymph node metastasis in thyroid cancer. Finally, we will conclude that AI technology holds great promise for improving the accuracy of thyroid disease ultrasound diagnosis and discuss the potential prospects of AI in this field.
Collapse
Affiliation(s)
- Chun-Li Cao
- Department of Ultrasound, The First Affiliated Hospital of Shihezi University, Shihezi, China
- NHC Key Laboratory of Prevention and Treatment of Central Asia High Incidence Diseases, First Affiliated Hospital, School of Medicine, Shihezi University, Shihezi, China
| | - Qiao-Li Li
- Department of Ultrasound, The First Affiliated Hospital of Shihezi University, Shihezi, China
- NHC Key Laboratory of Prevention and Treatment of Central Asia High Incidence Diseases, First Affiliated Hospital, School of Medicine, Shihezi University, Shihezi, China
| | - Jin Tong
- Department of Ultrasound, The First Affiliated Hospital of Shihezi University, Shihezi, China
| | - Li-Nan Shi
- Department of Ultrasound, The First Affiliated Hospital of Shihezi University, Shihezi, China
- NHC Key Laboratory of Prevention and Treatment of Central Asia High Incidence Diseases, First Affiliated Hospital, School of Medicine, Shihezi University, Shihezi, China
| | - Wen-Xiao Li
- Department of Ultrasound, The First Affiliated Hospital of Shihezi University, Shihezi, China
- NHC Key Laboratory of Prevention and Treatment of Central Asia High Incidence Diseases, First Affiliated Hospital, School of Medicine, Shihezi University, Shihezi, China
| | - Ya Xu
- Department of Ultrasound, The First Affiliated Hospital of Shihezi University, Shihezi, China
| | - Jing Cheng
- Department of Ultrasound, The First Affiliated Hospital of Shihezi University, Shihezi, China
| | - Ting-Ting Du
- Department of Ultrasound, The First Affiliated Hospital of Shihezi University, Shihezi, China
| | - Jun Li
- Department of Ultrasound, The First Affiliated Hospital of Shihezi University, Shihezi, China
- NHC Key Laboratory of Prevention and Treatment of Central Asia High Incidence Diseases, First Affiliated Hospital, School of Medicine, Shihezi University, Shihezi, China
| | - Xin-Wu Cui
- Department of Medical Ultrasound, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei, China
| |
Collapse
|
6
|
Ozturk AC, Haznedar H, Haznedar B, Ilgan S, Erogul O, Kalinli A. Differentiation of Benign and Malignant Thyroid Nodules with ANFIS by Using Genetic Algorithm and Proposing a Novel CAD-Based Risk Stratification System of Thyroid Nodules. Diagnostics (Basel) 2023; 13:diagnostics13040740. [PMID: 36832228 PMCID: PMC9954959 DOI: 10.3390/diagnostics13040740] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2022] [Revised: 01/27/2023] [Accepted: 02/06/2023] [Indexed: 02/17/2023] Open
Abstract
The thyroid nodule risk stratification guidelines used in the literature are based on certain well-known sonographic features of nodules and are still subjective since the application of these characteristics strictly depends on the reading physician. These guidelines classify nodules according to the sub-features of limited sonographic signs. This study aims to overcome these limitations by examining the relationships of a wide range of ultrasound (US) signs in the differential diagnosis of nodules by using artificial intelligence methods. An innovative method based on training Adaptive-Network Based Fuzzy Inference Systems (ANFIS) by using Genetic Algorithm (GA) is used to differentiate malignant from benign thyroid nodules. The comparison of the results from the proposed method to the results from the commonly used derivative-based algorithms and Deep Neural Network (DNN) methods yielded that the proposed method is more successful in differentiating malignant from benign thyroid nodules. Furthermore, a novel computer aided diagnosis (CAD) based risk stratification system for the thyroid nodule's US classification that is not present in the literature is proposed.
Collapse
Affiliation(s)
- Ahmet Cankat Ozturk
- Institute of Natural Science, Department of Biomedical Engineering, TOBB University of Economics and Technology, 06560 Ankara, Türkiye
- Correspondence:
| | - Hilal Haznedar
- Institute of Natural Science, Department of Computer Engineering, Erciyes University, 38280 Kayseri, Türkiye
| | - Bulent Haznedar
- Department of Computer Engineering, Gaziantep University, 27310 Gaziantep, Türkiye
| | - Seyfettin Ilgan
- Department of Nuclear Medicine, Ankara Guven Hospital, 06540 Ankara, Türkiye
| | - Osman Erogul
- Institute of Natural Science, Department of Biomedical Engineering, TOBB University of Economics and Technology, 06560 Ankara, Türkiye
| | - Adem Kalinli
- Presidency Office, Rectorate, Middle East Technical University, 06800 Ankara, Türkiye
| |
Collapse
|
7
|
Zhou L, Zheng LL, Zhang CJ, Wei HF, Xu LL, Zhang MR, Li Q, He GF, Ghamor-Amegavi EP, Li SY. Comparison of S-Detect and thyroid imaging reporting and data system classifications in the diagnosis of cytologically indeterminate thyroid nodules. Front Endocrinol (Lausanne) 2023; 14:1098031. [PMID: 36761203 PMCID: PMC9902707 DOI: 10.3389/fendo.2023.1098031] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/14/2022] [Accepted: 01/11/2023] [Indexed: 01/26/2023] Open
Abstract
Purpose The aim of this study was to investigate the value of S-Detect for predicting the malignant risk of cytologically indeterminate thyroid nodules (CITNs). Methods The preoperative prediction of 159 CITNs (Bethesda III, IV and V) were performed using S-Detect, Thyroid Imaging Reporting and Data System of American College of Radiology (ACR TI-RADS) and Chinese TI-RADS (C-TIRADS). First, Linear-by-Linear Association test and chi-square test were used to analyze the malignant risk of CITNs. McNemar's test and receiver operating characteristic curve were used to compare the diagnostic efficacy of S-Detect and the two TI-RADS classifications for CITNs. In addition, the McNemar's test was used to compare the diagnostic accuracy of the above three methods for different pathological types of nodules. Results The maximum diameter of the benign nodules was significantly larger than that of malignant nodules [0.88(0.57-1.42) vs 0.57(0.46-0.81), P=0.002]. The risk of malignant CITNs in Bethesda system and the two TI-RADS classifications increased with grade (all P for trend<0.001). In all the enrolled CITNs, the diagnostic results of S-Detect were significantly different from those of ACR TI-RADS and C-TIRADS, respectively (P=0.021 and P=0.007). The sensitivity and accuracy of S-Detect [95.9%(90.1%-98.5%) and 88.1%(81.7%-92.5%)] were higher than those of ACR TI-RADS [87.6%(80.1%-92.7%) and 81.8%(74.7%-87.3%)] (P=0.006 and P=0.021) and C-TIRADS [84.3%(76.3%-90.0%) and 78.6%(71.3%-84.5%)] (P=0.001 and P=0.001). Moreover, the negative predictive value and the area under curve value of S-Detect [82.8% (63.5%-93.5%) and 0.795%(0.724%-0.855%)] was higher than that of C-TIRADS [54.8%(38.8%-69.8%) and 0.724%(0.648%-0.792%] (P=0.024 and P=0.035). However, the specificity and positive predictive value of S-Detect were similar to those of ACR TI-RADS (P=1.000 and P=0.154) and C-TIRADS (P=1.000 and P=0.072). There was no significant difference in all the evaluated indicators between ACR TI-RADS and C-TIRADS (all P>0.05). The diagnostic accuracy of S-Detect (97.4%) for papillary thyroid carcinoma (PTC) was higher than that of ACR TI-RADS (90.4%) and C-TIRADS (87.8%) (P=0.021 and P=0.003). Conclusion The diagnostic performance of S-Detect in differentiating CITNs was similar to ACR TI-RADS and superior to C-TIRADS, especially for PTC.
Collapse
Affiliation(s)
- Ling Zhou
- Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| | - Lin-lin Zheng
- Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| | - Chuan-ju Zhang
- Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| | - Hong-fen Wei
- Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| | - Li-long Xu
- Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| | - Mu-rui Zhang
- Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| | - Qiang Li
- Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| | - Gao-fei He
- Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| | | | - Shi-yan Li
- Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| |
Collapse
|
8
|
Gong L, Zhou P, Li JL, Liu WG. Investigating the diagnostic efficiency of a computer-aided diagnosis system for thyroid nodules in the context of Hashimoto's thyroiditis. Front Oncol 2023; 12:941673. [PMID: 36686823 PMCID: PMC9850089 DOI: 10.3389/fonc.2022.941673] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2022] [Accepted: 12/09/2022] [Indexed: 01/07/2023] Open
Abstract
Objectives This study aims to investigate the efficacy of a computer-aided diagnosis (CAD) system in distinguishing between benign and malignant thyroid nodules in the context of Hashimoto's thyroiditis (HT) and to evaluate the role of the CAD system in reducing unnecessary biopsies of benign lesions. Methods We included a total of 137 nodules from 137 consecutive patients (mean age, 43.5 ± 11.8 years) who were histopathologically diagnosed with HT. The two-dimensional ultrasound images and videos of all thyroid nodules were analyzed by the CAD system and two radiologists with different experiences according to ACR TI-RADS. The diagnostic cutoff values of ACR TI-RADS were divided into two categories (TR4 and TR5), and then the sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) of the CAD system and the junior and senior radiologists were compared in both cases. Moreover, ACR TI-RADS classification was revised according to the results of the CAD system, and the efficacy of recommended fine-needle aspiration (FNA) was evaluated by comparing the unnecessary biopsy rate and the malignant rate of punctured nodules. Results The accuracy, sensitivity, specificity, PPV, and NPV of the CAD system were 0.876, 0.905, 0.830, 0.894, and 0.846, respectively. With TR4 as the cutoff value, the AUCs of the CAD system and the junior and senior radiologists were 0.867, 0.628, and 0.722, respectively, and the CAD system had the highest AUC (P < 0.0001). With TR5 as the cutoff value, the AUCs of the CAD system and the junior and senior radiologists were 0.867, 0.654, and 0.812, respectively, and the CAD system had a higher AUC than the junior radiologist (P < 0.0001) but comparable to the senior radiologist (P = 0.0709). With the assistance of the CAD system, the number of TR4 nodules was decreased by both junior and senior radiologists, the malignant rate of punctured nodules increased by 30% and 22%, and the unnecessary biopsies of benign lesions were both reduced by nearly half. Conclusions The CAD system based on deep learning can improve the diagnostic performance of radiologists in identifying benign and malignant thyroid nodules in the context of Hashimoto's thyroiditis and can play a role in FNA recommendations to reduce unnecessary biopsy rates.
Collapse
|
9
|
Wang D, Zhao CK, Wang HX, Lu F, Li XL, Guo LH, Sun LP, Fu HJ, Zhang YF, Xu HX. Ultrasound-based computer-aided diagnosis for cytologically indeterminate thyroid nodules with different radiologists. Clin Hemorheol Microcirc 2022; 82:217-230. [PMID: 35848013 DOI: 10.3233/ch-221423] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/27/2022]
Abstract
PURPOSE To evaluate a computer-aided diagnosis (CAD) technique in predicting malignancy for cytologically indeterminate thyroid nodules (TNs) as compared with different experienced radiologists. METHOD 436 patients with 436 cytologically indeterminate TNs on fine-needle aspiration cytology (FNAC) were included and all were confirmed by surgical pathology. They were retrospectively analyzed with respect to ultrasound (US) characteristics using a commercially available CAD system (AmCAD-UT; AmCad BioMed, Taiwan, China) and reviewed by one junior and one senior radiologists.The CAD system and different experienced radiologists stratified the risk of malignancy using ACR TI-RADS category. The diagnostic performance by different experienced radiologists independently and after consulting the CAD (different experienced radiologists + CAD) and by the CAD alone were compared. RESULTS The different experienced radiologists showed significantly higher specificities than the CAD system alone. The combination of radiologist and CAD system showed improved diagnostic performance with an AUC (Area under the curve) of 0.740 in the senior radiologist and 0.677 in the junior radiologist, as compared with CAD (AUC: 0.585) alone (all P < 0.05). The combination of senior radiologist and CAD system had the highest diagnostic performance (AUC: 0.740) and specificity (68.9%) compared to the others (all P < 0.05). CONCLUSION The CAD system may play the potential role as a decision-making assistant alongside radiologists for differential diagnosis of TNs with indeterminate cytology.
Collapse
Affiliation(s)
- Dan Wang
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, School of Medicine, Tongji University, Shanghai, China.,Ultrasound Research and Education Institute, Clinical Research Center for Interventional Medicine, School of Medicine, Tongji University, Shanghai, China.,Shanghai Engineering Research Center of Ultrasound Diagnosis and Treatment, Shanghai, China
| | - Chong-Ke Zhao
- Department of Ultrasound, Zhongshan Hospital, Institute of Ultrasound in Medicine and Engineering, Fudan University, Shanghai, China
| | - Han-Xiang Wang
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, School of Medicine, Tongji University, Shanghai, China.,Ultrasound Research and Education Institute, Clinical Research Center for Interventional Medicine, School of Medicine, Tongji University, Shanghai, China.,Shanghai Engineering Research Center of Ultrasound Diagnosis and Treatment, Shanghai, China
| | - Feng Lu
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, School of Medicine, Tongji University, Shanghai, China.,Ultrasound Research and Education Institute, Clinical Research Center for Interventional Medicine, School of Medicine, Tongji University, Shanghai, China.,Shanghai Engineering Research Center of Ultrasound Diagnosis and Treatment, Shanghai, China
| | - Xiao-Long Li
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, School of Medicine, Tongji University, Shanghai, China.,Ultrasound Research and Education Institute, Clinical Research Center for Interventional Medicine, School of Medicine, Tongji University, Shanghai, China.,Shanghai Engineering Research Center of Ultrasound Diagnosis and Treatment, Shanghai, China
| | - Le-Hang Guo
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, School of Medicine, Tongji University, Shanghai, China.,Ultrasound Research and Education Institute, Clinical Research Center for Interventional Medicine, School of Medicine, Tongji University, Shanghai, China.,Shanghai Engineering Research Center of Ultrasound Diagnosis and Treatment, Shanghai, China
| | - Li-Ping Sun
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, School of Medicine, Tongji University, Shanghai, China.,Ultrasound Research and Education Institute, Clinical Research Center for Interventional Medicine, School of Medicine, Tongji University, Shanghai, China.,Shanghai Engineering Research Center of Ultrasound Diagnosis and Treatment, Shanghai, China
| | - Hui-Jun Fu
- Department of Pathology, Shanghai Tenth People's Hospital, Tongji University School of Medicine, Shanghai, China
| | - Yi-Feng Zhang
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, School of Medicine, Tongji University, Shanghai, China.,Ultrasound Research and Education Institute, Clinical Research Center for Interventional Medicine, School of Medicine, Tongji University, Shanghai, China.,Shanghai Engineering Research Center of Ultrasound Diagnosis and Treatment, Shanghai, China
| | - Hui-Xiong Xu
- Department of Ultrasound, Zhongshan Hospital, Institute of Ultrasound in Medicine and Engineering, Fudan University, Shanghai, China
| |
Collapse
|
10
|
Lee SE, Lee E, Kim EK, Yoon JH, Park VY, Youk JH, Kwak JY. Application of Artificial Intelligence Computer-Assisted Diagnosis Originally Developed for Thyroid Nodules to Breast Lesions on Ultrasound. J Digit Imaging 2022; 35:1699-1707. [PMID: 35902445 PMCID: PMC9712894 DOI: 10.1007/s10278-022-00680-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/26/2021] [Revised: 06/27/2022] [Accepted: 07/11/2022] [Indexed: 10/16/2022] Open
Abstract
As thyroid and breast cancer have several US findings in common, we applied an artificial intelligence computer-assisted diagnosis (AI-CAD) software originally developed for thyroid nodules to breast lesions on ultrasound (US) and evaluated its diagnostic performance. From January 2017 to December 2017, 1042 breast lesions (mean size 20.2 ± 11.8 mm) of 1001 patients (mean age 45.9 ± 12.9 years) who underwent US-guided core-needle biopsy were included. An AI-CAD software that was previously trained and validated with thyroid nodules using the convolutional neural network was applied to breast nodules. There were 665 benign breast lesions (63.0%) and 391 breast cancers (37.0%). The area under the receiver operating characteristic curve (AUROC) of AI-CAD to differentiate breast lesions was 0.678 (95% confidence interval: 0.649, 0.707). After fine-tuning AI-CAD with 1084 separate breast lesions, the diagnostic performance of AI-CAD markedly improved (AUC 0.841). This was significantly higher than that of radiologists when the cutoff category was BI-RADS 4a (AUC 0.621, P < 0.001), but lower when the cutoff category was BI-RADS 4b (AUC 0.908, P < 0.001). When applied to breast lesions, the diagnostic performance of an AI-CAD software that had been developed for differentiating malignant and benign thyroid nodules was not bad. However, an organ-specific approach guarantees better diagnostic performance despite the similar US features of thyroid and breast malignancies.
Collapse
Affiliation(s)
- Si Eun Lee
- Department of Radiology, Yongin Severance Hospital, Research Institute of Radiological Science, Yonsei University College of Medicine, Seoul, Korea
| | - Eunjung Lee
- Department of Computational Science and Engineering, Yonsei University, Seoul, Korea
| | - Eun-Kyung Kim
- Department of Radiology, Yongin Severance Hospital, Research Institute of Radiological Science, Yonsei University College of Medicine, Seoul, Korea
| | - Jung Hyun Yoon
- Department of Radiology, Severance Hospital, Research Institute of Radiological Science, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Korea
| | - Vivian Youngjean Park
- Department of Radiology, Severance Hospital, Research Institute of Radiological Science, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Korea
| | - Ji Hyun Youk
- Department of Radiology, Gangnam Severance Hospital, Yonsei University College of Medicine, Seoul, Korea
| | - Jin Young Kwak
- Department of Radiology, Severance Hospital, Research Institute of Radiological Science, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Korea.
| |
Collapse
|
11
|
He LT, Chen FJ, Zhou DZ, Zhang YX, Li YS, Tang MX, Tang JX, Liu S, Chen ZJ, Tang Q. A Comparison of the Performances of Artificial Intelligence System and Radiologists in the Ultrasound Diagnosis of Thyroid Nodules. Curr Med Imaging 2022; 18:1369-1377. [PMID: 35466880 DOI: 10.2174/1573405618666220422132251] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2021] [Revised: 02/16/2022] [Accepted: 02/21/2022] [Indexed: 01/25/2023]
Abstract
AIMS The purpose of this paper is to prospectively evaluate the performance of an artificial intelligence (AI) system in diagnosing thyroid nodules and to assess its potential value in comparison with the performance of radiologists with different levels of experience, as well as the factors affecting its diagnostic accuracy. BACKGROUND In recent years, medical imaging diagnosis using AI has become a popular topic in clinical application research. OBJECTIVE This study aimed to evaluate the performance of an AI system in diagnosing thyroid nodules and compare it with the performance levels of different radiologists. METHODS This study involved 426 patients screened for thyroid nodules at the First Affiliated Hospital of Guangzhou Medical University between July 2017 and March 2019. All of the nodules were evaluated by radiologists with various levels of experience and an AI system. The diagnostic performances of two junior and two senior radiologists, an AI system, and an AI-assisted junior radiologist were compared, as were their diagnostic results with respect to nodules of different sizes. RESULTS The senior radiologists, the AI system, and the AI-assisted junior radiologist performed better than the junior radiologist (p < 0.05). The area under the curves of the AI system and the AI-assisted junior radiologist were similar to the curve of the senior radiologists (p > 0.05). The diagnostic results concerning the two nodule sizes showed that the diagnostic error rates of the AI system, junior radiologists, and senior radiologists for nodules with a maximum diameter of ≤1 cm (Dmax ≤ 1 cm) were higher than those for nodules with a maximum diameter of 1 cm (Dmax > 1 cm) (23.4% vs. 12.1%, p = 0.002; 26.6% vs. 7.3%, p < 0.001; and 38.3% vs. 14.6%, p < 0.001). CONCLUSION The AI system is a decision-making tool that could potentially improve the diagnostic efficiency of junior radiologists. Micronodules with Dmax ≤ 1cm were significantly correlated with diagnostic accuracy; accordingly, more micronodules of this size, in particular, should be added to the AI system as training samples. Other: The system could be a potential decision-making tool for effectively improving the diagnostic efficiency of junior radiologists in the community.
Collapse
Affiliation(s)
- Lian-Tu He
- Department of Ultrasound, First Affiliated Hospital of Guangzhou Medical University, Guangzhou 510120, China
| | - Feng-Juan Chen
- Department of Ultrasound, Guangzhou University of Traditional Chinese Medicine First Affiliated Hospital, Guangzhou 510405, China
| | - Da-Zhi Zhou
- Department of Ultrasound, First Affiliated Hospital of Guangzhou Medical University, Guangzhou 510120, China
| | - Yu-Xin Zhang
- Department of Ultrasound, First Affiliated Hospital of Guangzhou Medical University, Guangzhou 510120, China
| | - Ying-Shan Li
- Department of Ultrasound, First Affiliated Hospital of Guangzhou Medical University, Guangzhou 510120, China
| | - Min-Xuan Tang
- Department of Ultrasound, First Affiliated Hospital of Guangzhou Medical University, Guangzhou 510120, China
| | - Jia-Xin Tang
- Department of Respiratory Disease, The State Key Laboratory of Respiratory Disease, China Clinical Research Centre for Respiratory Disease, Guangzhou Institute of Respiratory Disease, First Affiliated Hospital of Guangzhou Medical University, Guangzhou 510120, China
| | - Shuo Liu
- Shenzhen Mindray Bio-Medical Electronics Co., Ltd., Shenzhen 518057, China
| | - Zhi-Jie Chen
- Shenzhen Mindray Bio-Medical Electronics Co., Ltd., Shenzhen 518057, China
| | - Qing Tang
- Department of Ultrasound, First Affiliated Hospital of Guangzhou Medical University, Guangzhou 510120, China
| |
Collapse
|
12
|
A comparison of artificial intelligence versus radiologists in the diagnosis of thyroid nodules using ultrasonography: a systematic review and meta-analysis. Eur Arch Otorhinolaryngol 2022; 279:5363-5373. [PMID: 35767056 DOI: 10.1007/s00405-022-07436-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2022] [Accepted: 05/06/2022] [Indexed: 11/03/2022]
Abstract
BACKGROUND Thyroid nodules are common. Ultrasonography (US) is the first investigation for thyroid nodules. Artificial Intelligence (AI) is widely integrated into medical diagnosis to provide additional information. The primary objective of this study was to accumulate the pooled sensitivity and specificity between all available AI and radiologists using thyroid US imaging. The secondary objective was to compare AI's diagnostic performance to that of radiologists. MATERIALS AND METHODS A systematic review meta-analysis. PubMed, Scopus, Web of Science, and Cochrane Library data were searched for studies from inception until June 11, 2020. RESULTS Twenty five studies were included in this meta-analysis. The pooled sensitivity and specificity of AI were 0.86 (95% CI 0.81-0.91) and 0.78 (95% CI 0.73-0.83), respectively. The pooled sensitivity and specificity of radiologists were 0.85 (95% CI 0.80-0.89) and 0.82 (95% CI 0.77-0.86), respectively. The accuracy of AI and radiologists is equivalent in terms of AUC [AI 0.89 (95% CI 0.86-0.92), radiologist 0.91 (95% CI 0.88-0.93)]. The diagnostic odd ratio (DOR) between AI 23.10 (95% CI 14.20-37.58) and radiologists 27.12 (95% CI 17.45-42.16) had no statistically significant difference (P = 0.56). Meta-regression analysis revealed that Deep Learning AI had significantly greater sensitivity and specificity than classic machine learning AI (P < 0.001). CONCLUSION AI demonstrated comparable performance to radiologists in diagnosing benign and malignant thyroid nodules using ultrasonography. Additional research to establish its equivalency should be conducted.
Collapse
|
13
|
Han X, Chang L, Song K, Cheng L, Li M, Wei X. Multitask network for thyroid nodule diagnosis based on TI-RADS. Med Phys 2022; 49:5064-5080. [PMID: 35608232 DOI: 10.1002/mp.15724] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2021] [Revised: 04/19/2022] [Accepted: 05/11/2022] [Indexed: 11/05/2022] Open
Abstract
PURPOSE Assessment of thyroid nodules is usually relied on the experience of the radiologist and is time consuming. Classification model of thyroid nodules can not only reduce the burden on physicians but also provide objective recommendations. However, most classification models based on deep learning simply give a prediction result of the benignity or malignancy of nodules thus physicians have no way of knowing how the deep learning gets the prediction result due to the black-box nature of neural networks. In this work, we integrate the explainability directly into the outputs generated by the model through combining TI-RADS. The inference process of the proposed method is consistent with doctor's clinical diagnosis process, therefore, doctors can better explain the diagnosis results of the model to the patient. METHODS A multitask network based on TI-RADS (MTN-TI-RADS) for the classification of thyroid nodules is proposed. In this network, a set of TI-RADS classifications of nodules is first obtained by multitask learning, then the TI-RADS points and the corresponding risk levels are calculated, finally, nodules are classified as benign and malignant. The classification process through the network is consistent with the diagnostic process of physician, thus the results of classification can be easily understood by physicians. In addition, the attention modules are introduced to the spatial and channel domains to let the network focus more on critical features. RESULTS To verify the classification performance of our method, we compared the results obtained through our method with the results of the radiologist's evaluation. For the 781 test nodules in the internal dataset and the 886 test nodules in the external dataset, the sensitivity and specificity of MTN-TI-RADS were 0.988, 0.912 in internal dataset, 0.949, 0.930 in external dataset, versus the senior radiologist of 0.925 (P < 0.001), 0.816 (P = 0.005) and 0.910(P = 0.009), 0.836 (P < 0.001), respectively. And the area under the receiver operating characteristic curve (AUC) of MTN-TI-RADS was 0.981 in internal dataset, 0.973 in external dataset, versus the senior radiologist of 0.905, 0.923. For the internal dataset, we also computed the accuracy of the risk level (TR1 to TR5) and the mean absolute error (MAE). The accuracy of the risk level of the proposed method is 78%, and the MAE is 1.30. The MAE of the total points (0 to 14 points) is 1.30. CONCLUSIONS An effective and result-interpretable end-to-end thyroid nodule classification network (MTN-TI-RADS) is proposed. MTN-TI-RADS has superior ability to classify malignant and benign thyroid nodules compared to senior radiologists. Based on MTN-TI-RADS, a classification model with strong interpretation and a high degree of physician trust is constructed. The proposed classification network is consistent with the diagnosis process of physicians, thus is more reliable and interpretable, and has great potential for clinical application. This article is protected by copyright. All rights reserved.
Collapse
Affiliation(s)
- Xiaohong Han
- College of Data Science, Taiyuan University of Technology, Taiyuan, China
| | - Luchen Chang
- Department of Diagnostic and Therapeutic Ultrasonography, Tianjin Medical University Cancer Institute and Hospital, National Clinical Research Center for Cancer, Key laboratory of Cancer Prevention and Therapy, Tianjin's Clinical Research Center for Cancer, Tianjin, China
| | - Ke Song
- College of Data Science, Taiyuan University of Technology, Taiyuan, China
| | - Longlong Cheng
- China Electronics Cloud Brain(Tianjin) Technology CO. LTD
| | - Minghui Li
- China Electronics Cloud Brain(Tianjin) Technology CO. LTD
| | - Xi Wei
- Department of Diagnostic and Therapeutic Ultrasonography, Tianjin Medical University Cancer Institute and Hospital, National Clinical Research Center for Cancer, Key laboratory of Cancer Prevention and Therapy, Tianjin's Clinical Research Center for Cancer, Tianjin, China
| |
Collapse
|
14
|
Hu M, Zong S, Xu N, Li J, Xia C, Yu F, Zhu Q, Zhao H. The Value of Thyroid Ultrasound Computer-Aided Diagnosis System in the Evaluation of Thyroid Nodules With Concurrent Hashimoto's Thyroiditis. JOURNAL OF ULTRASOUND IN MEDICINE : OFFICIAL JOURNAL OF THE AMERICAN INSTITUTE OF ULTRASOUND IN MEDICINE 2022; 41:1117-1124. [PMID: 34382688 DOI: 10.1002/jum.15801] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/14/2021] [Revised: 06/07/2021] [Accepted: 07/11/2021] [Indexed: 06/13/2023]
Abstract
OBJECTIVE To investigate the value of computer-aided diagnosis (CAD) system in assessing thyroid nodules concurrent with Hashimoto's thyroiditis (HT). METHODS Totally 148 patients with 193 thyroid nodules were enrolled. A radiologist assessed the nodules using a thyroid ultrasound CAD system. Additionally, the nodules were evaluated by one experienced radiologist alone, and one training radiologist without and with CAD assistance. The diagnostic performance was compared between the CAD system and the experienced radiologist, and the training radiologist without and with CAD assistance. RESULTS The CAD system demonstrated a similar sensitivity to that of the experienced radiologist in diagnosing thyroid cancers (89.8% versus 92.4%, P > .05). The specificity and accuracy of the CAD system were lower than that of the experienced radiologist in assessing the nodules with diffusedly altered glands (specificity, 60.0% versus 81.7%, P = .007; accuracy, 77.5% versus 88.1%, P = .011). With CAD assistance, the training radiologist had improved sensitivity and accuracy that increased to 87.9% and 86.8% in classifying nodules with sonographically evident HT (both P = .012). CONCLUSION The CAD system has comparable sensitivity, but lower specificity compared with the experienced radiologist in diagnosing thyroid malignancies concurrent with HT. For a radiologist with less experience, the CAD system can help improve the diagnostic performance by increasing sensitivity and accuracy in assessing thyroid nodules with diffusely altered parenchyma.
Collapse
Affiliation(s)
- Minxia Hu
- Department of Diagnostic Ultrasound, Beijing Tongren Hospital, Capital Medical University, Beijing, PR China
| | - Suting Zong
- Department of Diagnostic Ultrasound, Beijing Tongren Hospital, Capital Medical University, Beijing, PR China
| | - Ning Xu
- Department of Diagnostic Ultrasound, Beijing Tongren Hospital, Capital Medical University, Beijing, PR China
| | - Jinzhen Li
- Department of Diagnostic Ultrasound, Beijing Tongren Hospital, Capital Medical University, Beijing, PR China
| | - Chunxia Xia
- Department of Diagnostic Ultrasound, Beijing Tongren Hospital, Capital Medical University, Beijing, PR China
| | - Fengxia Yu
- Department of Diagnostic Ultrasound, Beijing Tongren Hospital, Capital Medical University, Beijing, PR China
| | - Qiang Zhu
- Department of Diagnostic Ultrasound, Beijing Tongren Hospital, Capital Medical University, Beijing, PR China
| | - Hanxue Zhao
- Department of Diagnostic Ultrasound, Beijing Tongren Hospital, Capital Medical University, Beijing, PR China
| |
Collapse
|
15
|
Cleere EF, Davey MG, O’Neill S, Corbett M, O’Donnell JP, Hacking S, Keogh IJ, Lowery AJ, Kerin MJ. Radiomic Detection of Malignancy within Thyroid Nodules Using Ultrasonography-A Systematic Review and Meta-Analysis. Diagnostics (Basel) 2022; 12:diagnostics12040794. [PMID: 35453841 PMCID: PMC9027085 DOI: 10.3390/diagnostics12040794] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2022] [Revised: 03/22/2022] [Accepted: 03/22/2022] [Indexed: 02/04/2023] Open
Abstract
Background: Despite investigation, 95% of thyroid nodules are ultimately benign. Radiomics is a field that uses radiological features to inform individualized patient care. We aimed to evaluate the diagnostic utility of radiomics in classifying undetermined thyroid nodules into benign and malignant using ultrasonography (US). Methods: A diagnostic test accuracy systematic review and meta-analysis was performed in accordance with PRISMA guidelines. Sensitivity, specificity, and area under curve (AUC) delineating benign and malignant lesions were recorded. Results: Seventy-five studies including 26,373 patients and 46,175 thyroid nodules met inclusion criteria. Males accounted for 24.6% of patients, while 75.4% of patients were female. Radiomics provided a pooled sensitivity of 0.87 (95% CI: 0.86−0.87) and a pooled specificity of 0.84 (95% CI: 0.84−0.85) for characterizing benign and malignant lesions. Using convolutional neural network (CNN) methods, pooled sensitivity was 0.85 (95% CI: 0.84−0.86) and pooled specificity was 0.82 (95% CI: 0.82−0.83); significantly lower than studies using non-CNN: sensitivity 0.90 (95% CI: 0.89−0.90) and specificity 0.88 (95% CI: 0.87−0.89) (p < 0.05). The diagnostic ability of radiologists and radiomics were comparable for both sensitivity (OR 0.98) and specificity (OR 0.95). Conclusions: Radiomic analysis using US provides a reproducible, reliable evaluation of undetermined thyroid nodules when compared to current best practice.
Collapse
Affiliation(s)
- Eoin F. Cleere
- The Lambe Institute for Translational Research, National University of Ireland, H91 YR71 Galway, Ireland; (M.G.D.); (A.J.L.); (M.J.K.)
- Department of Otolaryngology, Galway University Hospitals, H91 YR71 Galway, Ireland; (M.C.); (I.J.K.)
- Correspondence:
| | - Matthew G. Davey
- The Lambe Institute for Translational Research, National University of Ireland, H91 YR71 Galway, Ireland; (M.G.D.); (A.J.L.); (M.J.K.)
| | - Shane O’Neill
- Department of Breast and Endocrine Surgery, Galway University Hospitals, H91 YR71 Galway, Ireland;
| | - Mel Corbett
- Department of Otolaryngology, Galway University Hospitals, H91 YR71 Galway, Ireland; (M.C.); (I.J.K.)
| | - John P O’Donnell
- Department of Radiology, Galway University Hospitals, H91 YR71 Galway, Ireland;
| | - Sean Hacking
- Department of Pathology and Laboratory Medicine, Warren Alpert Medical School of Brown University, Providence, RI 02903, USA;
| | - Ivan J. Keogh
- Department of Otolaryngology, Galway University Hospitals, H91 YR71 Galway, Ireland; (M.C.); (I.J.K.)
| | - Aoife J. Lowery
- The Lambe Institute for Translational Research, National University of Ireland, H91 YR71 Galway, Ireland; (M.G.D.); (A.J.L.); (M.J.K.)
- Department of Breast and Endocrine Surgery, Galway University Hospitals, H91 YR71 Galway, Ireland;
| | - Michael J. Kerin
- The Lambe Institute for Translational Research, National University of Ireland, H91 YR71 Galway, Ireland; (M.G.D.); (A.J.L.); (M.J.K.)
- Department of Breast and Endocrine Surgery, Galway University Hospitals, H91 YR71 Galway, Ireland;
| |
Collapse
|
16
|
Lu J, Ouyang X, Shen X, Liu T, Cui Z, Wang Q, Shen D. GAN-guided Deformable Attention Network for Identifying Thyroid Nodules in Ultrasound Images. IEEE J Biomed Health Inform 2022; 26:1582-1590. [PMID: 35196250 DOI: 10.1109/jbhi.2022.3153559] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Early detection and identification of malignant thyroid nodules, a vital precursory to the treatment, is a difficult task even for experienced clinicians. Many Computer-Aided Diagnose (CAD) systems have been developed to assist clinicians in performing this task on ultrasonic images. Learning-based CAD systems for thyroid nodules generally accommodate both nodule detection/segmentation and fine-grained classification for its malignancy, and prior researches often treat aforementioned tasks in separate stages, leading to additional computational costs. In this paper, we utilize an online class activation mapping (CAM) mechanism to guide the network to learn discriminative features for identifying thyroid nodules in ultrasound images, called \textit{CAM attention network}. It takes nodule masks as localization cues for direct spatial attention of the classification module, thereby avoiding isolated training for classification. Meanwhile, we propose a deformable convolution module to add offsets to the regular grid sampling locations in the standard convolution, guiding the network to capture more discriminative features of nodule areas. Furthermore, we use a generative adversarial network (GAN) [1] to ensure reliable deformations of nodules from the deformable convolution module. Our proposed CAM attention network has already achieved the 2nd place in the classification task of TN-SCUI 2020, a MICCAI 2020 Challenge with the largest set of thyroid nodule ultrasound images according to our knowledge. The further inclusion of our proposed GAN-guided deformable module allows for capturing more fine-grained features between benign and malignant nodules, and further improves the classification accuracy to a new state-of-the-art level.
Collapse
|
17
|
Liang X, Huang Y, Cai Y, Liao J, Chen Z. A Computer-Aided Diagnosis System and Thyroid Imaging Reporting and Data System for Dual Validation of Ultrasound-Guided Fine-Needle Aspiration of Indeterminate Thyroid Nodules. Front Oncol 2021; 11:611436. [PMID: 34692466 PMCID: PMC8529148 DOI: 10.3389/fonc.2021.611436] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2020] [Accepted: 09/16/2021] [Indexed: 12/02/2022] Open
Abstract
Purpose The fully automatic AI-Sonic computer-aided design (CAD) system was employed for the detection and diagnosis of benign and malignant thyroid nodules. The aim of this study was to investigate the efficiency of the AI-Sonic CAD system with the use of a deep learning algorithm to improve the diagnostic accuracy of ultrasound-guided fine-needle aspiration (FNA). Methods A total of 138 thyroid nodules were collected from 124 patients and diagnosed by an expert, a novice, and the Thyroid Imaging Reporting and Data System (TI-RADS). Diagnostic efficiency and feasibility were compared among the expert, novice, and CAD system. The application of the CAD system to enhance the diagnostic efficiency of novices was assessed. Moreover, with the experience of the expert as the gold standard, the values of features detected by the CAD system were also analyzed. The efficiency of FNA was compared among the expert, novice, and CAD system to determine whether the CAD system is helpful for the management of FNA. Result In total, 56 malignant and 82 benign thyroid nodules were collected from the 124 patients (mean age, 46.4 ± 12.1 years; range, 12–70 years). The diagnostic area under the curve of the CAD system, expert, and novice were 0.919, 0.891, and 0.877, respectively (p < 0.05). In regard to feature detection, there was no significant differences in the margin and composition between the benign and malignant nodules (p > 0.05), while echogenicity and the existence of echogenic foci were of great significance (p < 0.05). For the recommendation of FNA, the results showed that the CAD system had better performance than the expert and novice (p < 0.05). Conclusions Precise diagnosis and recommendation of FNA are continuing hot topics for thyroid nodules. The CAD system based on deep learning had better accuracy and feasibility for the diagnosis of thyroid nodules, and was useful to avoid unnecessary FNA. The CAD system is potentially an effective auxiliary approach for diagnosis and asymptomatic screening, especially in developing areas.
Collapse
Affiliation(s)
- Xiaowen Liang
- Department of Ultrasound Medicine, The Third Affiliated Hospital of Guangzhou Medical University, Guangzhou, China
| | - Yingmin Huang
- Department of Ultrasound Medicine, The Third Affiliated Hospital of Guangzhou Medical University, Guangzhou, China
| | - Yongyi Cai
- Department of Ultrasound, Liwan Center Hospital of Guangzhou, Guangzhou, China
| | - Jianyi Liao
- Department of Ultrasound Medicine, The Third Affiliated Hospital of Guangzhou Medical University, Guangzhou, China
| | - Zhiyi Chen
- Department of Ultrasound Medicine, The Third Affiliated Hospital of Guangzhou Medical University, Guangzhou, China.,The First Affiliated Hospital, Medical Imaging Centre, Hengyang Medical School, University of South China, Hengyang, China
| |
Collapse
|
18
|
Diagnosing thyroid nodules with atypia of undetermined significance/follicular lesion of undetermined significance cytology with the deep convolutional neural network. Sci Rep 2021; 11:20048. [PMID: 34625636 PMCID: PMC8501016 DOI: 10.1038/s41598-021-99622-0] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2021] [Accepted: 09/24/2021] [Indexed: 11/08/2022] Open
Abstract
To compare the diagnostic performances of physicians and a deep convolutional neural network (CNN) predicting malignancy with ultrasonography images of thyroid nodules with atypia of undetermined significance (AUS)/follicular lesion of undetermined significance (FLUS) results on fine-needle aspiration (FNA). This study included 202 patients with 202 nodules ≥ 1 cm AUS/FLUS on FNA, and underwent surgery in one of 3 different institutions. Diagnostic performances were compared between 8 physicians (4 radiologists, 4 endocrinologists) with varying experience levels and CNN, and AUS/FLUS subgroups were analyzed. Interobserver variability was assessed among the 8 physicians. Of the 202 nodules, 158 were AUS, and 44 were FLUS; 86 were benign, and 116 were malignant. The area under the curves (AUCs) of the 8 physicians and CNN were 0.680–0.722 and 0.666, without significant differences (P > 0.05). In the subgroup analysis, the AUCs for the 8 physicians and CNN were 0.657–0.768 and 0.652 for AUS, 0.469–0.674 and 0.622 for FLUS. Interobserver agreements were moderate (k = 0.543), substantial (k = 0.652), and moderate (k = 0.455) among the 8 physicians, 4 radiologists, and 4 endocrinologists. For thyroid nodules with AUS/FLUS cytology, the diagnostic performance of CNN to differentiate malignancy with US images was comparable to that of physicians with variable experience levels.
Collapse
|
19
|
Guang Y, He W, Ning B, Zhang H, Yin C, Zhao M, Nie F, Huang P, Zhang RF, Yong Q, Guo Y, Yuan J, Wang Y, Yuan L, Ruan L, Yu T, Song H, Zhang Y. Deep learning-based carotid plaque vulnerability classification with multicentre contrast-enhanced ultrasound video: a comparative diagnostic study. BMJ Open 2021; 11:e047528. [PMID: 34452961 PMCID: PMC8404444 DOI: 10.1136/bmjopen-2020-047528] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/20/2022] Open
Abstract
OBJECTIVES The aim of this study was to evaluate the performance of deep learning-based detection and classification of carotid plaque (DL-DCCP) in carotid plaque contrast-enhanced ultrasound (CEUS). METHODS AND ANALYSIS A prospective multicentre study was conducted to assess vulnerability in patients with carotid plaque. Data from 547 potentially eligible patients were prospectively enrolled from 10 hospitals, and 205 patients with CEUS video were finally enrolled for analysis. The area under the receiver operating characteristic curve (AUC) was used to evaluate the effectiveness of DL-DCCP and two experienced radiologists who manually examined the CEUS video (RA-CEUS) in diagnosing and classifying carotid plaque vulnerability. To evaluate the influence of dynamic video input on the performance of the algorithm, a state-of-the-art deep convolutional neural network (CNN) model for static images (Xception) was compared with DL-DCCP for both training and holdout validation cohorts. RESULTS The AUCs of DL-DCCP were significantly better than those of the experienced radiologists for both the training and holdout validation cohorts (training, DL-DCCP vs RA-CEUS, AUC: 0.85 vs 0.69, p<0.01; holdout validation, DL-DCCP vs RA-CEUS, AUC: 0.87 vs 0.66, p<0.01), that is, also better than the best deep CNN model Xception we had performed, for both the training and holdout validation cohorts (training, DL-DCCP vs Xception, AUC:0.85 vs 0.82, p<0.01; holdout validation, DL-DCCP vs Xception, AUC: 0.87 vs 0.77, p<0.01). CONCLUSION DL-DCCP shows better overall performance in assessing the vulnerability of carotid atherosclerotic plaques than RA-CEUS. Moreover, with a more powerful network structure and better utilisation of video information, DL-DCCP provided greater diagnostic accuracy than a state-of-the-art static CNN model. TRIAL REGISTRATION NUMBER ChiCTR1900021846.
Collapse
Affiliation(s)
- Yang Guang
- Department of Ultrasound, Beijing Tiantan Hospital, Beijing, China
| | - Wen He
- Department of Ultrasound, Beijing Tiantan Hospital, Beijing, China
| | - Bin Ning
- Department of Ultrasound, Beijing Tiantan Hospital, Beijing, China
| | - Hongxia Zhang
- Department of Ultrasound, Beijing Tiantan Hospital, Beijing, China
| | - Chen Yin
- Department of R&D, CHISON Medical Technologies Co Ltd, Wuxi, China
| | - Mingchang Zhao
- Department of R&D, CHISON Medical Technologies Co Ltd, Wuxi, China
| | - Fang Nie
- Department of Ultrasound, Lanzhou University Second Hospital, Lanzhou, Gansu, China
| | - Pintong Huang
- Department of Ultrasound, Zhejiang University School of Medicine Second Affiliated Hospital, Hangzhou, Zhejiang, China
| | - Rui-Fang Zhang
- Department of Ultrasound, Zhengzhou University First Affiliated Hospital, Zhengzhou, Henan, China
| | - Qiang Yong
- Department of Ultrasound, Beijing An Zhen Hospital, Chaoyang-qu, Beijing, China
| | - Yanli Guo
- Department of Ultrasound, Third Military Medical University Southwest Hospital, Chongqing, China
| | - Jianjun Yuan
- Department of Ultrasound, Henan Provincial People's Hospital, Zhengzhou, Henan, China
| | - Yicheng Wang
- Department of Ultrasound, Hebei North University Basic Medical College, Zhangjiakou, Hebei, China
| | - Lijun Yuan
- Department of Ultrasound, Tangdu Hospital Fourth Military Medical University, Xi'an, Shaanxi, China
| | - Litao Ruan
- Department of Ultrasound, Xi'an Jiaotong University Medical College First Affiliated Hospital, Xi'an, Shaanxi, China
| | - Tengfei Yu
- Department of Ultrasound, Beijing Tiantan Hospital, Beijing, China
| | - Haiman Song
- Department of Ultrasound, Beijing Tiantan Hospital, Beijing, China
| | - Yukang Zhang
- Department of Ultrasound, Beijing Tiantan Hospital, Beijing, China
| |
Collapse
|
20
|
Xia X, Feng B, Wang J, Hua Q, Yang Y, Sheng L, Mou Y, Hu W. Deep Learning for Differentiating Benign From Malignant Parotid Lesions on MR Images. Front Oncol 2021; 11:632104. [PMID: 34249680 PMCID: PMC8262843 DOI: 10.3389/fonc.2021.632104] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2020] [Accepted: 06/07/2021] [Indexed: 12/29/2022] Open
Abstract
Purpose/Objectives(s) Salivary gland tumors are a rare, histologically heterogeneous group of tumors. The distinction between malignant and benign tumors of the parotid gland is clinically important. This study aims to develop and evaluate a deep-learning network for diagnosing parotid gland tumors via the deep learning of MR images. Materials/Methods Two hundred thirty-three patients with parotid gland tumors were enrolled in this study. Histology results were available for all tumors. All patients underwent MRI scans, including T1-weighted, CE-T1-weighted and T2-weighted imaging series. The parotid glands and tumors were segmented on all three MR image series by a radiologist with 10 years of clinical experience. A total of 3791 parotid gland region images were cropped from the MR images. A label (pleomorphic adenoma and Warthin tumor, malignant tumor or free of tumor), which was based on histology results, was assigned to each image. To train the deep-learning model, these data were randomly divided into a training dataset (90%, comprising 3035 MR images from 212 patients: 714 pleomorphic adenoma images, 558 Warthin tumor images, 861 malignant tumor images, and 902 images free of tumor) and a validation dataset (10%, comprising 275 images from 21 patients: 57 pleomorphic adenoma images, 36 Warthin tumor images, 93 malignant tumor images, and 89 images free of tumor). A modified ResNet model was developed to classify these images. The input images were resized to 224x224 pixels, including four channels (T1-weighted tumor images only, T2-weighted tumor images only, CE-T1-weighted tumor images only and parotid gland images). Random image flipping and contrast adjustment were used for data enhancement. The model was trained for 1200 epochs with a learning rate of 1e-6, and the Adam optimizer was implemented. It took approximately 2 hours to complete the whole training procedure. The whole program was developed with PyTorch (version 1.2). Results The model accuracy with the training dataset was 92.94% (95% CI [0.91, 0.93]). The micro-AUC was 0.98. The experimental results showed that the accuracy of the final algorithm in the diagnosis and staging of parotid cancer was 82.18% (95% CI [0.77, 0.86]). The micro-AUC was 0.93. Conclusion The proposed model may be used to assist clinicians in the diagnosis of parotid tumors. However, future larger-scale multicenter studies are required for full validation.
Collapse
Affiliation(s)
- Xianwu Xia
- Department of Radiation Oncology, Fudan University Shanghai Cancer Center, Shanghai, China.,Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, China.,Department of Oncology Intervention, The Affiliated Municipal Hospital of Taizhou University, Taizhou, China
| | - Bin Feng
- Department of Radiation Oncology, Fudan University Shanghai Cancer Center, Shanghai, China.,Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, China
| | - Jiazhou Wang
- Department of Radiation Oncology, Fudan University Shanghai Cancer Center, Shanghai, China.,Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, China
| | - Qianjin Hua
- Department of Oncology Intervention, The Affiliated Municipal Hospital of Taizhou University, Taizhou, China
| | - Yide Yang
- Department of Infectious Disease, The Affiliated Municipal Hospital of Taizhou University, Taizhou, China
| | - Liang Sheng
- Department of Radiology, The Affiliated Municipal Hospital of Taizhou University, Taizhou, China
| | - Yonghua Mou
- Department of Hepatobiliary Surgery, The Affiliated Municipal Hospital of Taizhou University, Taizhou, China
| | - Weigang Hu
- Department of Radiation Oncology, Fudan University Shanghai Cancer Center, Shanghai, China.,Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, China
| |
Collapse
|
21
|
Vadhiraj VV, Simpkin A, O’Connell J, Singh Ospina N, Maraka S, O’Keeffe DT. Ultrasound Image Classification of Thyroid Nodules Using Machine Learning Techniques. MEDICINA (KAUNAS, LITHUANIA) 2021; 57:527. [PMID: 34074037 PMCID: PMC8225215 DOI: 10.3390/medicina57060527] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/26/2021] [Revised: 05/10/2021] [Accepted: 05/18/2021] [Indexed: 02/07/2023]
Abstract
Background and Objectives: Thyroid nodules are lumps of solid or liquid-filled tumors that form inside the thyroid gland, which can be malignant or benign. Our aim was to test whether the described features of the Thyroid Imaging Reporting and Data System (TI-RADS) could improve radiologists' decision making when integrated into a computer system. In this study, we developed a computer-aided diagnosis system integrated into multiple-instance learning (MIL) that would focus on benign-malignant classification. Data were available from the Universidad Nacional de Colombia. Materials and Methods: There were 99 cases (33 Benign and 66 malignant). In this study, the median filter and image binarization were used for image pre-processing and segmentation. The grey level co-occurrence matrix (GLCM) was used to extract seven ultrasound image features. These data were divided into 87% training and 13% validation sets. We compared the support vector machine (SVM) and artificial neural network (ANN) classification algorithms based on their accuracy score, sensitivity, and specificity. The outcome measure was whether the thyroid nodule was benign or malignant. We also developed a graphic user interface (GUI) to display the image features that would help radiologists with decision making. Results: ANN and SVM achieved an accuracy of 75% and 96% respectively. SVM outperformed all the other models on all performance metrics, achieving higher accuracy, sensitivity, and specificity score. Conclusions: Our study suggests promising results from MIL in thyroid cancer detection. Further testing with external data is required before our classification model can be employed in practice.
Collapse
Affiliation(s)
- Vijay Vyas Vadhiraj
- School of Medicine, College of Medicine Nursing and Health Sciences, National University of Ireland Galway, H91 TK33 Galway, Ireland; (J.O.); (D.T.O.)
- Health Innovation Via Engineering Laboratory, Cúram SFI Research Centre for Medical Devices, Lambe Institute for Translational Research, National University of Ireland Galway, H91 TK33 Galway, Ireland
| | - Andrew Simpkin
- School of Mathematics, Statistics and Applied Maths, National University of Ireland, H91 TK33 Galway, Ireland;
| | - James O’Connell
- School of Medicine, College of Medicine Nursing and Health Sciences, National University of Ireland Galway, H91 TK33 Galway, Ireland; (J.O.); (D.T.O.)
- Health Innovation Via Engineering Laboratory, Cúram SFI Research Centre for Medical Devices, Lambe Institute for Translational Research, National University of Ireland Galway, H91 TK33 Galway, Ireland
| | - Naykky Singh Ospina
- Division of Endocrinology, Department of Medicine, University of Florida, Gainesville, FL 3210, USA;
| | - Spyridoula Maraka
- Division of Endocrinology and Metabolism, University of Arkansas for Medical Sciences, Little Rock, AR 72205, USA;
- Medicine Section, Central Arkansas Veterans Healthcare System, Little Rock, AR 72205, USA
| | - Derek T. O’Keeffe
- School of Medicine, College of Medicine Nursing and Health Sciences, National University of Ireland Galway, H91 TK33 Galway, Ireland; (J.O.); (D.T.O.)
- Health Innovation Via Engineering Laboratory, Cúram SFI Research Centre for Medical Devices, Lambe Institute for Translational Research, National University of Ireland Galway, H91 TK33 Galway, Ireland
- Lero, SFI Centre for Software Research, National University of Ireland Galway, H91 TK33 Galway, Ireland
| |
Collapse
|
22
|
Wu GG, Lv WZ, Yin R, Xu JW, Yan YJ, Chen RX, Wang JY, Zhang B, Cui XW, Dietrich CF. Deep Learning Based on ACR TI-RADS Can Improve the Differential Diagnosis of Thyroid Nodules. Front Oncol 2021; 11:575166. [PMID: 33987082 PMCID: PMC8111071 DOI: 10.3389/fonc.2021.575166] [Citation(s) in RCA: 24] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2020] [Accepted: 04/07/2021] [Indexed: 12/12/2022] Open
Abstract
Objective The purpose of this study was to improve the differentiation between malignant and benign thyroid nodules using deep learning (DL) in category 4 and 5 based on the Thyroid Imaging Reporting and Data System (TI-RADS, TR) from the American College of Radiology (ACR). Design and Methods From June 2, 2017 to April 23, 2019, 2082 thyroid ultrasound images from 1396 consecutive patients with confirmed pathology were retrospectively collected, of which 1289 nodules were category 4 (TR4) and 793 nodules were category 5 (TR5). Ninety percent of the B-mode ultrasound images were applied for training and validation, and the residual 10% and an independent external dataset for testing purpose by three different deep learning algorithms. Results In the independent test set, the DL algorithm of best performance got an AUC of 0.904, 0.845, 0.829 in TR4, TR5, and TR4&5, respectively. The sensitivity and specificity of the optimal model was 0.829, 0.831 on TR4, 0.846, 0.778 on TR5, 0.790, 0.779 on TR4&5, versus the radiologists of 0.686 (P=0.108), 0.766 (P=0.101), 0.677 (P=0.211), 0.750 (P=0.128), and 0.680 (P=0.023), 0.761 (P=0.530), respectively. Conclusions The study demonstrated that DL could improve the differentiation of malignant from benign thyroid nodules and had significant potential for clinical application on TR4 and TR5.
Collapse
Affiliation(s)
- Ge-Ge Wu
- Sino-German Tongji-Caritas Research Center of Ultrasound in Medicine, Department of Medical Ultrasound, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Wen-Zhi Lv
- Department of Artificial Intelligence, Julei Technology Company, Wuhan, China
| | - Rui Yin
- Department of Ultrasound, Affiliated Renhe Hospital of China Three Gorges University, Yichang, China
| | - Jian-Wei Xu
- Department of Ultrasound, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China
| | - Yu-Jing Yan
- Sino-German Tongji-Caritas Research Center of Ultrasound in Medicine, Department of Medical Ultrasound, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Rui-Xue Chen
- Department of Ultrasound, Wuchang Hospital, Wuhan University of Science and Technology, Wuhan, China
| | - Jia-Yu Wang
- Sino-German Tongji-Caritas Research Center of Ultrasound in Medicine, Department of Medical Ultrasound, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Bo Zhang
- Department of Ultrasonic Imaging, Xiangya Hospital, Central South University, Changsha, China
| | - Xin-Wu Cui
- Sino-German Tongji-Caritas Research Center of Ultrasound in Medicine, Department of Medical Ultrasound, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Christoph F Dietrich
- Department of General Internal Medicine, Kliniken Hirslanden Beau-Site, Bern, Switzerland
| |
Collapse
|
23
|
Zhu J, Zhang S, Yu R, Liu Z, Gao H, Yue B, Liu X, Zheng X, Gao M, Wei X. An efficient deep convolutional neural network model for visual localization and automatic diagnosis of thyroid nodules on ultrasound images. Quant Imaging Med Surg 2021; 11:1368-1380. [PMID: 33816175 DOI: 10.21037/qims-20-538] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/04/2023]
Abstract
Background The aim of this study was to construct a deep convolutional neural network (CNN) model for localization and diagnosis of thyroid nodules on ultrasound and evaluate its diagnostic performance. Methods We developed and trained a deep CNN model called the Brief Efficient Thyroid Network (BETNET) using 16,401 ultrasound images. According to the parameters of the model, we developed a computer-aided diagnosis (CAD) system to localize and differentiate thyroid nodules. The validation dataset (1,000 images) was used to compare the diagnostic performance of the model using three state-of-the-art algorithms. We used an internal test set (300 images) to evaluate the BETNET model by comparing it with diagnoses from five radiologists with varying degrees of experience in thyroid nodule diagnosis. Lastly, we demonstrated the general applicability of our artificial intelligence (AI) system for diagnosing thyroid cancer in an external test set (1,032 images). Results The BETNET model accurately detected thyroid nodules in visualization experiments. The model demonstrated higher values for area under the receiver operating characteristic (AUC-ROC) curve [0.983, 95% confidence interval (CI): 0.973-0.990], sensitivity (99.19%), accuracy (98.30%), and Youden index (0.9663) than the three state-of-the-art algorithms (P<0.05). In the internal test dataset, the diagnostic accuracy of the BETNET model was 91.33%, which was markedly higher than the accuracy of one experienced (85.67%) and two less experienced radiologists (77.67% and 69.33%). The area under the ROC curve of the BETNET model (0.951) was similar to that of the two highly skilled radiologists (0.940 and 0.953) and significantly higher than that of one experienced and two less experienced radiologists (P<0.01). The kappa coefficient of the BETNET model and the pathology results showed good agreement (0.769). In addition, the BETNET model achieved an excellent diagnostic performance (AUC =0.970, 95% CI: 0.958-0.980) when applied to ultrasound images from another independent hospital. Conclusions We developed a deep learning model which could accurately locate and automatically diagnose thyroid nodules on ultrasound images. The BETNET model exhibited better diagnostic performance than three state-of-the-art algorithms, which in turn performed similarly in diagnosis as the experienced radiologists. The BETNET model has the potential to be applied to ultrasound images from other hospitals.
Collapse
Affiliation(s)
- Jialin Zhu
- Department of Diagnostic and Therapeutic Ultrasonography, Tianjin Medical University Cancer Institute and Hospital, National Clinical Research Center of Cancer, Key Laboratory of Cancer Prevention and Therapy, Tianjin's Clinical Research Center for Cancer, Tianjin, China
| | - Sheng Zhang
- Department of Diagnostic and Therapeutic Ultrasonography, Tianjin Medical University Cancer Institute and Hospital, National Clinical Research Center of Cancer, Key Laboratory of Cancer Prevention and Therapy, Tianjin's Clinical Research Center for Cancer, Tianjin, China
| | - Ruiguo Yu
- College of Intelligence and Computing, Tianjin University, Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin Key Laboratory of Advanced Networking, Tianjin, China
| | - Zhiqiang Liu
- College of Intelligence and Computing, Tianjin University, Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin Key Laboratory of Advanced Networking, Tianjin, China
| | - Hongyan Gao
- Tianjin Xiqing District Women and Children's Health and Family Planning Service Center, Tianjin, China
| | - Bing Yue
- Department of Diagnostic and Therapeutic Ultrasonography, Tianjin Medical University Cancer Institute and Hospital, National Clinical Research Center of Cancer, Key Laboratory of Cancer Prevention and Therapy, Tianjin's Clinical Research Center for Cancer, Tianjin, China
| | - Xun Liu
- Department of Ultrasonography, the Fifth Central Hospital of Tianjin, Tianjin, China
| | - Xiangqian Zheng
- Department of Thyroid and Neck Tumor, Tianjin Medical University Cancer Institute and Hospital, National Clinical Research Center of Cancer, Key Laboratory of Cancer Prevention and Therapy, Tianjin's Clinical Research Center for Cancer, Tianjin, China
| | - Ming Gao
- Department of Thyroid and Neck Tumor, Tianjin Medical University Cancer Institute and Hospital, National Clinical Research Center of Cancer, Key Laboratory of Cancer Prevention and Therapy, Tianjin's Clinical Research Center for Cancer, Tianjin, China
| | - Xi Wei
- Department of Diagnostic and Therapeutic Ultrasonography, Tianjin Medical University Cancer Institute and Hospital, National Clinical Research Center of Cancer, Key Laboratory of Cancer Prevention and Therapy, Tianjin's Clinical Research Center for Cancer, Tianjin, China
| |
Collapse
|
24
|
Hou Y, Chen C, Zhang L, Zhou W, Lu Q, Jia X, Zhang J, Guo C, Qin Y, Zhu L, Zuo M, Xiao J, Huang L, Zhan W. Using Deep Neural Network to Diagnose Thyroid Nodules on Ultrasound in Patients With Hashimoto's Thyroiditis. Front Oncol 2021; 11:614172. [PMID: 33796455 PMCID: PMC8008116 DOI: 10.3389/fonc.2021.614172] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2020] [Accepted: 01/28/2021] [Indexed: 01/08/2023] Open
Abstract
Objective The aim of this study is to develop a model using Deep Neural Network (DNN) to diagnose thyroid nodules in patients with Hashimoto’s Thyroiditis. Methods In this retrospective study, we included 2,932 patients with thyroid nodules who underwent thyroid ultrasonogram in our hospital from January 2017 to August 2019. 80% of them were included as training set and 20% as test set. Nodules suspected for malignancy underwent FNA or surgery for pathological results. Two DNN models were trained to diagnose thyroid nodules, and we chose the one with better performance. The features of nodules as well as parenchyma around nodules will be learned by the model to achieve better performance under diffused parenchyma. 10-fold cross-validation and an independent test set were used to evaluate the performance of the algorithm. The performance of the model was compared with that of the three groups of radiologists with clinical experience of <5 years, 5–10 years, >10 years respectively. Results In total, 9,127 images were collected from 2,932 patients with 7,301 images for the training set and 1,806 for the test set. 56% of the patients enrolled had Hashimoto’s Thyroiditis. The model achieved an AUC of 0.924 for distinguishing malignant and benign nodules in the test set. It showed similar performance under diffused thyroid parenchyma and normal parenchyma with sensitivity of 0.881 versus 0.871 (p = 0.938) and specificity of 0.846 versus 0.822 (p = 0.178). In patients with HT, the model achieved an AUC of 0.924 to differentiate malignant and benign nodules which was significantly higher than that of the three groups of radiologists (AUC = 0.824, 0.857, 0.863 respectively, p < 0.05). Conclusion The model showed high performance in diagnosing thyroid nodules under both normal and diffused parenchyma. In patients with Hashimoto’s Thyroiditis, the model showed a better performance compared to radiologists with various years of experience.
Collapse
Affiliation(s)
- Yiqing Hou
- Department of Ultrasound Diagnosis, Ruijin Hospital Affiliated to Shanghai Jiaotong University, Shanghai, China
| | - Chao Chen
- Ping An Technology (Shenzhen) Co., Ltd., Shenzhen, China
| | - Lu Zhang
- Department of Ultrasound Diagnosis, Ruijin Hospital Affiliated to Shanghai Jiaotong University, Shanghai, China
| | - Wei Zhou
- Department of Ultrasound Diagnosis, Ruijin Hospital Affiliated to Shanghai Jiaotong University, Shanghai, China
| | - Qinyang Lu
- Ping An Technology (Shenzhen) Co., Ltd., Shenzhen, China
| | - Xiaohong Jia
- Department of Ultrasound Diagnosis, Ruijin Hospital Affiliated to Shanghai Jiaotong University, Shanghai, China
| | - Jingwen Zhang
- Department of Ultrasound Diagnosis, Ruijin Hospital Affiliated to Shanghai Jiaotong University, Shanghai, China
| | - Cen Guo
- Ping An Technology (Shenzhen) Co., Ltd., Shenzhen, China
| | - Yuxiang Qin
- Ping An Technology (Shenzhen) Co., Ltd., Shenzhen, China
| | - Lifeng Zhu
- Computer Centre, Ruijin Hospital Affiliated to Shanghai Jiaotong University, Shanghai, China
| | - Ming Zuo
- Computer Centre, Ruijin Hospital Affiliated to Shanghai Jiaotong University, Shanghai, China
| | - Jing Xiao
- Ping An Technology (Shenzhen) Co., Ltd., Shenzhen, China
| | - Lingyun Huang
- Ping An Technology (Shenzhen) Co., Ltd., Shenzhen, China
| | - Weiwei Zhan
- Department of Ultrasound Diagnosis, Ruijin Hospital Affiliated to Shanghai Jiaotong University, Shanghai, China
| |
Collapse
|
25
|
Ye FY, Lyu GR, Li SQ, You JH, Wang KJ, Cai ML, Su QC. Diagnostic Performance of Ultrasound Computer-Aided Diagnosis Software Compared with That of Radiologists with Different Levels of Expertise for Thyroid Malignancy: A Multicenter Prospective Study. ULTRASOUND IN MEDICINE & BIOLOGY 2021; 47:114-124. [PMID: 33239154 DOI: 10.1016/j.ultrasmedbio.2020.09.019] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/26/2020] [Revised: 09/19/2020] [Accepted: 09/22/2020] [Indexed: 06/11/2023]
Abstract
The aim of the work described here was to evaluate the diagnostic performance of ultrasound thyroid computer-aided diagnosis (CAD) software. This multicenter prospective study included 494 patients (565 thyroid nodules) who underwent surgery or biopsy after ultrasonography at four hospitals from January 2019 to September 2019. The diagnostic performance metrics of different readers were calculated and compared with the pathologic results. The sensitivity of CAD was outstanding and was equivalent to that of a senior radiologist (90.51% vs. 88.47%, p > 0.05). The area under the curve of CAD was equivalent to that of a junior radiologist (0.748 vs. 0.739, p > 0.05). However, the specificity was only 49.63%, which was lower than those of the three radiologists (75.56%, 85.93% and 90.37% for the junior, intermediate and senior radiologists, respectively). The diagnostic performance of the junior radiologist was significantly improved with the aid of CAD (junior + CAD). The sensitivity and area under the curve of junior + CAD were improved from 72.20% to 89.93% and from 0.739 to 0.816, respectively (both p values <0.05), and the positive predictive value, negative predictive value and κ coefficient improved from 76.3% to 78.6%, 82.0% to 86.8% and 0.394 to 0.511, respectively. Though specificity slightly decreased from 75.56% to 73.33%, the difference was not statistically significant (p > 0.05). In general, the clinical application value of CAD is promising, and its instrumental value for junior radiologists is significant.
Collapse
Affiliation(s)
- Feng-Ying Ye
- Department of Ultrasound, Second Affiliated Hospital of Fujian Medical University, Quanzhou, China
| | - Guo-Rong Lyu
- Department of Clinical Medicine, Quanzhou Medical College, Quanzhou, China.
| | - Shang-Qing Li
- Department of Clinical Medicine, Quanzhou Medical College, Quanzhou, China
| | - Jian-Hong You
- Department of Ultrasound, Zhongshan Hospital Affiliated to Xiamen University, Xiamen, China
| | - Kang-Jian Wang
- Department of Ultrasound, Zhangzhou Affiliated Hospital of Fujian Medical University, Zhangzhou, China
| | - Ming-Li Cai
- Department of Ultrasound, Jinjiang City Hospital, Jinjiang, China
| | - Qi-Chen Su
- Department of Ultrasound, Second Affiliated Hospital of Fujian Medical University, Quanzhou, China
| |
Collapse
|
26
|
Chung SR, Baek JH, Lee MK, Ahn Y, Choi YJ, Sung TY, Song DE, Kim TY, Lee JH. Computer-Aided Diagnosis System for the Evaluation of Thyroid Nodules on Ultrasonography: Prospective Non-Inferiority Study according to the Experience Level of Radiologists. Korean J Radiol 2020; 21:369-376. [PMID: 32090529 PMCID: PMC7039724 DOI: 10.3348/kjr.2019.0581] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2019] [Accepted: 11/04/2019] [Indexed: 12/27/2022] Open
Abstract
Objective To determine whether a computer-aided diagnosis (CAD) system for the evaluation of thyroid nodules is non-inferior to radiologists with different levels of experience. Materials and Methods Patients with thyroid nodules with a decisive diagnosis of benign or malignant nodule were consecutively enrolled from November 2017 to September 2018. Three radiologists with different levels of experience (1 month, 4 years, and 7 years) in thyroid ultrasound (US) reviewed the thyroid US with and without using the CAD system. Statistical analyses included non-inferiority testing of the diagnostic accuracy for malignant thyroid nodules between the CAD system and the three radiologists with a non-inferiority margin of 10%, comparison of the diagnostic performance, and the added value of the CAD system to the radiologists. Results Altogether, 197 patients were included in the study cohort. The diagnostic accuracy of the CAD system (88.48%, 95% confidence interval [CI] = 82.65–92.53) was non-inferior to that of the radiologists with less experience (1 month and 4 year) of thyroid US (83.03%, 95% CI = 76.52–88.02; p < 0.001), whereas it was inferior to that of the experienced radiologist (7 years) (95.76%, 95% CI = 91.37–97.96; p = 0.138). The sensitivity and negative predictive value of the CAD system were significantly higher than those of the less-experienced radiologists were, whereas no significant difference was found with those of the experienced radiologist. A combination of US and the CAD system significantly improved sensitivity and negative predictive value, although the specificity and positive predictive value deteriorated for the less-experienced radiologists. Conclusion The CAD system may offer support for decision-making in the diagnosis of malignant thyroid nodules for operators who have less experience with thyroid US.
Collapse
Affiliation(s)
- Sae Rom Chung
- Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Korea
| | - Jung Hwan Baek
- Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Korea.
| | - Min Kyoung Lee
- Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Korea
| | - Yura Ahn
- Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Korea
| | - Young Jun Choi
- Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Korea
| | - Tae Yon Sung
- Department of Surgery, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Korea
| | - Dong Eun Song
- Department of Pathology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Korea
| | - Tae Yong Kim
- Department of Endocrinology and Metabolism, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Korea
| | - Jeong Hyun Lee
- Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Korea
| |
Collapse
|
27
|
Zhang Y, Wu Q, Chen Y, Wang Y. A Clinical Assessment of an Ultrasound Computer-Aided Diagnosis System in Differentiating Thyroid Nodules With Radiologists of Different Diagnostic Experience. Front Oncol 2020; 10:557169. [PMID: 33042840 PMCID: PMC7518212 DOI: 10.3389/fonc.2020.557169] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2020] [Accepted: 08/17/2020] [Indexed: 01/18/2023] Open
Abstract
Introduction This study aimed to assess the diagnostic performance and the added value to radiologists of different levels of a computer-aided diagnosis (CAD) system for the detection of thyroid cancers. Methods 303 patients who underwent thyroidectomy from October 2018 to July 2019 were retrospectively reviewed. The diagnostic performance of the senior radiologist, the junior radiologist, and the CAD system were compared. The added value of the CAD system was assessed and subgroup analyses were performed according to the size of thyroid nodules. Results In total, 186 malignant thyroid nodules, and 179 benign thyroid nodules were included; 168 were papillary thyroid carcinoma (PTC), 7 were medullary thyroid carcinoma (MTC), 11 were follicular carcinoma (FTC), 127 were follicular adenoma (FA) and 52 were nodular goiters. The CAD system showed a comparable specificity as the senior radiologist (86.0% vs. 86.0%, p > 0.99), but a lower sensitivity and a lower area under the receiver operating characteristic (AUROC) curve (sensitivity: 71.5% vs. 95.2%, p < 0.001; AUROC: 0.788 vs. 0.906, p < 0.001). The CAD system improved the diagnostic sensitivities of both the senior and the junior radiologists (97.8% vs. 95.2%, p = 0.063; 88.2% vs. 75.3%, p < 0.001). Conclusion The use of the CAD system using artificial intelligence is a potential tool to distinguish malignant thyroid nodules and is preferable to serve as a second opinion for less experienced radiologists to improve their diagnosis performance.
Collapse
Affiliation(s)
- Yichun Zhang
- Department of Ultrasound in Medicine, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University, Shanghai, China.,Shanghai Institute of Ultrasound in Medicine, Shanghai, China
| | - Qiong Wu
- Department of Ultrasound in Medicine, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University, Shanghai, China.,Shanghai Institute of Ultrasound in Medicine, Shanghai, China
| | - Yutong Chen
- Department of Ultrasound in Medicine, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University, Shanghai, China.,Shanghai Institute of Ultrasound in Medicine, Shanghai, China
| | - Yan Wang
- Department of Ultrasound in Medicine, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University, Shanghai, China.,Shanghai Institute of Ultrasound in Medicine, Shanghai, China
| |
Collapse
|
28
|
Xu L, Gao J, Wang Q, Yin J, Yu P, Bai B, Pei R, Chen D, Yang G, Wang S, Wan M. Computer-Aided Diagnosis Systems in Diagnosing Malignant Thyroid Nodules on Ultrasonography: A Systematic Review and Meta-Analysis. Eur Thyroid J 2020; 9:186-193. [PMID: 32903956 PMCID: PMC7445671 DOI: 10.1159/000504390] [Citation(s) in RCA: 32] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/08/2019] [Revised: 10/25/2019] [Indexed: 12/16/2022] Open
Abstract
BACKGROUND Computer-aided diagnosis (CAD) systems are being applied to the ultrasonographic diagnosis of malignant thyroid nodules, but it remains controversial whether the systems add any accuracy for radiologists. OBJECTIVE To determine the accuracy of CAD systems in diagnosing malignant thyroid nodules. METHODS PubMed, EMBASE, and the Cochrane Library were searched for studies on the diagnostic performance of CAD systems. The diagnostic performance was assessed by pooled sensitivity and specificity, and their accuracy was compared with that of radiologists. The present systematic review was registered in PROSPERO (CRD42019134460). RESULTS Nineteen studies with 4,781 thyroid nodules were included. Both the classic machine learning- and the deep learning-based CAD system had good performance in diagnosing malignant thyroid nodules (classic machine learning: sensitivity 0.86 [95% CI 0.79-0.92], specificity 0.85 [95% CI 0.77-0.91], diagnostic odds ratio (DOR) 37.41 [95% CI 24.91-56.20]; deep learning: sensitivity 0.89 [95% CI 0.81-0.93], specificity 0.84 [95% CI 0.75-0.90], DOR 40.87 [95% CI 18.13-92.13]). The diagnostic performance of the deep learning-based CAD system was comparable to that of the radiologists (sensitivity 0.87 [95% CI 0.78-0.93] vs. 0.87 [95% CI 0.85-0.89], specificity 0.85 [95% CI 0.76-0.91] vs. 0.87 [95% CI 0.81-0.91], DOR 40.12 [95% CI 15.58-103.33] vs. DOR 44.88 [95% CI 30.71-65.57]). CONCLUSIONS The CAD systems demonstrated good performance in diagnosing malignant thyroid nodules. However, experienced radiologists may still have an advantage over CAD systems during real-time diagnosis.
Collapse
Affiliation(s)
- Lei Xu
- Key Laboratory of Biomedical Information Engineering of the Ministry of Education, Department of Biomedical Engineering, School of Life Science and Technology, Xi'an Jiaotong University, Xi'an, China
- Xi'an Hospital of Traditional Chinese Medicine, Xi'an, China
| | - Junling Gao
- Key Laboratory of Biomedical Information Engineering of the Ministry of Education, Department of Biomedical Engineering, School of Life Science and Technology, Xi'an Jiaotong University, Xi'an, China
| | - Quan Wang
- Laboratory of Surgical Oncology, Peking University People's Hospital, Peking University, Beijing, China
| | - Jichao Yin
- Xi'an Hospital of Traditional Chinese Medicine, Xi'an, China
| | - Pengfei Yu
- Xijing Hospital, Fourth Military Medical University, Xi'an, China
| | - Bin Bai
- Xijing Hospital, Fourth Military Medical University, Xi'an, China
| | - Ruixia Pei
- Xi'an Hospital of Traditional Chinese Medicine, Xi'an, China
| | - Dingzhang Chen
- Xijing Hospital, Fourth Military Medical University, Xi'an, China
| | - Guochun Yang
- Xi'an Hospital of Traditional Chinese Medicine, Xi'an, China
| | - Shiqi Wang
- Xijing Hospital, Fourth Military Medical University, Xi'an, China
- **Shiqi Wang, Xijing Hospital, Fourth Military Medical University, Changlexi St. 127, Xi'an 710032 (China), E-Mail
| | - Mingxi Wan
- Key Laboratory of Biomedical Information Engineering of the Ministry of Education, Department of Biomedical Engineering, School of Life Science and Technology, Xi'an Jiaotong University, Xi'an, China
- *Mingxi Wan, Key Laboratory of Biomedical Information Engineering of the Ministry of Education, Department of Biomedical Engineering, School of Life Science and Technology, Xi'an Jiaotong University, Xianningxi St. 28, Xi'an 710049 (China), E-Mail
| |
Collapse
|
29
|
Sun C, Zhang Y, Chang Q, Liu T, Zhang S, Wang X, Guo Q, Yao J, Sun W, Niu L. Evaluation of a deep learning-based computer-aided diagnosis system for distinguishing benign from malignant thyroid nodules in ultrasound images. Med Phys 2020; 47:3952-3960. [PMID: 32473030 DOI: 10.1002/mp.14301] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2020] [Revised: 05/18/2020] [Accepted: 05/20/2020] [Indexed: 11/09/2022] Open
Abstract
PURPOSE Computer-aided diagnosis (CAD) systems assist in solving subjective diagnosis problems that typically rely on personal experience. A CAD system has been developed to differentiate malignant thyroid nodules from benign thyroid nodules in ultrasound images based on deep learning methods. The diagnostic performance was compared between the CAD system and the experienced attending radiologists. METHODS The ultrasound image dataset for training the CAD system included 651 malignant nodules and 386 benign nodules while the database for testing included 422 malignant nodules and 128 benign nodules. All the nodules were confirmed by pathology results. In the proposed CAD system, a support vector machine (SVM) is used for classification and fused features which combined the deep features extracted by a convolutional neural network (CNN) with the hand-crafted features such as the histogram of oriented gradient (HOG), local binary patterns (LBP), and scale invariant feature transform (SIFT) were obtained. The optimal feature subset was formed by selecting these fused features based on the maximum class separation distance and used as the training sample for the SVM. RESULTS The accuracy, sensitivity, and specificity of the CAD system were 92.5%, 96.4%, and 83.1%, respectively, which were higher than those of the experienced attending radiologists. The areas under the ROC curves of the CAD system and the attending radiologists were 0.881 and 0.819, respectively. CONCLUSIONS The CAD system for thyroid nodules exhibited a better diagnostic performance than experienced attending radiologists. The CAD system could be a reliable supplementary tool to diagnose thyroid nodules using ultrasonography. Macroscopic features in ultrasound images, such as the margins and shape of thyroid nodules, could influence the diagnostic efficiency of the CAD system.
Collapse
Affiliation(s)
- Chao Sun
- Department of Ultrasound, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China
| | - Yukang Zhang
- Department of Ultrasound, Beijing Tiantan Hospital, Capital Medical University, Beijing, 100050, China
| | - Qing Chang
- Department of Ultrasound, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China
| | - Tianjiao Liu
- Department of Electronic Engineering, Tsinghua University, Beijing, 100084, China
| | - Shaohang Zhang
- Department of Ultrasound, Beijing Haidian Hospital, Haidian Section of Peking University Third Hospital, Beijing, 100080, China
| | - Xi Wang
- Department of Ultrasound, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China
| | - Qianqian Guo
- Department of Ultrasound, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China
| | - Jinpeng Yao
- Department of Ultrasound, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China
| | - Weidong Sun
- Department of Electronic Engineering, Tsinghua University, Beijing, 100084, China
| | - Lijuan Niu
- Department of Ultrasound, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China
| |
Collapse
|
30
|
Kim DH, Chung SR, Choi SH, Kim KW. Accuracy of thyroid imaging reporting and data system category 4 or 5 for diagnosing malignancy: a systematic review and meta-analysis. Eur Radiol 2020; 30:5611-5624. [PMID: 32356157 DOI: 10.1007/s00330-020-06875-w] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2020] [Revised: 04/01/2020] [Accepted: 04/08/2020] [Indexed: 12/19/2022]
Abstract
OBJECTIVES To determine the accuracies of the American College of Radiology (ACR)-thyroid imaging reporting and data systems (TIRADS), Korean (K)-TIRADS, and European (EU)-TIRADS for diagnosing malignancy in thyroid nodules. METHODS Original studies reporting the diagnostic accuracy of TIRADS for determining malignancy on ultrasound were identified in MEDLINE and EMBASE up to June 23, 2019. The meta-analytic summary sensitivity and specificity were obtained for TIRADS category 5 (TR-5) and category 4 or 5 (TR-4/5), using a bivariate random effects model. To explore study heterogeneity, meta-regression analyses were performed. RESULTS Of the 34 eligible articles (37,585 nodules), 25 used ACR-TIRADS, 12 used K-TIRADS, and seven used EU-TIRADS. For TR-5, the meta-analytic sensitivity was highest for EU-TIRADS (78% [95% confidence interval, 64-88%]), followed by ACR-TIRADS (70% [61-79%]) and K-TIRADS (64% [58-70%]), although the differences were not significant. K-TIRADS showed the highest meta-analytic specificity (93% [91-95%]), which was similar to ACR-TIRADS (89% [85-92%]) and EU-TIRADS (89% [77-95%]). For TR-4/5, all three TIRADS systems had sensitivities higher than 90%. K-TIRADS had the highest specificity (61% [50-72%]), followed by ACR-TIRADS (49% [43-56%]) and EU-TIRADS (48% [35-62%]), although the differences were not significant. Considerable threshold effects were noted with ACR- and K-TIRADS (p ≤ 0.01), with subject enrollment, country of origin, experience level of reviewer, number of patients, and clarity of blinding in review being the main causes of heterogeneity (p ≤ 0.05). CONCLUSIONS There was no significant difference among these three international TIRADS, but the trend toward higher sensitivity with EU-TIRADS and higher specificity with K-TIRADS. KEY POINTS • For TIRADS category 5, the meta-analytic sensitivity was highest for the EU-TIRADS, followed by the ACR-TIRADS and the K-TIRADS, although the differences were not significant. • For TIRADS category 5, K-TIRADS showed the highest meta-analytic specificity, which was similar to ACR-TIRADS and EU-TIRADS. • Considerable threshold effects were noted with ACR- and K-TIRADS, with subject enrollment, country of origin, experience level of reviewer, number of patients, and clarity of blinding in review being the main causes of heterogeneity.
Collapse
Affiliation(s)
- Dong Hwan Kim
- Department of Radiology, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea, 222 Banpo-daero, Seocho-gu, Seoul, 06591, Republic of Korea
| | - Sae Rom Chung
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, 88 Olympic-Ro 43-Gil, Songpa-Gu, Seoul, 05505, Republic of Korea
| | - Sang Hyun Choi
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, 88 Olympic-Ro 43-Gil, Songpa-Gu, Seoul, 05505, Republic of Korea.
| | - Kyung Won Kim
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, 88 Olympic-Ro 43-Gil, Songpa-Gu, Seoul, 05505, Republic of Korea
| |
Collapse
|
31
|
Computer-aided diagnostic system for thyroid nodule sonographic evaluation outperforms the specificity of less experienced examiners. J Ultrasound 2020; 23:169-174. [PMID: 32246401 DOI: 10.1007/s40477-020-00453-y] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2019] [Accepted: 03/13/2020] [Indexed: 12/15/2022] Open
Abstract
PURPOSE Computer-aided diagnosis (CAD) may improve interobserver agreement in the risk stratification of thyroid nodules. This study aims to evaluate the performance of the Korean Thyroid Imaging Reporting and Data System (K-TIRADS) classification as estimated by an expert radiologist, a senior resident, a medical student, and a CAD system, as well as the interobserver agreement among them. METHODS Between July 2016 and 2018, 107 nodules (size 5-40 mm, 27 malignant) were classified according to the K-TIRADS by an expert radiologist and CAD software. A third-year resident and a medical student with basic imaging training, both blinded to previous findings, retrospectively estimated the K-TIRADS classification. The diagnostic performance was calculated, including sensitivity, specificity, positive and negative predictive values, and the area under the receiver operating characteristic curve. RESULTS The CAD system and the expert achieved a sensitivity of 70.37% (95% CI 49.82-86.25%) and 81.48% (61.92-93.7%) and a specificity of 87.50% (78.21-93.84%) and 88.75% (79.72-94.72%), respectively. The specificity of the student was significantly lower (76.25% [65.42-85.05%], p = 0.02). CONCLUSION In our opinion, the CAD evaluation of thyroid nodules stratification risk has a potential role in a didactic field and does not play a real and effective role in the clinical field, where not only images but also specialistic medical practice is fundamental to achieve a diagnosis based on family history, genetics, lab tests, and so on. The CAD system may be useful for less experienced operators as its specificity was significantly higher.
Collapse
|
32
|
Barczyński M, Stopa-Barczyńska M, Wojtczak B, Czarniecka A, Konturek A. Clinical validation of S-Detect TM mode in semi-automated ultrasound classification of thyroid lesions in surgical office. Gland Surg 2020; 9:S77-S85. [PMID: 32175248 DOI: 10.21037/gs.2019.12.23] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
Background In recent years well-recognized scientific societies introduced guidelines for ultrasound (US) malignancy risk stratification of thyroid nodules. These guidelines categorize the risk of malignancy in relation to a combination of several US features. Based on these US image lexicons an US-based computer-aided diagnosis (CAD) systems were developed. Nevertheless, their clinical utility has not been evaluated in any study of surgeon-performed office US of the thyroid. Hence, the aim of this pilot study was to validate s-DetectTM mode in semi-automated US classification of thyroid lesions during surgeon-performed office US. Methods This is a prospective study of 50 patients who underwent surgeon-performed thyroid US (basic US skills without CAD vs. with CAD vs. expert US skills without CAD) in the out-patient office as part of the preoperative workup. The real-time CAD system software using artificial intelligence (S-DetectTM for Thyroid; Samsung Medison Co.) was integrated into the RS85 US system. Primary outcome was CAD system added-value to the surgeon-performed office US evaluation. Secondary outcomes were: diagnostic accuracy of CAD system, intra and interobserver variability in the US assessment of thyroid nodules. Surgical pathology report was used to validate the pre-surgical diagnosis. Results CAD system added-value to thyroid assessment by a surgeon with basic US skills was equal to 6% (overall accuracy of 82% for evaluation with CAD vs. 76% for evaluation without CAD system; P<0.001), and final diagnosis was different than predicted by US assessment in 3 patients (1 more true-positive and 2 more true-negative results). However, CAD system was inferior to thyroid assessment by a surgeon with expert US skills in 6 patients who had false-positive results (P<0.001). Conclusions The sensitivity and negative predictive value of CAD system for US classification of thyroid lesions were similar as surgeon with expert US skills whereas specificity and positive predictive value were significantly inferior but markedly better than judgement of a surgeon with basic US skills alone.
Collapse
Affiliation(s)
- Marcin Barczyński
- Department of Endocrine Surgery, Third Chair of General Surgery, Jagiellonian University Medical College, Kraków, Poland
| | - Małgorzata Stopa-Barczyńska
- Clinical Ward of General Surgery and Oncology, Gabriel Narutowicz Memorial Municipal Hospital, Kraków, Poland
| | - Beata Wojtczak
- Department of General, Minimally Invasive and Endocrine Surgery, Wroclaw Medical University, Wroclaw, Poland
| | - Agnieszka Czarniecka
- Department of Oncological and Reconstructive Surgery, M. Sklodowska-Curie Institute - Oncology Centre, Gliwice, Poland
| | - Aleksander Konturek
- Department of Endocrine Surgery, Third Chair of General Surgery, Jagiellonian University Medical College, Kraków, Poland
| |
Collapse
|
33
|
Convolutional Neural Network for Breast and Thyroid Nodules Diagnosis in Ultrasound Imaging. BIOMED RESEARCH INTERNATIONAL 2020; 2020:1763803. [PMID: 32420322 PMCID: PMC7199615 DOI: 10.1155/2020/1763803] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/09/2019] [Revised: 11/11/2019] [Accepted: 12/14/2019] [Indexed: 12/12/2022]
Abstract
Objective The incidence of superficial organ diseases has increased rapidly in recent years. New methods such as computer-aided diagnosis (CAD) are widely used to improve diagnostic efficiency. Convolutional neural networks (CNNs) are one of the most popular methods, and further improvements of CNNs should be considered. This paper aims to develop a multiorgan CAD system based on CNNs for classifying both thyroid and breast nodules and investigate the impact of this system on the diagnostic efficiency of different preprocessing approaches. Methods The training and validation sets comprised randomly selected thyroid and breast nodule images. The data were subgrouped into 4 models according to the different preprocessing methods (depending on segmentation and the classification method). A prospective data set was selected to verify the clinical value of the CNN model by comparison with ultrasound guidelines. Diagnostic efficiency was assessed based on receiver operating characteristic (ROC) curves. Results Among the 4 models, the CNN model using segmented images for classification achieved the best result. For the validation set, the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, and area under the curve (AUC) of our CNN model were 84.9%, 69.0%, 62.5%, 88.2%, 75.0%, and 0.769, respectively. There was no statistically significant difference between the CNN model and the ultrasound guidelines. The combination of the two methods achieved superior diagnostic efficiency compared with their use individually. Conclusions The study demonstrates the probability, feasibility, and clinical value of CAD in the ultrasound diagnosis of multiple organs. The use of segmented images and classification by the nature of the disease are the main factors responsible for the improvement of the CNN model. Moreover, the combination of the CNN model and ultrasound guidelines results in better diagnostic performance, which will contribute to the improved diagnostic efficiency of CAD systems.
Collapse
|
34
|
Singh Ospina N, Iñiguez-Ariza NM, Castro MR. Thyroid nodules: diagnostic evaluation based on thyroid cancer risk assessment. BMJ 2020; 368:l6670. [PMID: 31911452 DOI: 10.1136/bmj.l6670] [Citation(s) in RCA: 57] [Impact Index Per Article: 14.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
Thyroid nodules are extremely common and can be detected by sensitive imaging in more than 60% of the general population. They are often identified in patients without symptoms who are undergoing evaluation for other medical complaints. Indiscriminate evaluation of thyroid nodules with thyroid biopsy could cause a harmful epidemic of diagnoses of thyroid cancer, but inadequate selection of thyroid nodules for biopsy can lead to missed diagnoses of clinically relevant thyroid cancer. Recent clinical guidelines advocate a more conservative approach in the evaluation of thyroid nodules based on risk assessment for thyroid cancer, as determined by clinical and ultrasound features to guide the need for biopsy. Moreover, newer evidence suggests that for patients with indeterminate thyroid biopsy results, a combined assessment including the initial ultrasound risk stratification or other ancillary testing (molecular markers, second opinion on thyroid cytology) can further clarify the risk of thyroid cancer and the management strategies. This review summarizes the clinical importance of adequate evaluation of thyroid nodules, focuses on the clinical evidence for diagnostic tests that can clarify the risk of thyroid cancer, and highlights the importance of considering the patient's values and preferences when deciding on management strategies in the setting of uncertainty about the risk of thyroid cancer.
Collapse
Affiliation(s)
- Naykky Singh Ospina
- Division of Endocrinology, Department of Medicine, University of Florida, Gainesville, FL, USA
| | - Nicole M Iñiguez-Ariza
- Department of Endocrinology and Metabolism, Instituto Nacional de Ciencias Médicas y Nutrición Salvador Zubirán, Mexico City, Mexico
| | - M Regina Castro
- Division of Endocrinology, Department of Medicine, Mayo Clinic, Rochester, MN, USA
| |
Collapse
|
35
|
Jin Z, Zhu Y, Zhang S, Xie F, Zhang M, Zhang Y, Tian X, Zhang J, Luo Y, Cao J. Ultrasound Computer-Aided Diagnosis (CAD) Based on the Thyroid Imaging Reporting and Data System (TI-RADS) to Distinguish Benign from Malignant Thyroid Nodules and the Diagnostic Performance of Radiologists with Different Diagnostic Experience. Med Sci Monit 2020; 26:e918452. [PMID: 31929498 PMCID: PMC6977643 DOI: 10.12659/msm.918452] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2022] Open
Abstract
<strong>BACKGROUND</strong> The diagnosis of thyroid cancer and distinguishing benign from malignant thyroid nodules by junior radiologists can be challenging. This study aimed to develop a computer-aided diagnosis (CAD) system based on the Thyroid Imaging Reporting and Data System (TI-RADS) to distinguish benign from malignant thyroid nodules by analyzing ultrasound images to improve the diagnostic performance of junior radiologists. <strong>MATERIAL AND METHODS</strong> A modified TI-RADS based on a convolutional neural network (CNN) was used to develop the CAD system. This retrospective study reviewed 789 thyroid nodules from 695 patients and included radiologists with different diagnostic experience. Five study groups included the CAD group, the junior radiologist group, the intermediate-level radiologist group, the senior radiologist group, and the group in which the junior radiologist used the CAD system. The ultrasound findings were reviewed and compared with the histopathology diagnosis. <strong>RESULTS</strong> The CAD system for the diagnosis of thyroid cancer showed an accuracy of 80.35%, a sensitivity of 80.64%, a specificity of 80.13%, a positive predictive value (PPV) of 76.02%, a negative predictive value (NPV) of 84.12%, and an area under the receiver operating characteristic (ROC) curve (AUC) of 0.87. The accuracy of the junior radiologists in diagnosing thyroid cancer using CAD was similar to that of intermediate-level radiologists (79.21% <i>vs</i>. 77.57%; P=0.427). <strong>CONCLUSIONS</strong> The use of ultrasound CAD based on the TI-RADS showed potential for distinguishing between benign and malignant thyroid nodules and improved the diagnostic performance of junior radiologists.
Collapse
Affiliation(s)
- Zhuang Jin
- Department of Ultrasound, Chinese People's Liberation Army (PLA) General Hospital, Beijing, China (mainland).,Department of Ultrasound, General Hospital of Northern Theater Command, Shenyang, Liaoning, China (mainland).,Medical School of Chinese People's Liberation Army (PLA), Beijing, China (mainland)
| | - Yaqiong Zhu
- Department of Ultrasound, Chinese People's Liberation Army (PLA) General Hospital, Beijing, China (mainland).,Nankai University, Tianjin, China (mainland)
| | | | - Fang Xie
- Department of Ultrasound, Chinese People's Liberation Army (PLA) General Hospital, Beijing, China (mainland)
| | - Mingbo Zhang
- Department of Ultrasound, Chinese People's Liberation Army (PLA) General Hospital, Beijing, China (mainland)
| | - Ying Zhang
- Nankai University, Tianjin, China (mainland)
| | - Xiaoqi Tian
- Nankai University, Tianjin, China (mainland)
| | - Jue Zhang
- Peking University, Beijing, China (mainland)
| | - Yukun Luo
- Department of Ultrasound, Chinese People's Liberation Army (PLA) General Hospital, Beijing, China (mainland).,Medical School of Chinese People's Liberation Army (PLA), Beijing, China (mainland)
| | - Junying Cao
- Department of Ultrasound, General Hospital of Northern Theater Command, Shenyang, Liaoning, China (mainland)
| |
Collapse
|
36
|
Seifert P, Görges R, Zimny M, Kreissl MC, Schenke S. Interobserver agreement and efficacy of consensus reading in Kwak-, EU-, and ACR-thyroid imaging recording and data systems and ATA guidelines for the ultrasound risk stratification of thyroid nodules. Endocrine 2020; 67:143-154. [PMID: 31741167 DOI: 10.1007/s12020-019-02134-1] [Citation(s) in RCA: 34] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/12/2019] [Accepted: 11/07/2019] [Indexed: 02/07/2023]
Abstract
PURPOSE To investigate the interobserver agreement (IA) and the impact of consensus reading using four risk stratification systems for thyroid nodules (TN). METHODS Four experienced specialists independently rated US images of 80 TN according to the Kwak-TIRADS, EU-TIRADS, ACR TI-RADS, and ATA Guidelines. The cases were randomly extracted from a prospectively acquired database (n > 1500 TN). The observers were blinded to clinical data. This study was divided into two sessions (S1 and S2) with 40 image sets each. After every session, a consensus reading was carried out (C1, C2). Subsequently, the effect of C1 was tested in S2 with 40 new cases followed by C2. Fleiss' kappa (κ) was calculated for S1 and S2 to estimate the IA and learning curves. The results of C1 and C2 were used as reference for diagnostic accuracy calculations. RESULTS IA significantly increased (p < 0.01) after C1 with κ values of 0.375 (0.615), 0.411 (0.596), 0.321 (0.569), and 0.410 (0.583) for the Kwak-TIRADS, EU-TIRADS, ACR TI-RADS, and ATA Guidelines in S1 (S2), respectively. ROC analysis (C1 + C2) revealed similar areas under the curve (AUC) for the Kwak-TIRADS, EU-TIRADS, ACR TI-RADS, and ATA Guidelines (0.635, 0.675, 0.694, and 0.654, respectively, n.s.). AUC did not increase from C1 (0.677 ± 0.010) to C2 (0.632 ± 0.052, n.s.). ATA Guidelines were not applicable in five cases. CONCLUSIONS IA and diagnostic accuracy were very similar for the four investigated risk stratification systems. Consensus reading sessions significantly improved the IA but did not affect the diagnostic accuracy.
Collapse
Affiliation(s)
- Philipp Seifert
- Department of Nuclear Medicine, Jena University Hospital, Jena, Germany.
| | - Rainer Görges
- Department of Nuclear Medicine, Essen University Hospital, Essen, Germany
- Joint Practice for Nuclear Medicine, Duisburg/Moers, Duisburg, Germany
| | - Michael Zimny
- Institute for Nuclear Medicine Hanau, Giessen, Germany
| | - Michael C Kreissl
- Division of Nuclear Medicine, Department of Radiology and Nuclear Medicine, Magdeburg University Hospital, Magdeburg, Germany
| | - Simone Schenke
- Institute for Nuclear Medicine Hanau, Giessen, Germany
- Division of Nuclear Medicine, Department of Radiology and Nuclear Medicine, Magdeburg University Hospital, Magdeburg, Germany
| |
Collapse
|
37
|
Differentiation of thyroid nodules on US using features learned and extracted from various convolutional neural networks. Sci Rep 2019; 9:19854. [PMID: 31882683 PMCID: PMC6934479 DOI: 10.1038/s41598-019-56395-x] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2019] [Accepted: 12/11/2019] [Indexed: 11/08/2022] Open
Abstract
Thyroid nodules are a common clinical problem. Ultrasonography (US) is the main tool used to sensitively diagnose thyroid cancer. Although US is non-invasive and can accurately differentiate benign and malignant thyroid nodules, it is subjective and its results inevitably lack reproducibility. Therefore, to provide objective and reliable information for US assessment, we developed a CADx system that utilizes convolutional neural networks and the machine learning technique. The diagnostic performances of 6 radiologists and 3 representative results obtained from the proposed CADx system were compared and analyzed.
Collapse
|
38
|
Liu T, Guo Q, Lian C, Ren X, Liang S, Yu J, Niu L, Sun W, Shen D. Automated detection and classification of thyroid nodules in ultrasound images using clinical-knowledge-guided convolutional neural networks. Med Image Anal 2019; 58:101555. [DOI: 10.1016/j.media.2019.101555] [Citation(s) in RCA: 73] [Impact Index Per Article: 14.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2019] [Revised: 06/10/2019] [Accepted: 09/04/2019] [Indexed: 01/25/2023]
|
39
|
Park VY, Han K, Seong YK, Park MH, Kim EK, Moon HJ, Yoon JH, Kwak JY. Diagnosis of Thyroid Nodules: Performance of a Deep Learning Convolutional Neural Network Model vs. Radiologists. Sci Rep 2019; 9:17843. [PMID: 31780753 PMCID: PMC6882804 DOI: 10.1038/s41598-019-54434-1] [Citation(s) in RCA: 40] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2019] [Accepted: 11/08/2019] [Indexed: 01/05/2023] Open
Abstract
Computer-aided diagnosis (CAD) systems hold potential to improve the diagnostic accuracy of thyroid ultrasound (US). We aimed to develop a deep learning-based US CAD system (dCAD) for the diagnosis of thyroid nodules and compare its performance with those of a support vector machine (SVM)-based US CAD system (sCAD) and radiologists. dCAD was developed by using US images of 4919 thyroid nodules from three institutions. Its diagnostic performance was prospectively evaluated between June 2016 and February 2017 in 286 nodules, and was compared with those of sCAD and radiologists, using logistic regression with the generalized estimating equation. Subgroup analyses were performed according to experience level and separately for small thyroid nodules 1-2 cm. There was no difference in overall sensitivity, specificity, positive predictive value (PPV), negative predictive value and accuracy (all p > 0.05) between radiologists and dCAD. Radiologists and dCAD showed higher specificity, PPV, and accuracy than sCAD (all p < 0.001). In small nodules, experienced radiologists showed higher specificity, PPV and accuracy than sCAD (all p < 0.05). In conclusion, dCAD showed overall comparable diagnostic performance with radiologists and assessed thyroid nodules more effectively than sCAD, without loss of sensitivity.
Collapse
Affiliation(s)
- Vivian Y Park
- Department of Radiology, Severance Hospital, Research Institute of Radiological Science, Yonsei University College of Medicine, Seoul, Korea
| | - Kyunghwa Han
- Department of Radiology, Severance Hospital, Research Institute of Radiological Science, Yonsei University College of Medicine, Seoul, Korea
| | - Yeong Kyeong Seong
- Health & Medical Equipment Business, Samsung Electronics Co., Ltd., Seoul, Korea
| | - Moon Ho Park
- Health & Medical Equipment Business, Samsung Electronics Co., Ltd., Seoul, Korea
| | - Eun-Kyung Kim
- Department of Radiology, Severance Hospital, Research Institute of Radiological Science, Yonsei University College of Medicine, Seoul, Korea
| | - Hee Jung Moon
- Department of Radiology, Severance Hospital, Research Institute of Radiological Science, Yonsei University College of Medicine, Seoul, Korea
| | - Jung Hyun Yoon
- Department of Radiology, Severance Hospital, Research Institute of Radiological Science, Yonsei University College of Medicine, Seoul, Korea
| | - Jin Young Kwak
- Department of Radiology, Severance Hospital, Research Institute of Radiological Science, Yonsei University College of Medicine, Seoul, Korea.
| |
Collapse
|
40
|
Chambara N, Ying M. The Diagnostic Efficiency of Ultrasound Computer-Aided Diagnosis in Differentiating Thyroid Nodules: A Systematic Review and Narrative Synthesis. Cancers (Basel) 2019; 11:E1759. [PMID: 31717365 PMCID: PMC6896127 DOI: 10.3390/cancers11111759] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2019] [Revised: 11/03/2019] [Accepted: 11/06/2019] [Indexed: 12/20/2022] Open
Abstract
Computer-aided diagnosis (CAD) techniques have emerged to complement qualitative assessment in the diagnosis of benign and malignant thyroid nodules. The aim of this review was to summarize the current evidence on the diagnostic performance of various ultrasound CAD in characterizing thyroid nodules. PUBMED, EMBASE and Cochrane databases were searched for studies published until August 2019. The Quality Assessment of Studies of Diagnostic Accuracy included in Systematic Review 2 (QUADAS-2) tool was used to assess the methodological quality of the studies. Reported diagnostic performance data were analyzed and discussed. Fourteen studies with 2232 patients and 2675 thyroid nodules met the inclusion criteria. The study quality based on QUADAS-2 assessment was moderate. At best performance, grey scale CAD had a sensitivity of 96.7% while Doppler CAD was 90%. Combined techniques of qualitative grey scale features and Doppler CAD assessment resulted in overall increased sensitivity (92%) and optimal specificity (85.1%). The experience of the CAD user, nodule size and the thyroid malignancy risk stratification system used for interpretation were the main potential factors affecting diagnostic performance outcomes. The diagnostic performance of CAD of thyroid ultrasound is comparable to that of qualitative visual assessment; however, combined techniques have the potential for better optimized diagnostic accuracy.
Collapse
Affiliation(s)
| | - Michael Ying
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong, SAR, China;
| |
Collapse
|
41
|
Zhao WJ, Fu LR, Huang ZM, Zhu JQ, Ma BY. Effectiveness evaluation of computer-aided diagnosis system for the diagnosis of thyroid nodules on ultrasound: A systematic review and meta-analysis. Medicine (Baltimore) 2019; 98:e16379. [PMID: 31393347 PMCID: PMC6709241 DOI: 10.1097/md.0000000000016379] [Citation(s) in RCA: 32] [Impact Index Per Article: 6.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/05/2023] Open
Abstract
BACKGROUND More and more automated efficient ultrasound image analysis techniques, such as ultrasound-based computer-aided diagnosis system (CAD), were developed to obtain accurate, reproducible, and more objective diagnosis results for thyroid nodules. So far, whether the diagnostic performance of existing CAD systems can reach the diagnostic level of experienced radiologists is still controversial. The aim of the meta-analysis was to evaluate the accuracy of CAD for thyroid nodules' diagnosis by reviewing current literatures and summarizing the research status. METHODS A detailed literature search on PubMed, Embase, and Cochrane Libraries for articles published until December 2018 was carried out. The diagnostic performances of CAD systems vs radiologist were evaluated by meta-analysis. We determined the sensitivity and the specificity across studies, calculated positive and negative likelihood ratios and constructed summary receiver-operating characteristic (SROC) curves. Meta-analysis of studies was performed using a mixed-effect, hierarchical logistic regression model. RESULTS Five studies with 536 patients and 723 thyroid nodules were included in this meta-analysis. The pooled sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and diagnostic odds ratio (DOR) for CAD system were 0.87 (95% confidence interval [CI], 0.73-0.94), 0.79 (95% CI 0.63-0.89), 4.1 (95% CI 2.5-6.9), 0.17 (95% CI 0.09-0.32), and 25 (95% CI 15-42), respectively. The SROC curve indicated that the area under the curve was 0.90 (95% CI 0.87-0.92). The pooled sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and DOR for experienced radiologists were 0.82 (95% CI 0.69-0.91), 0.83 (95% CI 0.76-0.89), 4.9 (95% CI 3.4-7.0), 0.22 (95% CI 0.12-0.38), and 23 (95% CI 11-46), respectively. The SROC curve indicated that the area under the curve was 0.96 (95% CI 0.94-0.97). CONCLUSION The sensitivity of the CAD system in the diagnosis of thyroid nodules was similar to that of experienced radiologists. However, the CAD system had lower specificity and DOR than experienced radiologists. The CAD system may play the potential role as a decision-making assistant alongside radiologists in the thyroid nodules' diagnosis. Future technical improvements would be helpful to increase the accuracy as well as diagnostic efficiency.
Collapse
Affiliation(s)
- Wan-Jun Zhao
- Department of Thyroid & Parathyroid Surgery, West China Hospital
| | - Lin-Ru Fu
- West China School of Medicine, Sichuan University, Sichuan
| | - Zhi-Mian Huang
- Business College, New York University in Shanghai, Shanghai
| | - Jing-Qiang Zhu
- Department of Thyroid & Parathyroid Surgery, West China Hospital
| | - Bu-Yun Ma
- Department of Ultrasonography, West China Hospital, Sichuan University, Sichuan, China
| |
Collapse
|
42
|
Perros P. A decade of thyroidology. Hormones (Athens) 2018; 17:491-495. [PMID: 30306416 PMCID: PMC6294812 DOI: 10.1007/s42000-018-0068-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/11/2018] [Accepted: 07/18/2018] [Indexed: 11/26/2022]
Abstract
Significant scientific progress has been achieved in the past decade in thyroidology driven by scholarly enquiry, unmet patient needs, and investment by the pharmaceutical and diagnostics industry. In this review, nine publications have been selected for their impact in pushing the frontiers of knowledge and understanding. They include new perspectives in the diagnosis, pathophysiology, epidemiology and management of thyroid cancer, understanding of thyroid hormone physiology, and new treatments for Graves' orbitopathy.
Collapse
Affiliation(s)
- Petros Perros
- Newcastle upon Tyne Hospitals NHS Foundation Trust, Newcastle University, Newcastle upon Tyne, UK.
- Department of Endocrinology, Level 6, Leazes Wing, Royal Victoria Infirmary, Newcastle upon Tyne, NE1 4LP, UK.
| |
Collapse
|
43
|
Gitto S, Grassi G, De Angelis C, Monaco CG, Sdao S, Sardanelli F, Sconfienza LM, Mauri G. A computer-aided diagnosis system for the assessment and characterization of low-to-high suspicion thyroid nodules on ultrasound. Radiol Med 2018; 124:118-125. [PMID: 30244368 DOI: 10.1007/s11547-018-0942-z] [Citation(s) in RCA: 38] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2018] [Accepted: 09/11/2018] [Indexed: 12/25/2022]
Abstract
AIM OF THE STUDY To compare the diagnostic performance of a commercially available computer-aided diagnosis (CAD) system for thyroid ultrasound (US) with that of a non-computer-aided radiologist in the characterization of low-to-high suspicion thyroid nodules. METHODS This retrospective study included a consecutive series of adult patients referred for US-guided fine-needle aspiration biopsy (FNAB) of a thyroid nodule. All patients were eligible for thyroid nodule FNAB according to the current international guidelines. An interventional radiologist experienced in thyroid imaging acquired the US images subsequently used for post-processing, performed FNAB and provided the US features of each nodule. A radiology resident and an endocrinology resident in consensus performed post-processing using the CAD system to assess the same nodule characteristics. The diagnostic performance and agreement of US features between the CAD system and the radiologist were compared. RESULTS Sixty-two patients (50 F; age 60 ± 12 years) were enrolled: 77.4% (48/62) of thyroid nodules were benign, 22.6% (14/62) were undetermined to malignant and required follow-up or surgery. Interobserver agreement between the CAD system and the radiologist was substantial for orientation (K = 0.69), fair for composition (K = 0.36), echogenicity (K = 0.36), K-TIRADS (K = 0.29), and slight for margins (K = 0.03). The radiologist demonstrated a significantly higher sensitivity than the CAD system (78.6% vs. 21.4%; P = 0.008), while there was no statistical difference in specificity (66.7% vs. 81.3%; P = 0.065). CONCLUSION This CAD system is less sensitive than an experienced radiologist and showed slight-to-substantial agreement with the radiologist for the characterization of thyroid nodules. Although it is an innovative tool with good potential, additional efforts are needed to improve its diagnostic performance.
Collapse
Affiliation(s)
- Salvatore Gitto
- Scuola di Specializzazione in Radiodiagnostica, Università degli Studi di Milano, via Festa del Perdono 7, 20122, Milan, Italy.
| | - Giorgia Grassi
- Scuola di Specializzazione in Endocrinologia e Malattie del Metabolismo, Università degli Studi di Milano, Milan, Italy
| | - Chiara De Angelis
- Scuola di Specializzazione in Radiodiagnostica, Università degli Studi di Milano, via Festa del Perdono 7, 20122, Milan, Italy
| | - Cristian Giuseppe Monaco
- Scuola di Specializzazione in Radiodiagnostica, Università degli Studi di Milano, via Festa del Perdono 7, 20122, Milan, Italy
| | - Silvana Sdao
- Fondazione IRCCS Istituto Nazionale dei Tumori, Milan, Italy
| | - Francesco Sardanelli
- Servizio di Radiologia, IRCCS Policlinico San Donato, San Donato Milanese, Italy.,Dipartimento di Scienze Biomediche per la Salute, Università degli Studi di Milano, Milan, Italy
| | - Luca Maria Sconfienza
- Dipartimento di Scienze Biomediche per la Salute, Università degli Studi di Milano, Milan, Italy.,Unità Operativa di Radiologia Diagnostica ed Interventistica, IRCCS Istituto Ortopedico Galeazzi, Milan, Italy
| | - Giovanni Mauri
- Divisione di Radiologia Interventistica, IEO, Istituto Europeo di Oncologia IRCCS, Milano, Italy
| |
Collapse
|