1
|
Liu W, Lin C, Chen D, Niu L, Zhang R, Pi Z. Shape-margin knowledge augmented network for thyroid nodule segmentation and diagnosis. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 244:107999. [PMID: 38194766 DOI: 10.1016/j.cmpb.2023.107999] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/04/2023] [Revised: 11/11/2023] [Accepted: 12/26/2023] [Indexed: 01/11/2024]
Abstract
BACKGROUND AND OBJECTIVE Thyroid nodule segmentation is a crucial step in the diagnostic procedure of physicians and computer-aided diagnosis systems. However, prevailing studies often treat segmentation and diagnosis as independent tasks, overlooking the intrinsic relationship between these processes. The sequencial steps of these independent tasks in computer-aided diagnosis systems may lead to the accumulation of errors. Therefore, it is worth combining them as a whole by exploring the relationship between thyroid nodule segmentation and diagnosis. According to the diagnostic procedure of thyroid imaging reporting and data system (TI-RADS), the assessment of shape and margin characteristics is the prerequisite for radiologists to discriminate benign and malignant thyroid nodules. Inspired by TI-RADS, this study aims to integrate these tasks into a cohesive process, leveraging the insights from TI-RADS, thereby enhancing the accuracy and interpretability of thyroid nodule analysis. METHODS Specifically, this paper proposes a shape-margin knowledge augmented network (SkaNet) for simultaneous thyroid nodule segmentation and diagnosis. Due to the visual feature similarities between segmentation and diagnosis, SkaNet shares visual features in the feature extraction stage and then utilizes a dual-branch architecture to perform thyroid nodule segmentation and diagnosis tasks respectively. In the shared feature extraction, the combination of convolutional feature maps and self-attention maps allows to exploitation of both local information and global patterns in thyroid nodule images. To enhance effective discriminative features, an exponential mixture module is introduced, combining convolutional feature maps and self-attention maps through exponential weighting. Then, SkaNet is jointly optimized by a knowledge augmented multi-task loss function with a constraint penalty term. The constraint penalty term embeds shape and margin characteristics through numerical computations, establishing a vital relationship between thyroid nodule diagnosis results and segmentation masks. RESULTS We evaluate the proposed approach on a public thyroid ultrasound dataset (DDTI) and a locally collected thyroid ultrasound dataset. The experimental results reveal the value of our contributions and demonstrate that our approach can yield significant improvements compared with state-of-the-art counterparts. CONCLUSIONS SkaNet highlights the potential of combining thyroid nodule segmentation and diagnosis with knowledge augmented learning into a unified framework, which captures the key shape and margin characteristics for discriminating benign and malignant thyroid nodules. Our findings suggest promising insights for advancing computer-aided diagnosis joint with segmentation.
Collapse
Affiliation(s)
- Weihua Liu
- School of Medical Technology, Beijing Institute of Technology, 5 Zhongguancun South Street, Haidian, 100081, Beijing, China; AthenaEyesCO., LTD., Building 14, No. 39 Jianshan Road, Changsha, 410205, Hunan, China.
| | - Chaochao Lin
- School of Computer Science and Technology, Beijing Institute of Technology, 5 Zhongguancun South Street, Haidian, 100081, Beijing, China.
| | - Duanduan Chen
- School of Medical Technology, Beijing Institute of Technology, 5 Zhongguancun South Street, Haidian, 100081, Beijing, China.
| | - Lijuan Niu
- Department of Ultrasound, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 17 Nanli, Panjiayuan, Chaoyang, 100021, Beijing, China.
| | - Rui Zhang
- Department of Ultrasound, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 17 Nanli, Panjiayuan, Chaoyang, 100021, Beijing, China.
| | - Zhaoqiong Pi
- Xiangya School of Medicine, Central South University, No. 172, Tongzipo Road, Changsha, 410083, Hunan, China.
| |
Collapse
|
2
|
Yoon J, Lee E, Lee HS, Cho S, Son J, Kwon H, Yoon JH, Park VY, Lee M, Rho M, Kim D, Kwak JY. Learnability of Thyroid Nodule Assessment on Ultrasonography: Using a Big Data Set. ULTRASOUND IN MEDICINE & BIOLOGY 2023; 49:2581-2589. [PMID: 37758528 DOI: 10.1016/j.ultrasmedbio.2023.08.026] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/15/2023] [Revised: 08/29/2023] [Accepted: 08/31/2023] [Indexed: 09/29/2023]
Abstract
OBJECTIVE The aims of the work described here were to evaluate the learnability of thyroid nodule assessment on ultrasonography (US) using a big data set of US images and to evaluate the diagnostic utilities of artificial intelligence computer-aided diagnosis (AI-CAD) used by readers with varying experience to differentiate benign and malignant thyroid nodules. METHODS Six college freshmen independently studied the "learning set" composed of images of 13,560 thyroid nodules, and their diagnostic performance was evaluated after their daily learning sessions using the "test set" composed of images of 282 thyroid nodules. The diagnostic performance of two residents and an experienced radiologist was evaluated using the same "test set." After an initial diagnosis, all readers once again evaluated the "test set" with the assistance of AI-CAD. RESULTS Diagnostic performance of almost all students increased after the learning program. Although the mean areas under the receiver operating characteristic curves (AUROCs) of residents and the experienced radiologist were significantly higher than those of students, the AUROCs of five of the six students did not differ significantly compared with that of the one resident. With the assistance of AI-CAD, sensitivity significantly increased in three students, specificity in one student, accuracy in four students and AUROC in four students. Diagnostic performance of the two residents and the experienced radiologist was better with the assistance of AI-CAD. CONCLUSION A self-learning method using a big data set of US images has potential as an ancillary tool alongside traditional training methods. With the assistance of AI-CAD, the diagnostic performance of readers with varying experience in thyroid imaging could be further improved.
Collapse
Affiliation(s)
- Jiyoung Yoon
- Department of Radiology and Research Institute of Radiological Science, Yonsei University College of Medicine, Seoul, Korea
| | - Eunjung Lee
- School of Mathematics and Computing (Computational Science and Engineering), Yonsei University, Seoul, Korea
| | - Hye Sun Lee
- Biostatistics Collaboration Unit, Yonsei Biomedical Research Institute, Yonsei University College of Medicine, Seoul, Korea
| | - Sangwoo Cho
- Yonsei University College of Medicine, Seoul, Korea
| | - JinWoo Son
- Department of Radiology and Research Institute of Radiological Science, Yonsei University College of Medicine, Seoul, Korea
| | - Hyuk Kwon
- Department of Radiology and Research Institute of Radiological Science, Yonsei University College of Medicine, Seoul, Korea
| | - Jung Hyun Yoon
- Department of Radiology and Research Institute of Radiological Science, Yonsei University College of Medicine, Seoul, Korea
| | - Vivian Youngjean Park
- Department of Radiology and Research Institute of Radiological Science, Yonsei University College of Medicine, Seoul, Korea
| | - Minah Lee
- Department of Radiology and Research Institute of Radiological Science, Yonsei University College of Medicine, Seoul, Korea
| | - Miribi Rho
- Department of Radiology and Research Institute of Radiological Science, Yonsei University College of Medicine, Seoul, Korea
| | - Daham Kim
- Department of Internal Medicine, Yonsei Cancer Center, Yonsei University College of Medicine, Seoul, Korea
| | - Jin Young Kwak
- Department of Radiology and Research Institute of Radiological Science, Yonsei University College of Medicine, Seoul, Korea.
| |
Collapse
|
3
|
Gao Y, Zhang B, Zhao D, Li S, Rong C, Sun M, Wu X. Automatic Segmentation and Radiomics for Identification and Activity Assessment of CTE Lesions in Crohn's Disease. Inflamm Bowel Dis 2023:izad285. [PMID: 38011673 DOI: 10.1093/ibd/izad285] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Indexed: 11/29/2023]
Abstract
BACKGROUND The purpose of this article is to develop a deep learning automatic segmentation model for the segmentation of Crohn's disease (CD) lesions in computed tomography enterography (CTE) images. Additionally, the radiomics features extracted from the segmented CD lesions will be analyzed and multiple machine learning classifiers will be built to distinguish CD activity. METHODS This was a retrospective study with 2 sets of CTE image data. Segmentation datasets were used to establish nnU-Net neural network's automatic segmentation model. The classification dataset was processed using the automatic segmentation model to obtain segmentation results and extract radiomics features. The most optimal features were then selected to build 5 machine learning classifiers to distinguish CD activity. The performance of the automatic segmentation model was evaluated using the Dice similarity coefficient, while the performance of the machine learning classifier was evaluated using the area under the curve, sensitivity, specificity, and accuracy. RESULTS The segmentation dataset had 84 CTE examinations of CD patients (mean age 31 ± 13 years , 60 males), and the classification dataset had 193 (mean age 31 ± 12 years , 136 males). The deep learning segmentation model achieved a Dice similarity coefficient of 0.824 on the testing set. The logistic regression model showed the best performance among the 5 classifiers in the testing set, with an area under the curve, sensitivity, specificity, and accuracy of 0.862, 0.697, 0.840, and 0.759, respectively. CONCLUSION The automated segmentation model accurately segments CD lesions, and machine learning classifier distinguishes CD activity well. This method can assist radiologists in promptly and precisely evaluating CD activity.
Collapse
Affiliation(s)
- Yankun Gao
- Department of Radiology, The First Affiliated Hospital of Anhui Medical University, Hefei, China
| | - Bo Zhang
- Department of Radiology, The First Affiliated Hospital of Anhui Medical University, Hefei, China
| | - Dehan Zhao
- Department of Precision Machinery and Precision Instruments, University of Science and Technology of China, Hefei, China
| | - Shuai Li
- Department of Radiology, The First Affiliated Hospital of Anhui Medical University, Hefei, China
| | - Chang Rong
- Department of Radiology, The First Affiliated Hospital of Anhui Medical University, Hefei, China
| | - Mingzhai Sun
- Suzhou Institute for Advanced Research, University of Science and Technology of China, Suzhou, China
| | - Xingwang Wu
- Department of Radiology, The First Affiliated Hospital of Anhui Medical University, Hefei, China
| |
Collapse
|
4
|
Wang Y, Yu M, He M, Zhang G, Zhang L, Zhang B. Diagnostic value of a computer-assisted diagnosis system for the ultrasound features in thyroid nodules. ARCHIVES OF ENDOCRINOLOGY AND METABOLISM 2023; 68:e220501. [PMID: 37948567 PMCID: PMC10916796 DOI: 10.20945/2359-4292-2022-0501] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/04/2023] [Accepted: 03/15/2023] [Indexed: 11/12/2023]
Abstract
Objective To explore the diagnostic value of the TUIAS (SW_TH01/II) computer-aided diagnosis (CAD) software system for the ultrasound Thyroid Imaging Reporting and Data System (TI-RADS) features in thyroid nodules. Materials and methods This retrospective study enrolled patients with thyroid nodules in Shanghai East Hospital between January 2017 and October 2021. The novel CAD software (SW_TH01/II) and three sonographers performed a qualitative analysis of the ultrasound TI-RADS features in aspect ratio, margin irregularity, margin smoothness, calcification, and echogenicity of the thyroid nodules. Results A total of 225 patients were enrolled. The accuracy, sensitivity, and specificity of the CAD software in "aspect ratio" were 95.6%, 96.2%, and 95.4%, in "margin irregularity" were 90.7%, 90.5%, and 90.9%, in "margin smoothness" were 85.8%, 88.5%, and 83.0%, in "calcification" were 83.6%, 81.7%, and 82.0%, in "homogeneity" were 88.9%, 90.6%, and 82.2%, in "major echo" were 85.3%, 88.0%, and 85.4%, and in "contains very hypoechoic echo" were 92.0%, 90.0%, and 92.4%. The analysis time of the CAD software was significantly shorter than for the sonographers (2.7 ± 1.6 vs. 29.7 ± 12.7 s, P < 0.001). Conclusion The CAD system achieved high accuracy in describing thyroid nodule features. It might assist in clinical thyroid nodule analysis.
Collapse
Affiliation(s)
- Yiwei Wang
- Graduate School of Dalian Medical University, Dalian, Liaoning, China,
| | - Ming Yu
- Tend. AI Medical Technology, China
| | | | | | - Libo Zhang
- Shanghai East Hospital, Department of Ultrasound in Medicine, Shanghai, China
| | - Bo Zhang
- Shanghai East Hospital, Tongji University School of Medicine, Department of Ultrasound in Medicine, Shanghai, China
| |
Collapse
|
5
|
Wang Y, Xu L, Lu W, Kong X, Shi K, Wang L, Kong D. Clinical evaluation of malignancy diagnosis of rare thyroid carcinomas by an artificial intelligent automatic diagnosis system. Endocrine 2023; 80:93-99. [PMID: 36462146 DOI: 10.1007/s12020-022-03269-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/16/2022] [Accepted: 11/18/2022] [Indexed: 12/04/2022]
Abstract
PURPOSE To evaluate the application value of a generally trained artificial intelligence (AI) automatic diagnosis system in the malignancy diagnosis of rare thyroid carcinomas, such as follicular thyroid carcinoma, medullary thyroid carcinoma, primary thyroid lymphoma and anaplastic thyroid carcinoma and compare the diagnostic performance with radiologists of different experience levels. METHODS We retrospectively studied 342 patients with 378 thyroid nodules that included 196 rare malignant nodules by using postoperative pathology as the gold standard, and compared the diagnostic performances of three radiologists (one junior, one mid-level, one senior) and that of AI automatic diagnosis system. RESULTS The accuracy of the AI system in malignancy diagnosis was 0.825, which was significantly higher than that of all three radiologists and higher than the best radiologist in this study by a margin of 0.097 with P-value of 2.252 × 10-16. The mid-level radiologist and senior radiologist had higher sensitivity (0.857 and 0.959) than that of the AI system (0.847) at the cost of having much lower specificity (0.533, 0.478 versus 0.802). The junior radiologist showed relatively balanced sensitivity and specificity (0.816 and 0.549) but both were lower than that of the AI system. CONCLUSIONS The generally trained AI automatic diagnosis system showed high accuracy in the differential diagnosis of begin nodules and rare malignancy nodules. It may assist radiologists for screening of rare malignancy nodules that even senior radiologists are not acquainted with.
Collapse
Affiliation(s)
- Yuan Wang
- School of Mathematical Sciences, Zhejiang University, Hangzhou, China
| | - Lei Xu
- Zhejiang Qiushi Institute for Mathematical Medicine, Hangzhou, China
- South and North Lake Institute for Medical Artificial Intelligence, Haiyan, China
- Zhejiang Society for Mathematical Medicine, Hangzhou, China
| | - Wenliang Lu
- School of Mathematical Sciences, Zhejiang University, Hangzhou, China
| | - Xiangkai Kong
- Department of Ultrasonography, The Cancer Hospital of the University of Chinese Academy of Sciences (Zhejiang Cancer Hospital), Institute of Basic Medicine and Cancer, Chinese Academy of Sciences, Hangzhou, China
| | - Kaiyuan Shi
- Department of Ultrasonography, The Cancer Hospital of the University of Chinese Academy of Sciences (Zhejiang Cancer Hospital), Institute of Basic Medicine and Cancer, Chinese Academy of Sciences, Hangzhou, China
| | - Liping Wang
- Zhejiang Society for Mathematical Medicine, Hangzhou, China.
- Department of Ultrasonography, The Cancer Hospital of the University of Chinese Academy of Sciences (Zhejiang Cancer Hospital), Institute of Basic Medicine and Cancer, Chinese Academy of Sciences, Hangzhou, China.
| | - Dexing Kong
- School of Mathematical Sciences, Zhejiang University, Hangzhou, China.
- Zhejiang Qiushi Institute for Mathematical Medicine, Hangzhou, China.
- South and North Lake Institute for Medical Artificial Intelligence, Haiyan, China.
- Zhejiang Society for Mathematical Medicine, Hangzhou, China.
| |
Collapse
|
6
|
Gong L, Zhou P, Li JL, Liu WG. Investigating the diagnostic efficiency of a computer-aided diagnosis system for thyroid nodules in the context of Hashimoto's thyroiditis. Front Oncol 2023; 12:941673. [PMID: 36686823 PMCID: PMC9850089 DOI: 10.3389/fonc.2022.941673] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2022] [Accepted: 12/09/2022] [Indexed: 01/07/2023] Open
Abstract
Objectives This study aims to investigate the efficacy of a computer-aided diagnosis (CAD) system in distinguishing between benign and malignant thyroid nodules in the context of Hashimoto's thyroiditis (HT) and to evaluate the role of the CAD system in reducing unnecessary biopsies of benign lesions. Methods We included a total of 137 nodules from 137 consecutive patients (mean age, 43.5 ± 11.8 years) who were histopathologically diagnosed with HT. The two-dimensional ultrasound images and videos of all thyroid nodules were analyzed by the CAD system and two radiologists with different experiences according to ACR TI-RADS. The diagnostic cutoff values of ACR TI-RADS were divided into two categories (TR4 and TR5), and then the sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) of the CAD system and the junior and senior radiologists were compared in both cases. Moreover, ACR TI-RADS classification was revised according to the results of the CAD system, and the efficacy of recommended fine-needle aspiration (FNA) was evaluated by comparing the unnecessary biopsy rate and the malignant rate of punctured nodules. Results The accuracy, sensitivity, specificity, PPV, and NPV of the CAD system were 0.876, 0.905, 0.830, 0.894, and 0.846, respectively. With TR4 as the cutoff value, the AUCs of the CAD system and the junior and senior radiologists were 0.867, 0.628, and 0.722, respectively, and the CAD system had the highest AUC (P < 0.0001). With TR5 as the cutoff value, the AUCs of the CAD system and the junior and senior radiologists were 0.867, 0.654, and 0.812, respectively, and the CAD system had a higher AUC than the junior radiologist (P < 0.0001) but comparable to the senior radiologist (P = 0.0709). With the assistance of the CAD system, the number of TR4 nodules was decreased by both junior and senior radiologists, the malignant rate of punctured nodules increased by 30% and 22%, and the unnecessary biopsies of benign lesions were both reduced by nearly half. Conclusions The CAD system based on deep learning can improve the diagnostic performance of radiologists in identifying benign and malignant thyroid nodules in the context of Hashimoto's thyroiditis and can play a role in FNA recommendations to reduce unnecessary biopsy rates.
Collapse
|
7
|
Zhou L, Zheng LL, Zhang CJ, Wei HF, Xu LL, Zhang MR, Li Q, He GF, Ghamor-Amegavi EP, Li SY. Comparison of S-Detect and thyroid imaging reporting and data system classifications in the diagnosis of cytologically indeterminate thyroid nodules. Front Endocrinol (Lausanne) 2023; 14:1098031. [PMID: 36761203 PMCID: PMC9902707 DOI: 10.3389/fendo.2023.1098031] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/14/2022] [Accepted: 01/11/2023] [Indexed: 01/26/2023] Open
Abstract
PURPOSE The aim of this study was to investigate the value of S-Detect for predicting the malignant risk of cytologically indeterminate thyroid nodules (CITNs). METHODS The preoperative prediction of 159 CITNs (Bethesda III, IV and V) were performed using S-Detect, Thyroid Imaging Reporting and Data System of American College of Radiology (ACR TI-RADS) and Chinese TI-RADS (C-TIRADS). First, Linear-by-Linear Association test and chi-square test were used to analyze the malignant risk of CITNs. McNemar's test and receiver operating characteristic curve were used to compare the diagnostic efficacy of S-Detect and the two TI-RADS classifications for CITNs. In addition, the McNemar's test was used to compare the diagnostic accuracy of the above three methods for different pathological types of nodules. RESULTS The maximum diameter of the benign nodules was significantly larger than that of malignant nodules [0.88(0.57-1.42) vs 0.57(0.46-0.81), P=0.002]. The risk of malignant CITNs in Bethesda system and the two TI-RADS classifications increased with grade (all P for trend<0.001). In all the enrolled CITNs, the diagnostic results of S-Detect were significantly different from those of ACR TI-RADS and C-TIRADS, respectively (P=0.021 and P=0.007). The sensitivity and accuracy of S-Detect [95.9%(90.1%-98.5%) and 88.1%(81.7%-92.5%)] were higher than those of ACR TI-RADS [87.6%(80.1%-92.7%) and 81.8%(74.7%-87.3%)] (P=0.006 and P=0.021) and C-TIRADS [84.3%(76.3%-90.0%) and 78.6%(71.3%-84.5%)] (P=0.001 and P=0.001). Moreover, the negative predictive value and the area under curve value of S-Detect [82.8% (63.5%-93.5%) and 0.795%(0.724%-0.855%)] was higher than that of C-TIRADS [54.8%(38.8%-69.8%) and 0.724%(0.648%-0.792%] (P=0.024 and P=0.035). However, the specificity and positive predictive value of S-Detect were similar to those of ACR TI-RADS (P=1.000 and P=0.154) and C-TIRADS (P=1.000 and P=0.072). There was no significant difference in all the evaluated indicators between ACR TI-RADS and C-TIRADS (all P>0.05). The diagnostic accuracy of S-Detect (97.4%) for papillary thyroid carcinoma (PTC) was higher than that of ACR TI-RADS (90.4%) and C-TIRADS (87.8%) (P=0.021 and P=0.003). CONCLUSION The diagnostic performance of S-Detect in differentiating CITNs was similar to ACR TI-RADS and superior to C-TIRADS, especially for PTC.
Collapse
Affiliation(s)
- Ling Zhou
- Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| | - Lin-Lin Zheng
- Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| | - Chuan-Ju Zhang
- Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| | - Hong-Fen Wei
- Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| | - Li-Long Xu
- Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| | - Mu-Rui Zhang
- Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| | - Qiang Li
- Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| | - Gao-Fei He
- Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| | | | - Shi-Yan Li
- Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, Hangzhou, China
| |
Collapse
|
8
|
Sorrenti S, Dolcetti V, Radzina M, Bellini MI, Frezza F, Munir K, Grani G, Durante C, D’Andrea V, David E, Calò PG, Lori E, Cantisani V. Artificial Intelligence for Thyroid Nodule Characterization: Where Are We Standing? Cancers (Basel) 2022; 14:cancers14143357. [PMID: 35884418 PMCID: PMC9315681 DOI: 10.3390/cancers14143357] [Citation(s) in RCA: 40] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2022] [Revised: 06/24/2022] [Accepted: 07/08/2022] [Indexed: 12/12/2022] Open
Abstract
Simple Summary In the present review, an up-to-date summary of the state of the art of artificial intelligence (AI) implementation for thyroid nodule characterization and cancer is provided. The opinion on the real effectiveness of AI systems remains controversial. Taking into consideration the largest and most scientifically valid studies, it is possible to state that AI provides results that are comparable or inferior to expert ultrasound specialists and radiologists. Promising data approve AI as a support tool and simultaneously highlight the need for a radiologist supervisory framework for AI provided results. Therefore, current solutions might be more suitable for educational purposes. Abstract Machine learning (ML) is an interdisciplinary sector in the subset of artificial intelligence (AI) that creates systems to set up logical connections using algorithms, and thus offers predictions for complex data analysis. In the present review, an up-to-date summary of the current state of the art regarding ML and AI implementation for thyroid nodule ultrasound characterization and cancer is provided, highlighting controversies over AI application as well as possible benefits of ML, such as, for example, training purposes. There is evidence that AI increases diagnostic accuracy and significantly limits inter-observer variability by using standardized mathematical algorithms. It could also be of aid in practice settings with limited sub-specialty expertise, offering a second opinion by means of radiomics and computer-assisted diagnosis. The introduction of AI represents a revolutionary event in thyroid nodule evaluation, but key issues for further implementation include integration with radiologist expertise, impact on workflow and efficiency, and performance monitoring.
Collapse
Affiliation(s)
- Salvatore Sorrenti
- Department of Surgical Sciences, “Sapienza” University of Rome, 00161 Rome, Italy; (S.S.); (V.D.); (E.L.)
| | - Vincenzo Dolcetti
- Department of Radiological, Anatomo-Pathological Sciences, “Sapienza” University of Rome, 00161 Rome, Italy; (V.D.); (V.C.)
| | - Maija Radzina
- Radiology Research Laboratory, Riga Stradins University, LV-1007 Riga, Latvia;
- Medical Faculty, University of Latvia, Diagnostic Radiology Institute, Paula Stradina Clinical University Hospital, LV-1007 Riga, Latvia
| | - Maria Irene Bellini
- Department of Surgical Sciences, “Sapienza” University of Rome, 00161 Rome, Italy; (S.S.); (V.D.); (E.L.)
- Correspondence:
| | - Fabrizio Frezza
- Department of Information Engineering, Electronics and Telecommunications, “Sapienza” University of Rome, 00184 Rome, Italy; (F.F.); (K.M.)
- Consorzio Nazionale Interuniversitario per le Telecomunicazioni (CNIT), Viale G.P. Usberti 181/A Sede Scientifica di Ingegneria-Palazzina 3, 43124 Parma, Italy
| | - Khushboo Munir
- Department of Information Engineering, Electronics and Telecommunications, “Sapienza” University of Rome, 00184 Rome, Italy; (F.F.); (K.M.)
| | - Giorgio Grani
- Department of Translational and Precision Medicine, “Sapienza” University of Rome, 00161 Rome, Italy; (G.G.); (C.D.); (E.D.)
| | - Cosimo Durante
- Department of Translational and Precision Medicine, “Sapienza” University of Rome, 00161 Rome, Italy; (G.G.); (C.D.); (E.D.)
| | - Vito D’Andrea
- Department of Surgical Sciences, “Sapienza” University of Rome, 00161 Rome, Italy; (S.S.); (V.D.); (E.L.)
| | - Emanuele David
- Department of Translational and Precision Medicine, “Sapienza” University of Rome, 00161 Rome, Italy; (G.G.); (C.D.); (E.D.)
| | - Pietro Giorgio Calò
- Department of Surgical Sciences, “Policlinico Universitario Duilio Casula”, University of Cagliari, 09042 Monserrato, Italy;
| | - Eleonora Lori
- Department of Surgical Sciences, “Sapienza” University of Rome, 00161 Rome, Italy; (S.S.); (V.D.); (E.L.)
| | - Vito Cantisani
- Department of Radiological, Anatomo-Pathological Sciences, “Sapienza” University of Rome, 00161 Rome, Italy; (V.D.); (V.C.)
| |
Collapse
|
9
|
A comparison of artificial intelligence versus radiologists in the diagnosis of thyroid nodules using ultrasonography: a systematic review and meta-analysis. Eur Arch Otorhinolaryngol 2022; 279:5363-5373. [PMID: 35767056 DOI: 10.1007/s00405-022-07436-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2022] [Accepted: 05/06/2022] [Indexed: 11/03/2022]
Abstract
BACKGROUND Thyroid nodules are common. Ultrasonography (US) is the first investigation for thyroid nodules. Artificial Intelligence (AI) is widely integrated into medical diagnosis to provide additional information. The primary objective of this study was to accumulate the pooled sensitivity and specificity between all available AI and radiologists using thyroid US imaging. The secondary objective was to compare AI's diagnostic performance to that of radiologists. MATERIALS AND METHODS A systematic review meta-analysis. PubMed, Scopus, Web of Science, and Cochrane Library data were searched for studies from inception until June 11, 2020. RESULTS Twenty five studies were included in this meta-analysis. The pooled sensitivity and specificity of AI were 0.86 (95% CI 0.81-0.91) and 0.78 (95% CI 0.73-0.83), respectively. The pooled sensitivity and specificity of radiologists were 0.85 (95% CI 0.80-0.89) and 0.82 (95% CI 0.77-0.86), respectively. The accuracy of AI and radiologists is equivalent in terms of AUC [AI 0.89 (95% CI 0.86-0.92), radiologist 0.91 (95% CI 0.88-0.93)]. The diagnostic odd ratio (DOR) between AI 23.10 (95% CI 14.20-37.58) and radiologists 27.12 (95% CI 17.45-42.16) had no statistically significant difference (P = 0.56). Meta-regression analysis revealed that Deep Learning AI had significantly greater sensitivity and specificity than classic machine learning AI (P < 0.001). CONCLUSION AI demonstrated comparable performance to radiologists in diagnosing benign and malignant thyroid nodules using ultrasonography. Additional research to establish its equivalency should be conducted.
Collapse
|
10
|
Radiologists with and without deep learning-based computer-aided diagnosis: comparison of performance and interobserver agreement for characterizing and diagnosing pulmonary nodules/masses. Eur Radiol 2022; 33:348-359. [PMID: 35751697 DOI: 10.1007/s00330-022-08948-4] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2022] [Revised: 05/01/2022] [Accepted: 06/08/2022] [Indexed: 11/04/2022]
Abstract
OBJECTIVES To compare the performance of radiologists in characterizing and diagnosing pulmonary nodules/masses with and without deep learning (DL)-based computer-aided diagnosis (CAD). METHODS We studied a total of 101 nodules/masses detected on CT performed between January and March 2018 at Osaka University Hospital (malignancy: 55 cases). SYNAPSE SAI Viewer V1.4 was used to analyze the nodules/masses. In total, 15 independent radiologists were grouped (n = 5 each) according to their experience: L (< 3 years), M (3-5 years), and H (> 5 years). The likelihoods of 15 characteristics, such as cavitation and calcification, and the diagnosis (malignancy) were evaluated by each radiologist with and without CAD, and the assessment time was recorded. The AUCs compared with the reference standard set by two board-certified chest radiologists were analyzed following the multi-reader multi-case method. Furthermore, interobserver agreement was compared using intraclass correlation coefficients (ICCs). RESULTS The AUCs for ill-defined boundary, irregular margin, irregular shape, calcification, pleural contact, and malignancy in all 15 radiologists, irregular margin and irregular shape in L and ill-defined boundary and irregular margin in M improved significantly (p < 0.05); no significant improvements were found in H. L showed the greatest increase in the AUC for malignancy (not significant). The ICCs improved in all groups and for nearly all items. The median assessment time was not prolonged by CAD. CONCLUSIONS DL-based CAD helps radiologists, particularly those with < 5 years of experience, to accurately characterize and diagnose pulmonary nodules/masses, and improves the reproducibility of findings among radiologists. KEY POINTS • Deep learning-based computer-aided diagnosis improves the accuracy of characterizing nodules/masses and diagnosing malignancy, particularly by radiologists with < 5 years of experience. • Computer-aided diagnosis increases not only the accuracy but also the reproducibility of the findings across radiologists.
Collapse
|
11
|
Personalized Diagnosis in Differentiated Thyroid Cancers by Molecular and Functional Imaging Biomarkers: Present and Future. Diagnostics (Basel) 2022; 12:diagnostics12040944. [PMID: 35453992 PMCID: PMC9030409 DOI: 10.3390/diagnostics12040944] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2022] [Revised: 04/04/2022] [Accepted: 04/08/2022] [Indexed: 11/17/2022] Open
Abstract
Personalized diagnosis can save unnecessary thyroid surgeries, in cases of indeterminate thyroid nodules, when clinicians tend to aggressively treat all these patients. Personalized diagnosis benefits from a combination of imagery and molecular biomarkers, as well as artificial intelligence algorithms, which are used more and more in our timeline. Functional imaging diagnosis such as SPECT, PET, or fused images (SPECT/CT, PET/CT, PET/MRI), is exploited at maximum in thyroid nodules, with a long history in the past and a bright future with many suitable radiotracers that could properly contribute to diagnosing malignancy in thyroid nodules. In this way, patients will be spared surgery complications, and apparently more expensive diagnostic workouts will financially compensate each patient and also the healthcare system. In this review we will summarize essential available diagnostic tools for malignant and benignant thyroid nodules, beginning with functional imaging, molecular analysis, and combinations of these two and other future strategies, including AI or NIS targeted gene therapy for thyroid carcinoma diagnosis and treatment as well.
Collapse
|
12
|
Wang B, Wan Z, Li C, Zhang M, Shi Y, Miao X, Jian Y, Luo Y, Yao J, Tian W. Identification of benign and malignant thyroid nodules based on dynamic AI ultrasound intelligent auxiliary diagnosis system. Front Endocrinol (Lausanne) 2022; 13:1018321. [PMID: 36237194 PMCID: PMC9551607 DOI: 10.3389/fendo.2022.1018321] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/13/2022] [Accepted: 09/06/2022] [Indexed: 11/22/2022] Open
Abstract
BACKGROUND Dynamic artificial intelligence (AI) ultrasound intelligent auxiliary diagnosis system (Dynamic AI) is a joint application of AI technology and medical imaging data, which can perform a real-time synchronous dynamic analysis of nodules. The aim of this study is to investigate the value of dynamic AI in differentiating benign and malignant thyroid nodules and its guiding significance for treatment strategies. METHODS The data of 607 patients with 1007 thyroid nodules who underwent surgical treatment were reviewed and analyzed, retrospectively. Dynamic AI was used to differentiate benign and malignant nodules. The diagnostic efficacy of dynamic AI was evaluated by comparing the results of dynamic AI examination, preoperative fine needle aspiration cytology (FNAC) and postoperative pathology of nodules with different sizes and properties in patients of different sexes and ages. RESULTS The sensitivity, specificity and accuracy of dynamic AI in the diagnosis of thyroid nodules were 92.21%, 83.20% and 89.97%, respectively, which were highly consistent with the postoperative pathological results (kappa = 0.737, p < 0.001). There is no statistical difference in accuracy between people with different ages and sexes and nodules of different sizes, which showed the good stability. The accuracy of dynamic AI in malignant nodules (92.21%) was significantly higher than that in benign nodules (83.20%) (p < 0.001). The specificity and positive predictive value were significantly higher, and the misdiagnosis rate was significantly lower in dynamic AI than that of preoperative ultrasound ACR TI-RADS (p < 0.001). The accuracy of dynamic AI in nodules with diameter ≤ 0.50 cm was significantly higher than that of preoperative ultrasound (p = 0.044). Compared with FNAC, the sensitivity (96.58%) and accuracy (94.06%) of dynamic AI were similar. CONCLUSIONS The dynamic AI examination has high diagnostic value for benign and malignant thyroid nodules, which can effectively assist surgeons in formulating scientific and reasonable individualized diagnosis and treatment strategies for patients.
Collapse
Affiliation(s)
- Bing Wang
- Senior Department of General Surgery, The First Medical Center of Chinese People's Liberation Army (PLA) General Hospital, Beijing, China
| | - Zheng Wan
- Senior Department of General Surgery, The First Medical Center of Chinese People's Liberation Army (PLA) General Hospital, Beijing, China
| | - Chen Li
- Senior Department of General Surgery, The First Medical Center of Chinese People's Liberation Army (PLA) General Hospital, Beijing, China
| | - Mingbo Zhang
- Department of Ultrasound, The First Medical Center of Chinese People's Liberation Army (PLA) General Hospital, Beijing, China
| | - YiLei Shi
- MedAI Technology (Wuxi) Co. Ltd, Wuxi, China
| | - Xin Miao
- Senior Department of General Surgery, The First Medical Center of Chinese People's Liberation Army (PLA) General Hospital, Beijing, China
| | - Yanbing Jian
- Senior Department of General Surgery, The First Medical Center of Chinese People's Liberation Army (PLA) General Hospital, Beijing, China
| | - Yukun Luo
- Department of Ultrasound, The First Medical Center of Chinese People's Liberation Army (PLA) General Hospital, Beijing, China
| | - Jing Yao
- Senior Department of General Surgery, The First Medical Center of Chinese People's Liberation Army (PLA) General Hospital, Beijing, China
- *Correspondence: Jing Yao, ; Wen Tian,
| | - Wen Tian
- Senior Department of General Surgery, The First Medical Center of Chinese People's Liberation Army (PLA) General Hospital, Beijing, China
- *Correspondence: Jing Yao, ; Wen Tian,
| |
Collapse
|
13
|
Diagnosing thyroid nodules with atypia of undetermined significance/follicular lesion of undetermined significance cytology with the deep convolutional neural network. Sci Rep 2021; 11:20048. [PMID: 34625636 PMCID: PMC8501016 DOI: 10.1038/s41598-021-99622-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2021] [Accepted: 09/24/2021] [Indexed: 11/08/2022] Open
Abstract
To compare the diagnostic performances of physicians and a deep convolutional neural network (CNN) predicting malignancy with ultrasonography images of thyroid nodules with atypia of undetermined significance (AUS)/follicular lesion of undetermined significance (FLUS) results on fine-needle aspiration (FNA). This study included 202 patients with 202 nodules ≥ 1 cm AUS/FLUS on FNA, and underwent surgery in one of 3 different institutions. Diagnostic performances were compared between 8 physicians (4 radiologists, 4 endocrinologists) with varying experience levels and CNN, and AUS/FLUS subgroups were analyzed. Interobserver variability was assessed among the 8 physicians. Of the 202 nodules, 158 were AUS, and 44 were FLUS; 86 were benign, and 116 were malignant. The area under the curves (AUCs) of the 8 physicians and CNN were 0.680–0.722 and 0.666, without significant differences (P > 0.05). In the subgroup analysis, the AUCs for the 8 physicians and CNN were 0.657–0.768 and 0.652 for AUS, 0.469–0.674 and 0.622 for FLUS. Interobserver agreements were moderate (k = 0.543), substantial (k = 0.652), and moderate (k = 0.455) among the 8 physicians, 4 radiologists, and 4 endocrinologists. For thyroid nodules with AUS/FLUS cytology, the diagnostic performance of CNN to differentiate malignancy with US images was comparable to that of physicians with variable experience levels.
Collapse
|
14
|
Artificial Intelligence in Thyroid Field-A Comprehensive Review. Cancers (Basel) 2021; 13:cancers13194740. [PMID: 34638226 PMCID: PMC8507551 DOI: 10.3390/cancers13194740] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2021] [Revised: 09/19/2021] [Accepted: 09/20/2021] [Indexed: 12/12/2022] Open
Abstract
Simple Summary The incidence of thyroid pathologies has been increasing worldwide. Historically, the detection of thyroid neoplasms relies on medical imaging analysis, depending mainly on the experience of clinicians. The advent of artificial intelligence (AI) techniques led to a remarkable progress in image-recognition tasks. AI represents a powerful tool that may facilitate understanding of thyroid pathologies, but actually, the diagnostic accuracy is uncertain. This article aims to provide an overview of the basic aspects, limitations and open issues of the AI methods applied to thyroid images. Medical experts should be familiar with the workflow of AI techniques in order to avoid misleading outcomes. Abstract Artificial intelligence (AI) uses mathematical algorithms to perform tasks that require human cognitive abilities. AI-based methodologies, e.g., machine learning and deep learning, as well as the recently developed research field of radiomics have noticeable potential to transform medical diagnostics. AI-based techniques applied to medical imaging allow to detect biological abnormalities, to diagnostic neoplasms or to predict the response to treatment. Nonetheless, the diagnostic accuracy of these methods is still a matter of debate. In this article, we first illustrate the key concepts and workflow characteristics of machine learning, deep learning and radiomics. We outline considerations regarding data input requirements, differences among these methodologies and their limitations. Subsequently, a concise overview is presented regarding the application of AI methods to the evaluation of thyroid images. We developed a critical discussion concerning limits and open challenges that should be addressed before the translation of AI techniques to the broad clinical use. Clarification of the pitfalls of AI-based techniques results crucial in order to ensure the optimal application for each patient.
Collapse
|
15
|
Luo P, Fang Z, Zhang P, Yang Y, Zhang H, Su L, Wang Z, Ren J. Radiomics Score Combined with ACR TI-RADS in Discriminating Benign and Malignant Thyroid Nodules Based on Ultrasound Images: A Retrospective Study. Diagnostics (Basel) 2021; 11:diagnostics11061011. [PMID: 34205943 PMCID: PMC8229428 DOI: 10.3390/diagnostics11061011] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2021] [Revised: 05/25/2021] [Accepted: 05/26/2021] [Indexed: 12/12/2022] Open
Abstract
This study aimed to explore the ability of combination model of ultrasound radiomics score (Rad-score) and the thyroid imaging, reporting and data system by the American College of Radiology (ACR TI-RADS) in predicting benign and malignant thyroid nodules (TNs). Up to 286 radiomics features were extracted from ultrasound images of TNs. By using the lowest probability of classification error and average correlation coefficients (POE + ACC) and the least absolute shrinkage and selection operator (LASSO), we finally selected four features to establish Rad-score (Vertl-RLNonUni, Vertl-GLevNonU, WavEnLH-s4 and WavEnHL-s5). DeLong’s test and decision curve analysis (DCA) showed that the method of combining Rad-score and ACR TI-RADS had the best performance (the area under the receiver operating characteristic curve (AUC = 0.913 (95% confidence interval (CI), 0.881–0.939) and 0.899 (95%CI, 0.840–0.942) in the training group and verification group, respectively), followed by ACR TI-RADS (AUC = 0.898 (95%CI, 0.863–0.926) and 0.870 (95%CI, 0.806–0.919) in the training group and verification group, respectively), and followed by Rad-score (AUC = 0.750 (95%CI, 0.704–0.792) and 0.750 (95%CI, 0.672–0.817) in the training group and verification group, respectively). We concluded that the ability of ultrasound Rad-score to distinguish benign and malignant TNs was not as good as that of ACR TI-RADS, and the ability of the combination model of Rad-score and ACR TI-RADS to discriminate benign and malignant TNs was better than ACR TI-RADS or Rad-score alone. Ultrasound Rad-score might play a potential role in improving the differentiation of malignant TNs from benign TNs.
Collapse
Affiliation(s)
- Peng Luo
- Department of Ultrasound, The Second Affiliated Hospital of Chongqing Medical University, Chongqing 400010, China; (P.L.); (P.Z.); (Y.Y.); (H.Z.); (L.S.); (Z.W.)
| | - Zheng Fang
- Department of Radiology, The Second Affiliated Hospital of Chongqing Medical University, Chongqing 400010, China;
| | - Ping Zhang
- Department of Ultrasound, The Second Affiliated Hospital of Chongqing Medical University, Chongqing 400010, China; (P.L.); (P.Z.); (Y.Y.); (H.Z.); (L.S.); (Z.W.)
| | - Yang Yang
- Department of Ultrasound, The Second Affiliated Hospital of Chongqing Medical University, Chongqing 400010, China; (P.L.); (P.Z.); (Y.Y.); (H.Z.); (L.S.); (Z.W.)
| | - Hua Zhang
- Department of Ultrasound, The Second Affiliated Hospital of Chongqing Medical University, Chongqing 400010, China; (P.L.); (P.Z.); (Y.Y.); (H.Z.); (L.S.); (Z.W.)
| | - Lei Su
- Department of Ultrasound, The Second Affiliated Hospital of Chongqing Medical University, Chongqing 400010, China; (P.L.); (P.Z.); (Y.Y.); (H.Z.); (L.S.); (Z.W.)
| | - Zhigang Wang
- Department of Ultrasound, The Second Affiliated Hospital of Chongqing Medical University, Chongqing 400010, China; (P.L.); (P.Z.); (Y.Y.); (H.Z.); (L.S.); (Z.W.)
| | - Jianli Ren
- Department of Ultrasound, The Second Affiliated Hospital of Chongqing Medical University, Chongqing 400010, China; (P.L.); (P.Z.); (Y.Y.); (H.Z.); (L.S.); (Z.W.)
- Correspondence:
| |
Collapse
|
16
|
Kim GR, Lee E, Kim HR, Yoon JH, Park VY, Kwak JY. Convolutional Neural Network to Stratify the Malignancy Risk of Thyroid Nodules: Diagnostic Performance Compared with the American College of Radiology Thyroid Imaging Reporting and Data System Implemented by Experienced Radiologists. AJNR Am J Neuroradiol 2021; 42:1513-1519. [PMID: 33985947 DOI: 10.3174/ajnr.a7149] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2020] [Accepted: 03/06/2021] [Indexed: 12/12/2022]
Abstract
BACKGROUND AND PURPOSE Comparison of the diagnostic performance for thyroid cancer on ultrasound between a convolutional neural network and visual assessment by radiologists has been inconsistent. Thus, we aimed to evaluate the diagnostic performance of the convolutional neural network compared with the American College of Radiology Thyroid Imaging Reporting and Data System (TI-RADS) for the diagnosis of thyroid cancer using ultrasound images. MATERIALS AND METHODS From March 2019 to September 2019, seven hundred sixty thyroid nodules (≥10 mm) in 757 patients were diagnosed as benign or malignant through fine-needle aspiration, core needle biopsy, or an operation. Experienced radiologists assessed the sonographic descriptors of the nodules, and 1 of 5 American College of Radiology TI-RADS categories was assigned. The convolutional neural network provided malignancy risk percentages for nodules based on sonographic images. Sensitivity, specificity, accuracy, positive predictive value, and negative predictive value were calculated with cutoff values using the Youden index and compared between the convolutional neural network and the American College of Radiology TI-RADS. Areas under the receiver operating characteristic curve were also compared. RESULTS Of 760 nodules, 176 (23.2%) were malignant. At an optimal threshold derived from the Youden index, sensitivity and negative predictive values were higher with the convolutional neural network than with the American College of Radiology TI-RADS (81.8% versus 73.9%, P = .009; 94.0% versus 92.2%, P = .046). Specificity, accuracy, and positive predictive values were lower with the convolutional neural network than with the American College of Radiology TI-RADS (86.1% versus 93.7%, P < .001; 85.1% versus 89.1%, P = .003; and 64.0% versus 77.8%, P < .001). The area under the curve of the convolutional neural network was higher than that of the American College of Radiology TI-RADS (0.917 versus 0.891, P = .017). CONCLUSIONS The convolutional neural network provided diagnostic performance comparable with that of the American College of Radiology TI-RADS categories assigned by experienced radiologists.
Collapse
Affiliation(s)
- G R Kim
- From the Department of Radiology (G.R.K., J.H.Y., V.Y.P., J.Y.K.), Severance Hospital, Research Institute of Radiological Science, Center for Clinical Imaging Data Science, Yonsei University College of Medicine, Seoul, Korea
| | - E Lee
- Department of Computational Science and Engineering (E.L.), Yonsei University, Seoul, Korea
| | - H R Kim
- Biostatistics Collaboration Unit (H.R.K.), Department of Biomedical Systems Informatics, Yonsei University College of Medicine, Seoul, Korea
| | - J H Yoon
- From the Department of Radiology (G.R.K., J.H.Y., V.Y.P., J.Y.K.), Severance Hospital, Research Institute of Radiological Science, Center for Clinical Imaging Data Science, Yonsei University College of Medicine, Seoul, Korea
| | - V Y Park
- From the Department of Radiology (G.R.K., J.H.Y., V.Y.P., J.Y.K.), Severance Hospital, Research Institute of Radiological Science, Center for Clinical Imaging Data Science, Yonsei University College of Medicine, Seoul, Korea
| | - J Y Kwak
- From the Department of Radiology (G.R.K., J.H.Y., V.Y.P., J.Y.K.), Severance Hospital, Research Institute of Radiological Science, Center for Clinical Imaging Data Science, Yonsei University College of Medicine, Seoul, Korea
| |
Collapse
|
17
|
The value of the Demetics ultrasound-assisted diagnosis system in the differential diagnosis of benign from malignant thyroid nodules and analysis of the influencing factors. Eur Radiol 2021; 31:7936-7944. [PMID: 33856523 DOI: 10.1007/s00330-021-07884-z] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2020] [Revised: 02/18/2021] [Accepted: 03/15/2021] [Indexed: 12/12/2022]
Abstract
OBJECTIVES To evaluate the value of Demetics and to explore whether Demetics can help radiologists with varying years of experience in the differential diagnosis of benign from malignant thyroid nodules. METHODS The clinical application value of Demetics was assessed by comparing the diagnostic accuracy of radiologists before and after applying Demetics. This retrospective analysis included 284 thyroid nodules that underwent pathological examinations. Two different combined methods were applied. Using method 1: the original TI-RADS classification was forcibly upgraded or downgraded by one level when Demetics classified the thyroid nodules as malignant or benign. Using method 2: the TI-RADS and benign or malignant classification of the thyroid nodules were flexibly adjusted after the physician learned the Demetics' results. RESULTS Demetics exhibited a higher sensitivity than did junior radiologist 1 (pD1 = 0.029) and was similar in sensitivity to the two senior radiologists. Demetics had a higher AUC than both junior radiologists (pD1 = 0.042, pD2 = 0.038) and an AUC similar to that of the senior radiologists. The sensitivity (p = 0.035) and AUC (p = 0.031) of junior radiologist 1 and the specificity (p < 0.001) and AUC (p = 0.026) of junior radiologist 2 improved with combined method 1. The AUC of junior radiologist 2 improved with combined method 2 (p = 0.045). The factors influencing the diagnostic results of Demetics include sonographic signs (echogenicity and echogenic foci), contrast of the image, and nodule size. CONCLUSION Demetics exhibited high sensitivity and accuracy in the differential diagnosis of benign from malignant thyroid nodules. Demetics could improve the diagnostic accuracy of junior radiologists. KEY POINTS • Demetics exhibited a high sensitivity and accuracy in the differential diagnosis of benign from malignant thyroid nodules. • Demetics could improve the diagnostic accuracy of junior radiologists in the differential diagnosis of benign from malignant thyroid nodules. • Factors influencing the diagnostic results of Demetics include the sonographic signs (echogenicity and echogenic foci), contrast of the image, and nodule size.
Collapse
|
18
|
Jin Z, Zhu Y, Zhang S, Xie F, Zhang M, Guo Y, Wang H, Zhu Q, Cao J, Luo Y. Diagnosis of thyroid cancer using a TI-RADS-based computer-aided diagnosis system: a multicenter retrospective study. Clin Imaging 2021; 80:43-49. [PMID: 34237590 DOI: 10.1016/j.clinimag.2020.12.012] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2020] [Revised: 11/18/2020] [Accepted: 12/01/2020] [Indexed: 10/22/2022]
Abstract
OBJECTIVE The purpose of this study was to use a computer-aided diagnosis (CAD) system based on the Thyroid Imaging, Reporting, and Data System (TI-RADS) to improve the diagnostic performance of thyroid cancer by analyzing clinical ultrasound imaging data. METHODS A retrospective diagnostic study of ultrasound image sets was conducted at five hospitals in China. A CAD system based on TI-RADS was applied in this study, and the diagnostic performance of CAD system was tested through multi-center data. The performance of the CAD system was compared with the consensus of three experienced radiologists. The interobserver agreement for cancer diagnosis was calculated between the CAD system and the consensus of the three experienced radiologists. RESULTS The CAD system performed well in the diagnosis of thyroid cancer, with an area under the curve (AUC) value of 0.902 (95% CI: 0.884-0.918), and obtained results similar to those of the three experienced radiologists. The CAD system performed better in the internal test set than in the external test set (AUC: 0.930 vs 0.877, respectively). The performance of the CAD system in the diagnosis of thyroid cancer for nodules of different sizes (<1 cm, 1-2 cm and ≥2 cm) was basically similar (accuracy: 84.6% vs 85% vs 84.2%). The CAD system can recognize 15 ultrasound features of thyroid nodules, most of which reached the level of 3 experienced radiologists (12/15, 85%). CONCLUSION The CAD system achieved an improved AUC and similar sensitivity and specificity in the diagnosis of thyroid cancer compared with the consensus of experienced radiologists.
Collapse
Affiliation(s)
- Zhuang Jin
- Department of Ultrasound, Chinese PLA General Hospital, No. 28, Fuxing Road, Haidian District, Beijing, China
| | - Yaqiong Zhu
- Department of Ultrasound, Chinese PLA General Hospital, No. 28, Fuxing Road, Haidian District, Beijing, China; Nankai University, No. 94 Weijin Road, Nankai District, Tianjin City, China
| | - Shijie Zhang
- Peking University, No. 5 Yiheyuan Road, Haidian District, Beijing 10087, China
| | - Fang Xie
- Department of Ultrasound, Chinese PLA General Hospital, No. 28, Fuxing Road, Haidian District, Beijing, China
| | - Mingbo Zhang
- Department of Ultrasound, Chinese PLA General Hospital, No. 28, Fuxing Road, Haidian District, Beijing, China
| | - Yanli Guo
- Department of Ultrasound, Southwest Hospital, Third Military Medical University (Army Medical University), Shapingba District, Chongqing, China
| | - Hui Wang
- Department of Ultrasound, China-Japan Union Hospital of Jilin University, Changchun 130000, Jilin, China
| | - Qiang Zhu
- Department of Diagnostic Ultrasound, Beijing Tongren Hospital, Capital Medical University, Beijing, China
| | - Junying Cao
- Department of Ultrasound, General Hospital of Northern Theater Command, No. 83, Wenhua Road, Shenhe District, Shenyang, Liaoning Province 110018, China.
| | - Yukun Luo
- Department of Ultrasound, Chinese PLA General Hospital, No. 28, Fuxing Road, Haidian District, Beijing, China.
| |
Collapse
|
19
|
Li LR, Du B, Liu HQ, Chen C. Artificial Intelligence for Personalized Medicine in Thyroid Cancer: Current Status and Future Perspectives. Front Oncol 2021; 10:604051. [PMID: 33634025 PMCID: PMC7899964 DOI: 10.3389/fonc.2020.604051] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2020] [Accepted: 12/21/2020] [Indexed: 12/12/2022] Open
Abstract
Thyroid cancers (TC) have increasingly been detected following advances in diagnostic methods. Risk stratification guided by refined information becomes a crucial step toward the goal of personalized medicine. The diagnosis of TC mainly relies on imaging analysis, but visual examination may not reveal much information and not enable comprehensive analysis. Artificial intelligence (AI) is a technology used to extract and quantify key image information by simulating complex human functions. This latent, precise information contributes to stratify TC on the distinct risk and drives tailored management to transit from the surface (population-based) to a point (individual-based). In this review, we started with several challenges regarding personalized care in TC, for example, inconsistent rating ability of ultrasound physicians, uncertainty in cytopathological diagnosis, difficulty in discriminating follicular neoplasms, and inaccurate prognostication. We then analyzed and summarized the advances of AI to extract and analyze morphological, textural, and molecular features to reveal the ground truth of TC. Consequently, their combination with AI technology will make individual medical strategies possible.
Collapse
Affiliation(s)
- Ling-Rui Li
- Department of Breast and Thyroid Surgery, Renmin Hospital of Wuhan University, Wuhan, China
| | - Bo Du
- School of Computer Science, Wuhan University, Wuhan, China.,Institute of Artificial Intelligence, Wuhan University, Wuhan, China
| | - Han-Qing Liu
- Department of Breast and Thyroid Surgery, Renmin Hospital of Wuhan University, Wuhan, China
| | - Chuang Chen
- Department of Breast and Thyroid Surgery, Renmin Hospital of Wuhan University, Wuhan, China
| |
Collapse
|
20
|
Chambara N, Liu SYW, Lo X, Ying M. Diagnostic performance evaluation of different TI-RADS using ultrasound computer-aided diagnosis of thyroid nodules: An experience with adjusted settings. PLoS One 2021; 16:e0245617. [PMID: 33449958 PMCID: PMC7810331 DOI: 10.1371/journal.pone.0245617] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2020] [Accepted: 01/04/2021] [Indexed: 01/25/2023] Open
Abstract
BACKGROUND Thyroid cancer diagnosis has evolved to include computer-aided diagnosis (CAD) approaches to overcome the limitations of human ultrasound feature assessment. This study aimed to evaluate the diagnostic performance of a CAD system in thyroid nodule differentiation using varied settings. METHODS Ultrasound images of 205 thyroid nodules from 198 patients were analysed in this retrospective study. AmCAD-UT software was used at default settings and 3 adjusted settings to diagnose the nodules. Six risk-stratification systems in the software were used to classify the thyroid nodules: The American Thyroid Association (ATA), American College of Radiology Thyroid Imaging, Reporting, and Data System (ACR-TIRADS), British Thyroid Association (BTA), European Union (EU-TIRADS), Kwak (2011) and the Korean Society of Thyroid Radiology (KSThR). The diagnostic performance of CAD was determined relative to the histopathology and/or cytology diagnosis of each nodule. RESULTS At the default setting, EU-TIRADS yielded the highest sensitivity, 82.6% and lowest specificity, 42.1% while the ATA-TIRADS yielded the highest specificity, 66.4%. Kwak had the highest AUROC (0.74) which was comparable to that of ACR, ATA, and KSThR TIRADS (0.72, 0.73, and 0.70 respectively). At a hyperechoic foci setting of 3.5 with other settings at median values; ATA had the best-balanced sensitivity, specificity and good AUROC (70.4%; 67.3% and 0.71 respectively). CONCLUSION The default setting achieved the best diagnostic performance with all TIRADS and was best for maximizing the sensitivity of EU-TIRADS. Adjusting the settings by only reducing the sensitivity to echogenic foci may be most helpful for improving specificity with minimal change in sensitivity.
Collapse
Affiliation(s)
- Nonhlanhla Chambara
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong, SAR, China
| | - Shirley Y. W. Liu
- Department of Surgery, Prince of Wales Hospital, Shatin, New Territories, Hong Kong, SAR, China
| | - Xina Lo
- Department of Surgery, North District Hospital, Sheung Shui, New Territories, Hong Kong SAR, China
| | - Michael Ying
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong, SAR, China
- * E-mail:
| |
Collapse
|
21
|
Li M, Wei L, Li F, Kan Y, Liang X, Zhang H, Liu J. High Risk Thyroid Nodule Discrimination and Management by Modified TI-RADS. Cancer Manag Res 2021; 13:225-234. [PMID: 33469366 PMCID: PMC7810593 DOI: 10.2147/cmar.s284370] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2020] [Accepted: 11/23/2020] [Indexed: 11/24/2022] Open
Abstract
Objective Thyroid cancer is the most common primary malignant disorder of the thyroid. We aimed to illustrate the modified TI-RADS report system for differentiating malignant thyroid nodules from benign ones, and especially its role in the management of high risk nodules. Methods In this retrospective study, 5,162 healthy individuals who underwent thyroid ultrasound according to modified TI-RADS from January 2014 to December 2014 were enrolled and followed up during the whole 5 years, and the medical data were investigated and reviewed. Results The total detection rate of thyroid nodules was 39.40%. The total detection rate of thyroid cancer was 0.66%. Most thyroid cancers were single-shot, located at unilateral, at early clinical stages, without lymph node metastases, and with low recurrence risk. All patients had thyroid papillary carcinoma, except one thyroid medullary carcinoma. Based on modified TI-RADS classification, at the end of 5 years follow-up, more changes of thyroid nodules grade status were observed in grades 4a and above. The higher the grade status, the more malignant advances were occurred. The modified TI-RADS report system played an instructional role in adding medical treatment choice and decision for clinicians. Conclusion The modified TI-RADS report system plays an important role in thyroid benign and malignant nodule identification and management.
Collapse
Affiliation(s)
- Menghui Li
- Cancer Prevention Center, Tianjin Medical University Cancer Institute and Hospital, Tianjin, People's Republic of China.,National Clinical Research Center for Cancer, Key Laboratory of Cancer Prevention and Therapy, Tianjin's Clinical Research Center for Cancer, Tianjin, People's Republic of China
| | - Lijuan Wei
- Cancer Prevention Center, Tianjin Medical University Cancer Institute and Hospital, Tianjin, People's Republic of China
| | - Fangxuan Li
- Cancer Prevention Center, Tianjin Medical University Cancer Institute and Hospital, Tianjin, People's Republic of China
| | - Yanyan Kan
- Cancer Prevention Center, Tianjin Medical University Cancer Institute and Hospital, Tianjin, People's Republic of China
| | - Xiaofeng Liang
- Cancer Prevention Center, Tianjin Medical University Cancer Institute and Hospital, Tianjin, People's Republic of China
| | - Huan Zhang
- Cancer Prevention Center, Tianjin Medical University Cancer Institute and Hospital, Tianjin, People's Republic of China
| | - Juntian Liu
- Cancer Prevention Center, Tianjin Medical University Cancer Institute and Hospital, Tianjin, People's Republic of China.,National Clinical Research Center for Cancer, Key Laboratory of Cancer Prevention and Therapy, Tianjin's Clinical Research Center for Cancer, Tianjin, People's Republic of China
| |
Collapse
|
22
|
S-Detect Software vs. EU-TIRADS Classification: A Dual-Center Validation of Diagnostic Performance in Differentiation of Thyroid Nodules. J Clin Med 2020; 9:jcm9082495. [PMID: 32756510 PMCID: PMC7464710 DOI: 10.3390/jcm9082495] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2020] [Revised: 07/23/2020] [Accepted: 07/30/2020] [Indexed: 12/12/2022] Open
Abstract
Computer-aided diagnosis (CAD) and other risk stratification systems may improve ultrasound image interpretation. This prospective study aimed to compare the diagnostic performance of CAD and the European Thyroid Imaging Reporting and Data System (EU-TIRADS) classification applied by physicians with S-Detect 2 software CAD based on Korean Thyroid Imaging Reporting and Data System (K-TIRADS) and combinations of both methods (MODELs 1 to 5). In all, 133 nodules from 88 patients referred to thyroidectomy with available histopathology or with unambiguous results of cytology were included. The S-Detect system, EU-TIRADS, and mixed MODELs 1–5 for the diagnosis of thyroid cancer showed a sensitivity of 89.4%, 90.9%, 84.9%, 95.5%, 93.9%, 78.9% and 93.9%; a specificity of 80.6%, 61.2%, 88.1%, 53.7%, 73.1%, 89.6% and 80.6%; a positive predictive value of 81.9%, 69.8%, 87.5%, 67%, 77.5%, 88.1% and 82.7%; a negative predictive value of 88.5%, 87.2%, 85.5%, 92.3%, 92.5%, 81.1% and 93.1%; and an accuracy of 85%, 75.9%, 86.5%, 74.4%, 83.5%, 84.2%, and 87.2%, respectively. Comparison showed superiority of the similar MODELs 1 and 5 over other mixed models as well as EU-TIRADS and S-Detect used alone (p-value < 0.05). S-Detect software is characterized with high sensitivity and good specificity, whereas EU-TIRADS has high sensitivity, but rather low specificity. The best diagnostic performance in malignant thyroid nodule (TN) risk stratification was obtained for the combined model of S-Detect (“possibly malignant” nodule) and simultaneously obtaining 4 or 5 points (MODEL 1) or exactly 5 points (MODEL 5) on the EU-TIRADS scale.
Collapse
|
23
|
Wang Y, Yue W, Li X, Liu S, Guo L, Xu H, Zhang H, Yang G. Comparison Study of Radiomics and Deep Learning-Based Methods for Thyroid Nodules Classification Using Ultrasound Images. IEEE ACCESS 2020. [DOI: 10.1109/access.2020.2980290] [Citation(s) in RCA: 31] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/07/2022]
|