1
|
Vahdati S, Khosravi B, Robinson KA, Rouzrokh P, Moassefi M, Akkus Z, Erickson BJ. A Multi-View Deep Learning Model for Thyroid Nodules Detection and Characterization in Ultrasound Imaging. Bioengineering (Basel) 2024; 11:648. [PMID: 39061730 PMCID: PMC11273835 DOI: 10.3390/bioengineering11070648] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2024] [Revised: 05/27/2024] [Accepted: 06/13/2024] [Indexed: 07/28/2024] Open
Abstract
Thyroid Ultrasound (US) is the primary method to evaluate thyroid nodules. Deep learning (DL) has been playing a significant role in evaluating thyroid cancer. We propose a DL-based pipeline to detect and classify thyroid nodules into benign or malignant groups relying on two views of US imaging. Transverse and longitudinal US images of thyroid nodules from 983 patients were collected retrospectively. Eighty-one cases were held out as a testing set, and the rest of the data were used in five-fold cross-validation (CV). Two You Look Only Once (YOLO) v5 models were trained to detect nodules and classify them. For each view, five models were developed during the CV, which was ensembled by using non-max suppression (NMS) to boost their collective generalizability. An extreme gradient boosting (XGBoost) model was trained on the outputs of the ensembled models for both views to yield a final prediction of malignancy for each nodule. The test set was evaluated by an expert radiologist using the American College of Radiology Thyroid Imaging Reporting and Data System (ACR-TIRADS). The ensemble models for each view achieved a mAP0.5 of 0.797 (transverse) and 0.716 (longitudinal). The whole pipeline reached an AUROC of 0.84 (CI 95%: 0.75-0.91) with sensitivity and specificity of 84% and 63%, respectively, while the ACR-TIRADS evaluation of the same set had a sensitivity of 76% and specificity of 34% (p-value = 0.003). Our proposed work demonstrated the potential possibility of a deep learning model to achieve diagnostic performance for thyroid nodule evaluation.
Collapse
Affiliation(s)
- Sanaz Vahdati
- Artificial Intelligence Laboratory, Department of Radiology, Mayo Clinic, 200 1st Street, SW, Rochester, MN 55905, USA
| | - Bardia Khosravi
- Artificial Intelligence Laboratory, Department of Radiology, Mayo Clinic, 200 1st Street, SW, Rochester, MN 55905, USA
| | - Kathryn A. Robinson
- Department of Radiology, Mayo Clinic, 200 1st Street, SW, Rochester, MN 55905, USA
| | - Pouria Rouzrokh
- Artificial Intelligence Laboratory, Department of Radiology, Mayo Clinic, 200 1st Street, SW, Rochester, MN 55905, USA
| | - Mana Moassefi
- Artificial Intelligence Laboratory, Department of Radiology, Mayo Clinic, 200 1st Street, SW, Rochester, MN 55905, USA
| | - Zeynettin Akkus
- Department of Laboratory Medicine and Pathology, Mayo Clinic, Jacksonville, FL 32224, USA
| | - Bradley J. Erickson
- Artificial Intelligence Laboratory, Department of Radiology, Mayo Clinic, 200 1st Street, SW, Rochester, MN 55905, USA
- Department of Radiology, Mayo Clinic, 200 1st Street, SW, Rochester, MN 55905, USA
| |
Collapse
|
2
|
Yang X, Zhang B, Liu Y, Lv Q, Guo J. Automatic Quantitative Assessment of Muscle Strength Based on Deep Learning and Ultrasound. ULTRASONIC IMAGING 2024:1617346241255590. [PMID: 38881032 DOI: 10.1177/01617346241255590] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2024]
Abstract
Skeletal muscle is a vital organ that promotes human movement and maintains posture. Accurate assessment of muscle strength is helpful to provide valuable insights for athletes' rehabilitation and strength training. However, traditional techniques rely heavily on the operator's expertise, which may affect the accuracy of the results. In this study, we propose an automated method to evaluate muscle strength using ultrasound and deep learning techniques. B-mode ultrasound data of biceps brachii of multiple athletes at different strength levels were collected and then used to train our deep learning model. To evaluate the effectiveness of this method, this study tested the contraction of the biceps brachii under different force levels. The classification accuracy of this method for grade 4 and grade 6 muscle strength reached 98% and 96%, respectively, and the overall average accuracy was 93% and 87%, respectively. The experimental results confirm that the innovative methods in this paper can accurately and effectively evaluate and classify muscle strength.
Collapse
Affiliation(s)
- Xiao Yang
- Key Laboratory of Ultrasound of Shaanxi Province, School of Physics and Information Technology, Shaanxi Normal University, Xi'an, China
| | - Beilei Zhang
- Key Laboratory of Ultrasound of Shaanxi Province, School of Physics and Information Technology, Shaanxi Normal University, Xi'an, China
| | - Ying Liu
- Key Laboratory of Ultrasound of Shaanxi Province, School of Physics and Information Technology, Shaanxi Normal University, Xi'an, China
| | - Qian Lv
- Key Laboratory of Ultrasound of Shaanxi Province, School of Physics and Information Technology, Shaanxi Normal University, Xi'an, China
| | - Jianzhong Guo
- Key Laboratory of Ultrasound of Shaanxi Province, School of Physics and Information Technology, Shaanxi Normal University, Xi'an, China
| |
Collapse
|
3
|
Radhachandran A, Kinzel A, Chen J, Sant V, Patel M, Masamed R, Arnold CW, Speier W. A multitask approach for automated detection and segmentation of thyroid nodules in ultrasound images. Comput Biol Med 2024; 170:107974. [PMID: 38244471 DOI: 10.1016/j.compbiomed.2024.107974] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2023] [Revised: 12/06/2023] [Accepted: 01/02/2024] [Indexed: 01/22/2024]
Abstract
An increase in the incidence and diagnosis of thyroid nodules and thyroid cancer underscores the need for a better approach to nodule detection and risk stratification in ultrasound (US) images that can reduce healthcare costs, patient discomfort, and unnecessary invasive procedures. However, variability in ultrasound technique and interpretation makes the diagnostic process partially subjective. Therefore, an automated approach that detects and segments nodules could improve performance on downstream tasks, such as risk stratification. Ultrasound studies were acquired from 280 patients at UCLA Health, totaling 9888 images, and annotated by collaborating radiologists. Current deep learning architectures for segmentation are typically semi-automated because they are evaluated solely on images known to have nodules and do not assess ability to identify suspicious images. However, the proposed multitask approach both detects suspicious images and segments potential nodules; this allows for a clinically translatable model that aptly parallels the workflow for thyroid nodule assessment. The multitask approach is centered on an anomaly detection (AD) module that can be integrated with any UNet architecture variant to improve image-level nodule detection. Of the evaluated multitask models, a UNet with a ImageNet pretrained encoder and AD achieved the highest F1 score of 0.839 and image-wide Dice similarity coefficient of 0.808 on the hold-out test set. Furthermore, models were evaluated on two external validations datasets to demonstrate generalizability and robustness to data variability. Ultimately, the proposed architecture is an automated multitask method that expands on previous methods by successfully both detecting and segmenting nodules in ultrasound.
Collapse
Affiliation(s)
- Ashwath Radhachandran
- Computational Diagnostics Lab, University of California, Los Angeles, Los Angeles, CA, USA; Department of Bioengineering, University of California, Los Angeles, Los Angeles, CA, USA
| | - Adam Kinzel
- Department of Radiology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Joseph Chen
- Department of Radiology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Vivek Sant
- Division of Endocrine Surgery, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Maitraya Patel
- Department of Radiology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Rinat Masamed
- Department of Radiology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Corey W Arnold
- Computational Diagnostics Lab, University of California, Los Angeles, Los Angeles, CA, USA; Department of Bioengineering, University of California, Los Angeles, Los Angeles, CA, USA; Department of Radiology, University of California, Los Angeles, Los Angeles, CA, USA; Department of Pathology and Laboratory Medicine, University of California, Los Angeles, Los Angeles, CA, USA
| | - William Speier
- Computational Diagnostics Lab, University of California, Los Angeles, Los Angeles, CA, USA; Department of Bioengineering, University of California, Los Angeles, Los Angeles, CA, USA.
| |
Collapse
|
4
|
Wang J, Yang X, Jia X, Xue W, Chen R, Chen Y, Zhu X, Liu L, Cao Y, Zhou J, Ni D, Gu N. Thyroid ultrasound diagnosis improvement via multi-view self-supervised learning and two-stage pre-training. Comput Biol Med 2024; 171:108087. [PMID: 38364658 DOI: 10.1016/j.compbiomed.2024.108087] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2023] [Revised: 01/04/2024] [Accepted: 01/27/2024] [Indexed: 02/18/2024]
Abstract
Thyroid nodule classification and segmentation in ultrasound images are crucial for computer-aided diagnosis; however, they face limitations owing to insufficient labeled data. In this study, we proposed a multi-view contrastive self-supervised method to improve thyroid nodule classification and segmentation performance with limited manual labels. Our method aligns the transverse and longitudinal views of the same nodule, thereby enabling the model to focus more on the nodule area. We designed an adaptive loss function that eliminates the limitations of the paired data. Additionally, we adopted a two-stage pre-training to exploit the pre-training on ImageNet and thyroid ultrasound images. Extensive experiments were conducted on a large-scale dataset collected from multiple centers. The results showed that the proposed method significantly improves nodule classification and segmentation performance with limited manual labels and outperforms state-of-the-art self-supervised methods. The two-stage pre-training also significantly exceeded ImageNet pre-training.
Collapse
Affiliation(s)
- Jian Wang
- Key Laboratory for Bio-Electromagnetic Environment and Advanced Medical Theranostics, School of Biomedical Engineering and Informatics, Nanjing Medical University, Nanjing, 211166, China
| | - Xin Yang
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, 518073, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, 518073, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, 518073, China
| | - Xiaohong Jia
- Department of Ultrasound, Ruijin Hospital, Shanghai Jiaotong University School of Medicine, Shanghai, 200025, China
| | - Wufeng Xue
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, 518073, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, 518073, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, 518073, China
| | - Rusi Chen
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, 518073, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, 518073, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, 518073, China
| | - Yanlin Chen
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, 518073, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, 518073, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, 518073, China
| | - Xiliang Zhu
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, 518073, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, 518073, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, 518073, China
| | - Lian Liu
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, 518073, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, 518073, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, 518073, China
| | - Yan Cao
- Shenzhen RayShape Medical Technology Co., Ltd, Shenzhen, 518051, China
| | - Jianqiao Zhou
- Department of Ultrasound, Ruijin Hospital, Shanghai Jiaotong University School of Medicine, Shanghai, 200025, China.
| | - Dong Ni
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, 518073, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, 518073, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, 518073, China.
| | - Ning Gu
- Key Laboratory for Bio-Electromagnetic Environment and Advanced Medical Theranostics, School of Biomedical Engineering and Informatics, Nanjing Medical University, Nanjing, 211166, China; Cardiovascular Disease Research Center, Nanjing Drum Tower Hospital, Affiliated Hospital of Medical School, Medical School, Nanjing University, Nanjing, 210093, China.
| |
Collapse
|
5
|
Ajilisa OA, Jagathy Raj VP, Sabu MK. A Deep Learning Framework for the Characterization of Thyroid Nodules from Ultrasound Images Using Improved Inception Network and Multi-Level Transfer Learning. Diagnostics (Basel) 2023; 13:2463. [PMID: 37510206 PMCID: PMC10378664 DOI: 10.3390/diagnostics13142463] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2023] [Revised: 07/07/2023] [Accepted: 07/13/2023] [Indexed: 07/30/2023] Open
Abstract
In the past few years, deep learning has gained increasingly widespread attention and has been applied to diagnosing benign and malignant thyroid nodules. It is difficult to acquire sufficient medical images, resulting in insufficient data, which hinders the development of an efficient deep-learning model. In this paper, we developed a deep-learning-based characterization framework to differentiate malignant and benign nodules from the thyroid ultrasound images. This approach improves the recognition accuracy of the inception network by combining squeeze and excitation networks with the inception modules. We have also integrated the concept of multi-level transfer learning using breast ultrasound images as a bridge dataset. This transfer learning approach addresses the issues regarding domain differences between natural images and ultrasound images during transfer learning. This paper aimed to investigate how the entire framework could help radiologists improve diagnostic performance and avoid unnecessary fine-needle aspiration. The proposed approach based on multi-level transfer learning and improved inception blocks achieved higher precision (0.9057 for the benign class and 0.9667 for the malignant class), recall (0.9796 for the benign class and 0.8529 for malignant), and F1-score (0.9412 for benign class and 0.9062 for malignant class). It also obtained an AUC value of 0.9537, which is higher than that of the single-level transfer learning method. The experimental results show that this model can achieve satisfactory classification accuracy comparable to experienced radiologists. Using this model, we can save time and effort as well as deliver potential clinical application value.
Collapse
Affiliation(s)
- O A Ajilisa
- Department of Computer Applications, Cochin University of Science and Technology, South Kalamassery, Kochi 682022, Kerala, India
| | - V P Jagathy Raj
- School of Management Studies, Cochin University of Science and Technology, South Kalamassery, Kochi 682022, Kerala, India
| | - M K Sabu
- Department of Computer Applications, Cochin University of Science and Technology, South Kalamassery, Kochi 682022, Kerala, India
| |
Collapse
|
6
|
Aversano L, Bernardi ML, Cimitile M, Maiellaro A, Pecori R. A systematic review on artificial intelligence techniques for detecting thyroid diseases. PeerJ Comput Sci 2023; 9:e1394. [PMID: 37346658 PMCID: PMC10280452 DOI: 10.7717/peerj-cs.1394] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Accepted: 04/21/2023] [Indexed: 06/23/2023]
Abstract
The use of artificial intelligence approaches in health-care systems has grown rapidly over the last few years. In this context, early detection of diseases is the most common area of application. In this scenario, thyroid diseases are an example of illnesses that can be effectively faced if discovered quite early. Detecting thyroid diseases is crucial in order to treat patients effectively and promptly, by saving lives and reducing healthcare costs. This work aims at systematically reviewing and analyzing the literature on various artificial intelligence-related techniques applied to the detection and identification of various diseases related to the thyroid gland. The contributions we reviewed are classified according to different viewpoints and taxonomies in order to highlight pros and cons of the most recent research in the field. After a careful selection process, we selected and reviewed 72 papers, analyzing them according to three main research questions, i.e., which diseases of the thyroid gland are detected by different artificial intelligence techniques, which datasets are used to perform the aforementioned detection, and what types of data are used to perform the detection. The review demonstrates that the majority of the considered papers deal with supervised methods to detect hypo- and hyperthyroidism. The average accuracy of detection is high (96.84%), but the usage of private and outdated datasets with a majority of clinical data is very common. Finally, we discuss the outcomes of the systematic review, pointing out advantages, disadvantages, and future developments in the application of artificial intelligence for thyroid diseases detection.
Collapse
Affiliation(s)
- Lerina Aversano
- Department of Engineering, University of Sannio, Benevento, Italy
| | | | - Marta Cimitile
- Dept. of Law and Digital Society, UnitelmaSapienza University, Rome, Italy
| | - Andrea Maiellaro
- Department of Engineering, University of Sannio, Benevento, Italy
| | - Riccardo Pecori
- Institute of Materials for Electronics and Magnetism, National Research Council, Parma, Italy
- SMARTEST Research Centre, eCampus University, Novedrate (CO), Italy
| |
Collapse
|
7
|
Göreke V. A Novel Deep-Learning-Based CADx Architecture for Classification of Thyroid Nodules Using Ultrasound Images. Interdiscip Sci 2023:10.1007/s12539-023-00560-4. [PMID: 36976511 PMCID: PMC10043860 DOI: 10.1007/s12539-023-00560-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2022] [Revised: 03/03/2023] [Accepted: 03/05/2023] [Indexed: 03/29/2023]
Abstract
Nodules of thyroid cancer occur in the cells of the thyroid as benign or malign types. Thyroid sonographic images are mostly used for diagnosis of thyroid cancer. The aim of this study is to introduce a computer-aided diagnosis system that can classify the thyroid nodules with high accuracy using the data gathered from ultrasound images. Acquisition and labeling of sub-images were performed by a specialist physician. Then the number of these sub-images were increased using data augmentation methods. Deep features were obtained from the images using a pre-trained deep neural network. The dimensions of the features were reduced and features were improved. The improved features were combined with morphological and texture features. This feature group was rated by a value called similarity coefficient value which was obtained from a similarity coefficient generator module. The nodules were classified as benign or malignant using a multi-layer deep neural network with a pre-weighting layer designed with a novel approach. In this study, a novel multi-layer computer-aided diagnosis system was proposed for thyroid cancer detection. In the first layer of the system, a novel feature extraction method based on the class similarity of images was developed. In the second layer, a novel pre-weighting layer was proposed by modifying the genetic algorithm. The proposed system showed superior performance in different metrics compared to the literature.
Collapse
Affiliation(s)
- Volkan Göreke
- Department of Computer Technologies, Sivas Vocational School of Technical Sciences, Sivas Cumhuriyet University, 58140, Sivas, Türkiye.
| |
Collapse
|
8
|
Radhachandran A, Kinzel A, Chen J, Sant V, Patel M, Masamed R, Arnold CW, Speier W. A Multitask Approach for Automated Detection and Segmentation of Thyroid Nodules in Ultrasound Images. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.01.31.23285223. [PMID: 36778410 PMCID: PMC9915831 DOI: 10.1101/2023.01.31.23285223] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 02/05/2023]
Abstract
An increase in the incidence and diagnosis of thyroid nodules and thyroid cancer underscores the need for a better approach to nodule detection and risk stratification in ultrasound (US) images that can reduce healthcare costs, patient discomfort, and unnecessary invasive procedures. However, variability in ultrasound technique and interpretation makes the diagnostic process partially subjective. Therefore, an automated approach that detects and segments nodules could improve performance on downstream tasks, such as risk stratification.Current deep learning architectures for segmentation are typically semi-automated because they are evaluated solely on images known to have nodules and do not assess ability to identify suspicious images. However, the proposed multitask approach both detects suspicious images and segments potential nodules; this allows for a clinically translatable model that aptly parallels the workflow for thyroid nodule assessment. The multitask approach is centered on an anomaly detection (AD) module that can be integrated with any U-Net architecture variant to improve image-level nodule detection. Ultrasound studies were acquired from 280 patients at UCLA Health, totaling 9,888 images, and annotated by collaborating radiologists. Of the evaluated models, a multi-scale UNet (MSUNet) with AD achieved the highest F1 score of 0.829 and image-wide Dice similarity coefficient of 0.782 on our hold-out test set. Furthermore, models were evaluated on two external validations datasets to demonstrate generalizability and robustness to data variability. Ultimately, the proposed architecture is an automated multitask method that expands on previous methods by successfully both detecting and segmenting nodules in ultrasound.
Collapse
Affiliation(s)
- Ashwath Radhachandran
- Computational Diagnostics Lab and Department of Bioengineering at the University of California, Los Angeles. The Computational Diagnostics Lab is located at 924 Westwood Blvd, Ste 420, Los Angeles, CA 90024, USA
| | - Adam Kinzel
- Department of Radiology at the University of California, Los Angeles
| | - Joseph Chen
- Department of Radiology at the University of California, Los Angeles
| | - Vivek Sant
- Section of Endocrine Surgery in the Department of Surgery at the University of California, Los Angeles
| | - Maitraya Patel
- Department of Radiology at the University of California, Los Angeles
| | - Rinat Masamed
- Department of Radiology at the University of California, Los Angeles
| | - Corey W Arnold
- Computational Diagnostics Lab, Department of Bioengineering, Department of Radiology and Department of Pathology and Laboratory Medicine at the University of California, Los Angeles
| | - William Speier
- Computational Diagnostics Lab and Department of Bioengineering at the University of California, Los Angeles. The Computational Diagnostics Lab is located at 924 Westwood Blvd, Ste 420, Los Angeles, CA 90024, USA
| |
Collapse
|
9
|
Chen Y, Zhang X, Li D, Park H, Li X, Liu P, Jin J, Shen Y. Automatic segmentation of thyroid with the assistance of the devised boundary improvement based on multicomponent small dataset. APPL INTELL 2023; 53:1-16. [PMID: 37363389 PMCID: PMC10015528 DOI: 10.1007/s10489-023-04540-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/23/2023] [Indexed: 03/17/2023]
Abstract
Deep learning has been widely considered in medical image segmentation. However, the difficulty of acquiring medical images and labels can affect the accuracy of the segmentation results for deep learning methods. In this paper, an automatic segmentation method is proposed by devising a multicomponent neighborhood extreme learning machine to improve the boundary attention region of the preliminary segmentation results. The neighborhood features are acquired by training U-Nets with the multicomponent small dataset, which consists of original thyroid ultrasound images, Sobel edge images and superpixel images. Afterward, the neighborhood features are selected by min-redundancy and max-relevance filter in the designed extreme learning machine, and the selected features are used to train the extreme learning machine to obtain supplementary segmentation results. Finally, the accuracy of the segmentation results is improved by adjusting the boundary attention region of the preliminary segmentation results with the supplementary segmentation results. This method combines the advantages of deep learning and traditional machine learning, boosting the accuracy of thyroid segmentation accuracy with a small dataset in a multigroup test.
Collapse
Affiliation(s)
- Yifei Chen
- Control Science and Engineering, Harbin Institute of Technology, Harbin, 150001 China
- Electrical Engineering, Korea Advanced Institute of Science and Technology, Daejeon, 34141 Korea
| | - Xin Zhang
- Control Science and Engineering, Harbin Institute of Technology, Harbin, 150001 China
| | - Dandan Li
- Control Science and Engineering, Harbin Institute of Technology, Harbin, 150001 China
| | - HyunWook Park
- Electrical Engineering, Korea Advanced Institute of Science and Technology, Daejeon, 34141 Korea
| | - Xinran Li
- Mathematics, Harbin Institute of Technology, Harbin, 150001 China
| | - Peng Liu
- Heilongjiang Provincial Key Laboratory of Trace Elements and Human Health, Harbin Medical University, Harbin, 150081 China
| | - Jing Jin
- Control Science and Engineering, Harbin Institute of Technology, Harbin, 150001 China
| | - Yi Shen
- Control Science and Engineering, Harbin Institute of Technology, Harbin, 150001 China
| |
Collapse
|
10
|
Boers T, Braak SJ, Rikken NET, Versluis M, Manohar S. Ultrasound imaging in thyroid nodule diagnosis, therapy, and follow-up: Current status and future trends. JOURNAL OF CLINICAL ULTRASOUND : JCU 2023. [PMID: 36655705 DOI: 10.1002/jcu.23430] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/14/2022] [Accepted: 01/10/2023] [Indexed: 06/17/2023]
Abstract
Ultrasound, the primary imaging modality in thyroid nodule management, suffers from drawbacks including: high inter- and intra-observer variability, limited field-of-view and limited functional imaging. Developments in ultrasound technologies are taking place to overcome these limitations, including three-dimensional-Doppler, -elastography, -nodule characteristics-extraction, and novel machine-learning algorithms. For thyroid ablative treatments and biopsies, perioperative use of three-dimensional ultrasound opens a new field of research. This review provides an overview of the current and future applications of ultrasound, and discusses the potential of new developments and trends that may improve the diagnosis, therapy, and follow-up of thyroid nodules.
Collapse
Affiliation(s)
- Tim Boers
- Multi-Modality Medical Imaging Group, TechMed Centre, University of Twente, Enschede, the Netherlands
| | - Sicco J Braak
- Department of Radiology, Ziekenhuisgroep Twente, Hengelo, the Netherlands
| | - Nicole E T Rikken
- Department of Endocrinology, Ziekenhuisgroep Twente, Hengelo, the Netherlands
| | - Michel Versluis
- Physics of Fluids Group, TechMed Centre, University of Twente, Enschede, the Netherlands
| | - Srirang Manohar
- Multi-Modality Medical Imaging Group, TechMed Centre, University of Twente, Enschede, the Netherlands
| |
Collapse
|
11
|
Lin X, Zhou X, Tong T, Nie X, Wang L, Zheng H, Li J, Xue E, Chen S, Zheng M, Chen C, Jiang H, Du M, Gao Q. A Super-resolution Guided Network for Improving Automated Thyroid Nodule Segmentation. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 227:107186. [PMID: 36334526 DOI: 10.1016/j.cmpb.2022.107186] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/14/2022] [Revised: 10/03/2022] [Accepted: 10/15/2022] [Indexed: 06/16/2023]
Abstract
BACKGROUND AND OBJECTIVE A thyroid nodule is an abnormal lump that grows in the thyroid gland, which is the early symptom of thyroid cancer. In order to diagnose and treat thyroid cancer at the earliest stage, it is desired to characterize the nodule accurately. Ultrasound thyroid nodules segmentation is a challenging task due to the speckle noise, intensity heterogeneity, low contrast and low resolution. In this paper, we propose a novel framework to improve the accuracy of thyroid nodules segmentation. METHODS Different from previous work, a super-resolution reconstruction network is firstly constructed to upscale the resolution of the input ultrasound image. After that, our proposed N-shape network is utilized to perform the segmentation task. The guidance of super-resolution reconstruction network can make the high-frequency information of the input thyroid ultrasound image richer and more comprehensive than the original image. Our N-shape network consists of several atrous spatial pyramid pooling blocks, a multi-scale input layer, a U-shape convolutional network with attention blocks and a proposed parallel atrous convolution(PAC) module. These modules are conducive to capture context information at multiple scales so that semantic features can be fully utilized for lesion segmentation. Especially, our proposed PAC module is beneficial to further improve the segmentation by extracting high-level semantic features from different receptive fields. We use the UTNI-2021 dataset for model training, validating and testing. RESULTS The experimental results show that our proposed method achieve a Dice value of 91.9%, a mIoU value of 87.0%, a Precision value of 88.0%, a Recall value 83.7% and a F1-score value of 84.3%, which outperforms most state-of-the-art methods. CONCLUSIONS Our method achieves the best performance on the UTNI-2021 dataset and provides a new way of ultrasound image segmentation. We believe that our method can provide doctors with reliable auxiliary diagnosis information in clinical practice.
Collapse
Affiliation(s)
- Xingtao Lin
- College of Physics and Information Engineering, Fuzhou University; Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University
| | - Xiaogen Zhou
- College of Physics and Information Engineering, Fuzhou University; Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University
| | - Tong Tong
- College of Physics and Information Engineering, Fuzhou University; Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University; Imperial Vision Technology.
| | - Xingqing Nie
- College of Physics and Information Engineering, Fuzhou University; Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University
| | - Luoyan Wang
- College of Physics and Information Engineering, Fuzhou University; Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University
| | - Haonan Zheng
- College of Physics and Information Engineering, Fuzhou University; Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University
| | - Jing Li
- College of Physics and Information Engineering, Fuzhou University; Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University
| | | | - Shun Chen
- Fujian Medical University Union Hospital.
| | | | - Cong Chen
- Fujian Medical University Union Hospital
| | - Haiyan Jiang
- Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University
| | - Min Du
- College of Physics and Information Engineering, Fuzhou University; Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University
| | - Qinquan Gao
- College of Physics and Information Engineering, Fuzhou University; Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University; Imperial Vision Technology
| |
Collapse
|
12
|
Khodabandelu S, Ghaemian N, Khafri S, Ezoji M, Khaleghi S. Development of a Machine Learning-Based Screening Method for Thyroid Nodules Classification by Solving the Imbalance Challenge in Thyroid Nodules Data. J Res Health Sci 2022; 22:e00555. [PMID: 36511373 PMCID: PMC10422153 DOI: 10.34172/jrhs.2022.90] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2022] [Revised: 07/23/2022] [Accepted: 08/02/2022] [Indexed: 12/15/2022] Open
Abstract
BACKGROUND This study aims to show the impact of imbalanced data and the typical evaluation methods in developing and misleading assessments of machine learning-based models for preoperative thyroid nodules screening. STUDY DESIGN A retrospective study. METHODS The ultrasonography features for 431 thyroid nodules cases were extracted from medical records of 313 patients in Babol, Iran. Since thyroid nodules are commonly benign, the relevant data are usually unbalanced in classes. It can lead to the bias of learning models toward the majority class. To solve it, a hybrid resampling method called the Smote-was used to creating balance data. Following that, the support vector classification (SVC) algorithm was trained by balance and unbalanced datasets as Models 2 and 3, respectively, in Python language programming. Their performance was then compared with the logistic regression model as Model 1 that fitted traditionally. RESULTS The prevalence of malignant nodules was obtained at 14% (n = 61). In addition, 87% of the patients in this study were women. However, there was no difference in the prevalence of malignancy for gender. Furthermore, the accuracy, area under the curve, and geometric mean values were estimated at 92.1%, 93.2%, and 76.8% for Model 1, 91.3%, 93%, and 77.6% for Model 2, and finally, 91%, 92.6% and 84.2% for Model 3, respectively. Similarly, the results identified Micro calcification, Taller than wide shape, as well as lack of ISO and hyperechogenicity features as the most effective malignant variables. CONCLUSION Paying attention to data challenges, such as data imbalances, and using proper criteria measures can improve the performance of machine learning models for preoperative thyroid nodules screening.
Collapse
Affiliation(s)
- Sajad Khodabandelu
- Student Research Committee, School of Medicine, Faculty of Health, Babol University of Medical Science, Babol, Iran
| | - Naser Ghaemian
- Department of Radiology, Babol University of Medical Sciences, Babol, Iran
| | - Soraya Khafri
- Research Center for Social Determinants of Health, Health Research Institute, Department of Biostatistics and Epidemiology, Faculty of Health, Babol University of Medical Sciences, Babol, Iran
| | - Mehdi Ezoji
- Faculty of Electrical and Computer Engineering, Babol Noshirvani University of Technology, Babol, Iran
| | - Sara Khaleghi
- Student Research Committee, School of Medicine, Faculty of Health, Babol University of Medical Science, Babol, Iran
| |
Collapse
|
13
|
Abstract
Machine learning (ML) methods are pervading an increasing number of fields of application because of their capacity to effectively solve a wide variety of challenging problems. The employment of ML techniques in ultrasound imaging applications started several years ago but the scientific interest in this issue has increased exponentially in the last few years. The present work reviews the most recent (2019 onwards) implementations of machine learning techniques for two of the most popular ultrasound imaging fields, medical diagnostics and non-destructive evaluation. The former, which covers the major part of the review, was analyzed by classifying studies according to the human organ investigated and the methodology (e.g., detection, segmentation, and/or classification) adopted, while for the latter, some solutions to the detection/classification of material defects or particular patterns are reported. Finally, the main merits of machine learning that emerged from the study analysis are summarized and discussed.
Collapse
|
14
|
Wang L, Zhou X, Nie X, Lin X, Li J, Zheng H, Xue E, Chen S, Chen C, Du M, Tong T, Gao Q, Zheng M. A Multi-Scale Densely Connected Convolutional Neural Network for Automated Thyroid Nodule Classification. Front Neurosci 2022; 16:878718. [PMID: 35663553 PMCID: PMC9160335 DOI: 10.3389/fnins.2022.878718] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2022] [Accepted: 04/13/2022] [Indexed: 11/13/2022] Open
Abstract
Automated thyroid nodule classification in ultrasound images is an important way to detect thyroid nodules and to make a more accurate diagnosis. In this paper, we propose a novel deep convolutional neural network (CNN) model, called n-ClsNet, for thyroid nodule classification. Our model consists of a multi-scale classification layer, multiple skip blocks, and a hybrid atrous convolution (HAC) block. The multi-scale classification layer first obtains multi-scale feature maps in order to make full use of image features. After that, each skip-block propagates information at different scales to learn multi-scale features for image classification. Finally, the HAC block is used to replace the downpooling layer so that the spatial information can be fully learned. We have evaluated our n-ClsNet model on the TNUI-2021 dataset. The proposed n-ClsNet achieves an average accuracy (ACC) score of 93.8% in the thyroid nodule classification task, which outperforms several representative state-of-the-art classification methods.
Collapse
Affiliation(s)
- Luoyan Wang
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
- Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University, Fuzhou, China
| | - Xiaogen Zhou
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
- Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University, Fuzhou, China
| | - Xingqing Nie
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
- Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University, Fuzhou, China
| | - Xingtao Lin
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
- Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University, Fuzhou, China
| | - Jing Li
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
- Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University, Fuzhou, China
| | - Haonan Zheng
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
- Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University, Fuzhou, China
| | - Ensheng Xue
- Fujian Medical University Union Hospital, Fuzhou, China
- Fujian Medical Ultrasound Research Institute, Fuzhou, China
| | - Shun Chen
- Fujian Medical University Union Hospital, Fuzhou, China
| | - Cong Chen
- Fujian Medical University Union Hospital, Fuzhou, China
| | - Min Du
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
- Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University, Fuzhou, China
| | - Tong Tong
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
- Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University, Fuzhou, China
| | - Qinquan Gao
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
- Fujian Key Lab of Medical Instrumentation & Pharmaceutical Technology, Fuzhou University, Fuzhou, China
- *Correspondence: Qinquan Gao
| | - Meijuan Zheng
- Fujian Medical University Union Hospital, Fuzhou, China
- Meijuan Zheng
| |
Collapse
|
15
|
Deep Learning Based Fast Screening Approach on Ultrasound Images for Thyroid Nodules Diagnosis. Diagnostics (Basel) 2021; 11:diagnostics11122209. [PMID: 34943444 PMCID: PMC8700062 DOI: 10.3390/diagnostics11122209] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2021] [Revised: 11/21/2021] [Accepted: 11/25/2021] [Indexed: 11/23/2022] Open
Abstract
Thyroid nodules are widespread in the United States and the rest of the world, with a prevalence ranging from 19 to 68%. The problem with nodules is whether they are malignant or benign. Ultrasonography is currently recommended as the initial modality for evaluating thyroid nodules. However, obtaining a good diagnosis from ultrasound imaging depends entirely on the radiologists levels of experience and other circumstances. There is a tremendous demand for automated and more reliable methods to screen ultrasound images more efficiently. This research proposes an efficient and quick detection deep learning approach for thyroid nodules. An open and publicly available dataset, Thyroid Digital Image Database (TDID), is used to determine the robustness of the suggested method. Each image is formatted into a pyramid tile-based data structure, which the proposed VGG-16 model evaluates to provide segmentation results for nodular detection. The proposed method adopts a top-down approach to hierarchically integrate high- and low-level features to distinguish nodules of varied sizes by employing fuse features effectively. The results demonstrated that the proposed method outperformed the U-Net model, achieving an accuracy of 99%, and was two times faster than the competitive model.
Collapse
|
16
|
Marosán-Vilimszky P, Szalai K, Horváth A, Csabai D, Füzesi K, Csány G, Gyöngy M. Automated Skin Lesion Classification on Ultrasound Images. Diagnostics (Basel) 2021; 11:1207. [PMID: 34359290 PMCID: PMC8303815 DOI: 10.3390/diagnostics11071207] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2021] [Accepted: 06/30/2021] [Indexed: 11/17/2022] Open
Abstract
The growing incidence of skin cancer makes computer-aided diagnosis tools for this group of diseases increasingly important. The use of ultrasound has the potential to complement information from optical dermoscopy. The current work presents a fully automatic classification framework utilizing fully-automated (FA) segmentation and compares it with classification using two semi-automated (SA) segmentation methods. Ultrasound recordings were taken from a total of 310 lesions (70 melanoma, 130 basal cell carcinoma and 110 benign nevi). A support vector machine (SVM) model was trained on 62 features, with ten-fold cross-validation. Six classification tasks were considered, namely all the possible permutations of one class versus one or two remaining classes. The receiver operating characteristic (ROC) area under the curve (AUC) as well as the accuracy (ACC) were measured. The best classification was obtained for the classification of nevi from cancerous lesions (melanoma, basal cell carcinoma), with AUCs of over 90% and ACCs of over 85% obtained with all segmentation methods. Previous works have either not implemented FA ultrasound-based skin cancer classification (making diagnosis more lengthy and operator-dependent), or are unclear in their classification results. Furthermore, the current work is the first to assess the effect of implementing FA instead of SA classification, with FA classification never degrading performance (in terms of AUC or ACC) by more than 5%.
Collapse
Affiliation(s)
- Péter Marosán-Vilimszky
- Faculty of Information Technology and Bionics, Pázmány Péter Catholic University, Práter u. 50/A, 1083 Budapest, Hungary; (A.H.); (M.G.)
- Dermus Kft., Sopron út 64, 1116 Budapest, Hungary; (D.C.); (K.F.); (G.C.)
| | - Klára Szalai
- Department of Dermatology, Venereology and Dermatooncology, Semmelweis University, Mária u. 41, 1085 Budapest, Hungary;
| | - András Horváth
- Faculty of Information Technology and Bionics, Pázmány Péter Catholic University, Práter u. 50/A, 1083 Budapest, Hungary; (A.H.); (M.G.)
| | - Domonkos Csabai
- Dermus Kft., Sopron út 64, 1116 Budapest, Hungary; (D.C.); (K.F.); (G.C.)
| | - Krisztián Füzesi
- Dermus Kft., Sopron út 64, 1116 Budapest, Hungary; (D.C.); (K.F.); (G.C.)
| | - Gergely Csány
- Dermus Kft., Sopron út 64, 1116 Budapest, Hungary; (D.C.); (K.F.); (G.C.)
| | - Miklós Gyöngy
- Faculty of Information Technology and Bionics, Pázmány Péter Catholic University, Práter u. 50/A, 1083 Budapest, Hungary; (A.H.); (M.G.)
- Dermus Kft., Sopron út 64, 1116 Budapest, Hungary; (D.C.); (K.F.); (G.C.)
| |
Collapse
|