1
|
Xu M, Ma Q, Zhang H, Kong D, Zeng T. MEF-UNet: An end-to-end ultrasound image segmentation algorithm based on multi-scale feature extraction and fusion. Comput Med Imaging Graph 2024; 114:102370. [PMID: 38513396 DOI: 10.1016/j.compmedimag.2024.102370] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2023] [Revised: 03/10/2024] [Accepted: 03/13/2024] [Indexed: 03/23/2024]
Abstract
Ultrasound image segmentation is a challenging task due to the complexity of lesion types, fuzzy boundaries, and low-contrast images along with the presence of noises and artifacts. To address these issues, we propose an end-to-end multi-scale feature extraction and fusion network (MEF-UNet) for the automatic segmentation of ultrasound images. Specifically, we first design a selective feature extraction encoder, including detail extraction stage and structure extraction stage, to precisely capture the edge details and overall shape features of the lesions. In order to enhance the representation capacity of contextual information, we develop a context information storage module in the skip-connection section, responsible for integrating information from adjacent two-layer feature maps. In addition, we design a multi-scale feature fusion module in the decoder section to merge feature maps with different scales. Experimental results indicate that our MEF-UNet can significantly improve the segmentation results in both quantitative analysis and visual effects.
Collapse
Affiliation(s)
- Mengqi Xu
- School of Mathematics and Statistics, Nanjing University of Information Science and Technology, Nanjing, Jiangsu, 210044, China
| | - Qianting Ma
- School of Mathematics and Statistics, Nanjing University of Information Science and Technology, Nanjing, Jiangsu, 210044, China.
| | - Huajie Zhang
- School of Mathematics and Statistics, Nanjing University of Information Science and Technology, Nanjing, Jiangsu, 210044, China
| | - Dexing Kong
- School of Mathematical Sciences, Zhejiang University, Hangzhou, Zhejiang, 310027, China
| | - Tieyong Zeng
- Department of Mathematics, The Chinese University of Hong Kong, Shatin, Hong Kong Special Administrative Region of China
| |
Collapse
|
2
|
Alasmawi H, Bricker L, Yaqub M. FUSC: Fetal Ultrasound Semantic Clustering of Second-Trimester Scans Using Deep Self-Supervised Learning. ULTRASOUND IN MEDICINE & BIOLOGY 2024; 50:703-711. [PMID: 38350787 DOI: 10.1016/j.ultrasmedbio.2024.01.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/25/2023] [Revised: 12/31/2023] [Accepted: 01/14/2024] [Indexed: 02/15/2024]
Abstract
OBJECTIVE The aim of this study was address the challenges posed by the manual labeling of fetal ultrasound images by introducing an unsupervised approach, the fetal ultrasound semantic clustering (FUSC) method. The primary objective was to automatically cluster a large volume of ultrasound images into various fetal views, reducing or eliminating the need for labor-intensive manual labeling. METHODS The FUSC method was developed by using a substantial data set comprising 88,063 images. The methodology involves an unsupervised clustering approach to categorize ultrasound images into diverse fetal views. The method's effectiveness was further evaluated on an additional, unseen data set consisting of 8187 images. The evaluation included assessment of the clustering purity, and the entire process is detailed to provide insights into the method's performance. RESULTS The FUSC method exhibited notable success, achieving >92% clustering purity on the evaluation data set of 8187 images. The results signify the feasibility of automatically clustering fetal ultrasound images without relying on manual labeling. The study showcases the potential of this approach in handling a large volume of ultrasound scans encountered in clinical practice, with implications for improving efficiency and accuracy in fetal ultrasound imaging. CONCLUSION The findings of this investigation suggest that the FUSC method holds significant promise for the field of fetal ultrasound imaging. By automating the clustering of ultrasound images, this approach has the potential to reduce the manual labeling burden, making the process more efficient. The results pave the way for advanced automated labeling solutions, contributing to the enhancement of clinical practices in fetal ultrasound imaging. Our code is available at https://github.com/BioMedIA-MBZUAI/FUSC.
Collapse
Affiliation(s)
- Hussain Alasmawi
- Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates.
| | - Leanne Bricker
- Abu Dhabi Health Services Company (SEHA), Abu Dhabi, United Arab Emirates
| | - Mohammad Yaqub
- Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates
| |
Collapse
|
3
|
Cai R, Liu Y, Sun Z, Wang Y, Wang Y, Li F, Jiang H. Deep-learning based segmentation of ultrasound adipose image for liposuction. Int J Med Robot 2023; 19:e2548. [PMID: 37448348 DOI: 10.1002/rcs.2548] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2023] [Revised: 06/25/2023] [Accepted: 06/27/2023] [Indexed: 07/15/2023]
Abstract
BACKGROUND To develop an automatic and reliable ultrasonic visual system for robot- or computer-assisted liposuction, we examined the use of deep learning for the segmentation of adipose ultrasound images in clinical and educational settings. METHODS To segment adipose layers, it is proposed to use an Attention Skip-Convolutions ResU-Net (Attention SCResU-Net) consisting of SC residual blocks, attention gates and U-Net architecture. Transfer learning is utilised to compensate for the deficiency of clinical data. The Bama pig and clinical human adipose ultrasound image datasets are utilized, respectively. RESULTS The final model obtains a Dice of 99.06 ± 0.95% and an ASD of 0.19 ± 0.18 mm on clinical datasets, outperforming other methods. By fine-tuning the eight deepest layers, accurate and stable segmentation results are obtained. CONCLUSIONS The new deep-learning method achieves the accurate and automatic segmentation of adipose ultrasound images in real-time, thereby enhancing the safety of liposuction and enabling novice surgeons to better control the cannula.
Collapse
Affiliation(s)
- Ruxin Cai
- Beihang University, School of Biological Science and Medical Engineering, Beijing, China
| | - Yanzhen Liu
- Beihang University, School of Biological Science and Medical Engineering, Beijing, China
| | - Zhibin Sun
- Beihang University, School of Biological Science and Medical Engineering, Beijing, China
| | - Yuneng Wang
- Chinese Academy of Medical Sciences and Peking Union Medical College, Plastic Surgery Hospital, Beijing, China
| | - Yu Wang
- Beihang University, School of Biological Science and Medical Engineering, Beijing, China
| | - Facheng Li
- Chinese Academy of Medical Sciences and Peking Union Medical College, Plastic Surgery Hospital, Beijing, China
| | - Haiyue Jiang
- Chinese Academy of Medical Sciences and Peking Union Medical College, Plastic Surgery Hospital, Beijing, China
| |
Collapse
|
4
|
Ding X, Liu Y, Zhao J, Wang R, Li C, Luo Q, Shen C. A novel wavelet-transform-based convolution classification network for cervical lymph node metastasis of papillary thyroid carcinoma in ultrasound images. Comput Med Imaging Graph 2023; 109:102298. [PMID: 37769402 DOI: 10.1016/j.compmedimag.2023.102298] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2023] [Revised: 08/29/2023] [Accepted: 08/29/2023] [Indexed: 09/30/2023]
Abstract
Preoperative assessment of cervical lymph nodes metastasis (CLNM) for accurate qualitative and locating diagnosis is important for choosing the best treatment option for patients with papillary thyroid cancer. Non-destructive, non-invasive ultrasound is currently the imaging method of choice for lymph node metastatic assessment. For lymph node characteristics and ultrasound images, this paper proposes a multitasking network framework for diagnosing metastatic lymph nodes in ultrasound images, in which localization module not only provides information on the location of lymph nodes to focus on the peripheral and self regions of lymph nodes, but also provides structural features of lymph nodes for subsequent classification module. In the classification module, we design a novel wavelet-transform-based convolution network. Wavelet transform is introduced into the deep learning convolution module to analyze ultrasound images in both spatial and frequency domains, which effectively enriches the feature information and improves the classification performance of the model without increasing the model parameters. We collected 510 patient data (N = 1376) from Shanghai Sixth People's Hospital regarding ultrasound lymph nodes in the neck, as well as used three publicly available ultrasound datasets, including SCUI2020 (N = 2914), DDTI (N = 480), and BUSI (N = 780). Compared to the optimal two-stage model, our model has improved its accuracy and AUC indexes by 5.83% and 4%, which outperforms the two-stage architectures and also surpasses the latest classification networks.
Collapse
Affiliation(s)
- Xuehai Ding
- School of Computer Engineering and Science, Shanghai University, Shangda Rd, Shanghai, 200444, China
| | - Yanting Liu
- School of Computer Engineering and Science, Shanghai University, Shangda Rd, Shanghai, 200444, China
| | - Junjuan Zhao
- School of Computer Engineering and Science, Shanghai University, Shangda Rd, Shanghai, 200444, China.
| | - Ren Wang
- Department of Ultrasound Medicine, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Yishan Rd, Shanghai, 200233, China
| | - Chengfan Li
- School of Computer Engineering and Science, Shanghai University, Shangda Rd, Shanghai, 200444, China
| | - Quanyong Luo
- Department of Nuclear Medicine, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Yishan Rd, Shanghai, 200233, China
| | - Chentian Shen
- Department of Nuclear Medicine, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Yishan Rd, Shanghai, 200233, China.
| |
Collapse
|
5
|
Peng T, Wu Y, Gu Y, Xu D, Wang C, Li Q, Cai J. Intelligent contour extraction approach for accurate segmentation of medical ultrasound images. Front Physiol 2023; 14:1177351. [PMID: 37675280 PMCID: PMC10479019 DOI: 10.3389/fphys.2023.1177351] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2023] [Accepted: 07/28/2023] [Indexed: 09/08/2023] Open
Abstract
Introduction: Accurate contour extraction in ultrasound images is of great interest for image-guided organ interventions and disease diagnosis. Nevertheless, it remains a problematic issue owing to the missing or ambiguous outline between organs (i.e., prostate and kidney) and surrounding tissues, the appearance of shadow artifacts, and the large variability in the shape of organs. Methods: To address these issues, we devised a method that includes four stages. In the first stage, the data sequence is acquired using an improved adaptive selection principal curve method, in which a limited number of radiologist defined data points are adopted as the prior. The second stage then uses an enhanced quantum evolution network to help acquire the optimal neural network. The third stage involves increasing the precision of the experimental outcomes after training the neural network, while using the data sequence as the input. In the final stage, the contour is smoothed using an explicable mathematical formula explained by the model parameters of the neural network. Results: Our experiments showed that our approach outperformed other current methods, including hybrid and Transformer-based deep-learning methods, achieving an average Dice similarity coefficient, Jaccard similarity coefficient, and accuracy of 95.7 ± 2.4%, 94.6 ± 2.6%, and 95.3 ± 2.6%, respectively. Discussion: This work develops an intelligent contour extraction approach on ultrasound images. Our approach obtained more satisfactory outcome compared with recent state-of-the-art approaches . The knowledge of precise boundaries of the organ is significant for the conservation of risk structures. Our developed approach has the potential to enhance disease diagnosis and therapeutic outcomes.
Collapse
Affiliation(s)
- Tao Peng
- School of Future Science and Engineering, Soochow University, Suzhou, China
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China
- Department of Radiation Oncology, UT Southwestern Medical Center, Dallas, TX, United States
| | - Yiyun Wu
- Department of Ultrasound, Jiangsu Province Hospital of Chinese Medicine, Nanjing, Jiangsu, China
| | - Yidong Gu
- Department of Medical Ultrasound, The Affiliated Suzhou Hospital of Nanjing Medical University, Suzhou Municipal Hospital, Suzhou, Jiangsu, China
| | - Daqiang Xu
- Department of Radiology, The Affiliated Suzhou Hospital of Nanjing Medical University, Suzhou Municipal Hospital, Suzhou, Jiangsu, China
| | - Caishan Wang
- Department of Ultrasound, The Second Affiliated Hospital of Soochow University, Suzhou, China
| | - Quan Li
- Center of Stomatology, The Second Affiliated Hospital of Soochow University, Suzhou, China
| | - Jing Cai
- Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China
| |
Collapse
|
6
|
Ramirez Zegarra R, Ghi T. Use of artificial intelligence and deep learning in fetal ultrasound imaging. ULTRASOUND IN OBSTETRICS & GYNECOLOGY : THE OFFICIAL JOURNAL OF THE INTERNATIONAL SOCIETY OF ULTRASOUND IN OBSTETRICS AND GYNECOLOGY 2023; 62:185-194. [PMID: 36436205 DOI: 10.1002/uog.26130] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/20/2022] [Revised: 11/06/2022] [Accepted: 11/21/2022] [Indexed: 06/16/2023]
Abstract
Deep learning is considered the leading artificial intelligence tool in image analysis in general. Deep-learning algorithms excel at image recognition, which makes them valuable in medical imaging. Obstetric ultrasound has become the gold standard imaging modality for detection and diagnosis of fetal malformations. However, ultrasound relies heavily on the operator's experience, making it unreliable in inexperienced hands. Several studies have proposed the use of deep-learning models as a tool to support sonographers, in an attempt to overcome these problems inherent to ultrasound. Deep learning has many clinical applications in the field of fetal imaging, including identification of normal and abnormal fetal anatomy and measurement of fetal biometry. In this Review, we provide a comprehensive explanation of the fundamentals of deep learning in fetal imaging, with particular focus on its clinical applicability. © 2022 International Society of Ultrasound in Obstetrics and Gynecology.
Collapse
Affiliation(s)
- R Ramirez Zegarra
- Department of Medicine and Surgery, Obstetrics and Gynecology Unit, University of Parma, Parma, Italy
| | - T Ghi
- Department of Medicine and Surgery, Obstetrics and Gynecology Unit, University of Parma, Parma, Italy
| |
Collapse
|
7
|
Fiorentino MC, Villani FP, Di Cosmo M, Frontoni E, Moccia S. A review on deep-learning algorithms for fetal ultrasound-image analysis. Med Image Anal 2023; 83:102629. [PMID: 36308861 DOI: 10.1016/j.media.2022.102629] [Citation(s) in RCA: 27] [Impact Index Per Article: 27.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2022] [Revised: 07/12/2022] [Accepted: 09/10/2022] [Indexed: 11/07/2022]
Abstract
Deep-learning (DL) algorithms are becoming the standard for processing ultrasound (US) fetal images. A number of survey papers in the field is today available, but most of them are focusing on a broader area of medical-image analysis or not covering all fetal US DL applications. This paper surveys the most recent work in the field, with a total of 153 research papers published after 2017. Papers are analyzed and commented from both the methodology and the application perspective. We categorized the papers into (i) fetal standard-plane detection, (ii) anatomical structure analysis and (iii) biometry parameter estimation. For each category, main limitations and open issues are presented. Summary tables are included to facilitate the comparison among the different approaches. In addition, emerging applications are also outlined. Publicly-available datasets and performance metrics commonly used to assess algorithm performance are summarized, too. This paper ends with a critical summary of the current state of the art on DL algorithms for fetal US image analysis and a discussion on current challenges that have to be tackled by researchers working in the field to translate the research methodology into actual clinical practice.
Collapse
Affiliation(s)
| | | | - Mariachiara Di Cosmo
- Department of Information Engineering, Università Politecnica delle Marche, Italy
| | - Emanuele Frontoni
- Department of Information Engineering, Università Politecnica delle Marche, Italy; Department of Political Sciences, Communication and International Relations, Università degli Studi di Macerata, Italy
| | - Sara Moccia
- The BioRobotics Institute and Department of Excellence in Robotics & AI, Scuola Superiore Sant'Anna, Italy
| |
Collapse
|
8
|
Chen X, Xu H, Qi Q, Sun C, Jin J, Zhao H, Wang X, Weng W, Wang S, Sui X, Wang Z, Dai C, Peng M, Wang D, Hao Z, Huang Y, Wang X, Duan L, Zhu Y, Hong N, Yang F. AI-based chest CT semantic segmentation algorithm enables semi-automated lung cancer surgery planning by recognizing anatomical variants of pulmonary vessels. Front Oncol 2022; 12:1021084. [PMID: 36324583 PMCID: PMC9621115 DOI: 10.3389/fonc.2022.1021084] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2022] [Accepted: 09/26/2022] [Indexed: 11/16/2022] Open
Abstract
Background The recognition of anatomical variants is essential in preoperative planning for lung cancer surgery. Although three-dimensional (3-D) reconstruction provided an intuitive demonstration of the anatomical structure, the recognition process remains fully manual. To render a semiautomated approach for surgery planning, we developed an artificial intelligence (AI)–based chest CT semantic segmentation algorithm that recognizes pulmonary vessels on lobular or segmental levels. Hereby, we present a retrospective validation of the algorithm comparing surgeons’ performance. Methods The semantic segmentation algorithm to be validated was trained on non-contrast CT scans from a single center. A retrospective pilot study was performed. An independent validation dataset was constituted by an arbitrary selection from patients who underwent lobectomy or segmentectomy in three institutions during Apr. 2020 to Jun. 2021. The golden standard of anatomical variants of each enrolled case was obtained via expert surgeons’ judgments based on chest CT, 3-D reconstruction, and surgical observation. The performance of the algorithm is compared against the performance of two junior thoracic surgery attendings based on chest CT. Results A total of 27 cases were included in this study. The overall case-wise accuracy of the AI model was 82.8% in pulmonary vessels compared to 78.8% and 77.0% for the two surgeons, respectively. Segmental artery accuracy was 79.7%, 73.6%, and 72.7%; lobular vein accuracy was 96.3%, 96.3%, and 92.6% by the AI model and two surgeons, respectively. No statistical significance was found. In subgroup analysis, the anatomic structure-wise analysis of the AI algorithm showed a significant difference in accuracies between different lobes (p = 0.012). Higher AI accuracy in the right-upper lobe (RUL) and left-lower lobe (LLL) arteries was shown. A trend of better performance in non-contrast CT was also detected. Most recognition errors by the algorithm were the misclassification of LA1+2 and LA3. Radiological parameters did not exhibit a significant impact on the performance of both AI and surgeons. Conclusion The semantic segmentation algorithm achieves the recognition of the segmental pulmonary artery and the lobular pulmonary vein. The performance of the model approximates that of junior thoracic surgery attendings. Our work provides a novel semiautomated surgery planning approach that is potentially beneficial to lung cancer patients.
Collapse
Affiliation(s)
- Xiuyuan Chen
- Department of Thoracic Surgery, Peking University People’s Hospital, Beijing, China
- Thoracic Oncology Institute, Peking University People’s Hospital, Beijing, China
| | - Hao Xu
- Department of Thoracic Surgery, Peking University People’s Hospital, Beijing, China
- Thoracic Oncology Institute, Peking University People’s Hospital, Beijing, China
| | - Qingyi Qi
- Department of Radiology, Peking University People’s Hospital, Beijing, China
| | - Chao Sun
- Department of Radiology, Peking University People’s Hospital, Beijing, China
| | - Jian Jin
- Department of Thoracic Surgery, Peking University People’s Hospital, Beijing, China
- Thoracic Oncology Institute, Peking University People’s Hospital, Beijing, China
| | - Heng Zhao
- Department of Thoracic Surgery, Peking University People’s Hospital, Beijing, China
- Thoracic Oncology Institute, Peking University People’s Hospital, Beijing, China
| | - Xun Wang
- Department of Thoracic Surgery, Peking University People’s Hospital, Beijing, China
- Thoracic Oncology Institute, Peking University People’s Hospital, Beijing, China
| | - Wenhan Weng
- Department of Thoracic Surgery, Peking University People’s Hospital, Beijing, China
- Thoracic Oncology Institute, Peking University People’s Hospital, Beijing, China
| | - Shaodong Wang
- Department of Thoracic Surgery, Peking University People’s Hospital, Beijing, China
- Thoracic Oncology Institute, Peking University People’s Hospital, Beijing, China
| | - Xizhao Sui
- Department of Thoracic Surgery, Peking University People’s Hospital, Beijing, China
- Thoracic Oncology Institute, Peking University People’s Hospital, Beijing, China
| | - Zhenfan Wang
- Department of Thoracic Surgery, Peking University People’s Hospital, Beijing, China
- Thoracic Oncology Institute, Peking University People’s Hospital, Beijing, China
| | - Chenyang Dai
- Thoracic Surgery Department, Shanghai Pulmonary Hospital, Shanghai, China
| | - Muyun Peng
- Thoracic Surgery Department, The Second Xiangya Hospital of Central South University, Changsha, China
| | - Dawei Wang
- Institute of Advanced Research, Infervision Medical Technology Co., Ltd, Beijing, China
| | - Zenghao Hao
- Institute of Advanced Research, Infervision Medical Technology Co., Ltd, Beijing, China
| | - Yafen Huang
- Institute of Advanced Research, Infervision Medical Technology Co., Ltd, Beijing, China
| | - Xiang Wang
- Thoracic Surgery Department, The Second Xiangya Hospital of Central South University, Changsha, China
| | - Liang Duan
- Thoracic Surgery Department, Shanghai Pulmonary Hospital, Shanghai, China
| | - Yuming Zhu
- Thoracic Surgery Department, Shanghai Pulmonary Hospital, Shanghai, China
| | - Nan Hong
- Department of Radiology, Peking University People’s Hospital, Beijing, China
| | - Fan Yang
- Department of Thoracic Surgery, Peking University People’s Hospital, Beijing, China
- Thoracic Oncology Institute, Peking University People’s Hospital, Beijing, China
- *Correspondence: Fan Yang,
| |
Collapse
|
9
|
Alzubaidi M, Agus M, Alyafei K, Althelaya KA, Shah U, Abd-Alrazaq AA, Anbar M, Makhlouf M, Househ M. Towards deep observation: A systematic survey on artificial intelligence techniques to monitor fetus via Ultrasound Images. iScience 2022; 25:104713. [PMID: 35856024 PMCID: PMC9287600 DOI: 10.1016/j.isci.2022.104713] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2022] [Revised: 06/09/2022] [Accepted: 06/28/2022] [Indexed: 11/26/2022] Open
Abstract
Several reviews have been conducted regarding artificial intelligence (AI) techniques to improve pregnancy outcomes. But they are not focusing on ultrasound images. This survey aims to explore how AI can assist with fetal growth monitoring via ultrasound image. We reported our findings using the guidelines for PRISMA. We conducted a comprehensive search of eight bibliographic databases. Out of 1269 studies 107 are included. We found that 2D ultrasound images were more popular (88) than 3D and 4D ultrasound images (19). Classification is the most used method (42), followed by segmentation (31), classification integrated with segmentation (16) and other miscellaneous methods such as object-detection, regression, and reinforcement learning (18). The most common areas that gained traction within the pregnancy domain were the fetus head (43), fetus body (31), fetus heart (13), fetus abdomen (10), and the fetus face (10). This survey will promote the development of improved AI models for fetal clinical applications. Artificial intelligence studies to monitor fetal development via ultrasound images Fetal issues categorized based on four categories — general, head, heart, face, abdomen The most used AI techniques are classification, segmentation, object detection, and RL The research and practical implications are included.
Collapse
|
10
|
Zeng W, Luo J, Cheng J, Lu Y. Efficient fetal ultrasound image segmentation for automatic head circumference measurement using a lightweight deep convolutional neural network. Med Phys 2022; 49:5081-5092. [PMID: 35536111 DOI: 10.1002/mp.15700] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2021] [Revised: 03/20/2022] [Accepted: 04/24/2022] [Indexed: 11/07/2022] Open
Abstract
PURPOSE Fetal head circumference (HC) is an important biometric parameter that can be used to assess fetal development in obstetric clinical practice. Most of existing methods use deep neural network to accomplish the task of automatic fetal HC measurement from two-dimensional ultrasound images, and some of them achieved relatively high prediction accuracy. However, few of these methods focused on optimizing model efficiency performance. Our purpose is to develop a more efficient approach for this task, which could help doctors measure HC faster and would be more suitable for deployment on devices with scarce computing resources. METHODS In this paper, we present a very lightweight deep convolutional neural network to achieve automatic fetal head segmentation from ultrasound images. By using sequential prediction network architecture, the proposed model could perform much faster inference while maintaining a high prediction accuracy. In addition, we used depthwise separable convolution to replace part of the standard convolution in the network and shrunk the input image to further improve model efficiency. After getting fetal head segmentation results, post-processing, including morphological processing and least-squares ellipse fitting, was applied to obtain the fetal HC. All experiments in this work were performed on a public dataset, HC18, with 999 fetal ultrasound images for training and 335 for testing. The dataset is publicly available on https://hc18.grand-challenge.org/ and the code for our method is also publicly available on https://github.com/ApeMocker/CSM-for-fetal-HC-measurement. RESULTS Our model has only 0.13 million [M] parameters and can achieve an inference speed of 28[ms] per frame on a CPU and 0.194 [ms] per frame on a GPU, which far exceeds all existing deep learning-based models as far as we know. Experimental results showed that the method achieved a mean absolute difference of 1.97 (±1.89) [mm] and a Dice similarity coefficient of 97.61(±1.72) [%] on HC18 test set, which were comparable to the state-of-the-art. CONCLUSION We presented a very lightweight deep learning-based model to realize fast and accurate fetal head segmentation from two-dimensional ultrasound image, which is then used for calculating the fetal HC. The proposed method could help obstetricians measure the fetal head circumference more efficiently with high accuracy, and has the potential to be applied to the situations where computing resources are relatively scarce. This article is protected by copyright. All rights reserved.
Collapse
Affiliation(s)
- Wen Zeng
- School of Biomedical Engineering, Shenzhen campus of Sun Yat-sen University, Shenzhen, Guangdong, 518107, China
| | - Jie Luo
- School of Biomedical Engineering, Shenzhen campus of Sun Yat-sen University, Shenzhen, Guangdong, 518107, China.,Key Laboratory of Sensing Technology and Biomedical Instrument of Guangdong Province, Guangdong Provincial Engineering and Technology Center of Advanced and Portable Medical Devices, Sun Yat-sen University, Guangzhou, China
| | - Jiaru Cheng
- School of Biomedical Engineering, Shenzhen campus of Sun Yat-sen University, Shenzhen, Guangdong, 518107, China
| | - Yiling Lu
- School of Biomedical Engineering, Shenzhen campus of Sun Yat-sen University, Shenzhen, Guangdong, 518107, China
| |
Collapse
|
11
|
Zhang L, Zhang J. Ultrasound image denoising using generative adversarial networks with residual dense connectivity and weighted joint loss. PeerJ Comput Sci 2022; 8:e873. [PMID: 35494868 PMCID: PMC9044345 DOI: 10.7717/peerj-cs.873] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2021] [Accepted: 01/11/2022] [Indexed: 06/12/2023]
Abstract
BACKGROUND Ultrasound imaging has been recognized as a powerful tool in clinical diagnosis. Nonetheless, the presence of speckle noise degrades the signal-to-noise of ultrasound images. Various denoising algorithms cannot fully reduce speckle noise and retain image features well for ultrasound imaging. The application of deep learning in ultrasound image denoising has attracted more and more attention in recent years. METHODS In the article, we propose a generative adversarial network with residual dense connectivity and weighted joint loss (GAN-RW) to avoid the limitations of traditional image denoising algorithms and surpass the most advanced performance of ultrasound image denoising. The denoising network is based on U-Net architecture which includes four encoder and four decoder modules. Each of the encoder and decoder modules is replaced with residual dense connectivity and BN to remove speckle noise. The discriminator network applies a series of convolutional layers to identify differences between the translated images and the desired modality. In the training processes, we introduce a joint loss function consisting of a weighted sum of the L1 loss function, binary cross-entropy with a logit loss function and perceptual loss function. RESULTS We split the experiments into two parts. First, experiments were performed on Berkeley segmentation (BSD68) datasets corrupted by a simulated speckle. Compared with the eight existing denoising algorithms, the GAN-RW achieved the most advanced despeckling performance in terms of the peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and subjective visual effect. When the noise level was 15, the average value of the GAN-RW increased by approximately 3.58% and 1.23% for PSNR and SSIM, respectively. When the noise level was 25, the average value of the GAN-RW increased by approximately 3.08% and 1.84% for PSNR and SSIM, respectively. When the noise level was 50, the average value of the GAN-RW increased by approximately 1.32% and 1.98% for PSNR and SSIM, respectively. Secondly, experiments were performed on the ultrasound images of lymph nodes, the foetal head, and the brachial plexus. The proposed method shows higher subjective visual effect when verifying on the ultrasound images. In the end, through statistical analysis, the GAN-RW achieved the highest mean rank in the Friedman test.
Collapse
Affiliation(s)
- Lun Zhang
- School of Information Science and Engineering, Yunnan University, Kunming, Yunnan, China
- Yunnan Vocational Institute of Energy Technology, Qujing, Yunnan, China
| | - Junhua Zhang
- School of Information Science and Engineering, Yunnan University, Kunming, Yunnan, China
| |
Collapse
|