1
|
Dubey G, Srivastava S, Jayswal AK, Saraswat M, Singh P, Memoria M. Fetal Ultrasound Segmentation and Measurements Using Appearance and Shape Prior Based Density Regression with Deep CNN and Robust Ellipse Fitting. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:247-267. [PMID: 38343234 DOI: 10.1007/s10278-023-00908-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Revised: 09/05/2023] [Accepted: 09/06/2023] [Indexed: 03/02/2024]
Abstract
Accurately segmenting the structure of the fetal head (FH) and performing biometry measurements, including head circumference (HC) estimation, stands as a vital requirement for addressing abnormal fetal growth during pregnancy under the expertise of experienced radiologists using ultrasound (US) images. However, accurate segmentation and measurement is a challenging task due to image artifact, incomplete ellipse fitting, and fluctuations due to FH dimensions over different trimesters. Also, it is highly time-consuming due to the absence of specialized features, which leads to low segmentation accuracy. To address these challenging tasks, we propose an automatic density regression approach to incorporate appearance and shape priors into the deep learning-based network model (DR-ASPnet) with robust ellipse fitting using fetal US images. Initially, we employed multiple pre-processing steps to remove unwanted distortions, variable fluctuations, and a clear view of significant features from the US images. Then some form of augmentation operation is applied to increase the diversity of the dataset. Next, we proposed the hierarchical density regression deep convolutional neural network (HDR-DCNN) model, which involves three network models to determine the complex location of FH for accurate segmentation during the training and testing processes. Then, we used post-processing operations using contrast enhancement filtering with a morphological operation model to smooth the region and remove unnecessary artifacts from the segmentation results. After post-processing, we applied the smoothed segmented result to the robust ellipse fitting-based least square (REFLS) method for HC estimation. Experimental results of the DR-ASPnet model obtain 98.86% dice similarity coefficient (DSC) as segmentation accuracy, and it also obtains 1.67 mm absolute distance (AD) as measurement accuracy compared to other state-of-the-art methods. Finally, we achieved a 0.99 correlation coefficient (CC) in estimating the measured and predicted HC values on the HC18 dataset.
Collapse
Affiliation(s)
- Gaurav Dubey
- Department of Computer Science, KIET Group of Institutions, Delhi-NCR, Ghaziabad, U.P, India
| | | | | | - Mala Saraswat
- Department of Computer Science, Bennett University, Greater Noida, India
| | - Pooja Singh
- Shiv Nadar University, Greater Noida, Uttar Pradesh, India
| | - Minakshi Memoria
- CSE Department, UIT, Uttaranchal University, Dehradun, Uttarakhand, India
| |
Collapse
|
2
|
Slimani S, Hounka S, Mahmoudi A, Rehah T, Laoudiyi D, Saadi H, Bouziyane A, Lamrissi A, Jalal M, Bouhya S, Akiki M, Bouyakhf Y, Badaoui B, Radgui A, Mhlanga M, Bouyakhf EH. Fetal biometry and amniotic fluid volume assessment end-to-end automation using Deep Learning. Nat Commun 2023; 14:7047. [PMID: 37923713 PMCID: PMC10624828 DOI: 10.1038/s41467-023-42438-5] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2023] [Accepted: 10/10/2023] [Indexed: 11/06/2023] Open
Abstract
Fetal biometry and amniotic fluid volume assessments are two essential yet repetitive tasks in fetal ultrasound screening scans, aiding in the detection of potentially life-threatening conditions. However, these assessment methods can occasionally yield unreliable results. Advances in deep learning have opened up new avenues for automated measurements in fetal ultrasound, demonstrating human-level performance in various fetal ultrasound tasks. Nevertheless, the majority of these studies are retrospective in silico studies, with a limited number including African patients in their datasets. In this study we developed and prospectively assessed the performance of deep learning models for end-to-end automation of fetal biometry and amniotic fluid volume measurements. These models were trained using a newly constructed database of 172,293 de-identified Moroccan fetal ultrasound images, supplemented with publicly available datasets. the models were then tested on prospectively acquired video clips from 172 pregnant people forming a consecutive series gathered at four healthcare centers in Morocco. Our results demonstrate that the 95% limits of agreement between the models and practitioners for the studied measurements were narrower than the reported intra- and inter-observer variability among expert human sonographers for all the parameters under study. This means that these models could be deployed in clinical conditions, to alleviate time-consuming, repetitive tasks, and make fetal ultrasound more accessible in limited-resource environments.
Collapse
Affiliation(s)
- Saad Slimani
- Deepecho, 10106, Rabat, Morocco.
- Ibn Rochd University Hospital, Hassan II University, 20100, Casablanca, Morocco.
| | - Salaheddine Hounka
- Telecommunications Systems Services and Networks lab (STRS Lab), INPT, 10112, Rabat, Morocco
| | - Abdelhak Mahmoudi
- Deepecho, 10106, Rabat, Morocco
- Ecole Normale Supérieure, LIMIARF, Mohammed V University in Rabat, 4014, Rabat, Morocco
| | | | - Dalal Laoudiyi
- Ibn Rochd University Hospital, Hassan II University, 20100, Casablanca, Morocco
| | - Hanane Saadi
- Mohammed VI University Hospital, 60049, Oujda, Morocco
| | - Amal Bouziyane
- Université Mohammed VI des Sciences de la Santé, Hôpital Universitaire Cheikh Khalifa, 82403, Casablanca, Morocco
| | - Amine Lamrissi
- Ibn Rochd University Hospital, Hassan II University, 20100, Casablanca, Morocco
| | - Mohamed Jalal
- Ibn Rochd University Hospital, Hassan II University, 20100, Casablanca, Morocco
| | - Said Bouhya
- Ibn Rochd University Hospital, Hassan II University, 20100, Casablanca, Morocco
| | | | | | - Bouabid Badaoui
- Laboratory of Biodiversity, Ecology, and Genome, Department of Biology, Faculty of Sciences, Mohammed V University in Rabat, 1014, Rabat, Morocco
- African Sustainable Agriculture Research Institute (ASARI), Mohammed VI Polytechnic University (UM6P), 43150, Laâyoune, Morocco
| | - Amina Radgui
- Telecommunications Systems Services and Networks lab (STRS Lab), INPT, 10112, Rabat, Morocco
| | - Musa Mhlanga
- Radboud Institute for Molecular Life Sciences, Epigenomics & Single Cell Biophysics, 6525 XZ, Nijmegen, the Netherlands
| | | |
Collapse
|
3
|
Gao J, Lao Q, Liu P, Yi H, Kang Q, Jiang Z, Wu X, Li K, Chen Y, Zhang L. Anatomically Guided Cross-Domain Repair and Screening for Ultrasound Fetal Biometry. IEEE J Biomed Health Inform 2023; 27:4914-4925. [PMID: 37486830 DOI: 10.1109/jbhi.2023.3298096] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/26/2023]
Abstract
Ultrasound based estimation of fetal biometry is extensively used to diagnose prenatal abnormalities and to monitor fetal growth, for which accurate segmentation of the fetal anatomy is a crucial prerequisite. Although deep neural network-based models have achieved encouraging results on this task, inevitable distribution shifts in ultrasound images can still result in severe performance drop in real world deployment scenarios. In this article, we propose a complete ultrasound fetal examination system to deal with this troublesome problem by repairing and screening the anatomically implausible results. Our system consists of three main components: A routine segmentation network, a fetal anatomical key points guided repair network, and a shape-coding based selective screener. Guided by the anatomical key points, our repair network has stronger cross-domain repair capabilities, which can substantially improve the outputs of the segmentation network. By quantifying the distance between an arbitrary segmentation mask to its corresponding anatomical shape class, the proposed shape-coding based selective screener can then effectively reject the entire implausible results that cannot be fully repaired. Extensive experiments demonstrate that our proposed framework has strong anatomical guarantee and outperforms other methods in three different cross-domain scenarios.
Collapse
|
4
|
Alzubaidi M, Agus M, Shah U, Makhlouf M, Alyafei K, Househ M. Ensemble Transfer Learning for Fetal Head Analysis: From Segmentation to Gestational Age and Weight Prediction. Diagnostics (Basel) 2022; 12:diagnostics12092229. [PMID: 36140628 PMCID: PMC9497941 DOI: 10.3390/diagnostics12092229] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2022] [Revised: 08/25/2022] [Accepted: 08/26/2022] [Indexed: 11/16/2022] Open
Abstract
Ultrasound is one of the most commonly used imaging methodologies in obstetrics to monitor the growth of a fetus during the gestation period. Specifically, ultrasound images are routinely utilized to gather fetal information, including body measurements, anatomy structure, fetal movements, and pregnancy complications. Recent developments in artificial intelligence and computer vision provide new methods for the automated analysis of medical images in many domains, including ultrasound images. We present a full end-to-end framework for segmenting, measuring, and estimating fetal gestational age and weight based on two-dimensional ultrasound images of the fetal head. Our segmentation framework is based on the following components: (i) eight segmentation architectures (UNet, UNet Plus, Attention UNet, UNet 3+, TransUNet, FPN, LinkNet, and Deeplabv3) were fine-tuned using lightweight network EffientNetB0, and (ii) a weighted voting method for building an optimized ensemble transfer learning model (ETLM). On top of that, ETLM was used to segment the fetal head and to perform analytic and accurate measurements of circumference and seven other values of the fetal head, which we incorporated into a multiple regression model for predicting the week of gestational age and the estimated fetal weight (EFW). We finally validated the regression model by comparing our result with expert physician and longitudinal references. We evaluated the performance of our framework on the public domain dataset HC18: we obtained 98.53% mean intersection over union (mIoU) as the segmentation accuracy, overcoming the state-of-the-art methods; as measurement accuracy, we obtained a 1.87 mm mean absolute difference (MAD). Finally we obtained a 0.03% mean square error (MSE) in predicting the week of gestational age and 0.05% MSE in predicting EFW.
Collapse
Affiliation(s)
- Mahmood Alzubaidi
- College of Science and Engineering, Hamad Bin Khalifa University, Doha P.O. Box 34110, Qatar
- Correspondence: (M.A.); (M.H.)
| | - Marco Agus
- College of Science and Engineering, Hamad Bin Khalifa University, Doha P.O. Box 34110, Qatar
| | - Uzair Shah
- College of Science and Engineering, Hamad Bin Khalifa University, Doha P.O. Box 34110, Qatar
| | - Michel Makhlouf
- Sidra Medical and Research Center, Sidra Medicine, Doha P.O. Box 26999, Qatar
| | - Khalid Alyafei
- Sidra Medical and Research Center, Sidra Medicine, Doha P.O. Box 26999, Qatar
| | - Mowafa Househ
- College of Science and Engineering, Hamad Bin Khalifa University, Doha P.O. Box 34110, Qatar
- Correspondence: (M.A.); (M.H.)
| |
Collapse
|
5
|
An improved semantic segmentation with region proposal network for cardiac defect interpretation. Neural Comput Appl 2022. [DOI: 10.1007/s00521-022-07217-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
6
|
Ashkani Chenarlogh V, Ghelich Oghli M, Shabanzadeh A, Sirjani N, Akhavan A, Shiri I, Arabi H, Sanei Taheri M, Tarzamni MK. Fast and Accurate U-Net Model for Fetal Ultrasound Image Segmentation. ULTRASONIC IMAGING 2022; 44:25-38. [PMID: 34986724 DOI: 10.1177/01617346211069882] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
U-Net based algorithms, due to their complex computations, include limitations when they are used in clinical devices. In this paper, we addressed this problem through a novel U-Net based architecture that called fast and accurate U-Net for medical image segmentation task. The proposed fast and accurate U-Net model contains four tuned 2D-convolutional, 2D-transposed convolutional, and batch normalization layers as its main layers. There are four blocks in the encoder-decoder path. The results of our proposed architecture were evaluated using a prepared dataset for head circumference and abdominal circumference segmentation tasks, and a public dataset (HC18-Grand challenge dataset) for fetal head circumference measurement. The proposed fast network significantly improved the processing time in comparison with U-Net, dilated U-Net, R2U-Net, attention U-Net, and MFP U-Net. It took 0.47 seconds for segmenting a fetal abdominal image. In addition, over the prepared dataset using the proposed accurate model, Dice and Jaccard coefficients were 97.62% and 95.43% for fetal head segmentation, 95.07%, and 91.99% for fetal abdominal segmentation. Moreover, we have obtained the Dice and Jaccard coefficients of 97.45% and 95.00% using the public HC18-Grand challenge dataset. Based on the obtained results, we have concluded that a fine-tuned and a simple well-structured model used in clinical devices can outperform complex models.
Collapse
Affiliation(s)
| | - Mostafa Ghelich Oghli
- Research and Development Department, Med Fanavarn Plus Co., Karaj, Iran
- Department of Cardiovascular Sciences, KU Leuven, Leuven, Belgium
| | - Ali Shabanzadeh
- Research and Development Department, Med Fanavarn Plus Co., Karaj, Iran
| | - Nasim Sirjani
- Research and Development Department, Med Fanavarn Plus Co., Karaj, Iran
| | - Ardavan Akhavan
- Research and Development Department, Med Fanavarn Plus Co., Karaj, Iran
| | - Isaac Shiri
- Division of Nuclear Medicine and Molecular Imaging, Geneva University Hospital, Geneva, Switzerland
| | - Hossein Arabi
- Division of Nuclear Medicine and Molecular Imaging, Geneva University Hospital, Geneva, Switzerland
| | - Morteza Sanei Taheri
- Department of Radiology, Shohada-e-Tajrish Hospital, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Mohammad Kazem Tarzamni
- Department of Radiology, Imam Reza Hospital, Tabriz University of Medical Sciences, Tabriz, Iran
| |
Collapse
|
7
|
Moccia S, Fiorentino MC, Frontoni E. Mask-R[Formula: see text]CNN: a distance-field regression version of Mask-RCNN for fetal-head delineation in ultrasound images. Int J Comput Assist Radiol Surg 2021; 16:1711-1718. [PMID: 34156608 PMCID: PMC8580944 DOI: 10.1007/s11548-021-02430-0] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2021] [Accepted: 06/04/2021] [Indexed: 11/03/2022]
Abstract
BACKGROUND AND OBJECTIVES Fetal head-circumference (HC) measurement from ultrasound (US) images provides useful hints for assessing fetal growth. Such measurement is performed manually during the actual clinical practice, posing issues relevant to intra- and inter-clinician variability. This work presents a fully automatic, deep-learning-based approach to HC delineation, which we named Mask-R[Formula: see text]CNN. It advances our previous work in the field and performs HC distance-field regression in an end-to-end fashion, without requiring a priori HC localization nor any postprocessing for outlier removal. METHODS Mask-R[Formula: see text]CNN follows the Mask-RCNN architecture, with a backbone inspired by feature-pyramid networks, a region-proposal network and the ROI align. The Mask-RCNN segmentation head is here modified to regress the HC distance field. RESULTS Mask-R[Formula: see text]CNN was tested on the HC18 Challenge dataset, which consists of 999 training and 335 testing images. With a comprehensive ablation study, we showed that Mask-R[Formula: see text]CNN achieved a mean absolute difference of 1.95 mm (standard deviation [Formula: see text] mm), outperforming other approaches in the literature. CONCLUSIONS With this work, we proposed an end-to-end model for HC distance-field regression. With our experimental results, we showed that Mask-R[Formula: see text]CNN may be an effective support for clinicians for assessing fetal growth.
Collapse
Affiliation(s)
- Sara Moccia
- The BioRobotics Institute, Scuola Superiore Sant'Anna, Pisa, Italy.
- Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy.
| | | | - Emanuele Frontoni
- Department of Information Engineering, Università Politecnica delle Marche, Ancona, Italy
| |
Collapse
|
8
|
Automatic linear measurements of the fetal brain on MRI with deep neural networks. Int J Comput Assist Radiol Surg 2021; 16:1481-1492. [PMID: 34185253 DOI: 10.1007/s11548-021-02436-8] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2021] [Accepted: 06/17/2021] [Indexed: 12/17/2022]
Abstract
PURPOSE Timely, accurate and reliable assessment of fetal brain development is essential to reduce short and long-term risks to fetus and mother. Fetal MRI is increasingly used for fetal brain assessment. Three key biometric linear measurements important for fetal brain evaluation are cerebral biparietal diameter (CBD), bone biparietal diameter (BBD), and trans-cerebellum diameter (TCD), obtained manually by expert radiologists on reference slices, which is time consuming and prone to human error. The aim of this study was to develop a fully automatic method computing the CBD, BBD and TCD measurements from fetal brain MRI. METHODS The input is fetal brain MRI volumes which may include the fetal body and the mother's abdomen. The outputs are the measurement values and reference slices on which the measurements were computed. The method, which follows the manual measurements principle, consists of five stages: (1) computation of a region of interest that includes the fetal brain with an anisotropic 3D U-Net classifier; (2) reference slice selection with a convolutional neural network; (3) slice-wise fetal brain structures segmentation with a multi-class U-Net classifier; (4) computation of the fetal brain midsagittal line and fetal brain orientation, and; (5) computation of the measurements. RESULTS Experimental results on 214 volumes for CBD, BBD and TCD measurements yielded a mean [Formula: see text] difference of 1.55 mm, 1.45 mm and 1.23 mm, respectively, and a Bland-Altman 95% confidence interval ([Formula: see text] of 3.92 mm, 3.98 mm and 2.25 mm, respectively. These results are similar to the manual inter-observer variability, and are consistent across gestational ages and brain conditions. CONCLUSIONS The proposed automatic method for computing biometric linear measurements of the fetal brain from MR imaging achieves human-level performance. It has the potential of being a useful method for the assessment of fetal brain biometry in normal and pathological cases, and of improving routine clinical practice.
Collapse
|
9
|
Zeng Y, Tsui PH, Wu W, Zhou Z, Wu S. Fetal Ultrasound Image Segmentation for Automatic Head Circumference Biometry Using Deeply Supervised Attention-Gated V-Net. J Digit Imaging 2021; 34:134-148. [PMID: 33483862 PMCID: PMC7887128 DOI: 10.1007/s10278-020-00410-5] [Citation(s) in RCA: 29] [Impact Index Per Article: 9.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2020] [Revised: 11/06/2020] [Accepted: 11/20/2020] [Indexed: 10/22/2022] Open
Abstract
Automatic computerized segmentation of fetal head from ultrasound images and head circumference (HC) biometric measurement is still challenging, due to the inherent characteristics of fetal ultrasound images at different semesters of pregnancy. In this paper, we proposed a new deep learning method for automatic fetal ultrasound image segmentation and HC biometry: deeply supervised attention-gated (DAG) V-Net, which incorporated the attention mechanism and deep supervision strategy into V-Net models. In addition, multi-scale loss function was introduced for deep supervision. The training set of the HC18 Challenge was expanded with data augmentation to train the DAG V-Net deep learning models. The trained models were used to automatically segment fetal head from two-dimensional ultrasound images, followed by morphological processing, edge detection, and ellipse fitting. The fitted ellipses were then used for HC biometric measurement. The proposed DAG V-Net method was evaluated on the testing set of HC18 (n = 355), in terms of four performance indices: Dice similarity coefficient (DSC), Hausdorff distance (HD), HC difference (DF), and HC absolute difference (ADF). Experimental results showed that DAG V-Net had a DSC of 97.93%, a DF of 0.09 ± 2.45 mm, an AD of 1.77 ± 1.69 mm, and an HD of 1.29 ± 0.79 mm. The proposed DAG V-Net method ranks fifth among the participants in the HC18 Challenge. By incorporating the attention mechanism and deep supervision, the proposed method yielded better segmentation performance than conventional U-Net and V-Net methods. Compared with published state-of-the-art methods, the proposed DAG V-Net had better or comparable segmentation performance. The proposed DAG V-Net may be used as a new method for fetal ultrasound image segmentation and HC biometry. The code of DAG V-Net will be made available publicly on https://github.com/xiaojinmao-code/ .
Collapse
Affiliation(s)
- Yan Zeng
- Department of Biomedical Engineering, Faculty of Environmental and Life Sciences, Beijing University of Technology, Beijing, China
| | - Po-Hsiang Tsui
- Department of Medical Imaging and Radiological Sciences, College of Medicine, Chang Gung University, Taoyuan, Taiwan
- Department of Medical Imaging and Intervention, Chang Gung Memorial Hospital, Linkou, Taoyuan, Taiwan
- Medical Imaging Research Center, Institute for Radiological Research, Chang Gung University and Chang Gung Memorial Hospital, Linkou, Taoyuan, Taiwan
| | - Weiwei Wu
- College of Biomedical Engineering, Capital Medical University, Beijing, China
| | - Zhuhuang Zhou
- Department of Biomedical Engineering, Faculty of Environmental and Life Sciences, Beijing University of Technology, Beijing, China.
| | - Shuicai Wu
- Department of Biomedical Engineering, Faculty of Environmental and Life Sciences, Beijing University of Technology, Beijing, China.
| |
Collapse
|