1
|
García-Mejido JA, Solis-Martín D, Martín-Morán M, Fernández-Conde C, Fernández-Palacín F, Sainz-Bueno JA. Applicability of Deep Learning to Dynamically Identify the Different Organs of the Pelvic Floor in the Midsagittal Plane. Int Urogynecol J 2024:10.1007/s00192-024-05841-0. [PMID: 38913129 DOI: 10.1007/s00192-024-05841-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2024] [Accepted: 05/01/2024] [Indexed: 06/25/2024]
Abstract
INTRODUCTION AND HYPOTHESIS The objective was to create and validate the usefulness of a convolutional neural network (CNN) for identifying different organs of the pelvic floor in the midsagittal plane via dynamic ultrasound. METHODS This observational and prospective study included 110 patients. Transperineal ultrasound scans were performed by an expert sonographer of the pelvic floor. A video of each patient was made that captured the midsagittal plane of the pelvic floor at rest and the change in the pelvic structures during the Valsalva maneuver. After saving the captured videos, we manually labeled the different organs in each video. Three different architectures were tested-UNet, FPN, and LinkNet-to determine which CNN model best recognized anatomical structures. The best model was trained with the 86 cases for the number of epochs determined by the stop criterion via cross-validation. The Dice Similarity Index (DSI) was used for CNN validation. RESULTS Eighty-six patients were included to train the CNN and 24 to test the CNN. After applying the trained CNN to the 24 test videos, we did not observe any failed segmentation. In fact, we obtained a DSI of 0.79 (95% CI: 0.73 - 0.82) as the median of the 24 test videos. When we studied the organs independently, we observed differences in the DSI of each organ. The poorest DSIs were obtained in the bladder (0.71 [95% CI: 0.70 - 0.73]) and uterus (0.70 [95% CI: 0.68 - 0.74]), whereas the highest DSIs were obtained in the anus (0.81 [95% CI: 0.80 - 0.86]) and levator ani muscle (0.83 [95% CI: 0.82 - 0.83]). CONCLUSIONS Our results show that it is possible to apply deep learning using a trained CNN to identify different pelvic floor organs in the midsagittal plane via dynamic ultrasound.
Collapse
Affiliation(s)
- José Antonio García-Mejido
- Department of Obstetrics and Gynecology, Valme University Hospital, Seville, Spain.
- Department of Surgery, Faculty of Medicine, University of Seville, Seville, Spain.
| | - David Solis-Martín
- Department of Computer Science and Artificial Intelligence, Faculty of Mathematics, University of Seville, Seville, Spain
| | - Marina Martín-Morán
- Department of Obstetrics and Gynecology, Valme University Hospital, Seville, Spain
| | | | | | - José Antonio Sainz-Bueno
- Department of Obstetrics and Gynecology, Valme University Hospital, Seville, Spain
- Department of Surgery, Faculty of Medicine, University of Seville, Seville, Spain
| |
Collapse
|
2
|
Qu E, Wu S, Zhang M, Huang Z, Zheng Z, Zhang X. Validation of a built-in software in automatically reconstructing the tomographic images of the levator ani muscle. Int Urogynecol J 2024; 35:175-181. [PMID: 38019307 DOI: 10.1007/s00192-023-05686-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Accepted: 10/31/2023] [Indexed: 11/30/2023]
Abstract
INTRODUCTION AND HYPOTHESIS Transperineal ultrasound (TPUS) is an effective tool for evaluating the integrity of the levator ani muscle (LAM). Several operating steps are required to obtain the standard multi-slice image of the LAM, which is experience dependent and time consuming. This study was aimed at evaluating the feasibility and reproducibility of the built-in software, Smart-pelvic™, in reconstructing standard tomographic images of LAM from 3D/4D TPUS volumes. METHODS This study was conducted at a tertiary teaching hospital, enrolling women who underwent TPUS. Tomographic images of the LAM were automatically reconstructed by Smart-pelvicTM and rated by two experienced observers as standard or nonstandard. The anteroposterior diameter (APD) of the levator hiatus was also measured on the mid-sagittal plane of the automatically and manually reconstructed images. The APD measurements of each approach were compared using Bland-Altman plots, and interclass correlation coefficient (ICC) was used to evaluate intra- and inter-observer reproducibility. Meanwhile, the time taken for the reconstruction process of both methods was also recorded. RESULTS The ultrasound volume of a total of 104 patients were included in this study. Using Smart-pelvicTM, the overall success rate of the tomographic image reconstruction was 98%. Regarding measurements of APD, the ICC between the automatic and manual reconstruction methods was 0.99 (0.98, 0.99). The average time taken for reconstruction per case was 2.65 ± 0.52 s and 22.08 ± 3.45 s, respectively. CONCLUSIONS Using Smart-pelvicTM to reconstruct tomographic images of LAM is feasible, and it can promote TPUS by reducing operator dependence and improving examination efficiency in a clinical setting.
Collapse
Affiliation(s)
- Enze Qu
- Department of Ultrasound, Third Affiliated Hospital of Sun Yat-Sen University, 600 Tianhe Road, Guangzhou, Guangdong Province, China
| | - Shuangyu Wu
- Department of Ultrasound, Third Affiliated Hospital of Sun Yat-Sen University, 600 Tianhe Road, Guangzhou, Guangdong Province, China
| | - Man Zhang
- Department of Ultrasound, Third Affiliated Hospital of Sun Yat-Sen University, 600 Tianhe Road, Guangzhou, Guangdong Province, China
| | - Zeping Huang
- Department of Ultrasound, Third Affiliated Hospital of Sun Yat-Sen University, 600 Tianhe Road, Guangzhou, Guangdong Province, China
| | - Zhijuan Zheng
- Department of Ultrasound, Third Affiliated Hospital of Sun Yat-Sen University, 600 Tianhe Road, Guangzhou, Guangdong Province, China
| | - Xinling Zhang
- Department of Ultrasound, Third Affiliated Hospital of Sun Yat-Sen University, 600 Tianhe Road, Guangzhou, Guangdong Province, China.
| |
Collapse
|
3
|
Rabbat N, Qureshi A, Hsu KT, Asif Z, Chitnis P, Shobeiri SA, Wei Q. Automated Segmentation of Levator Ani Muscle from 3D Endovaginal Ultrasound Images. Bioengineering (Basel) 2023; 10:894. [PMID: 37627779 PMCID: PMC10451809 DOI: 10.3390/bioengineering10080894] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2023] [Revised: 07/18/2023] [Accepted: 07/21/2023] [Indexed: 08/27/2023] Open
Abstract
Levator ani muscle (LAM) avulsion is a common complication of vaginal childbirth and is linked to several pelvic floor disorders. Diagnosing and treating these conditions require imaging of the pelvic floor and examination of the obtained images, which is a time-consuming process subjected to operator variability. In our study, we proposed using deep learning (DL) to automate the segmentation of the LAM from 3D endovaginal ultrasound images (EVUS) to improve diagnostic accuracy and efficiency. Over one thousand images extracted from the 3D EVUS data of healthy subjects and patients with pelvic floor disorders were utilized for the automated LAM segmentation. A U-Net model was implemented, with Intersection over Union (IoU) and Dice metrics being used for model performance evaluation. The model achieved a mean Dice score of 0.86, demonstrating a better performance than existing works. The mean IoU was 0.76, indicative of a high degree of overlap between the automated and manual segmentation of the LAM. Three other models including Attention UNet, FD-UNet and Dense-UNet were also applied on the same images which showed comparable results. Our study demonstrated the feasibility and accuracy of using DL segmentation with U-Net architecture to automate LAM segmentation to reduce the time and resources required for manual segmentation of 3D EVUS images. The proposed method could become an important component in AI-based diagnostic tools, particularly in low socioeconomic regions where access to healthcare resources is limited. By improving the management of pelvic floor disorders, our approach may contribute to better patient outcomes in these underserved areas.
Collapse
Affiliation(s)
- Nada Rabbat
- Department of Bioengineering, George Mason University, Fairfax, VA 22030, USA; (N.R.); (A.Q.); (K.-T.H.); (P.C.); (S.A.S.)
| | - Amad Qureshi
- Department of Bioengineering, George Mason University, Fairfax, VA 22030, USA; (N.R.); (A.Q.); (K.-T.H.); (P.C.); (S.A.S.)
| | - Ko-Tsung Hsu
- Department of Bioengineering, George Mason University, Fairfax, VA 22030, USA; (N.R.); (A.Q.); (K.-T.H.); (P.C.); (S.A.S.)
| | - Zara Asif
- Department of Bioengineering, George Mason University, Fairfax, VA 22030, USA; (N.R.); (A.Q.); (K.-T.H.); (P.C.); (S.A.S.)
| | - Parag Chitnis
- Department of Bioengineering, George Mason University, Fairfax, VA 22030, USA; (N.R.); (A.Q.); (K.-T.H.); (P.C.); (S.A.S.)
| | - Seyed Abbas Shobeiri
- Department of Bioengineering, George Mason University, Fairfax, VA 22030, USA; (N.R.); (A.Q.); (K.-T.H.); (P.C.); (S.A.S.)
- Inova Fairfax Hospital, Fairfax, VA 22042, USA
| | - Qi Wei
- Department of Bioengineering, George Mason University, Fairfax, VA 22030, USA; (N.R.); (A.Q.); (K.-T.H.); (P.C.); (S.A.S.)
| |
Collapse
|
4
|
Asif Z, Tomashev R, Peterkin V, Wei Q, Alshiek J, Yael B, Shobeiri SA. Levator ani muscle volume and architecture in normal vs. muscle damage patients using 3D endovaginal ultrasound: a pilot study. Int Urogynecol J 2023; 34:581-587. [PMID: 36173426 DOI: 10.1007/s00192-022-05366-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2022] [Accepted: 08/20/2022] [Indexed: 01/26/2023]
Abstract
INTRODUCTION AND HYPOTHESIS This study aimed to compare the difference in levator ani muscle (LAM) volumes between 'normal' and those with sonographically visualized LAM defects. We hypothesized that the 'muscle damage' group would have a significantly lower muscle volume. METHODS The study included patients who had undergone a 3D endovaginal ultrasound. The normal (NM) and damage (DM) muscle groups' architectural changes were evaluated based on anterior-posterior (AP), left-right (LR) diameter, and minimal levator hiatus (MLH) area. The puboanalis-puboperinealis (PA), puborectalis (PR), and pubococcygeus-iliococcygeus (PC) were manually segmented using 2.5 vs. 1.0 mm to find the optimal sequence and to compare the volumes between NM and DM groups. POPQs were compared between the NM and DM groups. RESULTS The 1.0-mm segmentation volumes created superior volume analysis. Comparing NM to the DM group showed no significant difference in LAM volume. Respectively, the mean total LAM volumes were 17.27 cm3 (SD = 3.97) and 17.04 cm3 (SD = 4.32), p = 0.79. The mean MLH measurements for both groups respectively were 10.06 cm2 (SD = 2.93) and 12.18 cm2 (SD = 2.93), indicating a significant difference (p = 0.01). POPQ analysis demonstrated statistically significant differences at Ba and Bp parameters suggesting that the DM group had worse prolapse (p = 0.05, 0.01, respectively). CONCLUSIONS While LAM volumes are similar, there is a significant difference in the physical architecture of the LAM and the POPQ parameters in muscle-damaged patients compared to the normal group.
Collapse
Affiliation(s)
- Zara Asif
- Department of Bioengineering, George Mason University, Fairfax, VA, USA
| | - Roni Tomashev
- Department of Obstetrics & Gynecology, INOVA Women's Hospital, 3300 Gallows Road, Second-floor South tower, Falls Church, VA, 22042-3307, USA
| | - Veronica Peterkin
- Department of Obstetrics & Gynecology, INOVA Women's Hospital, 3300 Gallows Road, Second-floor South tower, Falls Church, VA, 22042-3307, USA
| | - Qi Wei
- Department of Bioengineering, George Mason University, Fairfax, VA, USA
| | - Jonia Alshiek
- Department of Obstetrics & Gynecology, INOVA Women's Hospital, 3300 Gallows Road, Second-floor South tower, Falls Church, VA, 22042-3307, USA
| | - Baumfeld Yael
- Department of Obstetrics & Gynecology, INOVA Women's Hospital, 3300 Gallows Road, Second-floor South tower, Falls Church, VA, 22042-3307, USA
| | - S Abbas Shobeiri
- Department of Bioengineering, George Mason University, Fairfax, VA, USA. .,Department of Obstetrics & Gynecology, INOVA Women's Hospital, 3300 Gallows Road, Second-floor South tower, Falls Church, VA, 22042-3307, USA.
| |
Collapse
|
5
|
van den Noort F, Manzini C, Hofsteenge M, Sirmacek B, van der Vaart CH, Slump CH. Unsupervised convolutional autoencoders for 4D transperineal ultrasound classification. J Med Imaging (Bellingham) 2023; 10:014004. [PMID: 36785585 PMCID: PMC9921518 DOI: 10.1117/1.jmi.10.1.014004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2021] [Accepted: 01/23/2023] [Indexed: 02/13/2023] Open
Abstract
Purpose 4D Transperineal ultrasound (TPUS) is used to examine female pelvic floor disorders. Muscle movement, like performing a muscle contraction or a Valsalva maneuver, can be captured on TPUS. Our work investigates the possibility for unsupervised analysis and classification of the TPUS data. Approach An unsupervised 3D-convolutional autoencoder is trained to compress TPUS volume frames into a latent feature vector (LFV) of 128 elements. The (co)variance of the features are analyzed and statistical tests are performed to analyze how features contribute in storing contraction and Valsalva information. Further dimensionality reduction is applied (principal component analysis or a 2D-convolutional autoencoder) to the LFVs of the frames of the TPUS movie to compress the data and analyze the interframe movement. Clustering algorithms ( K -means clustering and Gaussian mixture models) are applied to this representation of the data to investigate the possibilities of unsupervised classification. Results The majority of the features show a significant difference between contraction and Valsalva. The (co)variance of the features from the LFVs was investigated and features most prominent in capturing muscle movement were identified. Furthermore, the first principal component of the frames from a single TPUS movie can be used to identify movement between the frames. The best classification results were obtained after applying principal component analysis and Gaussian mixture models to the LFVs of the TPUS movies, yielding a 91.2% accuracy. Conclusion Unsupervised analysis and classification of TPUS data yields relevant information about the type and amount of muscle movement present.
Collapse
Affiliation(s)
- Frieda van den Noort
- University of Twente, Technical Medical Centre, Robotics and Mechatronics, Faculty of Electrical Engineering Mathematics and Computer Science, Enschede, The Netherlands
| | - Claudia Manzini
- University Medical Centre Utrecht, Department of Obstetrics and Gynecology, Utrecht, The Netherlands
| | - Merijn Hofsteenge
- University of Twente, Technical Medical Centre, Robotics and Mechatronics, Faculty of Electrical Engineering Mathematics and Computer Science, Enschede, The Netherlands
| | - Beril Sirmacek
- Saxion University of Applied Sciences, School of Creative Technology, Smart Cities Group, Enschede, The Netherlands
| | - Carl H. van der Vaart
- University Medical Centre Utrecht, Department of Obstetrics and Gynecology, Utrecht, The Netherlands
| | - Cornelis H. Slump
- University of Twente, Technical Medical Centre, Robotics and Mechatronics, Faculty of Electrical Engineering Mathematics and Computer Science, Enschede, The Netherlands
| |
Collapse
|
6
|
Xia W, Ameri G, Fakim D, Akhuanzada H, Raza MZ, Shobeiri SA, McLean L, Chen ECS. Automatic Plane of Minimal Hiatal Dimensions Extraction From 3D Female Pelvic Floor Ultrasound. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:3873-3883. [PMID: 35984794 DOI: 10.1109/tmi.2022.3199968] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
There is an increasing interest in the applications of 3D ultrasound imaging of the pelvic floor to improve the diagnosis, treatment, and surgical planning of female pelvic floor dysfunction (PFD). Pelvic floor biometrics are obtained on an oblique image plane known as the plane of minimal hiatal dimensions (PMHD). Identifying this plane requires the detection of two anatomical landmarks, the pubic symphysis and anorectal angle. The manual detection of the anatomical landmarks and the PMHD in 3D pelvic ultrasound requires expert knowledge of the pelvic floor anatomy, and is challenging, time-consuming, and subject to human error. These challenges have hindered the adoption of such quantitative analysis in the clinic. This work presents an automatic approach to identify the anatomical landmarks and extract the PMHD from 3D pelvic ultrasound volumes. To demonstrate clinical utility and a complete automated clinical task, an automatic segmentation of the levator-ani muscle on the extracted PMHD images was also performed. Experiments using 73 test images of patients during a pelvic muscle resting state showed that this algorithm has the capability to accurately identify the PMHD with an average Dice of 0.89 and an average mean boundary distance of 2.25mm. Further evaluation of the PMHD detection algorithm using 35 images of patients performing pelvic muscle contraction resulted in an average Dice of 0.88 and an average mean boundary distance of 2.75mm. This work had the potential to pave the way towards the adoption of ultrasound in the clinic and development of personalized treatment for PFD.
Collapse
|
7
|
van den Noort F, Manzini C, van der Vaart CH, van Limbeek MAJ, Slump CH, Grob ATM. Automatic identification and segmentation of slice of minimal hiatal dimensions in transperineal ultrasound volumes. ULTRASOUND IN OBSTETRICS & GYNECOLOGY : THE OFFICIAL JOURNAL OF THE INTERNATIONAL SOCIETY OF ULTRASOUND IN OBSTETRICS AND GYNECOLOGY 2022; 60:570-576. [PMID: 34767663 PMCID: PMC9828486 DOI: 10.1002/uog.24810] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/29/2021] [Revised: 10/07/2021] [Accepted: 10/26/2021] [Indexed: 05/31/2023]
Abstract
OBJECTIVE To develop and validate a tool for automatic selection of the slice of minimal hiatal dimensions (SMHD) and segmentation of the urogenital hiatus (UH) in transperineal ultrasound (TPUS) volumes. METHODS Manual selection of the SMHD and segmentation of the UH was performed in TPUS volumes of 116 women with symptomatic pelvic organ prolapse (POP). These data were used to train two deep-learning algorithms. The first algorithm was trained to provide an estimation of the position of the SMHD. Based on this estimation, a slice was selected and fed into the second algorithm, which performed automatic segmentation of the UH. From this segmentation, measurements of the UH area (UHA), anteroposterior diameter (APD) and coronal diameter (CD) were computed automatically. The mean absolute distance between manually and automatically selected SMHD, the overlap (dice similarity index (DSI)) between manual and automatic UH segmentation and the intraclass correlation coefficient (ICC) between manual and automatic UH measurements were assessed on a test set of 30 TPUS volumes. RESULTS The mean absolute distance between manually and automatically selected SMHD was 0.20 cm. All DSI values between manual and automatic UH segmentations were above 0.85. The ICC values between manual and automatic UH measurements were 0.94 (95% CI, 0.87-0.97) for UHA, 0.92 (95% CI, 0.78-0.97) for APD and 0.82 (95% CI, 0.66-0.91) for CD, demonstrating excellent agreement. CONCLUSIONS Our deep-learning algorithms allowed reliable automatic selection of the SMHD and UH segmentation in TPUS volumes of women with symptomatic POP. These algorithms can be implemented in the software of TPUS machines, thus reducing clinical analysis time and simplifying the examination of TPUS data for research and clinical purposes. © 2021 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of International Society of Ultrasound in Obstetrics and Gynecology.
Collapse
Affiliation(s)
- F. van den Noort
- Robotics and Mechatronics, Faculty of Electrical Engineering, Mathematics and Computer Science, Technical Medical CentreUniversity of TwenteEnschedeThe Netherlands
| | - C. Manzini
- Department of Obstetrics and GynecologyUniversity Medical Centre UtrechtUtrechtThe Netherlands
| | - C. H. van der Vaart
- Department of Obstetrics and GynecologyUniversity Medical Centre UtrechtUtrechtThe Netherlands
| | - M. A. J. van Limbeek
- Dynamics of Complex Fluids, Max Planck Institute for Dynamics and Self‐OrganizationGöttingenGermany
| | - C. H. Slump
- Robotics and Mechatronics, Faculty of Electrical Engineering, Mathematics and Computer Science, Technical Medical CentreUniversity of TwenteEnschedeThe Netherlands
| | - A. T. M. Grob
- Multi‐Modality Medical Imaging, Faculty of Science and TechnologyTechnical Medical Centre, University of TwenteEnschedeThe Netherlands
| |
Collapse
|
8
|
Diagnosing gastrointestinal diseases from endoscopy images through a multi-fused CNN with auxiliary layers, alpha dropouts, and a fusion residual block. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103683] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
9
|
Bonmati E, Hu Y, Grimwood A, Johnson GJ, Goodchild G, Keane MG, Gurusamy K, Davidson B, Clarkson MJ, Pereira SP, Barratt DC. Voice-Assisted Image Labeling for Endoscopic Ultrasound Classification Using Neural Networks. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:1311-1319. [PMID: 34962866 DOI: 10.1109/tmi.2021.3139023] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Ultrasound imaging is a commonly used technology for visualising patient anatomy in real-time during diagnostic and therapeutic procedures. High operator dependency and low reproducibility make ultrasound imaging and interpretation challenging with a steep learning curve. Automatic image classification using deep learning has the potential to overcome some of these challenges by supporting ultrasound training in novices, as well as aiding ultrasound image interpretation in patient with complex pathology for more experienced practitioners. However, the use of deep learning methods requires a large amount of data in order to provide accurate results. Labelling large ultrasound datasets is a challenging task because labels are retrospectively assigned to 2D images without the 3D spatial context available in vivo or that would be inferred while visually tracking structures between frames during the procedure. In this work, we propose a multi-modal convolutional neural network (CNN) architecture that labels endoscopic ultrasound (EUS) images from raw verbal comments provided by a clinician during the procedure. We use a CNN composed of two branches, one for voice data and another for image data, which are joined to predict image labels from the spoken names of anatomical landmarks. The network was trained using recorded verbal comments from expert operators. Our results show a prediction accuracy of 76% at image level on a dataset with 5 different labels. We conclude that the addition of spoken commentaries can increase the performance of ultrasound image classification, and eliminate the burden of manually labelling large EUS datasets necessary for deep learning applications.
Collapse
|
10
|
Huang Z, Qu E, Meng Y, Zhang M, Wei Q, Bai X, Zhang X. Deep learning-based pelvic levator hiatus segmentation from ultrasound images. Eur J Radiol Open 2022; 9:100412. [PMID: 35345817 PMCID: PMC8956942 DOI: 10.1016/j.ejro.2022.100412] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2022] [Revised: 02/27/2022] [Accepted: 03/09/2022] [Indexed: 11/29/2022] Open
Abstract
Purpose To automatically segment and measure the levator hiatus with a deep learning approach and evaluate the performance between algorithms, sonographers, and different devices. Methods Three deep learning models (UNet-ResNet34, HR-Net, and SegNet) were trained with 360 images and validated with 42 images. The trained models were tested with two test sets. The first set included 138 images to evaluate the performance between the algorithms and sonographers. An independent dataset including 679 images assessed the performances of algorithms between different ultrasound devices. Four metrics were used for evaluation: DSC, HDD, the relative error of segmentation area, and the absolute error of segmentation area. Results The UNet model outperformed HR-Net and SegNet. It could achieve a mean DSC of 0.964 for the first test set and 0.952 for the independent test set. UNet was creditable compared with three senior sonographers with a noninferiority test in the first test set and equivalent in the two test sets collected by different devices. On average, it took two seconds to process one case with a GPU and 2.4 s with a CPU. Conclusions The deep learning approach has good performance for levator hiatus segmentation and good generalization ability on independent test sets. This automatic levator hiatus segmentation approach could help shorten the clinical examination time and improve consistency.
Collapse
Affiliation(s)
- Zeping Huang
- Department of Ultrasound, the Third Affiliated Hospital of Sun Yat-sen University, 600 Tianhe Road, Guangzhou 510630, China
| | - Enze Qu
- Department of Ultrasound, the Third Affiliated Hospital of Sun Yat-sen University, 600 Tianhe Road, Guangzhou 510630, China
| | - Yishuang Meng
- Philips (China) Investment Co. Ltd, 6F, Building A2, 718 Lingshi Road, Shanghai 200072, China
| | - Man Zhang
- Department of Ultrasound, the Third Affiliated Hospital of Sun Yat-sen University, 600 Tianhe Road, Guangzhou 510630, China
| | - Qiuwen Wei
- Philips (China) Investment Co. Ltd, 6F, Building A2, 718 Lingshi Road, Shanghai 200072, China
| | - Xianghui Bai
- Philips (China) Investment Co. Ltd, 6F, Building A2, 718 Lingshi Road, Shanghai 200072, China
| | - Xinling Zhang
- Department of Ultrasound, the Third Affiliated Hospital of Sun Yat-sen University, 600 Tianhe Road, Guangzhou 510630, China
| |
Collapse
|
11
|
Williams H, Cattani L, Van Schoubroeck D, Yaqub M, Sudre C, Vercauteren T, D'Hooge J, Deprest J. Automatic Extraction of Hiatal Dimensions in 3-D Transperineal Pelvic Ultrasound Recordings. ULTRASOUND IN MEDICINE & BIOLOGY 2021; 47:3470-3479. [PMID: 34538535 DOI: 10.1016/j.ultrasmedbio.2021.08.009] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/04/2021] [Revised: 08/04/2021] [Accepted: 08/11/2021] [Indexed: 06/13/2023]
Abstract
The aims of this work were to create a robust automatic software tool for measurement of the levator hiatal area on transperineal ultrasound (TPUS) volumes and to measure the potential reduction in variability and time taken for analysis in a clinical setting. The proposed tool automatically detects the C-plane (i.e., the plane of minimal hiatal dimensions) from a 3-D TPUS volume and subsequently uses the extracted plane to automatically segment the levator hiatus, using a convolutional neural network. The automatic pipeline was tested using 73 representative TPUS volumes. Reference hiatal outlines were obtained manually by two experts and compared with the pipeline's automated outlines. The Hausdorff distance, area, a clinical quality score, C-plane angle and C-plane Euclidean distance were used to evaluate C-plane detection and quantify levator hiatus segmentation accuracy. A visual Turing test was created to compare the performance of the software with that of the expert, based on the visual assessment of C-plane and hiatal segmentation quality. The overall time taken to extract the hiatal area with both measurement methods (i.e., manual and automatic) was measured. Each metric was calculated both for computer-observer differences and for inter-and intra-observer differences. The automatic method gave results similar to those of the expert when determining the hiatal outline from a TPUS volume. Indeed, the hiatal area measured by the algorithm and by an expert were within the intra-observer variability. Similarly, the method identified the C-plane with an accuracy of 5.76 ± 5.06° and 6.46 ± 5.18 mm in comparison to the inter-observer variability of 9.39 ± 6.21° and 8.48 ± 6.62 mm. The visual Turing test suggested that the automatic method identified the C-plane position within the TPUS volume visually as well as the expert. The average time taken to identify the C-plane and segment the hiatal area manually was 2 min and 35 ± 17 s, compared with 35 ± 4 s for the automatic result. This study presents a method for automatically measuring the levator hiatal area using artificial intelligence-based methodologies whereby the C-plane within a TPUS volume is detected and subsequently traced for the levator hiatal outline. The proposed solution was determined to be accurate, relatively quick, robust and reliable and, importantly, to reduce time and expertise required for pelvic floor disorder assessment.
Collapse
Affiliation(s)
- Helena Williams
- Department of Development and Regeneration, Cluster Urogenital Surgery, Biomedical Sciences, KU Leuven; School of Biomedical Engineering & Imaging Sciences, King's College London, United Kingdom; Department of Cardiovascular Sciences, KU Leuven, Leuven, Belgium.
| | - Laura Cattani
- Department of Development and Regeneration, Cluster Urogenital Surgery, Biomedical Sciences, KU Leuven; Clinical Department of Obstetrics and Gynaecology, UZ Leuven, Leuven, Belgium
| | - Dominique Van Schoubroeck
- Department of Development and Regeneration, Cluster Urogenital Surgery, Biomedical Sciences, KU Leuven; Clinical Department of Obstetrics and Gynaecology, UZ Leuven, Leuven, Belgium
| | - Mohammad Yaqub
- Department of Computer Vision, Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates
| | - Carole Sudre
- School of Biomedical Engineering & Imaging Sciences, King's College London, United Kingdom
| | - Tom Vercauteren
- School of Biomedical Engineering & Imaging Sciences, King's College London, United Kingdom
| | - Jan D'Hooge
- Department of Cardiovascular Sciences, KU Leuven, Leuven, Belgium
| | - Jan Deprest
- Department of Development and Regeneration, Cluster Urogenital Surgery, Biomedical Sciences, KU Leuven; Clinical Department of Obstetrics and Gynaecology, UZ Leuven, Leuven, Belgium
| |
Collapse
|
12
|
Feng F, Ashton‐Miller JA, DeLancey JOL, Luo J. Convolutional neural network‐based pelvic floor structure segmentation using magnetic resonance imaging in pelvic organ prolapse. Med Phys 2020; 47:4281-4293. [DOI: 10.1002/mp.14377] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2019] [Revised: 06/18/2020] [Accepted: 06/22/2020] [Indexed: 11/11/2022] Open
Affiliation(s)
- Fei Feng
- University of Michigan‐Shanghai Jiao Tong University Joint Institute Shanghai Jiao Tong University Shanghai200240China
| | | | - John O. L. DeLancey
- Department of Obstetrics and Gynecology University of Michigan Ann Arbor MI48109USA
| | - Jiajia Luo
- Biomedical Engineering Department Peking University Beijing100191China
| |
Collapse
|
13
|
Ghavami N, Hu Y, Gibson E, Bonmati E, Emberton M, Moore CM, Barratt DC. Automatic segmentation of prostate MRI using convolutional neural networks: Investigating the impact of network architecture on the accuracy of volume measurement and MRI-ultrasound registration. Med Image Anal 2019; 58:101558. [PMID: 31526965 PMCID: PMC7985677 DOI: 10.1016/j.media.2019.101558] [Citation(s) in RCA: 32] [Impact Index Per Article: 6.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2019] [Revised: 09/10/2019] [Accepted: 09/10/2019] [Indexed: 12/27/2022]
Abstract
Convolutional neural networks (CNNs) have recently led to significant advances in automatic segmentations of anatomical structures in medical images, and a wide variety of network architectures are now available to the research community. For applications such as segmentation of the prostate in magnetic resonance images (MRI), the results of the PROMISE12 online algorithm evaluation platform have demonstrated differences between the best-performing segmentation algorithms in terms of numerical accuracy using standard metrics such as the Dice score and boundary distance. These small differences in the segmented regions/boundaries outputted by different algorithms may potentially have an unsubstantial impact on the results of downstream image analysis tasks, such as estimating organ volume and multimodal image registration, which inform clinical decisions. This impact has not been previously investigated. In this work, we quantified the accuracy of six different CNNs in segmenting the prostate in 3D patient T2-weighted MRI scans and compared the accuracy of organ volume estimation and MRI-ultrasound (US) registration errors using the prostate segmentations produced by different networks. Networks were trained and tested using a set of 232 patient MRIs with labels provided by experienced clinicians. A statistically significant difference was found among the Dice scores and boundary distances produced by these networks in a non-parametric analysis of variance (p < 0.001 and p < 0.001, respectively), where the following multiple comparison tests revealed that the statistically significant difference in segmentation errors were caused by at least one tested network. Gland volume errors (GVEs) and target registration errors (TREs) were then estimated using the CNN-generated segmentations. Interestingly, there was no statistical difference found in either GVEs or TREs among different networks, (p = 0.34 and p = 0.26, respectively). This result provides a real-world example that these networks with different segmentation performances may potentially provide indistinguishably adequate registration accuracies to assist prostate cancer imaging applications. We conclude by recommending that the differences in the accuracy of downstream image analysis tasks that make use of data output by automatic segmentation methods, such as CNNs, within a clinical pipeline should be taken into account when selecting between different network architectures, in addition to reporting the segmentation accuracy.
Collapse
Affiliation(s)
- Nooshin Ghavami
- Centre for Medical Image Computing, Department of Medical Physics and Biomedical Engineering, University College London, London, UK; Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK.
| | - Yipeng Hu
- Centre for Medical Image Computing, Department of Medical Physics and Biomedical Engineering, University College London, London, UK; Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK
| | - Eli Gibson
- Centre for Medical Image Computing, Department of Medical Physics and Biomedical Engineering, University College London, London, UK; Siemens Healthineers, Princeton, USA
| | - Ester Bonmati
- Centre for Medical Image Computing, Department of Medical Physics and Biomedical Engineering, University College London, London, UK; Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK
| | - Mark Emberton
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK; Division of Surgery & Interventional Science, University College London, London, UK
| | - Caroline M Moore
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK; Division of Surgery & Interventional Science, University College London, London, UK
| | - Dean C Barratt
- Centre for Medical Image Computing, Department of Medical Physics and Biomedical Engineering, University College London, London, UK; Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London, UK
| |
Collapse
|
14
|
van den Noort F, van der Vaart CH, Grob ATM, van de Waarsenburg MK, Slump CH, van Stralen M. Deep learning enables automatic quantitative assessment of puborectalis muscle and urogenital hiatus in plane of minimal hiatal dimensions. ULTRASOUND IN OBSTETRICS & GYNECOLOGY : THE OFFICIAL JOURNAL OF THE INTERNATIONAL SOCIETY OF ULTRASOUND IN OBSTETRICS AND GYNECOLOGY 2019; 54:270-275. [PMID: 30461079 PMCID: PMC6772057 DOI: 10.1002/uog.20181] [Citation(s) in RCA: 23] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/22/2018] [Revised: 10/12/2018] [Accepted: 11/15/2018] [Indexed: 05/05/2023]
Abstract
OBJECTIVES To measure the length, width and area of the urogenital hiatus (UH), and the length and mean echogenicity (MEP) of the puborectalis muscle (PRM), automatically and observer-independently, in the plane of minimal hiatal dimensions on transperineal ultrasound (TPUS) images, by automatic segmentation of the UH and the PRM using deep learning. METHODS In 1318 three- and four-dimensional (3D/4D) TPUS volume datasets from 253 nulliparae at 12 and 36 weeks' gestation, two-dimensional (2D) images in the plane of minimal hiatal dimensions with the PRM at rest, on maximum contraction and on maximum Valsalva maneuver, were obtained manually and the UH and PRM were segmented manually. In total, 713 of the images were used to train a convolutional neural network (CNN) to segment automatically the UH and PRM in the plane of minimal hiatal dimensions. In the remainder of the dataset (test set 1 (TS1); 601 images, four having been excluded), the performance of the CNN was evaluated by comparing automatic and manual segmentations. The performance of the CNN was also tested on 117 images from an independent dataset (test set 2 (TS2); two images having been excluded) from 40 nulliparae at 12 weeks' gestation, which were acquired and segmented manually by a different observer. The success of automatic segmentation was assessed visually. Based on the CNN segmentations, the following clinically relevant parameters were measured: the length, width and area of the UH, the length of the PRM and MEP. The overlap (Dice similarity index (DSI)) and surface distance (mean absolute distance (MAD) and Hausdorff distance (HDD)) between manual and CNN segmentations were measured to investigate their similarity. For the measured clinically relevant parameters, the intraclass correlation coefficients (ICCs) between manual and CNN results were determined. RESULTS Fully automatic CNN segmentation was successful in 99.0% and 93.2% of images in TS1 and TS2, respectively. DSI, MAD and HDD showed good overlap and distance between manual and CNN segmentations in both test sets. This was reflected in the respective ICC values in TS1 and TS2 for the length (0.96 and 0.95), width (0.77 and 0.87) and area (0.96 and 0.91) of the UH, the length of the PRM (0.87 and 0.73) and MEP (0.95 and 0.97), which showed good to very good agreement. CONCLUSION Deep learning can be used to segment automatically and reliably the PRM and UH on 2D ultrasound images of the nulliparous pelvic floor in the plane of minimal hiatal dimensions. These segmentations can be used to measure reliably UH dimensions as well as PRM length and MEP. © 2018 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of the International Society of Ultrasound in Obstetrics and Gynecology.
Collapse
Affiliation(s)
- F. van den Noort
- Robotics and Mechatronics, Faculty of Electrical Engineering, Mathematics and Computer Science, Technical Medical CenterUniversity of TwenteEnschedeThe Netherlands
- Department of Reproductive Medicine and GynecologyUniversity Medical CenterUtrechtThe Netherlands
| | - C. H. van der Vaart
- Department of Reproductive Medicine and GynecologyUniversity Medical CenterUtrechtThe Netherlands
| | - A. T. M. Grob
- Multi‐modality Medical Imaging, Faculty of Science and Technology, Technical Medical CenterUniversity of TwenteEnschedeThe Netherlands
| | - M. K. van de Waarsenburg
- Department of Reproductive Medicine and GynecologyUniversity Medical CenterUtrechtThe Netherlands
| | - C. H. Slump
- Robotics and Mechatronics, Faculty of Electrical Engineering, Mathematics and Computer Science, Technical Medical CenterUniversity of TwenteEnschedeThe Netherlands
| | - M. van Stralen
- Imaging DivisionUniversity Medical Center UtrechtUtrechtThe Netherlands
| |
Collapse
|