Automatic segmentation method of pelvic floor levator hiatus in ultrasound using a self-normalizing neural network

Applicability of Deep Learning to Dynamically Identify the Different Organs of the Pelvic Floor in the Midsagittal Plane

INTRODUCTION AND HYPOTHESIS

The objective was to create and validate the usefulness of a convolutional neural network (CNN) for identifying different organs of the pelvic floor in the midsagittal plane via dynamic ultrasound.

METHODS

This observational and prospective study included 110 patients. Transperineal ultrasound scans were performed by an expert sonographer of the pelvic floor. A video of each patient was made that captured the midsagittal plane of the pelvic floor at rest and the change in the pelvic structures during the Valsalva maneuver. After saving the captured videos, we manually labeled the different organs in each video. Three different architectures were tested-UNet, FPN, and LinkNet-to determine which CNN model best recognized anatomical structures. The best model was trained with the 86 cases for the number of epochs determined by the stop criterion via cross-validation. The Dice Similarity Index (DSI) was used for CNN validation.

RESULTS

Eighty-six patients were included to train the CNN and 24 to test the CNN. After applying the trained CNN to the 24 test videos, we did not observe any failed segmentation. In fact, we obtained a DSI of 0.79 (95% CI: 0.73 - 0.82) as the median of the 24 test videos. When we studied the organs independently, we observed differences in the DSI of each organ. The poorest DSIs were obtained in the bladder (0.71 [95% CI: 0.70 - 0.73]) and uterus (0.70 [95% CI: 0.68 - 0.74]), whereas the highest DSIs were obtained in the anus (0.81 [95% CI: 0.80 - 0.86]) and levator ani muscle (0.83 [95% CI: 0.82 - 0.83]).

CONCLUSIONS

Our results show that it is possible to apply deep learning using a trained CNN to identify different pelvic floor organs in the midsagittal plane via dynamic ultrasound.

Validation of a built-in software in automatically reconstructing the tomographic images of the levator ani muscle

INTRODUCTION AND HYPOTHESIS

Transperineal ultrasound (TPUS) is an effective tool for evaluating the integrity of the levator ani muscle (LAM). Several operating steps are required to obtain the standard multi-slice image of the LAM, which is experience dependent and time consuming. This study was aimed at evaluating the feasibility and reproducibility of the built-in software, Smart-pelvic™, in reconstructing standard tomographic images of LAM from 3D/4D TPUS volumes.

METHODS

This study was conducted at a tertiary teaching hospital, enrolling women who underwent TPUS. Tomographic images of the LAM were automatically reconstructed by Smart-pelvicTM and rated by two experienced observers as standard or nonstandard. The anteroposterior diameter (APD) of the levator hiatus was also measured on the mid-sagittal plane of the automatically and manually reconstructed images. The APD measurements of each approach were compared using Bland-Altman plots, and interclass correlation coefficient (ICC) was used to evaluate intra- and inter-observer reproducibility. Meanwhile, the time taken for the reconstruction process of both methods was also recorded.

RESULTS

The ultrasound volume of a total of 104 patients were included in this study. Using Smart-pelvicTM, the overall success rate of the tomographic image reconstruction was 98%. Regarding measurements of APD, the ICC between the automatic and manual reconstruction methods was 0.99 (0.98, 0.99). The average time taken for reconstruction per case was 2.65 ± 0.52 s and 22.08 ± 3.45 s, respectively.

CONCLUSIONS

Using Smart-pelvicTM to reconstruct tomographic images of LAM is feasible, and it can promote TPUS by reducing operator dependence and improving examination efficiency in a clinical setting.

Automated Segmentation of Levator Ani Muscle from 3D Endovaginal Ultrasound Images

Levator ani muscle (LAM) avulsion is a common complication of vaginal childbirth and is linked to several pelvic floor disorders. Diagnosing and treating these conditions require imaging of the pelvic floor and examination of the obtained images, which is a time-consuming process subjected to operator variability. In our study, we proposed using deep learning (DL) to automate the segmentation of the LAM from 3D endovaginal ultrasound images (EVUS) to improve diagnostic accuracy and efficiency. Over one thousand images extracted from the 3D EVUS data of healthy subjects and patients with pelvic floor disorders were utilized for the automated LAM segmentation. A U-Net model was implemented, with Intersection over Union (IoU) and Dice metrics being used for model performance evaluation. The model achieved a mean Dice score of 0.86, demonstrating a better performance than existing works. The mean IoU was 0.76, indicative of a high degree of overlap between the automated and manual segmentation of the LAM. Three other models including Attention UNet, FD-UNet and Dense-UNet were also applied on the same images which showed comparable results. Our study demonstrated the feasibility and accuracy of using DL segmentation with U-Net architecture to automate LAM segmentation to reduce the time and resources required for manual segmentation of 3D EVUS images. The proposed method could become an important component in AI-based diagnostic tools, particularly in low socioeconomic regions where access to healthcare resources is limited. By improving the management of pelvic floor disorders, our approach may contribute to better patient outcomes in these underserved areas.

Levator ani muscle volume and architecture in normal vs. muscle damage patients using 3D endovaginal ultrasound: a pilot study

INTRODUCTION AND HYPOTHESIS

This study aimed to compare the difference in levator ani muscle (LAM) volumes between 'normal' and those with sonographically visualized LAM defects. We hypothesized that the 'muscle damage' group would have a significantly lower muscle volume.

METHODS

The study included patients who had undergone a 3D endovaginal ultrasound. The normal (NM) and damage (DM) muscle groups' architectural changes were evaluated based on anterior-posterior (AP), left-right (LR) diameter, and minimal levator hiatus (MLH) area. The puboanalis-puboperinealis (PA), puborectalis (PR), and pubococcygeus-iliococcygeus (PC) were manually segmented using 2.5 vs. 1.0 mm to find the optimal sequence and to compare the volumes between NM and DM groups. POPQs were compared between the NM and DM groups.

RESULTS

The 1.0-mm segmentation volumes created superior volume analysis. Comparing NM to the DM group showed no significant difference in LAM volume. Respectively, the mean total LAM volumes were 17.27 cm³ (SD = 3.97) and 17.04 cm³ (SD = 4.32), p = 0.79. The mean MLH measurements for both groups respectively were 10.06 cm² (SD = 2.93) and 12.18 cm² (SD = 2.93), indicating a significant difference (p = 0.01). POPQ analysis demonstrated statistically significant differences at Ba and Bp parameters suggesting that the DM group had worse prolapse (p = 0.05, 0.01, respectively).

CONCLUSIONS

While LAM volumes are similar, there is a significant difference in the physical architecture of the LAM and the POPQ parameters in muscle-damaged patients compared to the normal group.

Unsupervised convolutional autoencoders for 4D transperineal ultrasound classification

Purpose

4D Transperineal ultrasound (TPUS) is used to examine female pelvic floor disorders. Muscle movement, like performing a muscle contraction or a Valsalva maneuver, can be captured on TPUS. Our work investigates the possibility for unsupervised analysis and classification of the TPUS data.

Approach

An unsupervised 3D-convolutional autoencoder is trained to compress TPUS volume frames into a latent feature vector (LFV) of 128 elements. The (co)variance of the features are analyzed and statistical tests are performed to analyze how features contribute in storing contraction and Valsalva information. Further dimensionality reduction is applied (principal component analysis or a 2D-convolutional autoencoder) to the LFVs of the frames of the TPUS movie to compress the data and analyze the interframe movement. Clustering algorithms ( K -means clustering and Gaussian mixture models) are applied to this representation of the data to investigate the possibilities of unsupervised classification.

Results

The majority of the features show a significant difference between contraction and Valsalva. The (co)variance of the features from the LFVs was investigated and features most prominent in capturing muscle movement were identified. Furthermore, the first principal component of the frames from a single TPUS movie can be used to identify movement between the frames. The best classification results were obtained after applying principal component analysis and Gaussian mixture models to the LFVs of the TPUS movies, yielding a 91.2% accuracy.

Conclusion

Unsupervised analysis and classification of TPUS data yields relevant information about the type and amount of muscle movement present.

Automatic Plane of Minimal Hiatal Dimensions Extraction From 3D Female Pelvic Floor Ultrasound

There is an increasing interest in the applications of 3D ultrasound imaging of the pelvic floor to improve the diagnosis, treatment, and surgical planning of female pelvic floor dysfunction (PFD). Pelvic floor biometrics are obtained on an oblique image plane known as the plane of minimal hiatal dimensions (PMHD). Identifying this plane requires the detection of two anatomical landmarks, the pubic symphysis and anorectal angle. The manual detection of the anatomical landmarks and the PMHD in 3D pelvic ultrasound requires expert knowledge of the pelvic floor anatomy, and is challenging, time-consuming, and subject to human error. These challenges have hindered the adoption of such quantitative analysis in the clinic. This work presents an automatic approach to identify the anatomical landmarks and extract the PMHD from 3D pelvic ultrasound volumes. To demonstrate clinical utility and a complete automated clinical task, an automatic segmentation of the levator-ani muscle on the extracted PMHD images was also performed. Experiments using 73 test images of patients during a pelvic muscle resting state showed that this algorithm has the capability to accurately identify the PMHD with an average Dice of 0.89 and an average mean boundary distance of 2.25mm. Further evaluation of the PMHD detection algorithm using 35 images of patients performing pelvic muscle contraction resulted in an average Dice of 0.88 and an average mean boundary distance of 2.75mm. This work had the potential to pave the way towards the adoption of ultrasound in the clinic and development of personalized treatment for PFD.

Automatic identification and segmentation of slice of minimal hiatal dimensions in transperineal ultrasound volumes

OBJECTIVE

To develop and validate a tool for automatic selection of the slice of minimal hiatal dimensions (SMHD) and segmentation of the urogenital hiatus (UH) in transperineal ultrasound (TPUS) volumes.

METHODS

Manual selection of the SMHD and segmentation of the UH was performed in TPUS volumes of 116 women with symptomatic pelvic organ prolapse (POP). These data were used to train two deep-learning algorithms. The first algorithm was trained to provide an estimation of the position of the SMHD. Based on this estimation, a slice was selected and fed into the second algorithm, which performed automatic segmentation of the UH. From this segmentation, measurements of the UH area (UHA), anteroposterior diameter (APD) and coronal diameter (CD) were computed automatically. The mean absolute distance between manually and automatically selected SMHD, the overlap (dice similarity index (DSI)) between manual and automatic UH segmentation and the intraclass correlation coefficient (ICC) between manual and automatic UH measurements were assessed on a test set of 30 TPUS volumes.

RESULTS

The mean absolute distance between manually and automatically selected SMHD was 0.20 cm. All DSI values between manual and automatic UH segmentations were above 0.85. The ICC values between manual and automatic UH measurements were 0.94 (95% CI, 0.87-0.97) for UHA, 0.92 (95% CI, 0.78-0.97) for APD and 0.82 (95% CI, 0.66-0.91) for CD, demonstrating excellent agreement.

CONCLUSIONS

Our deep-learning algorithms allowed reliable automatic selection of the SMHD and UH segmentation in TPUS volumes of women with symptomatic POP. These algorithms can be implemented in the software of TPUS machines, thus reducing clinical analysis time and simplifying the examination of TPUS data for research and clinical purposes. © 2021 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of International Society of Ultrasound in Obstetrics and Gynecology.

Voice-Assisted Image Labeling for Endoscopic Ultrasound Classification Using Neural Networks

Ultrasound imaging is a commonly used technology for visualising patient anatomy in real-time during diagnostic and therapeutic procedures. High operator dependency and low reproducibility make ultrasound imaging and interpretation challenging with a steep learning curve. Automatic image classification using deep learning has the potential to overcome some of these challenges by supporting ultrasound training in novices, as well as aiding ultrasound image interpretation in patient with complex pathology for more experienced practitioners. However, the use of deep learning methods requires a large amount of data in order to provide accurate results. Labelling large ultrasound datasets is a challenging task because labels are retrospectively assigned to 2D images without the 3D spatial context available in vivo or that would be inferred while visually tracking structures between frames during the procedure. In this work, we propose a multi-modal convolutional neural network (CNN) architecture that labels endoscopic ultrasound (EUS) images from raw verbal comments provided by a clinician during the procedure. We use a CNN composed of two branches, one for voice data and another for image data, which are joined to predict image labels from the spoken names of anatomical landmarks. The network was trained using recorded verbal comments from expert operators. Our results show a prediction accuracy of 76% at image level on a dataset with 5 different labels. We conclude that the addition of spoken commentaries can increase the performance of ultrasound image classification, and eliminate the burden of manually labelling large EUS datasets necessary for deep learning applications.

Deep learning-based pelvic levator hiatus segmentation from ultrasound images

Purpose

To automatically segment and measure the levator hiatus with a deep learning approach and evaluate the performance between algorithms, sonographers, and different devices.

Methods

Three deep learning models (UNet-ResNet34, HR-Net, and SegNet) were trained with 360 images and validated with 42 images. The trained models were tested with two test sets. The first set included 138 images to evaluate the performance between the algorithms and sonographers. An independent dataset including 679 images assessed the performances of algorithms between different ultrasound devices. Four metrics were used for evaluation: DSC, HDD, the relative error of segmentation area, and the absolute error of segmentation area.

Results

The UNet model outperformed HR-Net and SegNet. It could achieve a mean DSC of 0.964 for the first test set and 0.952 for the independent test set. UNet was creditable compared with three senior sonographers with a noninferiority test in the first test set and equivalent in the two test sets collected by different devices. On average, it took two seconds to process one case with a GPU and 2.4 s with a CPU.

Conclusions

The deep learning approach has good performance for levator hiatus segmentation and good generalization ability on independent test sets. This automatic levator hiatus segmentation approach could help shorten the clinical examination time and improve consistency.

Automatic Extraction of Hiatal Dimensions in 3-D Transperineal Pelvic Ultrasound Recordings

The aims of this work were to create a robust automatic software tool for measurement of the levator hiatal area on transperineal ultrasound (TPUS) volumes and to measure the potential reduction in variability and time taken for analysis in a clinical setting. The proposed tool automatically detects the C-plane (i.e., the plane of minimal hiatal dimensions) from a 3-D TPUS volume and subsequently uses the extracted plane to automatically segment the levator hiatus, using a convolutional neural network. The automatic pipeline was tested using 73 representative TPUS volumes. Reference hiatal outlines were obtained manually by two experts and compared with the pipeline's automated outlines. The Hausdorff distance, area, a clinical quality score, C-plane angle and C-plane Euclidean distance were used to evaluate C-plane detection and quantify levator hiatus segmentation accuracy. A visual Turing test was created to compare the performance of the software with that of the expert, based on the visual assessment of C-plane and hiatal segmentation quality. The overall time taken to extract the hiatal area with both measurement methods (i.e., manual and automatic) was measured. Each metric was calculated both for computer-observer differences and for inter-and intra-observer differences. The automatic method gave results similar to those of the expert when determining the hiatal outline from a TPUS volume. Indeed, the hiatal area measured by the algorithm and by an expert were within the intra-observer variability. Similarly, the method identified the C-plane with an accuracy of 5.76 ± 5.06° and 6.46 ± 5.18 mm in comparison to the inter-observer variability of 9.39 ± 6.21° and 8.48 ± 6.62 mm. The visual Turing test suggested that the automatic method identified the C-plane position within the TPUS volume visually as well as the expert. The average time taken to identify the C-plane and segment the hiatal area manually was 2 min and 35 ± 17 s, compared with 35 ± 4 s for the automatic result. This study presents a method for automatically measuring the levator hiatal area using artificial intelligence-based methodologies whereby the C-plane within a TPUS volume is detected and subsequently traced for the levator hiatal outline. The proposed solution was determined to be accurate, relatively quick, robust and reliable and, importantly, to reduce time and expertise required for pelvic floor disorder assessment.

Automatic segmentation of prostate MRI using convolutional neural networks: Investigating the impact of network architecture on the accuracy of volume measurement and MRI-ultrasound registration

Convolutional neural networks (CNNs) have recently led to significant advances in automatic segmentations of anatomical structures in medical images, and a wide variety of network architectures are now available to the research community. For applications such as segmentation of the prostate in magnetic resonance images (MRI), the results of the PROMISE12 online algorithm evaluation platform have demonstrated differences between the best-performing segmentation algorithms in terms of numerical accuracy using standard metrics such as the Dice score and boundary distance. These small differences in the segmented regions/boundaries outputted by different algorithms may potentially have an unsubstantial impact on the results of downstream image analysis tasks, such as estimating organ volume and multimodal image registration, which inform clinical decisions. This impact has not been previously investigated. In this work, we quantified the accuracy of six different CNNs in segmenting the prostate in 3D patient T2-weighted MRI scans and compared the accuracy of organ volume estimation and MRI-ultrasound (US) registration errors using the prostate segmentations produced by different networks. Networks were trained and tested using a set of 232 patient MRIs with labels provided by experienced clinicians. A statistically significant difference was found among the Dice scores and boundary distances produced by these networks in a non-parametric analysis of variance (p < 0.001 and p < 0.001, respectively), where the following multiple comparison tests revealed that the statistically significant difference in segmentation errors were caused by at least one tested network. Gland volume errors (GVEs) and target registration errors (TREs) were then estimated using the CNN-generated segmentations. Interestingly, there was no statistical difference found in either GVEs or TREs among different networks, (p = 0.34 and p = 0.26, respectively). This result provides a real-world example that these networks with different segmentation performances may potentially provide indistinguishably adequate registration accuracies to assist prostate cancer imaging applications. We conclude by recommending that the differences in the accuracy of downstream image analysis tasks that make use of data output by automatic segmentation methods, such as CNNs, within a clinical pipeline should be taken into account when selecting between different network architectures, in addition to reporting the segmentation accuracy.

Deep learning enables automatic quantitative assessment of puborectalis muscle and urogenital hiatus in plane of minimal hiatal dimensions

OBJECTIVES

To measure the length, width and area of the urogenital hiatus (UH), and the length and mean echogenicity (MEP) of the puborectalis muscle (PRM), automatically and observer-independently, in the plane of minimal hiatal dimensions on transperineal ultrasound (TPUS) images, by automatic segmentation of the UH and the PRM using deep learning.

METHODS

In 1318 three- and four-dimensional (3D/4D) TPUS volume datasets from 253 nulliparae at 12 and 36 weeks' gestation, two-dimensional (2D) images in the plane of minimal hiatal dimensions with the PRM at rest, on maximum contraction and on maximum Valsalva maneuver, were obtained manually and the UH and PRM were segmented manually. In total, 713 of the images were used to train a convolutional neural network (CNN) to segment automatically the UH and PRM in the plane of minimal hiatal dimensions. In the remainder of the dataset (test set 1 (TS1); 601 images, four having been excluded), the performance of the CNN was evaluated by comparing automatic and manual segmentations. The performance of the CNN was also tested on 117 images from an independent dataset (test set 2 (TS2); two images having been excluded) from 40 nulliparae at 12 weeks' gestation, which were acquired and segmented manually by a different observer. The success of automatic segmentation was assessed visually. Based on the CNN segmentations, the following clinically relevant parameters were measured: the length, width and area of the UH, the length of the PRM and MEP. The overlap (Dice similarity index (DSI)) and surface distance (mean absolute distance (MAD) and Hausdorff distance (HDD)) between manual and CNN segmentations were measured to investigate their similarity. For the measured clinically relevant parameters, the intraclass correlation coefficients (ICCs) between manual and CNN results were determined.

RESULTS

Fully automatic CNN segmentation was successful in 99.0% and 93.2% of images in TS1 and TS2, respectively. DSI, MAD and HDD showed good overlap and distance between manual and CNN segmentations in both test sets. This was reflected in the respective ICC values in TS1 and TS2 for the length (0.96 and 0.95), width (0.77 and 0.87) and area (0.96 and 0.91) of the UH, the length of the PRM (0.87 and 0.73) and MEP (0.95 and 0.97), which showed good to very good agreement.

CONCLUSION

Deep learning can be used to segment automatically and reliably the PRM and UH on 2D ultrasound images of the nulliparous pelvic floor in the plane of minimal hiatal dimensions. These segmentations can be used to measure reliably UH dimensions as well as PRM length and MEP. © 2018 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of the International Society of Ultrasound in Obstetrics and Gynecology.