The segmentation of the left ventricle of the heart from ultrasound data using deep learning architectures and derivative-based search methods

The Promise of Artificial Intelligence-Assisted Point-of-Care Ultrasonography in Perioperative Care

The role of point-of-care ultrasonography in the perioperative setting has expanded rapidly over recent years. Revolutionizing this technology further is integrating artificial intelligence to assist clinicians in optimizing images, identifying anomalies, performing automated measurements and calculations, and facilitating diagnoses. Artificial intelligence can increase point-of-care ultrasonography efficiency and accuracy, making it an even more valuable point-of-care tool. Given this topic's importance and ever-changing landscape, this review discusses the latest trends to serve as an introduction and update in this area.

Boundary attention with multi-task consistency constraints for semi-supervised 2D echocardiography segmentation

The 2D echocardiography semantic automatic segmentation technique is important in clinical applications for cardiac function assessment and diagnosis of cardiac diseases. However, automatic segmentation of 2D echocardiograms also faces the problems of loss of image boundary information, loss of image localization information, and limitations in data acquisition and annotation. To address these issues, this paper proposes a semi-supervised echocardiography segmentation method. It consists of two models: (1) a boundary attention transformer net (BATNet) and (2) a multi-task level semi-supervised model with consistency constraints on boundary features (semi-BATNet). BATNet is able to capture the location and spatial information of the input feature maps by using the self-attention mechanism. The multi-task level semi-supervised model with boundary feature consistency constraints (semi-BATNet) encourages consistent predictions of boundary features at different scales from the student and teacher networks to calculate the multi-scale consistency loss for unlabeled data. The proposed semi-BATNet was extensively evaluated on the dataset of cardiac acquisitions for multi-structure ultrasound segmentation (CAMUS) and self-collected echocardiography dataset from the First Affiliated Hospital of Chongqing Medical University. Experimental results on the CAMUS dataset showed that when only 25% of the images are labeled, the proposed method greatly improved the segmentation performance by utilizing unlabeled images, and it also outperformed five state-of-the-art semi-supervised segmentation methods. Moreover, when only 50% of the images labeled, semi-BATNet achieved the Dice coefficient values of 0.936, the Jaccard similarity of 0.881 on self-collected echocardiography dataset. Semi-BATNet can complete a more accurate segmentation of cardiac structures in 2D echocardiograms, indicating that it has the potential to accurately and efficiently assist cardiologists.

A Data-Efficient Deep Learning Strategy for Tissue Characterization via Quantitative Ultrasound: Zone Training

Deep learning (DL) powered biomedical ultrasound imaging is an emerging research field where researchers adapt the image analysis capabilities of DL algorithms to biomedical ultrasound imaging settings. A major roadblock to wider adoption of DL powered biomedical ultrasound imaging is that acquisition of large and diverse datasets is expensive in clinical settings, which is a requirement for successful DL implementation. Hence, there is a constant need for developing data-efficient DL techniques to turn DL powered biomedical ultrasound imaging into reality. In this work, we develop a data-efficient DL training strategy for classifying tissues based on the ultrasonic backscattered RF data, i.e., quantitative ultrasound (QUS), which we named zone training. In zone training, we propose to divide the complete field of view of an ultrasound image into multiple zones associated with different regions of a diffraction pattern and then, train separate DL networks for each zone. The main advantage of zone training is that it requires less training data to achieve high accuracy. In this work, three different tissue-mimicking phantoms were classified by a DL network. The results demonstrated that zone training can require a factor of 2-3 less training data in low data regime to achieve similar classification accuracies compared to a conventional training strategy.

SILICOFCM platform, multiscale modeling of left ventricle from echocardiographic images and drug influence for cardiomyopathy disease

BACKGROUND AND OBJECTIVE

In silico clinical trials are the future of medicine and virtual testing and simulation are the future of medical engineering. The use of a computational platform can reduce costs and time required for developing new models of medical devices and drugs. The computational platform, which is one of the main results of the SILICOFCM project, was developed using state-of-the-art finite element modeling for macro simulation of fluid-structure interaction with micro modeling at the molecular level for drug interaction with the cardiac cells. SILICOFCM platform is using for risk prediction and optimal drug therapy of familial cardiomyopathy in a specific patient.

METHODS

In order to obtain 3D image reconstruction, the U-net architecture was used to determine geometric parameters for the left ventricle which were extracted from the echocardiographic apical and M-mode views. A micro-mechanics cellular model which includes three kinetic processes of sarcomeric proteins interactions was developed. It allows simulation of the drugs which are divided into three major groups defined by the principal action of each drug. Fluid-solid coupling for the left ventricle was presented. A nonlinear material model of the heart wall that was developed by using constitutive curves which include the stress-strain relationship was used.

RESULTS

The results obtained with the parametric model of the left ventricle where pressure-volume (PV) diagrams depend on the change of Ca2+ were presented. It directly affects the ejection fraction. The presented approach with the variation of the left ventricle (LV) geometry and simulations which include the influence of different parameters on the PV diagrams are directly interlinked with drug effects on the heart function. It includes different drugs such as Entresto and Digoxin that directly affect the cardiac PV diagrams and ejection fraction.

CONCLUSIONS

Computational platforms such as the SILICOFCM platform are novel tools for risk prediction of cardiac disease in a specific patient that will certainly open a new avenue for in silico clinical trials in the future.

Left ventricle analysis in echocardiographic images using transfer learning

The segmentation of cardiac boundaries, specifically Left Ventricle (LV) segmentation in 2D echocardiographic images, is a critical step in LV segmentation and cardiac function assessment. These images are generally of poor quality and present low contrast, making daily clinical delineation difficult, time-consuming, and often inaccurate. Thus, it is necessary to design an intelligent automatic endocardium segmentation system. The present work aims to examine and assess the performance of some deep learning-based architectures such as U-Net1, U-Net2, LinkNet, Attention U-Net, and TransUNet using the public CAMUS (Cardiac Acquisitions for Multi-structure Ultrasound Segmentation) dataset. The adopted approach emphasizes the advantage of using transfer learning and resorting to pre-trained backbones in the encoder part of a segmentation network for echocardiographic image analysis. The experimental findings indicated that the proposed framework with the [Formula: see text]-[Formula: see text] is quite promising; it outperforms other more recent approaches with a Dice similarity coefficient of 93.30% and a Hausdorff Distance of 4.01 mm. In addition, a good agreement between the clinical indices calculated from the automatic segmentation and those calculated from the ground truth segmentation. For instance, the mean absolute errors for the left ventricular end-diastolic volume, end-systolic volume, and ejection fraction are equal to 7.9 ml, 5.4 ml, and 6.6%, respectively. These results are encouraging and point out additional perspectives for further improvement.

Research progress and hotspot of the artificial intelligence application in the ultrasound during 2011-2021: A bibliometric analysis

Ultrasound, as a common clinical examination tool, inevitably has human errors due to the limitations of manual operation. Artificial intelligence is an advanced computer program that can solve this problem. Therefore, the relevant literature on the application of artificial intelligence in the ultrasonic field from 2011 to 2021 was screened by authors from the Web of Science Core Collection, which aims to summarize the trend of artificial intelligence application in the field of ultrasound, meanwhile, visualize and predict research hotspots. A total of 908 publications were included in the study. Overall, the number of global publications is on the rise, and studies on the application of artificial intelligence in the field of ultrasound continue to increase. China has made the largest contribution in this field. In terms of institutions, Fudan University has the most number of publications. Recently, IEEE Access is the most published journal. Suri J. S. published most of the articles and had the highest number of citations in this field (29 articles). It's worth noting that, convolutional neural networks (CNN), as a kind of deep learning algorithm, was considered to bring better image analysis and processing ability in recent most-cited articles. According to the analysis of keywords, the latest keyword is "COVID-19" (2020.8). The co-occurrence analysis of keywords by VOSviewer visually presented four clusters which consisted of "deep learning," "machine learning," "application in the field of visceral organs," and "application in the field of cardiovascular". The latest hot words of these clusters were "COVID-19; neural-network; hepatocellular carcinoma; atherosclerotic plaques". This study reveals the importance of multi-institutional and multi-field collaboration in promoting research progress.

Segmentation of Medical Image Using Novel Dilated Ghost Deep Learning Model

Image segmentation and computer vision are becoming more important in computer-aided design. A computer algorithm extracts image borders, colours, and textures. It also depletes resources. Technical knowledge is required to extract information about distinctive features. There is currently no medical picture segmentation or recognition software available. The proposed model has 13 layers and uses dilated convolution and max-pooling to extract small features. Ghost model deletes the duplicated features, makes the process easier, and reduces the complexity. The Convolution Neural Network (CNN) generates a feature vector map and improves the accuracy of area or bounding box proposals. Restructuring is required for healing. As a result, convolutional neural networks segment medical images. It is possible to acquire the beginning region of a segmented medical image. The proposed model gives better results as compared to the traditional models, it gives an accuracy of 96.05, Precision 98.2, and recall 95.78. The first findings are improved by thickening and categorising the image's pixels. Morphological techniques may be used to segment medical images. Experiments demonstrate that the recommended segmentation strategy is effective. This study rethinks medical image segmentation methods.

Generalizability and quality control of deep learning-based 2D echocardiography segmentation models in a large clinical dataset

Use of machine learning (ML) for automated annotation of heart structures from echocardiographic videos is an active research area, but understanding of comparative, generalizable performance among models is lacking. This study aimed to (1) assess the generalizability of five state-of-the-art ML-based echocardiography segmentation models within a large Geisinger clinical dataset, and (2) test the hypothesis that a quality control (QC) method based on segmentation uncertainty can further improve segmentation results. Five models were applied to 47,431 echocardiography studies that were independent from any training samples. Chamber volume and mass from model segmentations were compared to clinically-reported values. The median absolute errors (MAE) in left ventricular (LV) volumes and ejection fraction exhibited by all five models were comparable to reported inter-observer errors (IOE). MAE for left atrial volume and LV mass were similarly favorable to respective IOE for models trained for those tasks. A single model consistently exhibited the lowest MAE in all five clinically-reported measures. We leveraged the tenfold cross-validation training scheme of this best-performing model to quantify segmentation uncertainty. We observed that removing segmentations with high uncertainty from 14 to 71% studies reduced volume/mass MAE by 6-10%. The addition of convexity filters improved specificity, efficiently removing < 10% studies with large MAE (16-40%). In conclusion, five previously published echocardiography segmentation models generalized to a large, independent clinical dataset-segmenting one or multiple cardiac structures with overall accuracy comparable to manual analyses-with variable performance. Convexity-reinforced uncertainty QC efficiently improved segmentation performance and may further facilitate the translation of such models.

MallesNet: A multi-object assistance based network for brachial plexus segmentation in ultrasound images

Ultrasound-guided injection is widely used to help anesthesiologists perform anesthesia in peripheral nerve blockade (PNB). However, it is a daunting task to accurately identify nerve structure in ultrasound images even for the experienced anesthesiologists. In this paper, a Multi-object assistance based Brachial Plexus Segmentation Network, named MallesNet, is proposed to improve the nerve segmentation performance in ultrasound image with the assistance of simultaneously segmenting its surrounding anatomical structures (e.g., muscle, vein, and artery). The MallesNet is designed by following the framework of Mask R-CNN to implement the multi object identification and segmentation. Moreover, a spatial local contrast feature (SLCF) extraction module is proposed to compute contrast features at different scales to effectively obtain useful features for small objects. And the self-attention gate (SAG) is also utilized to capture the spatial relationships in different channels and further re-weight the channels in feature maps by following the design of non-local operation and channel attention. Furthermore, the upsampling mechanism in original Feature Pyramid Network (FPN) is improved by adopting the transpose convolution and skip concatenation to fine-tune the feature maps. The Ultrasound Brachial Plexus Dataset (UBPD) is also proposed to support the research on brachial plexus segmentation, which consists of 1055 ultrasound images with four objects (i.e., nerve, artery, vein and muscle) and their corresponding label masks. Extensive experimental results using UBPD dataset demonstrate that MallesNet can achieve a better segmentation performance on nerves structure and also on surrounding structures in comparison to other competing approaches.

LVNet: Lightweight Model for Left Ventricle Segmentation for Short Axis Views in Echocardiographic Imaging

Lightweight segmentation models are becoming more popular for fast diagnosis on small and low cost medical imaging devices. This study focuses on the segmentation of the left ventricle (LV) in cardiac ultrasound (US) images. A new lightweight model [LV network (LVNet)] is proposed for segmentation, which gives the benefits of requiring fewer parameters but with improved segmentation performance in terms of Dice score (DS). The proposed model is compared with state-of-the-art methods, such as UNet, MiniNetV2, and fully convolutional dense dilated network (FCdDN). The model proposed comes with a post-processing pipeline that further enhances the segmentation results. In general, the training is done directly using the segmentation mask as the output and the US image as the input of the model. A new strategy for segmentation is also introduced in addition to the direct training method used. Compared with the UNet model, an improvement in DS performance as high as 5% for segmentation with papillary (WP) muscles was found, while showcasing an improvement of 18.5% when the papillary muscles are excluded. The model proposed requires only 5% of the memory required by a UNet model. LVNet achieves a better trade-off between the number of parameters and its segmentation performance as compared with other conventional models. The developed codes are available at https://github.com/navchetanawasthi/Left_Ventricle_Segmentation.

Voice-Assisted Image Labeling for Endoscopic Ultrasound Classification Using Neural Networks

Ultrasound imaging is a commonly used technology for visualising patient anatomy in real-time during diagnostic and therapeutic procedures. High operator dependency and low reproducibility make ultrasound imaging and interpretation challenging with a steep learning curve. Automatic image classification using deep learning has the potential to overcome some of these challenges by supporting ultrasound training in novices, as well as aiding ultrasound image interpretation in patient with complex pathology for more experienced practitioners. However, the use of deep learning methods requires a large amount of data in order to provide accurate results. Labelling large ultrasound datasets is a challenging task because labels are retrospectively assigned to 2D images without the 3D spatial context available in vivo or that would be inferred while visually tracking structures between frames during the procedure. In this work, we propose a multi-modal convolutional neural network (CNN) architecture that labels endoscopic ultrasound (EUS) images from raw verbal comments provided by a clinician during the procedure. We use a CNN composed of two branches, one for voice data and another for image data, which are joined to predict image labels from the spoken names of anatomical landmarks. The network was trained using recorded verbal comments from expert operators. Our results show a prediction accuracy of 76% at image level on a dataset with 5 different labels. We conclude that the addition of spoken commentaries can increase the performance of ultrasound image classification, and eliminate the burden of manually labelling large EUS datasets necessary for deep learning applications.

Machine Learning-Based Automated Diagnostic Systems Developed for Heart Failure Prediction Using Different Types of Data Modalities: A Systematic Review and Future Directions

One of the leading causes of deaths around the globe is heart disease. Heart is an organ that is responsible for the supply of blood to each part of the body. Coronary artery disease (CAD) and chronic heart failure (CHF) often lead to heart attack. Traditional medical procedures (angiography) for the diagnosis of heart disease have higher cost as well as serious health concerns. Therefore, researchers have developed various automated diagnostic systems based on machine learning (ML) and data mining techniques. ML-based automated diagnostic systems provide an affordable, efficient, and reliable solutions for heart disease detection. Various ML, data mining methods, and data modalities have been utilized in the past. Many previous review papers have presented systematic reviews based on one type of data modality. This study, therefore, targets systematic review of automated diagnosis for heart disease prediction based on different types of modalities, i.e., clinical feature-based data modality, images, and ECG. Moreover, this paper critically evaluates the previous methods and presents the limitations in these methods. Finally, the article provides some future research directions in the domain of automated heart disease detection based on machine learning and multiple of data modalities.

Advanced Ultrasound and Photoacoustic Imaging in Cardiology

Cardiovascular diseases (CVDs) remain the leading cause of death worldwide. An effective management and treatment of CVDs highly relies on accurate diagnosis of the disease. As the most common imaging technique for clinical diagnosis of the CVDs, US imaging has been intensively explored. Especially with the introduction of deep learning (DL) techniques, US imaging has advanced tremendously in recent years. Photoacoustic imaging (PAI) is one of the most promising new imaging methods in addition to the existing clinical imaging methods. It can characterize different tissue compositions based on optical absorption contrast and thus can assess the functionality of the tissue. This paper reviews some major technological developments in both US (combined with deep learning techniques) and PA imaging in the application of diagnosis of CVDs.

Simultaneous Segmentation of Four Cardiac Chambers in Fetal Echocardiography

Accurate segmentation of cardiac chambers is helpful for the diagnosis of Congenital Heart Disease (CHD) in fetal echocardiography. Previous studies mainly focused on single cardiac chamber segmentation, which cannot provide sufficient information for the cardiologists. In this paper, we present an instance segmentation approach capable of segmenting four cardiac chambers accurately and simultaneously. A novel object proposal recovery strategy is further deployed to retrieve possible missing objects. To alleviate the shortage of medical data and further improve the segmentation performance, we utilize a rotation and distortion method for data augmentation. Experiments on a fetal echocardiography dataset of 319 fetuses demonstrate that the proposed approach can achieve superior performance according to common-used evaluation metrics.Clinical relevance-This can be used to help the cardiologists to better analyze the structure and function of the fetal heart.

Automated peak prominence-based iterative Dijkstras algorithm for segmentation of B-mode echocardiograms

We present a user-initialized, automated segmentation method for use with echocardiograms (echo). The method uses an iterative Dijkstra's algorithm, a strategic node selection, and a novel cost matrix formulation based on intensity peak prominence, termed the Prominence Iterative Dijkstras algorithm, or ProID. ProID is initialized with three user-input clicks per time-series scan. ProID was tested using artificial echo images representing five different systems. Results showed accurate LV contours and volume estimations as compared to the ground-truth for all systems. Using the CAMUS dataset, we demonstrate ProID maintained similar Dice similarity scores to other automated methods. ProID was then used to analyze a clinical cohort of 66 pediatric patients, including normal and diseased hearts. Output segmentations, end-diastolic, end-systolic volumes, and ejection fraction were compared against manual segmentations from two expert readers. ProID maintained an average Dice score of 0.93 when comparing against manual segmentation. Comparing the two expert readers, the manual segmentations maintained a score of 0.93 which increased to 0.95 when they used ProID. Thus, ProID reduced the inter-operator variability across the expert readers. Overall, this work demonstrates ProID yields accurate boundaries across age groups, disease states, and echo platforms with low computational cost and no need for training data.

Mitral Valve Segmentation Using Robust Nonnegative Matrix Factorization

Analyzing and understanding the movement of the mitral valve is of vital importance in cardiology, as the treatment and prevention of several serious heart diseases depend on it. Unfortunately, large amounts of noise as well as a highly varying image quality make the automatic tracking and segmentation of the mitral valve in two-dimensional echocardiographic videos challenging. In this paper, we present a fully automatic and unsupervised method for segmentation of the mitral valve in two-dimensional echocardiographic videos, independently of the echocardiographic view. We propose a bias-free variant of the robust non-negative matrix factorization (RNMF) along with a window-based localization approach, that is able to identify the mitral valve in several challenging situations. We improve the average f1-score on our dataset of 10 echocardiographic videos by 0.18 to a f1-score of 0.56.

A category attention instance segmentation network for four cardiac chambers segmentation in fetal echocardiography

Fetal echocardiography is an essential and comprehensive examination technique for the detection of fetal heart anomalies. Accurate cardiac chambers segmentation can assist cardiologists to analyze cardiac morphology and facilitate heart disease diagnosis. Previous research mainly focused on the segmentation of single cardiac chambers, such as left ventricle (LV) segmentation or left atrium (LA) segmentation. We propose a generic framework based on instance segmentation to segment the four cardiac chambers accurately and simultaneously. The proposed Category Attention Instance Segmentation Network (CA-ISNet) has three branches: a category branch for predicting the semantic category, a mask branch for segmenting the cardiac chambers, and a category attention branch for learning category information of instances. The category attention branch is used to correct instance misclassification of the category branch. In our collected dataset, which contains echocardiography images with four-chamber views of 319 fetuses, experimental results show our method can achieve superior segmentation performance against state-of-the-art methods. Specifically, using fivefold cross-validation, our model achieves Dice coefficients of 0.7956, 0.7619, 0.8199, 0.7470 for the four cardiac chambers, and with an average precision of 45.64%.

Deep Learning Application for Analyzing of Constituents and Their Correlations in the Interpretations of Medical Images

The need for time and attention, given by the doctor to the patient, due to the increased volume of medical data to be interpreted and filtered for diagnostic and therapeutic purposes has encouraged the development of the option to support, constructively and effectively, deep learning models. Deep learning (DL) has experienced an exponential development in recent years, with a major impact on interpretations of the medical image. This has influenced the development, diversification and increase of the quality of scientific data, the development of knowledge construction methods and the improvement of DL models used in medical applications. All research papers focus on description, highlighting, classification of one of the constituent elements of deep learning models (DL), used in the interpretation of medical images and do not provide a unified picture of the importance and impact of each constituent in the performance of DL models. The novelty in our paper consists primarily in the unitary approach, of the constituent elements of DL models, namely, data, tools used by DL architectures or specifically constructed DL architecture combinations and highlighting their "key" features, for completion of tasks in current applications in the interpretation of medical images. The use of "key" characteristics specific to each constituent of DL models and the correct determination of their correlations, may be the subject of future research, with the aim of increasing the performance of DL models in the interpretation of medical images.

Towards automatic diagnosis of rheumatic heart disease on echocardiographic exams through video-based deep learning

OBJECTIVE

Rheumatic heart disease (RHD) affects an estimated 39 million people worldwide and is the most common acquired heart disease in children and young adults. Echocardiograms are the gold standard for diagnosis of RHD, but there is a shortage of skilled experts to allow widespread screenings for early detection and prevention of the disease progress. We propose an automated RHD diagnosis system that can help bridge this gap.

MATERIALS AND METHODS

Experiments were conducted on a dataset with 11 646 echocardiography videos from 912 exams, obtained during screenings in underdeveloped areas of Brazil and Uganda. We address the challenges of RHD identification with a 3D convolutional neural network (C3D), comparing its performance with a 2D convolutional neural network (VGG16) that is commonly used in the echocardiogram literature. We also propose a supervised aggregation technique to combine video predictions into a single exam diagnosis.

RESULTS

The proposed approach obtained an accuracy of 72.77% for exam diagnosis. The results for the C3D were significantly better than the ones obtained by the VGG16 network for videos, showing the importance of considering the temporal information during the diagnostic. The proposed aggregation model showed significantly better accuracy than the majority voting strategy and also appears to be capable of capturing underlying biases in the neural network output distribution, balancing them for a more correct diagnosis.

CONCLUSION

Automatic diagnosis of echo-detected RHD is feasible and, with further research, has the potential to reduce the workload of experts, enabling the implementation of more widespread screening programs worldwide.

Residual Learning: A New Paradigm to Improve Deep Learning-Based Segmentation of the Left Ventricle in Magnetic Resonance Imaging Cardiac Images

BACKGROUND

Recently, magnetic resonance imaging (MRI) has become a useful tool for the early detection of heart failure. A vital step of this process is a valid measurement of the left ventricle's properties, which seriously depends on the accurate segmentation of the heart in captured images. Although various schemes have been tested for this segmentation so far, the latest proposed methods have used the concept of deep learning to estimate the range of the left ventricle in cardiac MRI images. While deep learning methods can lead to better results than their classical alternatives, but unfortunately, the gradient vanishing and exploding problems may hamper their efficiency for the accurate segmentation of the left ventricle in MRI heart images.

METHODS

In this article, a new concept called residual learning is utilized to improve the performance of deep learning schemes against gradient vanishing problems. For this purpose, the Residual Network of Residual Network (i.e., Residual of Residual) substructure is utilized inside the main deep learning architecture (e.g., Unet), which provides more significant detection indexes.

RESULTS AND CONCLUSION

The proposed method's performances and its alternatives were evaluated on Sunnybrook Cardiac Data as a reliable dataset in the left ventricle segmentation. The results show that the detection parameters are improved at least by 5%, 3.5%, 8.1%, and 11.4% compared to its deep alternatives in terms of Jaccard, Dice, precision, and false-positive rate indexes, respectively. These improvements were made when the recall parameter was reduced to a negligible value (i.e., approximately 1%). Overall, the proposed method can be used as a suitable tool for more accurate detection of the left ventricle in MRI images.

Ensemble machine learning approach for screening of coronary heart disease based on echocardiography and risk factors

Background

Extensive clinical evidence suggests that a preventive screening of coronary heart disease (CHD) at an earlier stage can greatly reduce the mortality rate. We use 64 two-dimensional speckle tracking echocardiography (2D-STE) features and seven clinical features to predict whether one has CHD.

Methods

We develop a machine learning approach that integrates a number of popular classification methods together by model stacking, and generalize the traditional stacking method to a two-step stacking method to improve the diagnostic performance.

Results

By borrowing strengths from multiple classification models through the proposed method, we improve the CHD classification accuracy from around 70–87.7% on the testing set. The sensitivity of the proposed method is 0.903 and the specificity is 0.843, with an AUC of 0.904, which is significantly higher than those of the individual classification models.

Conclusion

Our work lays a foundation for the deployment of speckle tracking echocardiography-based screening tools for coronary heart disease.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01535-5.

Artificial intelligence in ultrasound

Ultrasound (US), a flexible green imaging modality, is expanding globally as a first-line imaging technique in various clinical fields following with the continual emergence of advanced ultrasonic technologies and the well-established US-based digital health system. Actually, in US practice, qualified physicians should manually collect and visually evaluate images for the detection, identification and monitoring of diseases. The diagnostic performance is inevitably reduced due to the intrinsic property of high operator-dependence from US. In contrast, artificial intelligence (AI) excels at automatically recognizing complex patterns and providing quantitative assessment for imaging data, showing high potential to assist physicians in acquiring more accurate and reproducible results. In this article, we will provide a general understanding of AI, machine learning (ML) and deep learning (DL) technologies; We then review the rapidly growing applications of AI-especially DL technology in the field of US-based on the following anatomical regions: thyroid, breast, abdomen and pelvis, obstetrics heart and blood vessels, musculoskeletal system and other organs by covering image quality control, anatomy localization, object detection, lesion segmentation, and computer-aided diagnosis and prognosis evaluation; Finally, we offer our perspective on the challenges and opportunities for the clinical practice of biomedical AI systems in US.

Echocardiographic image multi-structure segmentation using Cardiac-SegNet

PURPOSE

Cardiac boundary segmentation of echocardiographic images is important for cardiac function assessment and disease diagnosis. However, it is challenging to segment cardiac ventricles due to the low contrast-to-noise ratio and speckle noise of the echocardiographic images. Manual segmentation is subject to interobserver variability and is too slow for real-time image-guided interventions. We aim to develop a deep learning-based method for automated multi-structure segmentation of echocardiographic images.

METHODS

We developed an anchor-free mask convolutional neural network (CNN), termed Cardiac-SegNet, which consists of three subnetworks, that is, a backbone, a fully convolutional one-state object detector (FCOS) head, and a mask head. The backbone extracts multi-level and multi-scale features from endocardium image. The FOCS head utilizes these features to detect and label the region-of-interests (ROIs) of the segmentation targets. Unlike the traditional mask regional CNN (Mask R-CNN) method, the FCOS head is anchor-free and can model the spatial relationship of the targets. The mask head utilizes a spatial attention strategy, which allows the network to highlight salient features to perform segmentation on each detected ROI. For evaluation, we investigated 450 patient datasets by a five-fold cross-validation and a hold-out test. The endocardium (LV_Endo ) and epicardium (LV_Epi ) of the left ventricle and left atrium (LA) were segmented and compared with manual contours using the Dice similarity coefficient (DSC), Hausdorff distance (HD), mean absolute distance (MAD), and center-of-mass distance (CMD).

RESULTS

Compared to U-Net and Mask R-CNN, our method achieved higher segmentation accuracy and fewer erroneous speckles. When our method was evaluated on a separate hold-out dataset at the end diastole (ED) and the end systole (ES) phases, the average DSC were 0.952 and 0.939 at ED and ES for the LV_Endo , 0.965 and 0.959 at ED and ES for the LV_Epi , and 0.924 and 0.926 at ED and ES for the LA. For patients with a typical image size of 549 × 788 pixels, the proposed method can perform the segmentation within 0.5 s.

CONCLUSION

We proposed a fast and accurate method to segment echocardiographic images using an anchor-free mask CNN.

ACTIVE CONTOUR MODELS-BASED SEGMENTATION OF LEFT VENTRICLE IN ULTRASOUND IMAGES FOR DIFFERENT AXES VIEWS

Segmentation of the left ventricle in ultrasound images for viewing through different axes is a critical aspect. This paper proposes the development of novel active contour models with shape constraint to segment the left ventricle in three different axis views of the ultrasound images. The shapes observed in all the axis views of the left ventricle were not similar. According to the cardiac cycle, the valve opening in the end-diastolic phase influenced the left ventricle segmentation; hence, a shape constraint was embedded in the active contour model to keep ventricle’s shape, especially in the Apical long-axis view and Apical four-chamber view. Furthermore, for different axes views, diverse active contour models were proposed to fit each situation. The shape constraint in each function for different views exhibited a specific shape during the function iteration. In order to speed up the algorithm evolution, previous results were used for the initialization of the present active contour. We evaluated the proposed method on 57 patients with three different views: Apical long-axis view, Apical four-chamber view and Short-axis view at the papillary muscle level and obtained the Dice similarity coefficients of [Formula: see text], [Formula: see text] and [Formula: see text] and the Hausdorff distance metrics of [Formula: see text], [Formula: see text] and [Formula: see text], respectively. The qualitative and quantitative evaluations showed an advantage of our method in terms of segmentation accuracy.

Shape prior model via dual subspace segment projection learning

BACKGROUND AND OBJECTIVE

Shape prior models play a vital role for segmentation in medical image analysis. These models are most effective when shape variations can be captured by a parametric distribution, and sufficient training data is available. However, in the absence of these conditions, results are invariably much poorer. In this paper, we propose a novel shape prior model, via dual subspace segment projection learning (DSSPL), to address these challenges.

METHODS

DSSPL serves to compose shapes from an ensemble of shape segments where each segment is formed using two subspaces: global shape subspace and segment-specific subspace, each necessary for extracting global shape patterns and local patterns, respectively. This ensures the proposed approach has general shape plausibility in regions of signal drop-out or missing boundary information, and also more localized flexibility. The learned projections are constrained with l_2,1 sparse norm terms to extract the most distinguishable features, while the reconstructive properties of DSSPL reduces information loss and leverages the subspaces to provide contiguous shapes without any post-processing.

RESULTS

Extensive analysis is performed on three databases from different medical imaging systems across X-Ray, MRI, and ultrasound. DSSPL outperforms all compared benchmarks in terms of shape generalization ability and segmentation performance.

CONCLUSIONS

We propose a new shape prior model for segmentation in medical image analysis to address the challenges of modelling complex organ shapes with low sample size training data.

Comparative analysis of active contour and convolutional neural network in rapid left-ventricle volume quantification using echocardiographic imaging

In cardiology, ultrasound is often used to diagnose heart disease associated with myocardial infarction. This study aims to develop robust segmentation techniques for segmenting the left ventricle (LV) in ultrasound images to check myocardium movement during heartbeat. The proposed technique utilizes machine learning (ML) techniques such as the active contour (AC) and convolutional neural networks (CNNs) for segmentation. Medical experts determine the consistency between the proposed ML approach, which is a state-of-the-art deep learning method, and the manual segmentation approach. These methods are compared in terms of performance indicators such as the ventricular area (VA), ventricular maximum diameter (VMXD), ventricular minimum diameter (VMID), and ventricular long axis angle (AVLA) measurements. Furthermore, the Dice similarity coefficient, Jaccard index, and Hausdorff distance are measured to estimate the agreement of the LV segmented results between the automatic and visual approaches. The obtained results indicate that the proposed techniques for LV segmentation are useful and practical. There is no significant difference between the use of AC and CNN in image segmentation; however, the AC method could obtain comparable accuracy as the CNN method using less training data and less run-time.

A Tour of Unsupervised Deep Learning for Medical Image Analysis

BACKGROUND

Interpretation of medical images for the diagnosis and treatment of complex diseases from high-dimensional and heterogeneous data remains a key challenge in transforming healthcare. In the last few years, both supervised and unsupervised deep learning achieved promising results in the area of medical image analysis. Several reviews on supervised deep learning are published, but hardly any rigorous review on unsupervised deep learning for medical image analysis is available.

OBJECTIVES

The objective of this review is to systematically present various unsupervised deep learning models, tools, and benchmark datasets applied to medical image analysis. Some of the discussed models are autoencoders and its other variants, Restricted Boltzmann machines (RBM), Deep belief networks (DBN), Deep Boltzmann machine (DBM), and Generative adversarial network (GAN). Further, future research opportunities and challenges of unsupervised deep learning techniques for medical image analysis are also discussed.

CONCLUSION

Currently, interpretation of medical images for diagnostic purposes is usually performed by human experts that may be replaced by computer-aided diagnosis due to advancement in machine learning techniques, including deep learning, and the availability of cheap computing infrastructure through cloud computing. Both supervised and unsupervised machine learning approaches are widely applied in medical image analysis, each of them having certain pros and cons. Since human supervisions are not always available or inadequate or biased, therefore, unsupervised learning algorithms give a big hope with lots of advantages for biomedical image analysis.

A survey of deep learning models in medical therapeutic areas

Artificial intelligence is a broad field that comprises a wide range of techniques, where deep learning is presently the one with the most impact. Moreover, the medical field is an area where data both complex and massive and the importance of the decisions made by doctors make it one of the fields in which deep learning techniques can have the greatest impact. A systematic review following the Cochrane recommendations with a multidisciplinary team comprised of physicians, research methodologists and computer scientists has been conducted. This survey aims to identify the main therapeutic areas and the deep learning models used for diagnosis and treatment tasks. The most relevant databases included were MedLine, Embase, Cochrane Central, Astrophysics Data System, Europe PubMed Central, Web of Science and Science Direct. An inclusion and exclusion criteria were defined and applied in the first and second peer review screening. A set of quality criteria was developed to select the papers obtained after the second screening. Finally, 126 studies from the initial 3493 papers were selected and 64 were described. Results show that the number of publications on deep learning in medicine is increasing every year. Also, convolutional neural networks are the most widely used models and the most developed area is oncology where they are used mainly for image analysis.

Deep learning in spatiotemporal cardiac imaging: A review of methodologies and clinical usability

The use of different cardiac imaging modalities such as MRI, CT or ultrasound enables the visualization and interpretation of altered morphological structures and function of the heart. In recent years, there has been an increasing interest in AI and deep learning that take into account spatial and temporal information in medical image analysis. In particular, deep learning tools using temporal information in image processing have not yet found their way into daily clinical practice, despite its presumed high diagnostic and prognostic value. This review aims to synthesize the most relevant deep learning methods and discuss their clinical usability in dynamic cardiac imaging using for example the complete spatiotemporal image information of the heart cycle. Selected articles were categorized according to the following indicators: clinical applications, quality of datasets, preprocessing and annotation, learning methods and training strategy, and test performance. Clinical usability was evaluated based on these criteria by classifying the selected papers into (i) clinical level, (ii) robust candidate and (iii) proof of concept applications. Interestingly, not a single one of the reviewed papers was classified as a "clinical level" study. Almost 39% of the articles achieved a "robust candidate" and as many as 61% a "proof of concept" status. In summary, deep learning in spatiotemporal cardiac imaging is still strongly research-oriented and its implementation in clinical application still requires considerable efforts. Challenges that need to be addressed are the quality of datasets together with clinical verification and validation of the performance achieved by the used method.

LU-Net: A Multistage Attention Network to Improve the Robustness of Segmentation of Left Ventricular Structures in 2-D Echocardiography

Segmentation of cardiac structures is one of the fundamental steps to estimate volumetric indices of the heart. This step is still performed semiautomatically in clinical routine and is, thus, prone to interobserver and intraobserver variabilities. Recent studies have shown that deep learning has the potential to perform fully automatic segmentation. However, the current best solutions still suffer from a lack of robustness in terms of accuracy and number of outliers. The goal of this work is to introduce a novel network designed to improve the overall segmentation accuracy of left ventricular structures (endocardial and epicardial borders) while enhancing the estimation of the corresponding clinical indices and reducing the number of outliers. This network is based on a multistage framework where both the localization and segmentation steps are optimized jointly through an end-to-end scheme. Results obtained on a large open access data set show that our method outperforms the current best-performing deep learning solution with a lighter architecture and achieved an overall segmentation accuracy lower than the intraobserver variability for the epicardial border (i.e., on average a mean absolute error of 1.5 mm and a Hausdorff distance of 5.1mm) with 11% of outliers. Moreover, we demonstrate that our method can closely reproduce the expert analysis for the end-diastolic and end-systolic left ventricular volumes, with a mean correlation of 0.96 and a mean absolute error of 7.6 ml. Concerning the ejection fraction of the left ventricle, results are more contrasted with a mean correlation coefficient of 0.83 and an absolute mean error of 5.0%, producing scores that are slightly below the intraobserver margin. Based on this observation, areas for improvement are suggested.

One Shot Segmentation: Unifying Rigid Detection and Non-Rigid Segmentation Using Elastic Regularization

This paper proposes a novel approach for the non-rigid segmentation of deformable objects in image sequences, which is based on one-shot segmentation that unifies rigid detection and non-rigid segmentation using elastic regularization. The domain of application is the segmentation of a visual object that temporally undergoes a rigid transformation (e.g., affine transformation) and a non-rigid transformation (i.e., contour deformation). The majority of segmentation approaches to solve this problem are generally based on two steps that run in sequence: a rigid detection, followed by a non-rigid segmentation. In this paper, we propose a new approach, where both the rigid and non-rigid segmentation are performed in a single shot using a sparse low-dimensional manifold that represents the visual object deformations. Given the multi-modality of these deformations, the manifold partitions the training data into several patches, where each patch provides a segmentation proposal during the inference process. These multiple segmentation proposals are merged using the classification results produced by deep belief networks (DBN) that compute the confidence on each segmentation proposal. Thus, an ensemble of DBN classifiers is used for estimating the final segmentation. Compared to current methods proposed in the field, our proposed approach is advantageous in four aspects: (i) it is a unified framework to produce rigid and non-rigid segmentations; (ii) it uses an ensemble classification process, which can help the segmentation robustness; (iii) it provides a significant reduction in terms of the number of dimensions of the rigid and non-rigid segmentations search spaces, compared to current approaches that divide these two problems; and (iv) this lower dimensionality of the search space can also reduce the need for large annotated training sets to be used for estimating the DBN models. Experiments on the problem of left ventricle endocardial segmentation from ultrasound images, and lip segmentation from frontal facial images using the extended Cohn-Kanade (CK+) database, demonstrate the potential of the methodology through qualitative and quantitative evaluations, and the ability to reduce the search and training complexities without a significant impact on the segmentation accuracy.

Cardiac Segmentation With Strong Anatomical Guarantees

Convolutional neural networks (CNN) have had unprecedented success in medical imaging and, in particular, in medical image segmentation. However, despite the fact that segmentation results are closer than ever to the inter-expert variability, CNNs are not immune to producing anatomically inaccurate segmentations, even when built upon a shape prior. In this paper, we present a framework for producing cardiac image segmentation maps that are guaranteed to respect pre-defined anatomical criteria, while remaining within the inter-expert variability. The idea behind our method is to use a well-trained CNN, have it process cardiac images, identify the anatomically implausible results and warp these results toward the closest anatomically valid cardiac shape. This warping procedure is carried out with a constrained variational autoencoder (cVAE) trained to learn a representation of valid cardiac shapes through a smooth, yet constrained, latent space. With this cVAE, we can project any implausible shape into the cardiac latent space and steer it toward the closest correct shape. We tested our framework on short-axis MRI as well as apical two and four-chamber view ultrasound images, two modalities for which cardiac shapes are drastically different. With our method, CNNs can now produce results that are both within the inter-expert variability and always anatomically plausible without having to rely on a shape prior.

ResDUnet: Residual Dilated UNet for Left Ventricle Segmentation from Echocardiographic Images

Echocardiography is the modality of choice for the assessment of left ventricle function. Left ventricle is responsible for pumping blood rich in oxygen to all body parts. Segmentation of this chamber from echocardiographic images is a challenging task, due to the ambiguous boundary and inhomogeneous intensity distribution. In this paper we propose a novel deep learning model named ResDUnet. The model is based on U-net incorporated with dilated convolution, where residual blocks are employed instead of the basic U-net units to ease the training process. Each block is enriched with squeeze and excitation unit for channel-wise attention and adaptive feature re-calibration. To tackle the problem of left ventricle shape and size variability, we chose to enrich the process of feature concatenation in U-net by integrating feature maps generated by cascaded dilation. Cascaded dilation broadens the receptive field size in comparison with traditional convolution, which allows the generation of multi-scale information which in turn results in a more robust segmentation. Performance measures were evaluated on a publicly available dataset of 500 patients with large variability in terms of quality and patients pathology. The proposed model shows a dice similarity increase of 8.4% when compared to deeplabv3 and 1.2% when compared to the basic U-net architecture. Experimental results demonstrate the potential use in clinical domain.

Deep pyramid local attention neural network for cardiac structure segmentation in two-dimensional echocardiography

Automatic semantic segmentation in 2D echocardiography is vital in clinical practice for assessing various cardiac functions and improving the diagnosis of cardiac diseases. However, two distinct problems have persisted in automatic segmentation in 2D echocardiography, namely the lack of an effective feature enhancement approach for contextual feature capture and lack of label coherence in category prediction for individual pixels. Therefore, in this study, we propose a deep learning model, called deep pyramid local attention neural network (PLANet), to improve the segmentation performance of automatic methods in 2D echocardiography. Specifically, we propose a pyramid local attention module to enhance features by capturing supporting information within compact and sparse neighboring contexts. We also propose a label coherence learning mechanism to promote prediction consistency for pixels and their neighbors by guiding the learning with explicit supervision signals. The proposed PLANet was extensively evaluated on the dataset of cardiac acquisitions for multi-structure ultrasound segmentation (CAMUS) and sub-EchoNet-Dynamic, which are two large-scale and public 2D echocardiography datasets. The experimental results show that PLANet performs better than traditional and deep learning-based segmentation methods on geometrical and clinical metrics. Moreover, PLANet can complete the segmentation of heart structures in 2D echocardiography in real time, indicating a potential to assist cardiologists accurately and efficiently.

Ultrasound Deep Learning for Wall Segmentation and Near-Wall Blood Flow Measurement

Studies of medical flow imaging have technical limitations for accurate analysis of blood flow dynamics and vessel wall interaction at arteries. We propose a new deep learning-based boundary detection and compensation (DL-BDC) technique in ultrasound (US) imaging. It can segment vessel boundaries by harnessing the convolutional neural network and wall motion compensation in the analysis of near-wall flow dynamics. The network enables training from real and synthetic US images together. The performance of the technique is validated through synthetic US images and tissue-mimicking phantom experiments. The neural network performs well with high Dice coefficients of over 0.94 and 0.9 for lumens and walls, outperforming previous segmentation techniques. Then, the performance of the wall motion compensation is examined for compliant phantoms. When DL-BDC is applied to flow influenced by wall motion, root-mean-square errors are less than 0.07%. The technique is utilized to analyze flow dynamics and wall interaction with varying elastic moduli of the phantoms. The results show that the flow dynamics and wall shear stress values are consistent with the expected values of the compliant phantoms, and their wall motion behavior is observed with pulse wave propagation. This strategy makes US imaging capable of simultaneous measurement of blood flow and vessel dynamics in human arteries for their accurate interaction analysis. DL-BDC can segment vessel walls fast, accurately, and robustly. It enables to measure the near-wall flow precisely by determining the vessel boundary dynamics. This approach can be beneficial in flow dynamics and wall interaction analyses in various biomedical applications.

Technology trends and applications of deep learning in ultrasonography: image quality enhancement, diagnostic support, and improving workflow efficiency

In this review of the most recent applications of deep learning to ultrasound imaging, the architectures of deep learning networks are briefly explained for the medical imaging applications of classification, detection, segmentation, and generation. Ultrasonography applications for image processing and diagnosis are then reviewed and summarized, along with some representative imaging studies of the breast, thyroid, heart, kidney, liver, and fetal head. Efforts towards workflow enhancement are also reviewed, with an emphasis on view recognition, scanning guide, image quality assessment, and quantification and measurement. Finally some future prospects are presented regarding image quality enhancement, diagnostic support, and improvements in workflow efficiency, along with remarks on hurdles, benefits, and necessary collaborations.

Generalizable fully automated multi-label segmentation of four-chamber view echocardiograms based on deep convolutional adversarial networks

A major issue in translation of the artificial intelligence platforms for automatic segmentation of echocardiograms to clinics is their generalizability. The present study introduces and verifies a novel generalizable and efficient fully automatic multi-label segmentation method for four-chamber view echocardiograms based on deep fully convolutional networks (FCNs) and adversarial training. For the first time, we used generative adversarial networks for pixel classification training, a novel method in machine learning not currently used for cardiac imaging, to overcome the generalization problem. The method's performance was validated against manual segmentations as the ground-truth. Furthermore, to verify our method's generalizability in comparison with other existing techniques, we compared our method's performance with a state-of-the-art method on our dataset in addition to an independent dataset of 450 patients from the CAMUS (cardiac acquisitions for multi-structure ultrasound segmentation) challenge. On our test dataset, automatic segmentation of all four chambers achieved a dice metric of 92.1%, 86.3%, 89.6% and 91.4% for LV, RV, LA and RA, respectively. LV volumes' correlation between automatic and manual segmentation were 0.94 and 0.93 for end-diastolic volume and end-systolic volume, respectively. Excellent agreement with chambers' reference contours and significant improvement over previous FCN-based methods suggest that generative adversarial networks for pixel classification training can effectively design generalizable fully automatic FCN-based networks for four-chamber segmentation of echocardiograms even with limited number of training data.

Cardiac point-of-care to cart-based ultrasound translation using constrained CycleGAN

PURPOSE

 The emerging market of cardiac handheld ultrasound (US) is on the rise. Despite the advantages in ease of access and the lower cost, a gap in image quality can still be observed between the echocardiography (echo) data captured by point-of-care ultrasound (POCUS) compared to conventional cart-based US, which limits the further adaptation of POCUS. In this work, we aim to present a machine learning solution based on recent advances in adversarial training to investigate the feasibility of translating POCUS echo images to the quality level of high-end cart-based US systems.

METHODS

 We propose a constrained cycle-consistent generative adversarial architecture for unpaired translation of cardiac POCUS to cart-based US data. We impose a structured shape-wise regularization via a critic segmentation network to preserve the underlying shape of the heart during quality translation. The proposed deep transfer model is constrained to the anatomy of the left ventricle (LV) in apical two-chamber (AP2) echo views.

RESULTS

 A total of 1089 echo studies from 841 patients are used in this study. The AP2 frames are captured by POCUS (Philips Lumify and Clarius) and cart-based (Philips iE33 and Vivid E9) US machines. The dataset of quality translation comprises a total of 441 echo studies from 395 patients. Data from both POCUS and cart-based systems of the same patient were available in 122 cases. The deep-quality transfer model is integrated into a pipeline for an automated cardiac evaluation task, namely segmentation of LV in AP2 view. By transferring the low-quality POCUS data to the cart-based US, a significant average improvement of 30% and 34 mm is obtained in the LV segmentation Dice score and Hausdorff distance metrics, respectively.

CONCLUSION

 This paper presents the feasibility of a machine learning solution to transform the image quality of POCUS data to that of high-quality high-end cart-based systems. The experiments show that by leveraging the quality translation through the proposed constrained adversarial training, the accuracy of automatic segmentation with POCUS data could be improved.

The Stacked-Ellipse Algorithm: An Ultrasound-Based 3-D Uterine Segmentation Tool for Enabling Adaptive Radiotherapy for Uterine Cervix Cancer

The stacked-ellipse (SE) algorithm was developed to rapidly segment the uterus on 3-D ultrasound (US) for the purpose of enabling US-guided adaptive radiotherapy (RT) for uterine cervix cancer patients. The algorithm was initialised manually on a single sagittal slice to provide a series of elliptical initialisation contours in semi-axial planes along the uterus. The elliptical initialisation contours were deformed according to US features such that they conformed to the uterine boundary. The uterus of 15 patients was scanned with 3-D US using the Clarity System (Elekta Ltd.) at multiple days during RT and manually contoured (n = 49 images and corresponding contours). The median (interquartile range) Dice similarity coefficient and mean surface-to-surface-distance between the SE algorithm and manual contours were 0.80 (0.03) and 3.3 (0.2) mm, respectively, which are within the ranges of reported inter-observer contouring variabilities. The SE algorithm could be implemented in adaptive RT to precisely segment the uterus on 3-D US.

Deep Learning for Cardiac Image Segmentation: A Review

Deep learning has become the most widely used approach for cardiac image segmentation in recent years. In this paper, we provide a review of over 100 cardiac image segmentation papers using deep learning, which covers common imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound and major anatomical structures of interest (ventricles, atria, and vessels). In addition, a summary of publicly available cardiac image datasets and code repositories are included to provide a base for encouraging reproducible research. Finally, we discuss the challenges and limitations with current deep learning-based approaches (scarcity of labels, model generalizability across different domains, interpretability) and suggest potential directions for future research.

Deep Learning-Based Automatic Endometrium Segmentation and Thickness Measurement for 2D Transvaginal Ultrasound

Endometrial thickness is closely related to gyneco-logical function and is an important biomarker in transvaginal ultrasound (TVUS) examinations for assessing female reproductive health. Manual measurement is time-consuming and subject to high inter- and intra- observer variability. In this paper, we present a fully automated endometrial thickness measurement method using deep learning. Our pipeline consists of: 1) endometrium segmentation using a VGG-based U-Net, and 2) endometrial thickness estimation using medial axis transformation. We conducted experimental studies on 137 2D TVUS cases (74/63 secretory phase/proliferative phase). On a test set of 27 cases/277 images, the segmentation Dice score is 0.83. For thickness measurement, we achieved mean absolute error of 1.23/1.38 mm and root mean squared error of 1.79/1.85 mm on two different test sets. The results are considered well within the clinically acceptable range of ±2 mm. Furthermore, our phase-stratified analysis shows that the measurement variance from the secretory phase is higher than that from the proliferative phase, largely due to the high variability of the endometrium appearance in the secretory phase. Future work will extend our current algorithm toward different clinical outcomes for a broader spectrum of clinical applications.