Echocardiographic image segmentation using deep Res-U network

MADR-Net: multi-level attention dilated residual neural network for segmentation of medical images

Medical image segmentation has made a significant contribution towards delivering affordable healthcare by facilitating the automatic identification of anatomical structures and other regions of interest. Although convolution neural networks have become prominent in the field of medical image segmentation, they suffer from certain limitations. In this study, we present a reliable framework for producing performant outcomes for the segmentation of pathological structures of 2D medical images. Our framework consists of a novel deep learning architecture, called deep multi-level attention dilated residual neural network (MADR-Net), designed to improve the performance of medical image segmentation. MADR-Net uses a U-Net encoder/decoder backbone in combination with multi-level residual blocks and atrous pyramid scene parsing pooling. To improve the segmentation results, channel-spatial attention blocks were added in the skip connection to capture both the global and local features and superseded the bottleneck layer with an ASPP block. Furthermore, we introduce a hybrid loss function that has an excellent convergence property and enhances the performance of the medical image segmentation task. We extensively validated the proposed MADR-Net on four typical yet challenging medical image segmentation tasks: (1) Left ventricle, left atrium, and myocardial wall segmentation from Echocardiogram images in the CAMUS dataset, (2) Skin cancer segmentation from dermoscopy images in ISIC 2017 dataset, (3) Electron microscopy in FIB-SEM dataset, and (4) Fluid attenuated inversion recovery abnormality from MR images in LGG segmentation dataset. The proposed algorithm yielded significant results when compared to state-of-the-art architectures such as U-Net, Residual U-Net, and Attention U-Net. The proposed MADR-Net consistently outperformed the classical U-Net by 5.43%, 3.43%, and 3.92% relative improvement in terms of dice coefficient, respectively, for electron microscopy, dermoscopy, and MRI. The experimental results demonstrate superior performance on single and multi-class datasets and that the proposed MADR-Net can be utilized as a baseline for the assessment of cross-dataset and segmentation tasks.

Deep learning supported echocardiogram analysis: A comprehensive review

An echocardiogram is a sophisticated ultrasound imaging technique employed to diagnose heart conditions. The transthoracic echocardiogram, one of the most prevalent types, is instrumental in evaluating significant cardiac diseases. However, interpreting its results heavily relies on the clinician's expertise. In this context, artificial intelligence has emerged as a vital tool for helping clinicians. This study critically analyzes key state-of-the-art research that uses deep learning techniques to automate transthoracic echocardiogram analysis and support clinical judgments. We have systematically organized and categorized articles that proffer solutions for view classification, enhancement of image quality and dataset, segmentation and identification of cardiac structures, detection of cardiac function abnormalities, and quantification of cardiac functions. We compared the performance of various deep learning approaches within each category, identifying the most promising methods. Additionally, we highlight limitations in current research and explore promising avenues for future exploration. These include addressing generalizability issues, incorporating novel AI approaches, and tackling the analysis of rare cardiac diseases.

Cross-Domain Echocardiography Segmentation with Multi-Space Joint Adaptation

The segmentation of the left ventricle endocardium (LVendo) and the left ventricle epicardium (LVepi) in echocardiography plays an important role in clinical diagnosis. Recently, deep neural networks have been the most commonly used approach for echocardiography segmentation. However, the performance of a well-trained segmentation network may degrade in unseen domain datasets due to the distribution shift of the data. Adaptation algorithms can improve the generalization of deep neural networks to different domains. In this paper, we present a multi-space adaptation-segmentation-joint framework, named MACS, for cross-domain echocardiography segmentation. It adopts a generative adversarial architecture; the generator fulfills the segmentation task and the multi-space discriminators align the two domains on both the feature space and output space. We evaluated the MACS method on two echocardiography datasets from different medical centers and vendors, the publicly available CAMUS dataset and our self-acquired dataset. The experimental results indicated that the MACS could handle unseen domain datasets well, without requirements for manual annotations, and improve the generalization performance by 2.2% in the Dice metric.

Real-time echocardiography image analysis and quantification of cardiac indices

Deep learning has a huge potential to transform echocardiography in clinical practice and point of care ultrasound testing by providing real-time analysis of cardiac structure and function. Automated echocardiography analysis is benefited through use of machine learning for tasks such as image quality assessment, view classification, cardiac region segmentation, and quantification of diagnostic indices. By taking advantage of high-performing deep neural networks, we propose a novel and eicient real-time system for echocardiography analysis and quantification. Our system uses a self-supervised modality-specific representation trained using a publicly available large-scale dataset. The trained representation is used to enhance the learning of target echo tasks with relatively small datasets. We also present a novel Trilateral Attention Network (TaNet) for real-time cardiac region segmentation. The proposed network uses a module for region localization and three lightweight pathways for encoding rich low-level, textural, and high-level features. Feature embeddings from these individual pathways are then aggregated for cardiac region segmentation. This network is fine-tuned using a joint loss function and training strategy. We extensively evaluate the proposed system and its components, which are echo view retrieval, cardiac segmentation, and quantification, using four echocardiography datasets. Our experimental results show a consistent improvement in the performance of echocardiography analysis tasks with enhanced computational eiciency that charts a path toward its adoption in clinical practice. Specifically, our results show superior real-time performance in retrieving good quality echo from individual cardiac view, segmenting cardiac chambers with complex overlaps, and extracting cardiac indices that highly agree with the experts' values. The source code of our implementation can be found in the project's GitHub page.

Segmentation of Echocardiography Based on Deep Learning Model

In order to achieve the classification of mitral regurgitation, a deep learning network VDS-UNET was designed to automatically segment the critical regions of echocardiography with three sections of apical two-chamber, apical three-chamber, and apical four-chamber. First, an expert-labeled dataset of 153 echocardiographic videos and 2183 images from 49 subjects was constructed. Then, the convolution layer in the VGG16 network was used to replace the contraction path in the original UNet network to extract image features, and depth supervision was added to the expansion path to achieve the segmentation of LA, LV, and MV. The results showed that the Dice coefficients of LA, LV, and MV were 0.935, 0.915, and 0.757, respectively. The proposed deep learning network can achieve simultaneous and accurate segmentation of LA, LV, and MV in multi-section echocardiography, laying a foundation for quantitative measurement of clinical parameters related to mitral regurgitation.

Explicit and automatic ejection fraction assessment on 2D cardiac ultrasound with a deep learning-based approach

BACKGROUND

Ejection fraction (EF) is a key parameter for assessing cardiovascular functions in cardiac ultrasound, but its manual assessment is time-consuming and subject to high inter and intra-observer variability. Deep learning-based methods have the potential to perform accurate fully automatic EF predictions but suffer from a lack of explainability and interpretability. This study proposes a fully automatic method to reliably and explicitly evaluate the biplane left ventricular EF on 2D echocardiography following the recommended modified Simpson's rule.

METHODS

A deep learning model was trained on apical 4 and 2-chamber echocardiography to segment the left ventricle and locate the mitral valve. Predicted segmentations are then validated with a statistical shape model, which detects potential failures that could impact the EF evaluation. Finally, the end-diastolic and end-systolic frames are identified based on the remaining LV segmentations' areas and EF is estimated on all available cardiac cycles.

RESULTS

Our approach was trained on a dataset of 783 patients. Its performances were evaluated on an internal and external dataset of respectively 200 and 450 patients. On the internal dataset, EF assessment achieved a mean absolute error of 6.10% and a bias of 1.56 ± 7.58% using multiple cardiac cycles. The approach evaluated EF with a mean absolute error of 5.39% and a bias of -0.74 ± 7.12% on the external dataset.

CONCLUSION

Following the recommended guidelines, we proposed an end-to-end fully automatic approach that achieves state-of-the-art performance in biplane EF evaluation while giving explicit details to clinicians.

Cardiac Segmentation Method Based on Domain Knowledge

Echocardiography plays an important role in the clinical diagnosis of cardiovascular diseases. Cardiac function assessment by echocardiography is a crucial process in daily cardiology. However, cardiac segmentation in echocardiography is a challenging task due to shadows and speckle noise. The traditional manual segmentation method is a time-consuming process and limited by inter-observer variability. In this paper, we present a fast and accurate echocardiographic automatic segmentation framework based on Convolutional neural networks (CNN). We propose FAUet, a segmentation method serially integrated U-Net with coordinate attention mechanism and domain feature loss from VGG19 pre-trained on the ImageNet dataset. The coordinate attention mechanism can capture long-range dependencies along one spatial direction and meanwhile preserve precise positional information along the other spatial direction. And the domain feature loss is more concerned with the topology of cardiac structures by exploiting their higher-level features. In this research, we use a two-dimensional echocardiogram (2DE) of 88 patients from two devices, Philips Epiq 7C and Mindray Resona 7T, to segment the left ventricle (LV), interventricular septal (IVS), and posterior left ventricular wall (PLVW). We also draw the gradient weighted class activation mapping (Grad-CAM) to improve the interpretability of the segmentation results. Compared with the traditional U-Net, the proposed segmentation method shows better performance. The mean Dice Score Coefficient (Dice) of LV, IVS, and PLVW of FAUet can achieve 0.932, 0.848, and 0.868, and the average Dice of the three objects can achieve 0.883. Statistical analysis showed that there is no significant difference between the segmentation results of the two devices. The proposed method can realize fast and accurate segmentation of 2DE with a low time cost. Combining coordinate attention module and feature loss with the original U-Net framework can significantly increase the performance of the algorithm.

Mitral Valve Segmentation Using Robust Nonnegative Matrix Factorization

Analyzing and understanding the movement of the mitral valve is of vital importance in cardiology, as the treatment and prevention of several serious heart diseases depend on it. Unfortunately, large amounts of noise as well as a highly varying image quality make the automatic tracking and segmentation of the mitral valve in two-dimensional echocardiographic videos challenging. In this paper, we present a fully automatic and unsupervised method for segmentation of the mitral valve in two-dimensional echocardiographic videos, independently of the echocardiographic view. We propose a bias-free variant of the robust non-negative matrix factorization (RNMF) along with a window-based localization approach, that is able to identify the mitral valve in several challenging situations. We improve the average f1-score on our dataset of 10 echocardiographic videos by 0.18 to a f1-score of 0.56.

Trilateral Attention Network for Real-Time Cardiac Region Segmentation

The accurate segmentation of cardiac images into anatomically meaningful regions is critical for the extraction of quantitative cardiac indices. The common pipeline for segmentation comprises regions of interest (ROIs) localization and segmentation stages that are independent of each other and typically performed using separate models. In this paper, we propose an end-to-end network, called Trilateral Attention Network (TaNet), for real-time region localization and segmentation. TaNet has a module for ROIs localization and three segmentation pathways: spatial pathway, handcrafted pathway, and context pathway. The localization module focuses segmentation attention on the desired region while learning the context relationship between different regions in the image. The localized regions are then sent to the three pathways for segmentation. The spatial pathway, which has regular convolutional kernels, is used to extract deep features at different levels of abstraction. The handcrafted pathway, which has hand-designed convolutional kernels, is used to extract a unique set of features complementary to the deep features. Finally, the context (or global) pathway is used to enlarge the receptive field. By jointly training TaNet for localization and segmentation, TaNet achieved superior performance, in terms of accuracy and speed, when evaluated on two echocardiography datasets for cardiac region segmentation.

Echocardiographic image multi-structure segmentation using Cardiac-SegNet

PURPOSE

Cardiac boundary segmentation of echocardiographic images is important for cardiac function assessment and disease diagnosis. However, it is challenging to segment cardiac ventricles due to the low contrast-to-noise ratio and speckle noise of the echocardiographic images. Manual segmentation is subject to interobserver variability and is too slow for real-time image-guided interventions. We aim to develop a deep learning-based method for automated multi-structure segmentation of echocardiographic images.

METHODS

We developed an anchor-free mask convolutional neural network (CNN), termed Cardiac-SegNet, which consists of three subnetworks, that is, a backbone, a fully convolutional one-state object detector (FCOS) head, and a mask head. The backbone extracts multi-level and multi-scale features from endocardium image. The FOCS head utilizes these features to detect and label the region-of-interests (ROIs) of the segmentation targets. Unlike the traditional mask regional CNN (Mask R-CNN) method, the FCOS head is anchor-free and can model the spatial relationship of the targets. The mask head utilizes a spatial attention strategy, which allows the network to highlight salient features to perform segmentation on each detected ROI. For evaluation, we investigated 450 patient datasets by a five-fold cross-validation and a hold-out test. The endocardium (LVEndo ) and epicardium (LVEpi ) of the left ventricle and left atrium (LA) were segmented and compared with manual contours using the Dice similarity coefficient (DSC), Hausdorff distance (HD), mean absolute distance (MAD), and center-of-mass distance (CMD).

RESULTS

Compared to U-Net and Mask R-CNN, our method achieved higher segmentation accuracy and fewer erroneous speckles. When our method was evaluated on a separate hold-out dataset at the end diastole (ED) and the end systole (ES) phases, the average DSC were 0.952 and 0.939 at ED and ES for the LVEndo , 0.965 and 0.959 at ED and ES for the LVEpi , and 0.924 and 0.926 at ED and ES for the LA. For patients with a typical image size of 549 × 788 pixels, the proposed method can perform the segmentation within 0.5 s.

CONCLUSION

We proposed a fast and accurate method to segment echocardiographic images using an anchor-free mask CNN.