Deep pyramid local attention neural network for cardiac structure segmentation in two-dimensional echocardiography

A Semi-supervised Four-Chamber Echocardiographic Video Segmentation Algorithm Based on Multilevel Edge Perception and Calibration Fusion

OBJECTIVE

Echocardiographic videos are commonly used for automatic semantic segmentation of endocardium, which is crucial in evaluating cardiac function and assisting doctors to make accurate diagnoses of heart disease. However, this task faces two distinct challenges: one is the edge blurring, which is caused by the presence of speckle noise or excessive de-noising operation, and the other is the lack of an effective feature fusion approach for multilevel features for obtaining accurate endocardium.

METHODS

In this study, a deep learning model, based on multilevel edge perception and calibration fusion is proposed to improve the segmentation performance. First, a multilevel edge perception module is proposed to comprehensively extract edge features through both a detail branch and a semantic branch to alleviate the adverse impact of noise. Second, a calibration fusion module is proposed that calibrates and integrates various features, including semantic and detailed information, to maximize segmentation performance. Furthermore, the features obtained from the calibration fusion module are stored by using a memory architecture to achieve semi-supervised segmentation through both labeled and unlabeled data.

RESULTS

Our method is evaluated on two public echocardiography video data sets, achieving average Dice coefficients of 93.05% and 93.93%, respectively. Additionally, we validated our method on a local hospital clinical data set, achieving a Pearson correlation of 0.765 for predicting left ventricular ejection fraction.

CONCLUSION

The proposed model effectively solves the challenges encountered in echocardiography by using semi-supervised networks, thereby improving the segmentation accuracy of the ventricles. This indicates that the proposed model can assist cardiologists in obtaining accurate and effective research and diagnostic results.

Deep Learning Based Automatic Left Ventricle Segmentation from the Transgastric Short-Axis View on Transesophageal Echocardiography: A Feasibility Study

Segmenting the left ventricle from the transgastric short-axis views (TSVs) on transesophageal echocardiography (TEE) is the cornerstone for cardiovascular assessment during perioperative management. Even for seasoned professionals, the procedure remains time-consuming and experience-dependent. The current study aims to evaluate the feasibility of deep learning for automatic segmentation by assessing the validity of different U-Net algorithms. A large dataset containing 1388 TSV acquisitions was retrospectively collected from 451 patients (32% women, average age 53.42 years) who underwent perioperative TEE between July 2015 and October 2023. With image preprocessing and data augmentation, 3336 images were included in the training set, 138 images in the validation set, and 138 images in the test set. Four deep neural networks (U-Net, Attention U-Net, UNet++, and UNeXt) were employed for left ventricle segmentation and compared in terms of the Jaccard similarity coefficient (JSC) and Dice similarity coefficient (DSC) on the test set, as well as the number of network parameters, training time, and inference time. The Attention U-Net and U-Net++ models performed better in terms of JSC (the highest average JSC: 86.02%) and DSC (the highest average DSC: 92.00%), the UNeXt model had the smallest network parameters (1.47 million), and the U-Net model had the least training time (6428.65 s) and inference time for a single image (101.75 ms). The Attention U-Net model outperformed the other three models in challenging cases, including the impaired boundary of left ventricle and the artifact of the papillary muscle. This pioneering exploration demonstrated the feasibility of deep learning for the segmentation of the left ventricle from TSV on TEE, which will facilitate an accelerated and objective alternative of cardiovascular assessment for perioperative management.

Multi-granularity learning of explicit geometric constraint and contrast for label-efficient medical image segmentation and differentiable clinical function assessment

Automated segmentation is a challenging task in medical image analysis that usually requires a large amount of manually labeled data. However, most current supervised learning based algorithms suffer from insufficient manual annotations, posing a significant difficulty for accurate and robust segmentation. In addition, most current semi-supervised methods lack explicit representations of geometric structure and semantic information, restricting segmentation accuracy. In this work, we propose a hybrid framework to learn polygon vertices, region masks, and their boundaries in a weakly/semi-supervised manner that significantly advances geometric and semantic representations. Firstly, we propose multi-granularity learning of explicit geometric structure constraints via polygon vertices (PolyV) and pixel-wise region (PixelR) segmentation masks in a semi-supervised manner. Secondly, we propose eliminating boundary ambiguity by using an explicit contrastive objective to learn a discriminative feature space of boundary contours at the pixel level with limited annotations. Thirdly, we exploit the task-specific clinical domain knowledge to differentiate the clinical function assessment end-to-end. The ground truth of clinical function assessment, on the other hand, can serve as auxiliary weak supervision for PolyV and PixelR learning. We evaluate the proposed framework on two tasks, including optic disc (OD) and cup (OC) segmentation along with vertical cup-to-disc ratio (vCDR) estimation in fundus images; left ventricle (LV) segmentation at end-diastolic and end-systolic frames along with ejection fraction (LVEF) estimation in two-dimensional echocardiography images. Experiments on nine large-scale datasets of the two tasks under different label settings demonstrate our model's superior performance on segmentation and clinical function assessment.

MADR-Net: multi-level attention dilated residual neural network for segmentation of medical images

Medical image segmentation has made a significant contribution towards delivering affordable healthcare by facilitating the automatic identification of anatomical structures and other regions of interest. Although convolution neural networks have become prominent in the field of medical image segmentation, they suffer from certain limitations. In this study, we present a reliable framework for producing performant outcomes for the segmentation of pathological structures of 2D medical images. Our framework consists of a novel deep learning architecture, called deep multi-level attention dilated residual neural network (MADR-Net), designed to improve the performance of medical image segmentation. MADR-Net uses a U-Net encoder/decoder backbone in combination with multi-level residual blocks and atrous pyramid scene parsing pooling. To improve the segmentation results, channel-spatial attention blocks were added in the skip connection to capture both the global and local features and superseded the bottleneck layer with an ASPP block. Furthermore, we introduce a hybrid loss function that has an excellent convergence property and enhances the performance of the medical image segmentation task. We extensively validated the proposed MADR-Net on four typical yet challenging medical image segmentation tasks: (1) Left ventricle, left atrium, and myocardial wall segmentation from Echocardiogram images in the CAMUS dataset, (2) Skin cancer segmentation from dermoscopy images in ISIC 2017 dataset, (3) Electron microscopy in FIB-SEM dataset, and (4) Fluid attenuated inversion recovery abnormality from MR images in LGG segmentation dataset. The proposed algorithm yielded significant results when compared to state-of-the-art architectures such as U-Net, Residual U-Net, and Attention U-Net. The proposed MADR-Net consistently outperformed the classical U-Net by 5.43%, 3.43%, and 3.92% relative improvement in terms of dice coefficient, respectively, for electron microscopy, dermoscopy, and MRI. The experimental results demonstrate superior performance on single and multi-class datasets and that the proposed MADR-Net can be utilized as a baseline for the assessment of cross-dataset and segmentation tasks.

Multi-Task Learning for Motion Analysis and Segmentation in 3D Echocardiography

Characterizing left ventricular deformation and strain using 3D+time echocardiography provides useful insights into cardiac function and can be used to detect and localize myocardial injury. To achieve this, it is imperative to obtain accurate motion estimates of the left ventricle. In many strain analysis pipelines, this step is often accompanied by a separate segmentation step; however, recent works have shown both tasks to be highly related and can be complementary when optimized jointly. In this work, we present a multi-task learning network that can simultaneously segment the left ventricle and track its motion between multiple time frames. Two task-specific networks are trained using a composite loss function. Cross-stitch units combine the activations of these networks by learning shared representations between the tasks at different levels. We also propose a novel shape-consistency unit that encourages motion propagated segmentations to match directly predicted segmentations. Using a combined synthetic and in-vivo 3D echocardiography dataset, we demonstrate that our proposed model can achieve excellent estimates of left ventricular motion displacement and myocardial segmentation. Additionally, we observe strong correlation of our image-based strain measurements with crystal-based strain measurements as well as good correspondence with SPECT perfusion mappings. Finally, we demonstrate the clinical utility of the segmentation masks in estimating ejection fraction and sphericity indices that correspond well with benchmark measurements.

Automatic multi-view pose estimation in focused cardiac ultrasound

Focused cardiac ultrasound (FoCUS) is a valuable point-of-care method for evaluating cardiovascular structures and function, but its scope is limited by equipment and operator's experience, resulting in primarily qualitative 2D exams. This study presents a novel framework to automatically estimate the 3D spatial relationship between standard FoCUS views. The proposed framework uses a multi-view U-Net-like fully convolutional neural network to regress line-based heatmaps representing the most likely areas of intersection between input images. The lines that best fit the regressed heatmaps are then extracted, and a system of nonlinear equations based on the intersection between view triplets is created and solved to determine the relative 3D pose between all input images. The feasibility and accuracy of the proposed pipeline were validated using a novel realistic in silico FoCUS dataset, demonstrating promising results. Interestingly, as shown in preliminary experiments, the estimation of the 2D images' relative poses enables the application of 3D image analysis methods and paves the way for 3D quantitative assessments in FoCUS examinations.

Deep learning supported echocardiogram analysis: A comprehensive review

An echocardiogram is a sophisticated ultrasound imaging technique employed to diagnose heart conditions. The transthoracic echocardiogram, one of the most prevalent types, is instrumental in evaluating significant cardiac diseases. However, interpreting its results heavily relies on the clinician's expertise. In this context, artificial intelligence has emerged as a vital tool for helping clinicians. This study critically analyzes key state-of-the-art research that uses deep learning techniques to automate transthoracic echocardiogram analysis and support clinical judgments. We have systematically organized and categorized articles that proffer solutions for view classification, enhancement of image quality and dataset, segmentation and identification of cardiac structures, detection of cardiac function abnormalities, and quantification of cardiac functions. We compared the performance of various deep learning approaches within each category, identifying the most promising methods. Additionally, we highlight limitations in current research and explore promising avenues for future exploration. These include addressing generalizability issues, incorporating novel AI approaches, and tackling the analysis of rare cardiac diseases.

Boundary attention with multi-task consistency constraints for semi-supervised 2D echocardiography segmentation

The 2D echocardiography semantic automatic segmentation technique is important in clinical applications for cardiac function assessment and diagnosis of cardiac diseases. However, automatic segmentation of 2D echocardiograms also faces the problems of loss of image boundary information, loss of image localization information, and limitations in data acquisition and annotation. To address these issues, this paper proposes a semi-supervised echocardiography segmentation method. It consists of two models: (1) a boundary attention transformer net (BATNet) and (2) a multi-task level semi-supervised model with consistency constraints on boundary features (semi-BATNet). BATNet is able to capture the location and spatial information of the input feature maps by using the self-attention mechanism. The multi-task level semi-supervised model with boundary feature consistency constraints (semi-BATNet) encourages consistent predictions of boundary features at different scales from the student and teacher networks to calculate the multi-scale consistency loss for unlabeled data. The proposed semi-BATNet was extensively evaluated on the dataset of cardiac acquisitions for multi-structure ultrasound segmentation (CAMUS) and self-collected echocardiography dataset from the First Affiliated Hospital of Chongqing Medical University. Experimental results on the CAMUS dataset showed that when only 25% of the images are labeled, the proposed method greatly improved the segmentation performance by utilizing unlabeled images, and it also outperformed five state-of-the-art semi-supervised segmentation methods. Moreover, when only 50% of the images labeled, semi-BATNet achieved the Dice coefficient values of 0.936, the Jaccard similarity of 0.881 on self-collected echocardiography dataset. Semi-BATNet can complete a more accurate segmentation of cardiac structures in 2D echocardiograms, indicating that it has the potential to accurately and efficiently assist cardiologists.

Diagnosing oral and maxillofacial diseases using deep learning

The classification and localization of odontogenic lesions from panoramic radiographs is a challenging task due to the positional biases and class imbalances of the lesions. To address these challenges, a novel neural network, DOLNet, is proposed that uses mutually influencing hierarchical attention across different image scales to jointly learn the global representation of the entire jaw and the local discrepancy between normal tissue and lesions. The proposed approach uses local attention to learn representations within a patch. From the patch-level representations, we generate inter-patch, i.e., global, attention maps to represent the positional prior of lesions in the whole image. Global attention enables the reciprocal calibration of path-level representations by considering non-local information from other patches, thereby improving the generation of whole-image-level representation. To address class imbalances, we propose an effective data augmentation technique that involves merging lesion crops with normal images, thereby synthesizing new abnormal cases for effective model training. Our approach outperforms recent studies, enhancing the classification performance by up to 42.4% and 44.2% in recall and F1 scores, respectively, and ensuring robust lesion localization with respect to lesion size variations and positional biases. Our approach further outperforms human expert clinicians in classification by 10.7 % and 10.8 % in recall and F1 score, respectively.

TC-SegNet: robust deep learning network for fully automatic two-chamber segmentation of two-dimensional echocardiography

Heart chamber quantification is an essential clinical task to analyze heart abnormalities by evaluating the heart volume estimated through the endocardial border of the chambers. A precise heart chamber segmentation algorithm using echocardiography is essential for improving the diagnosis of cardiac disease. This paper proposes a robust two chamber segmentation network (TC-SegNet) for echocardiography which follows a U-Net architecture and effectively incorporates the proposed modified skip connection, Atrous Spatial Pyramid Pooling (ASPP) modules and squeeze and excitation modules. The TC-SegNet is evaluated on the open-source fully annotated dataset of cardiac acquisitions for multi-structure ultrasound segmentation (CAMUS). The proposed TC-SegNet obtained an average value of F1-score of 0.91, an average Dice score of 0.9284 and an IoU score of 0.8322 which are higher than the reference models used here for comparison. Further, Pixel error (PE) of 1.5109 which are significantly less than the comparison models. The segmentation results and metrics show that the proposed model outperforms the state-of-the-art segmentation methods.

Segmentation of Anatomical Structures of the Left Heart from Echocardiographic Images Using Deep Learning

Knowledge about the anatomical structures of the left heart, specifically the atrium (LA) and ventricle (i.e., endocardium-Vendo-and epicardium-LVepi) is essential for the evaluation of cardiac functionality. Manual segmentation of cardiac structures from echocardiography is the baseline reference, but results are user-dependent and time-consuming. With the aim of supporting clinical practice, this paper presents a new deep-learning (DL)-based tool for segmenting anatomical structures of the left heart from echocardiographic images. Specifically, it was designed as a combination of two convolutional neural networks, the YOLOv7 algorithm and a U-Net, and it aims to automatically segment an echocardiographic image into LVendo, LVepi and LA. The DL-based tool was trained and tested on the Cardiac Acquisitions for Multi-Structure Ultrasound Segmentation (CAMUS) dataset of the University Hospital of St. Etienne, which consists of echocardiographic images from 450 patients. For each patient, apical two- and four-chamber views at end-systole and end-diastole were acquired and annotated by clinicians. Globally, our DL-based tool was able to segment LVendo, LVepi and LA, providing Dice similarity coefficients equal to 92.63%, 85.59%, and 87.57%, respectively. In conclusion, the presented DL-based tool proved to be reliable in automatically segmenting the anatomical structures of the left heart and supporting the cardiological clinical practice.

Cyclical Self-Supervision for Semi-Supervised Ejection Fraction Prediction From Echocardiogram Videos

Left-ventricular ejection fraction (LVEF) is an important indicator of heart failure. Existing methods for LVEF estimation from video require large amounts of annotated data to achieve high performance, e.g. using 10,030 labeled echocardiogram videos to achieve mean absolute error (MAE) of 4.10. Labeling these videos is time-consuming however and limits potential downstream applications to other heart diseases. This paper presents the first semi-supervised approach for LVEF prediction. Unlike general video prediction tasks, LVEF prediction is specifically related to changes in the left ventricle (LV) in echocardiogram videos. By incorporating knowledge learned from predicting LV segmentations into LVEF regression, we can provide additional context to the model for better predictions. To this end, we propose a novel Cyclical Self-Supervision (CSS) method for learning video-based LV segmentation, which is motivated by the observation that the heartbeat is a cyclical process with temporal repetition. Prediction masks from our segmentation model can then be used as additional input for LVEF regression to provide spatial context for the LV region. We also introduce teacher-student distillation to distill the information from LV segmentation masks into an end-to-end LVEF regression model that only requires video inputs. Results show our method outperforms alternative semi-supervised methods and can achieve MAE of 4.17, which is competitive with state-of-the-art supervised performance, using half the number of labels. Validation on an external dataset also shows improved generalization ability from using our method.

EchoEFNet: Multi-task deep learning network for automatic calculation of left ventricular ejection fraction in 2D echocardiography

Left ventricular ejection fraction (LVEF) is essential for evaluating left ventricular systolic function. However, its clinical calculation requires the physician to interactively segment the left ventricle and obtain the mitral annulus and apical landmarks. This process is poorly reproducible and error prone. In this study, we propose a multi-task deep learning network EchoEFNet. The network use ResNet50 with dilated convolution as the backbone to extract high-dimensional features while maintaining spatial features. The branching network used our designed multi-scale feature fusion decoder to segment the left ventricle and detect landmarks simultaneously. The LVEF was then calculated automatically and accurately using the biplane Simpson's method. The model was tested for performance on the public dataset CAMUS and private dataset CMUEcho. The experimental results showed that the geometrical metrics and percentage of correct keypoints of EchoEFNet outperformed other deep learning methods. The correlation between the predicted LVEF and true values on the CAMUS and CMUEcho datasets was 0.854 and 0.916, respectively.

RA-DENet: Reverse Attention and Distractions Elimination Network for polyp segmentation

To address the problems of polyps of different shapes, sizes, and colors, low-contrast polyps, various noise distractions, and blurred edges on colonoscopy, we propose the Reverse Attention and Distraction Elimination Network, which includes Improved Reverse Attention, Distraction Elimination, and Feature Enhancement. First, we input the images in the polyp image set, and use the five levels polyp features and the global polyp feature extracted from the Res2Net-based backbone as the input of the Improved Reverse Attention to obtain augmented representations of salient and non-salient regions to capture the different shapes of polyp and distinguish low-contrast polyps from background. Then, the augmented representations of salient and non-salient areas are fed into the Distraction Elimination to obtain the refined polyp feature without false positive and false negative distractions for eliminating noises. Finally, the extracted low-level polyp feature is used as the input of the Feature Enhancement to obtain the edge feature for supplementing missing edge information of polyp. The polyp segmentation result is output by connecting the edge feature with the refined polyp feature. The proposed method is evaluated on five polyp datasets and compared with the current polyp segmentation models. Our model improves the mDice to 0.760 on the most challenge dataset (ETIS).

Co-attention spatial transformer network for unsupervised motion tracking and cardiac strain analysis in 3D echocardiography

Myocardial ischemia/infarction causes wall-motion abnormalities in the left ventricle. Therefore, reliable motion estimation and strain analysis using 3D+time echocardiography for localization and characterization of myocardial injury is valuable for early detection and targeted interventions. Previous unsupervised cardiac motion tracking methods rely on heavily-weighted regularization functions to smooth out the noisy displacement fields in echocardiography. In this work, we present a Co-Attention Spatial Transformer Network (STN) for improved motion tracking and strain analysis in 3D echocardiography. Co-Attention STN aims to extract inter-frame dependent features between frames to improve the motion tracking in otherwise noisy 3D echocardiography images. We also propose a novel temporal constraint to further regularize the motion field to produce smooth and realistic cardiac displacement paths over time without prior assumptions on cardiac motion. Our experimental results on both synthetic and in vivo 3D echocardiography datasets demonstrate that our Co-Attention STN provides superior performance compared to existing methods. Strain analysis from Co-Attention STNs also correspond well with the matched SPECT perfusion maps, demonstrating the clinical utility for using 3D echocardiography for infarct localization.

MAEF-Net: Multi-attention efficient feature fusion network for left ventricular segmentation and quantitative analysis in two-dimensional echocardiography

The segmentation of cardiac chambers and the quantification of clinical functional metrics in dynamic echocardiography are the keys to the clinical diagnosis of heart disease. Identifying the end-diastolic frames (EDFs) and end-systolic frames (ESFs) and manually segmenting the left ventricle in the echocardiographic cardiac cycle before obtaining the left ventricular ejection fraction (LVEF) is a time-consuming and tedious task for clinicians. In this work, we proposed a deep learning-based fully automated echocardiographic analysis method. We proposed a multi-attention efficient feature fusion network (MAEF-Net) to automatically segment the left ventricle. Then, EDFs and ESFs in all cardiac cycles were automatically detected to compute LVEF. The MAEF-Net method used a multi-attention mechanism to guide the network to capture heartbeat features effectively, while suppressing noise, and incorporated deep supervision mechanism and spatial pyramid feature fusion to enhance feature extraction capabilities. The proposed method was validated on the public EchoNet-Dynamic dataset (n = 1226). The Dice similarity coefficient (DSC) of the left ventricular segmentation reached (93.10 ± 2.22)%, and the mean absolute error (MAE) of cardiac phase detection was (2.36 ± 2.23) frames. The MAE for predicting LVEF was 6.29 %. The proposed method was also validated on a private clinical dataset (n = 22). The DSC of the left ventricular segmentation reached (92.81 ± 2.85)%, and the MAE of cardiac phase detection was (2.25 ± 2.27) frames. The MAE for predicting LVEF was 5.91 %, and the Pearson correlation coefficient r reached 0.96. The proposed method may be used as a new method for automatic left ventricular segmentation and quantitative analysis in two-dimensional echocardiography. Our code and trained models will be made available publicly at https://github.com/xiaojinmao-code/MAEF-Net.

Segmentation of Echocardiography Based on Deep Learning Model

In order to achieve the classification of mitral regurgitation, a deep learning network VDS-UNET was designed to automatically segment the critical regions of echocardiography with three sections of apical two-chamber, apical three-chamber, and apical four-chamber. First, an expert-labeled dataset of 153 echocardiographic videos and 2183 images from 49 subjects was constructed. Then, the convolution layer in the VGG16 network was used to replace the contraction path in the original UNet network to extract image features, and depth supervision was added to the expansion path to achieve the segmentation of LA, LV, and MV. The results showed that the Dice coefficients of LA, LV, and MV were 0.935, 0.915, and 0.757, respectively. The proposed deep learning network can achieve simultaneous and accurate segmentation of LA, LV, and MV in multi-section echocardiography, laying a foundation for quantitative measurement of clinical parameters related to mitral regurgitation.

Explicit and automatic ejection fraction assessment on 2D cardiac ultrasound with a deep learning-based approach

BACKGROUND

Ejection fraction (EF) is a key parameter for assessing cardiovascular functions in cardiac ultrasound, but its manual assessment is time-consuming and subject to high inter and intra-observer variability. Deep learning-based methods have the potential to perform accurate fully automatic EF predictions but suffer from a lack of explainability and interpretability. This study proposes a fully automatic method to reliably and explicitly evaluate the biplane left ventricular EF on 2D echocardiography following the recommended modified Simpson's rule.

METHODS

A deep learning model was trained on apical 4 and 2-chamber echocardiography to segment the left ventricle and locate the mitral valve. Predicted segmentations are then validated with a statistical shape model, which detects potential failures that could impact the EF evaluation. Finally, the end-diastolic and end-systolic frames are identified based on the remaining LV segmentations' areas and EF is estimated on all available cardiac cycles.

RESULTS

Our approach was trained on a dataset of 783 patients. Its performances were evaluated on an internal and external dataset of respectively 200 and 450 patients. On the internal dataset, EF assessment achieved a mean absolute error of 6.10% and a bias of 1.56 ± 7.58% using multiple cardiac cycles. The approach evaluated EF with a mean absolute error of 5.39% and a bias of -0.74 ± 7.12% on the external dataset.

CONCLUSION

Following the recommended guidelines, we proposed an end-to-end fully automatic approach that achieves state-of-the-art performance in biplane EF evaluation while giving explicit details to clinicians.

Adaptive Contrast for Image Regression in Computer-Aided Disease Assessment

Image regression tasks for medical applications, such as bone mineral density (BMD) estimation and left-ventricular ejection fraction (LVEF) prediction, play an important role in computer-aided disease assessment. Most deep regression methods train the neural network with a single regression loss function like MSE or L1 loss. In this paper, we propose the first contrastive learning framework for deep image regression, namely AdaCon, which consists of a feature learning branch via a novel adaptive-margin contrastive loss and a regression prediction branch. Our method incorporates label distance relationships as part of the learned feature representations, which allows for better performance in downstream regression tasks. Moreover, it can be used as a plug-and-play module to improve performance of existing regression methods. We demonstrate the effectiveness of AdaCon on two medical image regression tasks, i.e., bone mineral density estimation from X-ray images and left-ventricular ejection fraction prediction from echocardiogram videos. AdaCon leads to relative improvements of 3.3% and 5.9% in MAE over state-of-the-art BMD estimation and LVEF prediction methods, respectively.

Cardiac Segmentation Method Based on Domain Knowledge

Echocardiography plays an important role in the clinical diagnosis of cardiovascular diseases. Cardiac function assessment by echocardiography is a crucial process in daily cardiology. However, cardiac segmentation in echocardiography is a challenging task due to shadows and speckle noise. The traditional manual segmentation method is a time-consuming process and limited by inter-observer variability. In this paper, we present a fast and accurate echocardiographic automatic segmentation framework based on Convolutional neural networks (CNN). We propose FAUet, a segmentation method serially integrated U-Net with coordinate attention mechanism and domain feature loss from VGG19 pre-trained on the ImageNet dataset. The coordinate attention mechanism can capture long-range dependencies along one spatial direction and meanwhile preserve precise positional information along the other spatial direction. And the domain feature loss is more concerned with the topology of cardiac structures by exploiting their higher-level features. In this research, we use a two-dimensional echocardiogram (2DE) of 88 patients from two devices, Philips Epiq 7C and Mindray Resona 7T, to segment the left ventricle (LV), interventricular septal (IVS), and posterior left ventricular wall (PLVW). We also draw the gradient weighted class activation mapping (Grad-CAM) to improve the interpretability of the segmentation results. Compared with the traditional U-Net, the proposed segmentation method shows better performance. The mean Dice Score Coefficient (Dice) of LV, IVS, and PLVW of FAUet can achieve 0.932, 0.848, and 0.868, and the average Dice of the three objects can achieve 0.883. Statistical analysis showed that there is no significant difference between the segmentation results of the two devices. The proposed method can realize fast and accurate segmentation of 2DE with a low time cost. Combining coordinate attention module and feature loss with the original U-Net framework can significantly increase the performance of the algorithm.

MCAL: An Anatomical Knowledge Learning Model for Myocardial Segmentation in 2-D Echocardiography

Segmentation of the left ventricular (LV) myocardium in 2-D echocardiography is essential for clinical decision making, especially in geometry measurement and index computation. However, segmenting the myocardium is a time-consuming process and challenging due to the fuzzy boundary caused by the low image quality. The ground-truth label is employed as pixel-level class associations or shape regulation in segmentation, which works limit for effective feature enhancement for 2-D echocardiography. We propose a training strategy named multiconstrained aggregate learning (referred to as MCAL), which leverages anatomical knowledge learned through ground-truth labels to infer segmented parts and discriminate boundary pixels. The new framework encourages the model to focus on the features in accordance with the learned anatomical representations, and the training objectives incorporate a boundary distance transform weight (BDTW) to enforce a higher weight value on the boundary region, which helps to improve the segmentation accuracy. The proposed method is built as an end-to-end framework with a top-down, bottom-up architecture with skip convolution fusion blocks and carried out on two datasets (our dataset and the public CAMUS dataset). The comparison study shows that the proposed network outperforms the other segmentation baseline models, indicating that our method is beneficial for boundary pixels discrimination in segmentation.

U-net architecture with embedded Inception-ResNet-v2 image encoding modules for automatic segmentation of organs-at-risk in head and neck cancer radiation therapy based on computed tomography scans

PURPOSE

The purpose of this study was to utilize a novel deep learning model to automatically contour critical organs on the Computed Tomography (CT) scans of head and neck cancer patients who underwent radiation therapy treatment and interpret the clinical suitability of the model results through activation mapping.

MATERIALS AND METHODS

This study included 25 critical organs that were delineated by expert radiation oncologists. Contoured medical images of 964 patients were sourced from 9 institutions through publicly available TCIA database. The proportion of training, validation, and testing samples for deep learning model development was 65%, 25%, and 10% respectively. The CT scans and segmentation masks were augmented with shift, scale, and rotate transformations. Additionally, medical images were pre-processed using contrast limited adaptive histogram equalization (CLAHE) to enhance soft tissue contrast while contours were subjected to morphological operations to ensure their structural integrity. The segmentation model was based on the U-Net architecture with embedded Inception-ResNet-v2 blocks. The model performance was evaluated with Dice Score, Jaccard Index, and Hausdorff Distances. The interpretability of the model was analyzed with Guided Gradient-weighted Class Activation Mapping.

RESULTS

The Dice Score, Jaccard Index, and mean Hausdorff Distance averaged over all structures and patients were 0.82±0.10, 0.71±0.10, and 1.51±1.17 mm respectively on the testing data sets. The Dice Scores for 86.4% of compared structures was within range or better than published interobserver variability derived from multi-institutional studies. The average model training time was 8 h per anatomical structure. The full segmentation of head and neck anatomy by the trained network required only 6.8 s per patient.

CONCLUSIONS

High accuracy obtained on a large, multi-institutional data set, short segmentation time and clinically-realistic prediction reasoning make the model proposed in this work a feasible solution for head and neck CT scan segmentation in a clinical environment.

Deep Attention and Graphical Neural Network for Multiple Sclerosis Lesion Segmentation from MR Imaging Sequences

The segmentation of multiple sclerosis (MS) lesions from MR imaging sequences remains a challenging task, due to the characteristics of variant shapes, scattered distributions and unknown numbers of lesions. However, the current automated MS segmentation methods with deep learning models face the challenges of (1) capturing the multiple scattered lesions in multiple regions and (2) delineating the global contour of variant lesions. To address these challenges, in this paper, we propose a novel attention and graph-driven network (DAG-Net), which incorporates (1) the spatial correlations for embracing the lesions in distant regions and (2) the global context for better representing lesions of variant features in a unified architecture. Firstly, the novel local attention coherence mechanism is designed to construct dynamic and expansible graphs for the spatial correlations between pixels and their proximities. Secondly, the proposed spatial-channel attention module enhances features to optimize the global contour delineation, by aggregating relevant features. Moreover, with the dynamic graphs, the learning process of the DAG-Net is interpretable, which in turns support the reliability of segmentation results. Extensive experiments were conducted on a public ISBI2015 dataset and an in-house dataset in comparison to state-of-the-art methods, based on the geometrical and clinical metrics. The experimental results validate the effectiveness of the proposed DAG-Net on segmenting variant and scatted lesions in multiple regions.

Multi-frame Attention Network for Left Ventricle Segmentation in 3D Echocardiography

Echocardiography is one of the main imaging modalities used to assess the cardiovascular health of patients. Among the many analyses performed on echocardiography, segmentation of left ventricle is crucial to quantify the clinical measurements like ejection fraction. However, segmentation of left ventricle in 3D echocardiography remains a challenging and tedious task. In this paper, we propose a multi-frame attention network to improve the performance of segmentation of left ventricle in 3D echocardiography. The multi-frame attention mechanism allows highly correlated spatiotemporal features in a sequence of images that come after a target image to be used to augment the performance of segmentation. Experimental results shown on 51 in vivo porcine 3D+time echocardiography images show that utilizing correlated spatiotemporal features significantly improves the performance of left ventricle segmentation when compared to other standard deep learning-based medical image segmentation models.