Deep learning-based carotid media-adventitia and lumen-intima boundary segmentation from three-dimensional ultrasound images

Pericarotid Fat as a Marker of Cerebrovascular Risk

Vascular inflammation is widely recognized as an important factor in the atherosclerotic process, particularly in terms of plaque development and progression. Conventional tests, such as measuring circulating inflammatory biomarkers, lack the precision to identify specific areas of vascular inflammation. In this context, noninvasive imaging modalities can detect perivascular fat changes, serving as a marker of vascular inflammation. This review aims to provide a comprehensive overview of the key concepts related to perivascular carotid fat and its pathophysiology. Additionally, we examine the existing literature on the association of pericarotid fat with features of plaque vulnerability and cerebrovascular events. Finally, we scrutinize the advantages and limitations of the noninvasive assessment of pericarotid fat.

Automatic Segmentation and Alignment of Uterine Shapes from 3D Ultrasound Data

BACKGROUND

The uterus is the most important organ in the female reproductive system. Its shape plays a critical role in fertility and pregnancy outcomes. Advances in medical imaging, such as 3D ultrasound, have significantly improved the exploration of the female genital tract, thereby enhancing gynecological healthcare. Despite well-documented data for organs like the liver and heart, large-scale studies on the uterus are lacking. Existing classifications, such as VCUAM and ESHRE/ESGE, provide different definitions for normal uterine shapes but are not based on real-world measurements. Moreover, the lack of comprehensive datasets significantly hinders research in this area. Our research, part of the larger NURSE study, aims to fill this gap by establishing the shape of a normal uterus using real-world 3D vaginal ultrasound scans. This will facilitate research into uterine shape abnormalities associated with infertility and recurrent miscarriages.

METHODS

We developed an automated system for the segmentation and alignment of uterine shapes from 3D ultrasound data, which consists of two steps: automatic segmentation of the uteri in 3D ultrasound scans using deep learning techniques, and alignment of the resulting shapes with standard geometrical approaches, enabling the extraction of the normal shape for future analysis. The system was trained and validated on a comprehensive dataset of 3D ultrasound images from multiple medical centers. Its performance was evaluated by comparing the automated results with manual annotations provided by expert clinicians.

RESULTS

The presented approach demonstrated high accuracy in segmenting and aligning uterine shapes from 3D ultrasound data. The segmentation achieved an average Dice similarity coefficient (DSC) of 0.90. Our method for aligning uterine shapes showed minimal translation and rotation errors compared to traditional methods, with the preliminary average shape exhibiting characteristics consistent with expert findings of a normal uterus.

CONCLUSION

We have presented an approach to automatically segment and align uterine shapes from 3D ultrasound data. We trained a deep learning nnU-Net model that achieved high accuracy and proposed an alignment method using a combination of standard geometrical techniques. Additionally, we have created a publicly available dataset of 3D transvaginal ultrasound volumes with manual annotations of uterine cavities to support further research and development in this field. The dataset and the trained models are available at https://github.com/UL-FRI-LGM/UterUS.

Deep learning-based fully automatic screening of carotid artery plaques in computed tomography angiography: a multicenter study

AIM

To develop and validate a deep learning (DL) algorithm for the automated detection and classification of carotid artery plaques (CAPs) on computed tomography angiography (CTA) images.

MATERIALS AND METHODS

This retrospective study enrolled 400 patients (300 in the Center Ⅰ and 100 in Ⅱ). Three radiologists co-labeled CAPs, and their revised calcification status (noncalcified, mixed, and calcified) was regarded as ground truth. Center Ⅰ patients were randomly divided into training and internal validation datasets, while Center Ⅱ patients served as the external validation dataset. Carotid artery regions were segmented using a modified 3D-UNet network, followed by CAPs detection and classification using a ResUNet-based architecture in a two-step DL system. The DL model's detection and classification performance were evaluated on the validation dataset using precision-recall curve, free-response receiver operating characteristic (fROC) curve, Cohen's kappa, and ROC curve analysis.

RESULTS

The DL model had achieved 83.4% sensitivity at 3.0 false positives (FPs)/CTA scan in internal validation and 78.9% in external validation. F1-scores were 0.764 and 0.769 at the optimal threshold, and area under fROC curves were 0.756 and 0.738, respectively, indicating good overall accuracy for CAP detection. The DL model also showed good performance for the ternary classification of CAPs, with Cohen's kappa achieved 0.728 and 0.703 in both validation datasets.

CONCLUSION

This study demonstrated the feasibility of using a fully automated DL-based algorithm for the detection and ternary classification of CAPs, which could be helpful for the workloads of radiologists.

Unsupervised shape-and-texture-based generative adversarial tuning of pre-trained networks for carotid segmentation from 3D ultrasound images

BACKGROUND

Vessel-wall volume and localized three-dimensional ultrasound (3DUS) metrics are sensitive to the change of carotid atherosclerosis in response to medical/dietary interventions. Manual segmentation of the media-adventitia boundary (MAB) and lumen-intima boundary (LIB) required to obtain these metrics is time-consuming and prone to observer variability. Although supervised deep-learning segmentation models have been proposed, training of these models requires a sizeable manually segmented training set, making larger clinical studies prohibitive.

PURPOSE

We aim to develop a method to optimize pre-trained segmentation models without requiring manual segmentation to supervise the fine-tuning process.

METHODS

We developed an adversarial framework called the unsupervised shape-and-texture generative adversarial network (USTGAN) to fine-tune a convolutional neural network (CNN) pre-trained on a source dataset for accurate segmentation of a target dataset. The network integrates a novel texture-based discriminator with a shape-based discriminator, which together provide feedback for the CNN to segment the target images in a similar way as the source images. The texture-based discriminator increases the accuracy of the CNN in locating the artery, thereby lowering the number of failed segmentations. Failed segmentation was further reduced by a self-checking mechanism to flag longitudinal discontinuity of the artery and by self-correction strategies involving surface interpolation followed by a case-specific tuning of the CNN. The U-Net was pre-trained by the source dataset involving 224 3DUS volumes with 136, 44, and 44 volumes in the training, validation and testing sets. The training of USTGAN involved the same training group of 136 volumes in the source dataset and 533 volumes in the target dataset. No segmented boundaries for the target cohort were available for training USTGAN. The validation and testing of USTGAN involved 118 and 104 volumes from the target cohort, respectively. The segmentation accuracy was quantified by Dice Similarity Coefficient (DSC), and incorrect localization rate (ILR). Tukey's Honestly Significant Difference multiple comparison test was employed to quantify the difference of DSCs between models and settings, wherep ≤ 0.05 $p\,\le \,0.05$ was considered statistically significant.

RESULTS

USTGAN attained a DSC of85.7 ± 13.0 $85.7\,\pm \,13.0$ % in LIB and86.2 ± 10.6 ${86.2}\,\pm \,{10.6}$ % in MAB, improving from the baseline performance of74.6 ± 30.7 ${74.6}\,\pm \,{30.7}$ % in LIB (p< 10 - 12 $<10^{-12}$ ) and75.7 ± 28.9 ${75.7}\,\pm \,{28.9}$ % in MAB (p< 10 - 12 $<10^{-12}$ ). Our approach outperformed six state-of-the-art domain-adaptation models (MAB:p ≤ 3.63 × 10 - 7 $p \le 3.63\,\times \,10^{-7}$ , LIB:p ≤ 9.34 × 10 - 8 $p\,\le \,9.34\,\times \,10^{-8}$ ). The proposed USTGAN also had the lowest ILR among the methods compared (LIB: 2.5%, MAB: 1.7%).

CONCLUSION

Our framework improves segmentation generalizability, thereby facilitating efficient carotid disease monitoring in multicenter trials and in clinics with less expertise in 3DUS imaging.

Software that combines deep learning, 3D reconstruction and CFD to analyze the state of carotid arteries from ultrasound imaging

BACKGROUND

Ultrasound is one of the non-invasive techniques that are used in clinical diagnostics of carotid artery disease.

OBJECTIVE

This paper presents software methodology that can be used in combination with this imaging technique to provide additional information about the state of patient-specific artery.

METHODS

Overall three modules are combined within the proposed methodology. A clinical dataset is used within the deep learning module to extract the contours of the carotid artery. This data is then used within the second module to perform the three-dimensional reconstruction of the geometry of the carotid bifurcation and ultimately this geometry is used within the third module, where the hemodynamic analysis is performed. The obtained distributions of hemodynamic quantities enable a more detailed analysis of the blood flow and state of the arterial wall and could be useful to predict further progress of present abnormalities in the carotid bifurcation.

RESULTS

The performance of the deep learning module was demonstrated through the high values of relevant common classification metric parameters. Also, the accuracy of the proposed methodology was shown through the validation of results for the reconstructed parameters against the clinically measured values.

CONCLUSION

The presented methodology could be used in combination with standard clinical ultrasound examination to quickly provide additional quantitative and qualitative information about the state of the patient's carotid bifurcation and thus ensure a treatment that is more adapted to the specific patient.

Development of a Three-Dimensional Carotid Ultrasound Image Segmentation Workflow for Improved Efficiency, Reproducibility and Accuracy in Measuring Vessel Wall and Plaque Volume and Thickness

Automated segmentation of carotid lumen-intima boundary (LIB) and media-adventitia boundary (MAB) by deep convolutional neural networks (CNN) from three-dimensional ultrasound (3DUS) images has made assessment and monitoring of carotid atherosclerosis more efficient than manual segmentation. However, training of CNN still requires manual segmentation of LIB and MAB. Therefore, there is a need to improve the efficiency of manual segmentation and develop strategies to improve segmentation accuracy by the CNN for serial monitoring of carotid atherosclerosis. One strategy to reduce segmentation time is to increase the interslice distance (ISD) between segmented axial slices of a 3DUS image while maintaining the segmentation reliability. We, for the first time, investigated the effect of ISD on the reproducibility of MAB and LIB segmentations. The intra-observer reproducibility of LIB and MAB segmentations at ISDs of 1 mm and 2 mm was not statistically significantly different, whereas the reproducibility at ISD = 3 mm was statistically lower. Therefore, we conclude that segmentation with an ISD of 2 mm provides sufficient reliability for CNN training. We further proposed training the CNN by the baseline images of the entire cohort of patients for automatic segmentation of the follow-up images acquired for the same cohort. We validated that segmentation with this time-based partitioning approach is more accurate than that produced by patient-based partitioning, especially at the carotid bifurcation. This study forms the basis for an efficient, reproducible, and accurate 3DUS workflow for serial monitoring of carotid atherosclerosis useful in risk stratification of cardiovascular events and in evaluating the efficacy of new treatments.

A Dual-Stream Centerline-Guided Network for Segmentation of the Common and Internal Carotid Arteries From 3D Ultrasound Images

Segmentation of the carotid section encompassing the common carotid artery (CCA), the bifurcation and the internal carotid artery (ICA) from three-dimensional ultrasound (3DUS) is required to measure the vessel wall volume (VWV) and localized vessel-wall-plus-plaque thickness (VWT), shown to be sensitive to treatment effect. We proposed an approach to combine a centerline extraction network (CHG-Net) and a dual-stream centerline-guided network (DSCG-Net) to segment the lumen-intima (LIB) and media-adventitia boundaries (MAB) from 3DUS images. Correct arterial location is essential for successful segmentation of the carotid section encompassing the bifurcation. We addressed this challenge by using the arterial centerline to enhance the localization accuracy of the segmentation network. The CHG-Net was developed to generate a heatmap indicating high probability regions for the centerline location, which was then integrated with the 3DUS image by the DSCG-Net to generate the MAB and LIB. The DSCG-Net includes a scale-based and a spatial attention mechanism to fuse multi-level features extracted by the encoder, and a centerline heatmap reconstruction side-branch connected to the end of the encoder to increase the generalization ability of the network. Experiments involving 224 3DUS volumes produce a Dice similarity coefficient (DSC) of 95.8±1.9% and 92.3±5.4% for CCA MAB and LIB, respectively, and 93.2±4.4% and 89.0±10.0% for ICA MAB and LIB, respectively. Our approach outperformed four state-of-the-art 3D CNN models, even after their performances were boosted by centerline guidance. The efficiency afforded by the framework would allow it to be incorporated into the clinical workflow for improved quantification of plaque change.

Ultrasound-Based Image Analysis for Predicting Carotid Artery Stenosis Risk: A Comprehensive Review of the Problem, Techniques, Datasets, and Future Directions

The carotid artery is a major blood vessel that supplies blood to the brain. Plaque buildup in the arteries can lead to cardiovascular diseases such as atherosclerosis, stroke, ruptured arteries, and even death. Both invasive and non-invasive methods are used to detect plaque buildup in the arteries, with ultrasound imaging being the first line of diagnosis. This paper presents a comprehensive review of the existing literature on ultrasound image analysis methods for detecting and characterizing plaque buildup in the carotid artery. The review includes an in-depth analysis of datasets; image segmentation techniques for the carotid artery plaque area, lumen area, and intima-media thickness (IMT); and plaque measurement, characterization, classification, and stenosis grading using deep learning and machine learning. Additionally, the paper provides an overview of the performance of these methods, including challenges in analysis, and future directions for research.

Assessment of atherosclerosis: should coronary calcium score and intima-media thickness be replaced by ultrasound measurement of carotid plaque burden and vessel wall volume?

PURPOSE OF REVIEW

To describe the uses of vessel wall volume (VWV) and measurement of carotid plaque burden, as total plaque area (TPA) and total plaque volume (TPV), and to contrast them with measurement of carotid intima-media thickness (IMT) and coronary calcium (CAC).

RECENT FINDINGS

Measurement of carotid plaque burden (CPB) is useful for risk stratification, research into the genetics and biology of atherosclerosis, for measuring effects of new therapies for atherosclerosis, and for treatment of high-risk patients with severe atherosclerosis. It is as predictive of risk as CAC, with important advantages. IMT is only a weak predictor of risk and changes so little over time that it is not useful for assessing effects of therapy.

SUMMARY

Measurement of CPB and VWV are far superior to measurement of carotid IMT in many ways, and should replace it. Vessel wall volume can be measured in persons with no plaque as an alternative to IMT. There are important advantages of CPB over coronary calcium; CPB should be more widely used in vascular prevention.

Ultrasonic imaging using conditional generative adversarial networks

The Full Matrix Capture (FMC) and Total Focusing Method (TFM) combination is often considered the gold standard in ultrasonic nondestructive testing, however it may be impractical due to the amount of time required to gather and process the FMC, particularly for high cadence inspections. This study proposes replacing conventional FMC acquisition and TFM processing with a single zero-degree plane wave (PW) insonification and a conditional Generative Adversarial Network (cGAN) trained to produce TFM-like images. Three models with different cGAN architectures and loss formulations were tested in different scenarios. Their performances were compared with conventional TFM computed from FMC. The proposed cGANs were able to recreate TFM-like images with the same resolution while improving the contrast in more than 94% of the reconstructions in comparison with conventional TFM reconstructions. Indeed, thanks to the use of a bias in the cGANs' training, the contrast was systematically increased through a reduction of the background noise level and the elimination of some artifacts. Finally, the proposed method led to a reduction of the computation time and file size by a factor of 120 and 75, respectively.

Carotid Vessel-Wall-Volume Ultrasound Measurement via a UNet++ Ensemble Algorithm Trained on Small Data Sets

Vessel wall volume (VWV) is a 3-D ultrasound measurement for the assessment of therapy in patients with carotid atherosclerosis. Deep learning can be used to segment the media-adventitia boundary (MAB) and lumen-intima boundary (LIB) and to quantify VWV automatically; however, it typically requires large training data sets with expert manual segmentation, which are difficult to obtain. In this study, a UNet++ ensemble approach was developed for automated VWV measurement, trained on five small data sets (n = 30 participants) and tested on 100 participants with clinically diagnosed coronary artery disease enrolled in a multicenter CAIN trial. The Dice similarity coefficient (DSC), average symmetric surface distance (ASSD), Pearson correlation coefficient (r), Bland-Altman plots and coefficient of variation (CoV) were used to evaluate algorithm segmentation accuracy, agreement and reproducibility. The UNet++ ensemble yielded DSCs of 91.07%-91.56% and 87.53%-89.44% and ASSDs of 0.10-0.11 mm and 0.33-0.39 mm for the MAB and LIB, respectively; the algorithm VWV measurements were correlated (r = 0.763-0.795, p < 0.001) with manual segmentations, and the CoV for VWV was 8.89%. In addition, the UNet++ ensemble trained on 30 participants achieved a performance similar to that of U-Net and Voxel-FCN trained on 150 participants. These results suggest that our approach could provide accurate and reproducible carotid VWV measurements using relatively small training data sets, supporting deep learning applications for monitoring atherosclerosis progression in research and clinical trials.

Method for Carotid Artery 3-D Ultrasound Image Segmentation Based on CSWin Transformer

Precise segmentation of carotid artery (CA) structure is an important prerequisite for the medical assessment and detection of carotid plaques. For automatic segmentation of the media-adventitia boundary (MAB) and lumen-intima boundary (LIB) in 3-D ultrasound images of the CA, a U-shaped CSWin transformer (U-CSWT) is proposed. Both the encoder and decoder of the U-CSWT are composed of hierarchical CSWT modules, which can capture rich global context information in the 3-D image. Experiments were performed on a 3-D ultrasound image data set of the CA, and the results indicate that the U-CSWT performs better than other convolutional neural network (CNN)-based and CNN-transformer hybrid methods. The model yields Dice coefficients of 94.6 ± 3.0% and 90.8 ± 5.1% for the MAB and LIB in the common carotid artery (CCA) and 92.9 ± 4.9% and 89.6 ± 6.2% for MAB and LIB in the bifurcation, respectively. Our U-CSWT is expected to become an effective method for automatic segmentation of 3-D ultrasound images of CA.

Ultrasound carotid plaque segmentation via image reconstruction-based self-supervised learning with limited training labels

Carotid total plaque area (TPA) is an important contributing measurement to the evaluation of stroke risk. Deep learning provides an efficient method for ultrasound carotid plaque segmentation and TPA quantification. However, high performance of deep learning requires datasets with many labeled images for training, which is very labor-intensive. Thus, we propose an image reconstruction-based self-supervised learning algorithm (IR-SSL) for carotid plaque segmentation when few labeled images are available. IR-SSL consists of pre-trained and downstream segmentation tasks. The pre-trained task learns region-wise representations with local consistency by reconstructing plaque images from randomly partitioned and disordered images. The pre-trained model is then transferred to the segmentation network as the initial parameters in the downstream task. IR-SSL was implemented with two networks, UNet++ and U-Net, and evaluated on two independent datasets of 510 carotid ultrasound images from 144 subjects at SPARC (London, Canada) and 638 images from 479 subjects at Zhongnan hospital (Wuhan, China). Compared to the baseline networks, IR-SSL improved the segmentation performance when trained on few labeled images (n = 10, 30, 50 and 100 subjects). For 44 SPARC subjects, IR-SSL yielded Dice-similarity-coefficients (DSC) of 80.14-88.84%, and algorithm TPAs were strongly correlated (r=0.962-0.993, p < 0.001) with manual results. The models trained on the SPARC images but applied to the Zhongnan dataset without retraining achieved DSCs of 80.61-88.18% and strong correlation with manual segmentation (r=0.852-0.978, p < 0.001). These results suggest that IR-SSL could improve deep learning when trained on small labeled datasets, making it useful for monitoring carotid plaque progression/regression in clinical use and trials.

Learning-based initialization for correntropy-based level sets to segment atherosclerotic plaque in ultrasound images

Carotid artery atherosclerosis is a significant cause of stroke. Ultrasound imaging has been widely used in the diagnosis of atherosclerosis. Therefore, segmenting the atherosclerotic carotid plaque in an ultrasound image is an important task. Accurate plaque segmentation is helpful for the measurement of carotid plaque burden. This study proposes an automatic method for atherosclerotic plaque segmentation by using correntropy-based level sets (CLS) with learning-based initialization. We introduce the CLS model, containing the point-based local bias-field corrected image fitting method and correntropy-based distance measurement, to overcome the limitations of the ultrasound images. A supervised learning algorithm is employed to solve the automatic initialization problem of the variational methods. The proposed atherosclerotic plaque segmentation method is validated on 29 carotid ultrasound images, obtaining a Dice ratio of 90.6 ± 1.9% and an overlap index of 83.6 ± 3.2%. Moreover, by comparing the standard deviation of each evaluation index, it can be found that the proposed method is more robust for segmenting the atherosclerotic plaque. Our work shows that our proposed method can be more helpful than other variational models for measuring the carotid plaque burden.

HRU-Net: A Transfer Learning Method for Carotid Artery Plaque Segmentation in Ultrasound Images

Carotid artery stenotic plaque segmentation in ultrasound images is a crucial means for the analysis of plaque components and vulnerability. However, segmentation of severe stenotic plaques remains a challenging task because of the heterogeneities of inter-plaques and intra-plaques, and obscure boundaries of plaques. In this paper, we propose an automated HRU-Net transfer learning method for segmenting carotid plaques, using the limited images. The HRU-Net is based on the U-Net encoder−decoder paradigm, and cross-domain knowledge is transferred for plaque segmentation by fine-tuning the pretrained ResNet-50. Moreover, a cropped-blood-vessel image augmentation is customized for the plaque position constraint during training only. Moreover, hybrid atrous convolutions (HACs) are designed to derive diverse long-range dependences for refined plaque segmentation that are used on high-level semantic layers to exploit the implicit discrimination features. The experiments are performed on 115 images; Firstly, the 10-fold cross-validation, using 40 images with severe stenosis plaques, shows that the proposed method outperforms some of the state-of-the-art CNN-based methods on Dice, IoU, Acc, and modified Hausdorff distance (MHD) metrics; the improvements on metrics of Dice and MHD are statistically significant (p < 0.05). Furthermore, our HRU-Net transfer learning method shows fine generalization performance on 75 new images with varying degrees of plaque stenosis, and it may be used as an alternative for automatic noisy plaque segmentation in carotid ultrasound images clinically.

CSM-Net: Automatic joint segmentation of intima-media complex and lumen in carotid artery ultrasound images

The intima-media thickness (IMT) is an effective biomarker for atherosclerosis, which is commonly measured by ultrasound technique. However, the intima-media complex (IMC) segmentation for the IMT is challenging due to confused IMC boundaries and various noises. In this paper, we propose a flexible method CSM-Net for the joint segmentation of IMC and Lumen in carotid ultrasound images. Firstly, the cascaded dilated convolutions combined with the squeeze-excitation module are introduced for exploiting more contextual features on the highest-level layer of the encoder. Furthermore, a triple spatial attention module is utilized for emphasizing serviceable features on each decoder layer. Besides, a multi-scale weighted hybrid loss function is employed to resolve the class-imbalance issues. The experiments are conducted on a private dataset of 100 images for IMC and Lumen segmentation, as well as on two public datasets of 1600 images for IMC segmentation. For the private dataset, our method obtain the IMC Dice, Lumen Dice, Precision, Recall, and F1 score of 0.814 ± 0.061, 0.941 ± 0.024, 0.911 ± 0.044, 0.916 ± 0.039, and 0.913 ± 0.027, respectively. For the public datasets, we obtain the IMC Dice, Precision, Recall, and F1 score of 0.885 ± 0.067, 0.885 ± 0.070, 0.894 ± 0.089, and 0.885 ± 0.067, respectively. The results demonstrate that the proposed method precedes some cutting-edge methods, and the ablation experiments show the validity of each module. The proposed method may be useful for the IMC segmentation of carotid ultrasound images in the clinic. Our code is publicly available at https://github.com/yuanyc798/US-IMC-code.

Unsupervised domain adaptation method for segmenting cross-sectional CCA images

BACKGROUND AND OBJECTIVES

Automatic vessel segmentation in ultrasound is challenging due to the quality of the ultrasound images, which is affected by attenuation, high level of speckle noise and acoustic shadowing. Recently, deep convolutional neural networks are increasing in popularity due to their great performance on image segmentation problems, including vessel segmentation. Traditionally, large labeled datasets are required to train a network that achieves high performance, and is able to generalize well to different orientations, transducers and ultrasound scanners. However, these large datasets are rare, given that it is challenging and time-consuming to acquire and manually annotate in-vivo data.

METHODS

In this work, we present a model-based, unsupervised domain adaptation method that consists of two stages. In the first stage, the network is trained on simulated ultrasound images, which have an accurate ground truth. In the second stage, the network continues training on in-vivo data in an unsupervised way, therefore not requiring the data to be labelled. Rather than using an adversarial neural network, prior knowledge on the elliptical shape of the segmentation mask is used to detect unexpected outputs.

RESULTS

The segmentation performance was quantified using manually segmented images as ground truth. Due to the proposed domain adaptation method, the median Dice similarity coefficient increased from 0 to 0.951, outperforming a domain adversarial neural network (median Dice 0.922) and a state-of-the-art Star-Kalman algorithm that was specifically designed for this dataset (median Dice 0.942).

CONCLUSIONS

The results show that it is feasible to first train a neural network on simulated data, and then apply model-based domain adaptation to further improve segmentation performance by training on unlabeled in-vivo data. This overcomes the limitation of conventional deep learning approaches to require large amounts of manually labeled in-vivo data. Since the proposed domain adaptation method only requires prior knowledge on the shape of the segmentation mask, performance can be explored in various domains and applications in future research.

Improving Image Quality for Single-Angle Plane Wave Ultrasound Imaging With Convolutional Neural Network Beamformer

Ultrafast ultrasound imaging based on plane wave (PW) compounding has been proposed for use in various clinical and preclinical applications, including shear wave imaging and super resolution blood flow imaging. Because the image quality afforded by PW imaging is highly dependent on the number of PW angles used for compounding, a tradeoff between image quality and frame rate occurs. In the present study, a convolutional neural network (CNN) beamformer based on a combination of the GoogLeNet and U-Net architectures was developed to replace the conventional delay-and-sum (DAS) algorithm to obtain high-quality images at a high frame rate. RF channel data are used as the inputs for the CNN beamformers. The outputs are in-phase and quadrature data. Simulations and phantom experiments revealed that the images predicted by the CNN beamformers had higher resolution and contrast than those predicted by conventional single-angle PW imaging with the DAS approach. In in vivo studies, the contrast-to-noise ratios (CNRs) of carotid artery images predicted by the CNN beamformers using three or five PWs as ground truths were approximately 12 dB in the transverse view, considerably higher than the CNR obtained using the DAS beamformer (3.9 dB). Most tissue speckle information was retained in the in vivo images produced by the CNN beamformers. In conclusion, only a single PW at 0° was fired, but the quality of the output image was proximal to that of an image generated using three or five PW angles. In other words, the quality-frame rate tradeoff of coherence compounding could be mitigated through the use of the proposed CNN for beamforming.

FRDD-Net: Automated Carotid Plaque Ultrasound Images Segmentation Using Feature Remapping and Dense Decoding

Automated segmentation and evaluation of carotid plaques ultrasound images is of great significance for the diagnosis and early intervention of high-risk groups of cardiovascular and cerebrovascular diseases. However, it remains challenging to develop such solutions due to the relatively low quality of ultrasound images and heterogenous characteristics of carotid plaques. To address those problems, in this paper, we propose a novel deep convolutional neural network, FRDD-Net, with an encoder–decoder architecture to automatically segment carotid plaques. We propose the feature remapping modules (FRMs) and incorporate them into the encoding and decoding blocks to ameliorate the reliability of acquired features. We also propose a new dense decoding mechanism as part of the decoder, thus promoting the utilization efficiency of encoded features. Additionally, we construct a compound loss function to train our network to further enhance its robustness in the face of numerous cases. We train and test our network in multiple carotid plaque ultrasound datasets and our method yields the best performance compared to other state-of-the-art methods. Further ablation studies consistently show the advancement of our proposed architecture.

A Generalized Approach for Automatic 3-D Geometry Assessment of Blood Vessels in Transverse Ultrasound Images Using Convolutional Neural Networks

Accurate 3-D geometries of arteries and veins are important clinical data for diagnosis of arterial disease and intervention planning. Automatic segmentation of vessels in the transverse view suffers from the low lateral resolution and contrast. Convolutional neural networks are a promising tool for automatic segmentation of medical images, outperforming the traditional segmentation methods with high robustness. In this study, we aim to create a general, robust, and accurate method to segment the lumen-wall boundary of healthy central and peripheral vessels in large field-of-view freehand ultrasound (US) datasets. Data were acquired using the freehand US, in combination with a probe tracker. A total of ±36 000 cross-sectional images, acquired in the common, internal, and external carotid artery ( N = 37 ), in the radial, ulnar artery, and cephalic vein ( N = 12 ), and in the femoral artery ( N = 5 ) were included. To create masks (of the lumen) for training data, a conventional automatic segmentation method was used. The neural networks were trained on: 1) data of all vessels and 2) the carotid artery only. The performance was compared and tested using an open-access dataset. The recall, precision, DICE, and intersection over union (IoU) were calculated. Overall, segmentation was successful in the carotid and peripheral arteries. The Multires U-net architecture performs best overall with DICE = 0.93 when trained on the total dataset. Future studies will focus on the inclusion of vascular pathologies.

Measurement of Intima-Media Thickness Depending on Intima Media Complex Segmentation by Deep Neural Networks

Intima Media Thickness (IMT) of the carotid artery is an important marker indicating the sign of cardiovascular disease. Automated measurement of IMT requires segmentation of intima media complex (IMC).Traditional methods which use shape, color and texture for classification have poor generalization capability. This paper proposes two models- the pipeline model and the end-to-end model using Convolutional Neural Networks (CNN) and auto encoder–decoder network respectively. CNN architecture is implemented and tested by varying the number of convolutional layer, size of the kernel as well as the number of kernels. Auto encoder–decoder performs pixel wise classification using two interconnected pathways for identifying the boundary of lumen-intima (LI) and media adventitia (MA). This helps in reconstruction of the segmented portion for measurement of IMT. Both methods are tested using a dataset of 550 subjects. The results clearly indicate that end-to-end model has an edge over the pipeline model exhibiting lesser deviation between the automated measurement and the measurement made by the radiologist. The pipeline model however has better segmentation accuracy when the size of the image used for training is small. The convolutional neural network with auto encoder–decoder proves robust through sparse representation, and faster learning with better generalization. Also, the experimental setup is analyzed by interconnecting Tensor flow simulated result with Raspberry PI and the outcomes are analyzed.

Optimization of psoriasis assessment system based on patch images

Psoriasis is a chronic inflammatory skin disease that occurs in various forms throughout the body and is associated with certain conditions such as heart disease, diabetes, and depression. The psoriasis area severity index (PASI) score, a tool used to evaluate the severity of psoriasis, is currently used in clinical trials and clinical research. The determination of severity is based on the subjective judgment of the clinician. Thus, the disease evaluation deviations are induced. Therefore, we propose optimal algorithms that can effectively segment the lesion area and classify the severity. In addition, a new dataset on psoriasis was built, including patch images of erythema and scaling. We performed psoriasis lesion segmentation and classified the disease severity. In addition, we evaluated the best-performing segmentation method and classifier and analyzed features that are highly related to the severity of psoriasis. In conclusion, we presented the optimal techniques for evaluating the severity of psoriasis. Our newly constructed dataset improved the generalization performance of psoriasis diagnosis and evaluation. It proposed an optimal system for specific evaluation indicators of the disease and a quantitative PASI scoring method. The proposed system can help to evaluate the severity of localized psoriasis more accurately.

Three-dimensional ultrasound assessment of effects of therapies on carotid atherosclerosis using vessel wall thickness maps

We present a new method for assessing the effects of therapies on atherosclerosis, by measuring the weighted average of carotid vessel-wall-plus-plaque thickness change (ΔVWT¯_Weighted) in 120 patients randomized to pomegranate juice/extract versus placebo. Three-dimensional ultrasound images were acquired at baseline and one year after. Three-dimensional VWT maps were reconstructed and then projected onto a carotid template to obtain two-dimensional VWT maps. Anatomic correspondence on the two-dimensional VWT maps was optimized to reduce misalignment for the same subject and across subjects. A weight was computed at each point on the two-dimensional VWT map to highlight anatomic locations likely to exhibit plaque progression/regression, resulting in ΔVWT¯_Weighted for each subject. The weighted average of VWT-Change measured from the two-dimensional VWT maps with correspondence alignment (ΔVWT¯_Weighted,MDL) detected a significant difference between the pomegranate and placebo groups (P = 0.008). This method improves the cost-effectiveness of proof-of-concept studies involving new therapies for atherosclerosis.

Deep Learning-Based Carotid Plaque Segmentation from B-Mode Ultrasound Images

Carotid ultrasound measurement of total plaque area (TPA) provides a method for quantifying carotid plaque burden and monitoring changes in carotid atherosclerosis in response to medical treatment. Plaque boundary segmentation is required to generate the TPA measurement; however, training of observers and manual delineation are time consuming. Thus, our objective was to develop an automated plaque segmentation method to generate TPA from longitudinal carotid ultrasound images. In this study, a deep learning-based method, modified U-Net, was used to train the segmentation model and generate TPA measurement. A total of 510 plaques from 144 patients were used in our study, where the Monte Carlo cross-validation was used by randomly splitting the data set into 2/3 and 1/3 for training and testing. Two observers were trained to manually delineate the 510 plaques separately, which were used as the ground-truth references. Two U-Net models (M1 and M2) were trained using the two different ground-truth data sets from the two observers to evaluate the accuracy, variability and sensitivity on the ground-truth data sets used for training our method. The results of the algorithm segmentations of the two models yielded strong agreement with the two manual segmentations with the Pearson correlation coefficient r = 0.989 (p < 0.0001) and r = 0.987 (p < 0.0001). Comparison of the U-Net and manual segmentations resulted in mean TPA differences of 0.05 ± 7.13 mm² (95% confidence interval: 14.02-13.02 mm²) and 0.8 ± 8.7 mm² (17.85-16.25 mm²) for the two models, which are small compared with the TPA range in our data set from 4.7 to 312.8 mm². Furthermore, the mean time to segment a plaque was only 8.3 ± 3.1 ms. The presented deep learning-based method described has sufficient accuracy with a short computation time and exhibits high agreement between the algorithm and manual TPA measurements, suggesting that the method could be used to measure TPA and to monitor the progression and regression of carotid atherosclerosis.

Multilevel Strip Pooling-Based Convolutional Neural Network for the Classification of Carotid Plaque Echogenicity

Carotid plaque echogenicity in ultrasound images has been found to be closely correlated with the risk of stroke in atherosclerotic patients. The automatic and accurate classification of carotid plaque echogenicity is of great significance for clinically estimating the stability of carotid plaques and predicting cardiovascular events. Existing convolutional neural networks (CNNs) can provide an automatic carotid plaque echogenicity classification; however, they require a fixed-size input image, while the carotid plaques are of varying sizes. Although cropping and scaling the input carotid plaque images is promising, it will cause content loss or distortion and hence reduce the classification accuracy. In this study, we redesign the spatial pyramid pooling (SPP) and propose multilevel strip pooling (MSP) for the automatic and accurate classification of carotid plaque echogenicity in the longitudinal section. The proposed MSP module can accept arbitrarily sized carotid plaques as input and capture a long-range informative context to improve the accuracy of classification. In our experiments, we implement an MSP-based CNN by using the visual geometry group (VGG) network as the backbone. A total of 1463 carotid plaques (335 echo-rich plaques, 405 intermediate plaques, and 723 echolucent plaques) were collected from Zhongnan Hospital of Wuhan University. The 5-fold cross-validation results show that the proposed MSP-based VGGNet achieves a sensitivity of 92.1%, specificity of 95.6%, accuracy of 92.1%, and F1-score of 92.1%. These results demonstrate that our approach provides a way to enhance the applicability of CNN by enabling the acceptance of arbitrary input sizes and improving the classification accuracy of carotid plaque echogenicity, which has a great potential for an efficient and objective risk assessment of carotid plaques in the clinic.

Coarse-to-fine multiplanar D-SEA UNet for automatic 3D carotid segmentation in CTA images

PURPOSE

Carotid artery atherosclerotic stenosis accounts for 18-25% of ischemic stroke. In the evaluation of carotid atherosclerotic lesions, the automatic, accurate and rapid segmentation of the carotid artery is a priority issue that needs to be addressed urgently. However, the carotid artery area occupies a small target in computed tomography angiography (CTA) images, which affect the segmentation accuracy.

METHODS

We proposed a coarse-to-fine segmentation pipeline with the Multiplanar D-SEA UNet to achieve fully automatic carotid artery segmentation on the entire 3D CTA images, and compared with other four neural networks (3D-UNet, RA-UNet, Isensee-UNet, Multiplanar-UNet) by assessing Dice, Jaccard similarity coefficient, sensitivity, area under the curve and average hausdorff distance.

RESULTS

Our proposed method can achieve a mean Dice score of 91.51% on the 68 neck CTA scans from Beijing Hospital, which remarkably outperforms state-of-the-art 3D image segmentation methods. And the C2F segmentation pipeline can effectively improve segmentation accuracy while avoiding resolution loss.

CONCLUSION

The proposed segmentation method can realize the fully automatic segmentation of the carotid artery and has robust performance with segmentation accuracy, which can be applied into plaque exfoliation and interventional surgery services. In addition, our method is easy to extend to other medical segmentation tasks with appropriate parameter settings.

Carotid Ultrasound Boundary Study (CUBS): An Open Multicenter Analysis of Computerized Intima-Media Thickness Measurement Systems and Their Clinical Impact

Common carotid intima-media thickness (CIMT) is a commonly used marker for atherosclerosis and is often computed in carotid ultrasound images. An analysis of different computerized techniques for CIMT measurement and their clinical impacts on the same patient data set is lacking. Here we compared and assessed five computerized CIMT algorithms against three expert analysts' manual measurements on a data set of 1088 patients from two centers. Inter- and intra-observer variability was assessed, and the computerized CIMT values were compared with those manually obtained. The CIMT measurements were used to assess the correlation with clinical parameters, cardiovascular event prediction through a generalized linear model and the Kaplan-Meier hazard ratio. CIMT measurements obtained with a skilled analyst's segmentation and the computerized segmentation were comparable in statistical analyses, suggesting they can be used interchangeably for CIMT quantification and clinical outcome investigation. To facilitate future studies, the entire data set used is made publicly available for the community at http://dx.doi.org/10.17632/fpv535fss7.1.

Segmentation of common and internal carotid arteries from 3D ultrasound images based on adaptive triple loss

PURPOSE

Vessel wall volume (VWV) and localized vessel-wall-plus-plaque thickness (VWT) measured from three-dimensional (3D) ultrasound (US) carotid images are sensitive to anti-atherosclerotic effects of medical/dietary treatments. VWV and VWT measurements require the lumen-intima (LIB) and media-adventitia boundaries (MAB) at the common and internal carotid arteries (CCA and ICA). However, most existing segmentation techniques were capable of segmenting the CCA only. An approach capable of segmenting the MAB and LIB from the CCA and ICA was required to accelerate VWV and VWT quantification.

METHODS

Segmentation for CCA and ICA was performed independently using the proposed two-channel U-Net, which was driven by a novel loss function known as the adaptive triple Dice loss (ADTL) function. The training set was augmented by interpolating manual segmentation along the longitudinal direction, thereby taking continuity of the artery into account. A test-time augmentation (TTA) approach was applied, in which segmentation was performed three times based on the input axial images and its flipped versions; the final segmentation was generated by pixel-wise majority voting.

RESULTS

Experiments involving 224 3DUS volumes produce a Dice similarity coefficient (DSC) of 95.1% ± 4.1% and 91.6% ± 6.6% for the MAB and LIB, in the CCA, respectively, and 94.2% ± 3.3% and 89.0% ± 8.1% for the MAB and LIB, in the ICA, respectively. TTA and ATDL independently contributed to a statistically significant improvement to all boundaries except the LIB in ICA.

CONCLUSIONS

The proposed two-channel U-Net with ADTL and TTA can segment the CCA and ICA accurately and efficiently from the 3DUS volume. Our approach has the potential to accelerate the transition of 3DUS measurements of carotid atherosclerosis to clinical research.

Multimodality carotid plaque tissue characterization and classification in the artificial intelligence paradigm: a narrative review for stroke application

Cardiovascular disease (CVD) is one of the leading causes of morbidity and mortality in the United States of America and globally. Carotid arterial plaque, a cause and also a marker of such CVD, can be detected by various non-invasive imaging modalities such as magnetic resonance imaging (MRI), computer tomography (CT), and ultrasound (US). Characterization and classification of carotid plaque-type in these imaging modalities, especially into symptomatic and asymptomatic plaque, helps in the planning of carotid endarterectomy or stenting. It can be challenging to characterize plaque components due to (I) partial volume effect in magnetic resonance imaging (MRI) or (II) varying Hausdorff values in plaque regions in CT, and (III) attenuation of echoes reflected by the plaque during US causing acoustic shadowing. Artificial intelligence (AI) methods have become an indispensable part of healthcare and their applications to the non-invasive imaging technologies such as MRI, CT, and the US. In this narrative review, three main types of AI models (machine learning, deep learning, and transfer learning) are analyzed when applied to MRI, CT, and the US. A link between carotid plaque characteristics and the risk of coronary artery disease is presented. With regard to characterization, we review tools and techniques that use AI models to distinguish carotid plaque types based on signal processing and feature strengths. We conclude that AI-based solutions offer an accurate and robust path for tissue characterization and classification for carotid artery plaque imaging in all three imaging modalities. Due to cost, user-friendliness, and clinical effectiveness, AI in the US has dominated the most.

Towards Clinical Application of Artificial Intelligence in Ultrasound Imaging

Artificial intelligence (AI) is being increasingly adopted in medical research and applications. Medical AI devices have continuously been approved by the Food and Drug Administration in the United States and the responsible institutions of other countries. Ultrasound (US) imaging is commonly used in an extensive range of medical fields. However, AI-based US imaging analysis and its clinical implementation have not progressed steadily compared to other medical imaging modalities. The characteristic issues of US imaging owing to its manual operation and acoustic shadows cause difficulties in image quality control. In this review, we would like to introduce the global trends of medical AI research in US imaging from both clinical and basic perspectives. We also discuss US image preprocessing, ingenious algorithms that are suitable for US imaging analysis, AI explainability for obtaining informed consent, the approval process of medical AI devices, and future perspectives towards the clinical application of AI-based US diagnostic support technologies.

Leveraging Machine Learning and Artificial Intelligence to Improve Peripheral Artery Disease Detection, Treatment, and Outcomes

Peripheral artery disease is an atherosclerotic disorder which, when present, portends poor patient outcomes. Low diagnosis rates perpetuate poor management, leading to limb loss and excess rates of cardiovascular morbidity and death. Machine learning algorithms and artificially intelligent systems have shown great promise in application to many areas in health care, such as accurately detecting disease, predicting patient outcomes, and automating image interpretation. Although the application of these technologies to peripheral artery disease are in their infancy, their promises are tremendous. In this review, we provide an introduction to important concepts in the fields of machine learning and artificial intelligence, detail the current state of how these technologies have been applied to peripheral artery disease, and discuss potential areas for future care enhancement with advanced analytics.

Artificial intelligence in ultrasound

Ultrasound (US), a flexible green imaging modality, is expanding globally as a first-line imaging technique in various clinical fields following with the continual emergence of advanced ultrasonic technologies and the well-established US-based digital health system. Actually, in US practice, qualified physicians should manually collect and visually evaluate images for the detection, identification and monitoring of diseases. The diagnostic performance is inevitably reduced due to the intrinsic property of high operator-dependence from US. In contrast, artificial intelligence (AI) excels at automatically recognizing complex patterns and providing quantitative assessment for imaging data, showing high potential to assist physicians in acquiring more accurate and reproducible results. In this article, we will provide a general understanding of AI, machine learning (ML) and deep learning (DL) technologies; We then review the rapidly growing applications of AI-especially DL technology in the field of US-based on the following anatomical regions: thyroid, breast, abdomen and pelvis, obstetrics heart and blood vessels, musculoskeletal system and other organs by covering image quality control, anatomy localization, object detection, lesion segmentation, and computer-aided diagnosis and prognosis evaluation; Finally, we offer our perspective on the challenges and opportunities for the clinical practice of biomedical AI systems in US.

Ensembled deep learning model outperforms human experts in diagnosing biliary atresia from sonographic gallbladder images

It is still challenging to make accurate diagnosis of biliary atresia (BA) with sonographic gallbladder images particularly in rural area without relevant expertise. To help diagnose BA based on sonographic gallbladder images, an ensembled deep learning model is developed. The model yields a patient-level sensitivity 93.1% and specificity 93.9% [with areas under the receiver operating characteristic curve of 0.956 (95% confidence interval: 0.928-0.977)] on the multi-center external validation dataset, superior to that of human experts. With the help of the model, the performances of human experts with various levels are improved. Moreover, the diagnosis based on smartphone photos of sonographic gallbladder images through a smartphone app and based on video sequences by the model still yields expert-level performances. The ensembled deep learning model in this study provides a solution to help radiologists improve the diagnosis of BA in various clinical application scenarios, particularly in rural and undeveloped regions with limited expertise.

Cascaded residual U-net for fully automatic segmentation of 3D carotid artery in high-resolution multi-contrast MR images

Accurate and automatic carotid artery segmentation for magnetic resonance (MR) images is eagerly expected, which can greatly assist a comprehensive study of atherosclerosis and accelerate the translation. Although many efforts have been made, identification of the inner lumen and outer wall in diseased vessels is still a challenging task due to complex vascular deformation, blurred wall boundary, and confusing componential expression. In this paper, we introduce a novel fully automatic 3D framework for simultaneously segmenting the carotid artery from high-resolution multi-contrast MR sequences based on deep learning. First, an optimal channel fitting structure is designed for identity mapping, and a novel 3D residual U-net is used as a basic network. Second, high-resolution MR images are trained using both patch-level and global-level strategies, and the two pre-segmentation results are optimized based on structural characteristics. Third, the optimized pre-segmentation results are cascaded with the patch-cropped MR volume data and trained to segment the carotid lumen and wall. Extensive experiments demonstrate the proposed method outperforms the state-of-the-art 3D Unet-based segmentation models.

Deep learning-based digitization of prostate brachytherapy needles in ultrasound images

PURPOSE

To develop, and evaluate the performance of, a deep learning-based three-dimensional (3D) convolutional neural network (CNN) artificial intelligence (AI) algorithm aimed at finding needles in ultrasound images used in prostate brachytherapy.

METHODS

Transrectal ultrasound (TRUS) image volumes from 1102 treatments were used to create a clinical ground truth (CGT) including 24422 individual needles that had been manually digitized by medical physicists during brachytherapy procedures. A 3D CNN U-net with 128 × 128 × 128 TRUS image volumes as input was trained using 17215 needle examples. Predictions of voxels constituting a needle were combined to yield a 3D linear function describing the localization of each needle in a TRUS volume. Manual and AI digitizations were compared in terms of the root-mean-square distance (RMSD) along each needle, expressed as median and interquartile range (IQR). The method was evaluated on a data set including 7207 needle examples. A subgroup of the evaluation data set (n = 188) was created, where the needles were digitized once more by a medical physicist (G1) trained in brachytherapy. The digitization procedure was timed.

RESULTS

The RMSD between the AI and CGT was 0.55 (IQR: 0.35-0.86) mm. In the smaller subset, the RMSD between AI and CGT was similar (0.52 [IQR: 0.33-0.79] mm) but significantly smaller (P < 0.001) than the difference of 0.75 (IQR: 0.49-1.20) mm between AI and G1. The difference between CGT and G1 was 0.80 (IQR: 0.48-1.18) mm, implying that the AI performed as well as the CGT in relation to G1. The mean time needed for human digitization was 10 min 11 sec, while the time needed for the AI was negligible.

CONCLUSIONS

A 3D CNN can be trained to identify needles in TRUS images. The performance of the network was similar to that of a medical physicist trained in brachytherapy. Incorporating a CNN for needle identification can shorten brachytherapy treatment procedures substantially.

Segmentation of 3D ultrasound carotid vessel wall using U-Net and segmentation average network

Segmentation of carotid vessel wall is required in vessel wall volume (VWV) and local vessel-wall-plus-plaque thickness (VWT) quantification of the carotid artery. Manual segmentation of the vessel wall is time-consuming and prone to interobserver variability. In this paper, we proposed a convolutional neural network (CNN) to segment the common carotid artery (CCA) from 3D carotid ultrasound images. The proposed CNN involves three U-Nets that segmented the 3D ultrasound (3DUS) images in the axial, lateral and frontal orientations. The segmentation maps generated by three U-Nets were consolidated by a novel segmentation average network (SAN) we proposed in this paper. The experimental results show that the proposed CNN improved the segmentation accuracies. Compared to only using U-Net alone, the proposed CNN improved the Dice similarity coefficient (DSC) for vessel wall segmentation from 64.8% to 67.5%, the sensitivity from 63.8% to 70.5%, and the area under receiver operator characteristic curve (AUC) from 0.89 to 0.94.

Ultrasound Deep Learning for Wall Segmentation and Near-Wall Blood Flow Measurement

Studies of medical flow imaging have technical limitations for accurate analysis of blood flow dynamics and vessel wall interaction at arteries. We propose a new deep learning-based boundary detection and compensation (DL-BDC) technique in ultrasound (US) imaging. It can segment vessel boundaries by harnessing the convolutional neural network and wall motion compensation in the analysis of near-wall flow dynamics. The network enables training from real and synthetic US images together. The performance of the technique is validated through synthetic US images and tissue-mimicking phantom experiments. The neural network performs well with high Dice coefficients of over 0.94 and 0.9 for lumens and walls, outperforming previous segmentation techniques. Then, the performance of the wall motion compensation is examined for compliant phantoms. When DL-BDC is applied to flow influenced by wall motion, root-mean-square errors are less than 0.07%. The technique is utilized to analyze flow dynamics and wall interaction with varying elastic moduli of the phantoms. The results show that the flow dynamics and wall shear stress values are consistent with the expected values of the compliant phantoms, and their wall motion behavior is observed with pulse wave propagation. This strategy makes US imaging capable of simultaneous measurement of blood flow and vessel dynamics in human arteries for their accurate interaction analysis. DL-BDC can segment vessel walls fast, accurately, and robustly. It enables to measure the near-wall flow precisely by determining the vessel boundary dynamics. This approach can be beneficial in flow dynamics and wall interaction analyses in various biomedical applications.

Measurement of carotid plaque burden

PURPOSE OF REVIEW

To describe the uses of measurement of carotid plaque burden, as total plaque area (TPA), total plaque volume (TPV), and vessel wall volume (VWV), which includes plaque burden and wall volume.

RECENT FINDINGS

Measurement of plaque burden is useful for risk stratification, research into the genetics and biology of atherosclerosis, for measuring effects of new therapies for atherosclerosis, and for treatment of high-risk patients with severe atherosclerosis.

SUMMARY

Measurement of plaque burden is far superior to measurement of carotid intima-media thickness (IMT) in many ways, and should replace it. Vessel wall volume can be measured in persons with no plaque as an alternative to IMT.

A Voxel-Based Fully Convolution Network and Continuous Max-Flow for Carotid Vessel-Wall-Volume Segmentation From 3D Ultrasound Images

Vessel-wall-volume (VWV) is an important three-dimensional ultrasound (3DUS) metric used in the assessment of carotid plaque burden and monitoring changes in carotid atherosclerosis in response to medical treatment. To generate the VWV measurement, we proposed an approach that combined a voxel-based fully convolution network (Voxel-FCN) and a continuous max-flow module to automatically segment the carotid media-adventitia (MAB) and lumen-intima boundaries (LIB) from 3DUS images. Voxel-FCN includes an encoder consisting of a general 3D CNN and a 3D pyramid pooling module to extract spatial and contextual information, and a decoder using a concatenating module with an attention mechanism to fuse multi-level features extracted by the encoder. A continuous max-flow algorithm is used to improve the coarse segmentation provided by the Voxel-FCN. Using 1007 3DUS images, our approach yielded a Dice-similarity-coefficient (DSC) of 93.2±3.0% for the MAB in the common carotid artery (CCA), and 91.9±5.0% in the bifurcation by comparing algorithm and expert manual segmentations. We achieved a DSC of 89.5±6.7% and 89.3±6.8% for the LIB in the CCA and the bifurcation respectively. The mean errors between the algorithm-and manually-generated VWVs were 0.2±51.2 mm³ for the CCA and -4.0±98.2 mm³ for the bifurcation. The algorithm segmentation accuracy was comparable to intra-observer manual segmentation but our approach required less than 1s, which will not alter the clinical work-flow as 10s is required to image one side of the neck. Therefore, we believe that the proposed method could be used clinically for generating VWV to monitor progression and regression of carotid plaques.

Deep Learning for Carotid Plaque Segmentation using a Dilated U-Net Architecture

Carotid plaque segmentation in ultrasound longitudinal B-mode images using deep learning is presented in this work. We report on 101 severely stenotic carotid plaque patients. A standard U-Net is compared with a dilated U-Net architecture in which the dilated convolution layers were used in the bottleneck. Both a fully automatic and a semi-automatic approach with a bounding box was implemented. The performance degradation in plaque segmentation due to errors in the bounding box is quantified. We found that the bounding box significantly improved the performance of the networks with U-Net Dice coefficients of 0.48 for automatic and 0.83 for semi-automatic segmentation of plaque. Similar results were also obtained for the dilated U-Net with Dice coefficients of 0.55 for automatic and 0.84 for semi-automatic when compared to manual segmentations of the same plaque by an experienced sonographer. A 5% error in the bounding box in both dimensions reduced the Dice coefficient to 0.79 and 0.80 for U-Net and dilated U-Net respectively.

Using a distance map and an active contour model to segment the carotid artery boundary from the lumen contour in proton density weighted magnetic resonance images

Segmentation methods have assumed an important role in image-based diagnosis of several cardiovascular diseases. Particularly, the segmentation of the boundary of the carotid artery is demanded in the detection and characterization of atherosclerosis and assessment of the disease progression. In this article, a fully automatic approach for the segmentation of the carotid artery boundary in Proton Density Weighted Magnetic Resonance Images is presented. The approach relies on the expansion of the lumen contour based on a distance map built using the gray-weighted distance relative to the center of the identified lumen region in the image under analysis. Then, a Snake model with a modified weighted external energy based on the combination of a balloon force along with a Gradient Vector Flow-based external energy is applied to the expanded contour towards the correct boundary of the carotid artery. The average values of the Dice coefficient, Polyline distance, mean contour distance and centroid distance found in the segmentation of 139 carotid arteries were 0.83 ± 0.11, 2.70 ± 1.69 pixels, 2.79 ± 1.89 pixels and 3.44 ± 2.82 pixels, respectively. The segmentation results of the proposed approach were also compared against the ones obtained by related approaches found in the literature, which confirmed the outstanding performance of the new approach. Additionally, the proposed weighted external energy for the Snake model was shown to be also robust to carotid arteries with large thickness and weak boundary image edges.

Bimodal Automated Carotid Ultrasound Segmentation Using Geometrically Constrained Deep Neural Networks

For asymptomatic patients suffering from carotid stenosis, the assessment of plaque morphology is an important clinical task which allows monitoring of the risk of plaque rupture and future incidents of stroke. Ultrasound Imaging provides a safe and non-invasive modality for this, and the segmentation of media-adventitia boundaries and lumen-intima boundaries of the Carotid artery form an essential part in this monitoring process. In this paper, we propose a novel Deep Neural Network as a fully automated segmentation tool, and its application in delineating both the media-adventitia boundary and the lumen-intima boundary. We develop a new geometrically constrained objective function as part of the Network's Stochastic Gradient Descent optimisation, thus tuning it to the problem at hand. Furthermore, we also apply a bimodal fusion of amplitude and phase congruency data proposed by us in previous work, as an input to the network, as the latter provides an intensity-invariant data source to the network. We finally report the segmentation performance of the network on transverse sections of the carotid. Tests are carried out on an augmented dataset of 81,000 images, and the results are compared to other studies by reporting the DICE coefficient of similarity, modified Hausdorff Distance, sensitivity and specificity. Our proposed modification is shown to yield improved results on the standard network over this larger dataset, with the advantage of it being fully automated. We conclude that Deep Neural Networks provide a reliable trained manner in which carotid ultrasound images may be automatically segmented, using amplitude data and intensity invariant phase congruency maps as a data source.

Deep morphology aided diagnosis network for segmentation of carotid artery vessel wall and diagnosis of carotid atherosclerosis on black-blood vessel wall MRI

PURPOSE

Early detection of carotid atherosclerosis on the vessel wall (VW) magnetic resonance imaging (MRI) (VW-MRI) images can prevent the progression of cardiovascular disease. However, the manual inspection process of the VW-MRI images is cumbersome and has low reproducibility. Therefore in this paper, by using the convolutional neural networks (CNNs), we develop a deep morphology aided diagnosis (DeepMAD) network for automated segmentation of the VW of carotid artery and for automated diagnosis of the carotid atherosclerosis with the black-blood (BB) VW-MRI (i.e., the T1-weighted MRI) in a slice-by-slice manner.

METHODS

The proposed DeepMAD network consists of a segmentation subnetwork and a diagnosis subnetwork for performing the segmentation and diagnosis tasks on the BB-VW-MRI images, where the manual labeled lumen area, the manual labeled outer wall area and the manual labeled lesion Types based on the modified American Heart Association (AHA) criteria are used as the ground-truth. Specifically, a deep U-shape CNN with a weighted fusion layer is designed as the segmentation subnetwork, where the lumen area and the outer wall area can be simultaneously segmented under the supervision of the triple Dice loss to provide the vessel wall map as morphological information. Then, the image stream from the BB-VWMRI image and the morphology stream from the obtained vessel wall map are extracted from two deep CNNs and combined to obtain the diagnosis results of atherosclerosis in the diagnosis subnetwork. In addition, the triple input set is formed by three carotid regions of interest (ROIs) from three consecutive slices of the MRI sequence and input to the DeepMAD network, where the first and last slices used as additional adjacent slices to provide 2.5D spatial information along the carotid artery centerline for the intermediate slice, which is the target slice for segmentation and diagnosis in the study.

RESULTS

Compared to other existing methods, the DeepMAD network can achieve promising segmentation performances (0.9594 Dice for the lumen and 0.9657 Dice for the outer wall) and better diagnosis Accuracy of the carotid atherosclerosis (0.9503 AUC and 0.8916 Accuracy) in the test dataset (including invisible subjects) from same source as the training dataset. In addition, the trained DeepMAD model can be successfully transferred to another test dataset for segmentation and diagnosis tasks with remarkable performance (0.9475 Dice for the lumen and 0.9542 Dice for the outer wall, 0. 9227 AUC and 0.8679 Accuracy for diagnosis).

CONCLUSIONS

Even without the intervention of reviewers required for previous works, the proposed DeepMAD network automatically segments the lumen and the outer wall together and diagnoses the carotid atherosclerosis with high performances. The DeepMAD network can be used in clinical trials to help radiologists get rid of tedious reading tasks, such as screening review to separate the normal carotid from the atherosclerotic arteries and outlining the vessel wall contours.