Sensitive three-dimensional ultrasound assessment of carotid atherosclerosis by weighted average of local vessel wall and plaque thickness change

Unsupervised shape-and-texture-based generative adversarial tuning of pre-trained networks for carotid segmentation from 3D ultrasound images

BACKGROUND

Vessel-wall volume and localized three-dimensional ultrasound (3DUS) metrics are sensitive to the change of carotid atherosclerosis in response to medical/dietary interventions. Manual segmentation of the media-adventitia boundary (MAB) and lumen-intima boundary (LIB) required to obtain these metrics is time-consuming and prone to observer variability. Although supervised deep-learning segmentation models have been proposed, training of these models requires a sizeable manually segmented training set, making larger clinical studies prohibitive.

PURPOSE

We aim to develop a method to optimize pre-trained segmentation models without requiring manual segmentation to supervise the fine-tuning process.

METHODS

We developed an adversarial framework called the unsupervised shape-and-texture generative adversarial network (USTGAN) to fine-tune a convolutional neural network (CNN) pre-trained on a source dataset for accurate segmentation of a target dataset. The network integrates a novel texture-based discriminator with a shape-based discriminator, which together provide feedback for the CNN to segment the target images in a similar way as the source images. The texture-based discriminator increases the accuracy of the CNN in locating the artery, thereby lowering the number of failed segmentations. Failed segmentation was further reduced by a self-checking mechanism to flag longitudinal discontinuity of the artery and by self-correction strategies involving surface interpolation followed by a case-specific tuning of the CNN. The U-Net was pre-trained by the source dataset involving 224 3DUS volumes with 136, 44, and 44 volumes in the training, validation and testing sets. The training of USTGAN involved the same training group of 136 volumes in the source dataset and 533 volumes in the target dataset. No segmented boundaries for the target cohort were available for training USTGAN. The validation and testing of USTGAN involved 118 and 104 volumes from the target cohort, respectively. The segmentation accuracy was quantified by Dice Similarity Coefficient (DSC), and incorrect localization rate (ILR). Tukey's Honestly Significant Difference multiple comparison test was employed to quantify the difference of DSCs between models and settings, wherep ≤ 0.05 $p\,\le \,0.05$ was considered statistically significant.

RESULTS

USTGAN attained a DSC of85.7 ± 13.0 $85.7\,\pm \,13.0$ % in LIB and86.2 ± 10.6 ${86.2}\,\pm \,{10.6}$ % in MAB, improving from the baseline performance of74.6 ± 30.7 ${74.6}\,\pm \,{30.7}$ % in LIB (p< 10 - 12 $<10^{-12}$ ) and75.7 ± 28.9 ${75.7}\,\pm \,{28.9}$ % in MAB (p< 10 - 12 $<10^{-12}$ ). Our approach outperformed six state-of-the-art domain-adaptation models (MAB:p ≤ 3.63 × 10 - 7 $p \le 3.63\,\times \,10^{-7}$ , LIB:p ≤ 9.34 × 10 - 8 $p\,\le \,9.34\,\times \,10^{-8}$ ). The proposed USTGAN also had the lowest ILR among the methods compared (LIB: 2.5%, MAB: 1.7%).

CONCLUSION

Our framework improves segmentation generalizability, thereby facilitating efficient carotid disease monitoring in multicenter trials and in clinics with less expertise in 3DUS imaging.

Development of a Three-Dimensional Carotid Ultrasound Image Segmentation Workflow for Improved Efficiency, Reproducibility and Accuracy in Measuring Vessel Wall and Plaque Volume and Thickness

Automated segmentation of carotid lumen-intima boundary (LIB) and media-adventitia boundary (MAB) by deep convolutional neural networks (CNN) from three-dimensional ultrasound (3DUS) images has made assessment and monitoring of carotid atherosclerosis more efficient than manual segmentation. However, training of CNN still requires manual segmentation of LIB and MAB. Therefore, there is a need to improve the efficiency of manual segmentation and develop strategies to improve segmentation accuracy by the CNN for serial monitoring of carotid atherosclerosis. One strategy to reduce segmentation time is to increase the interslice distance (ISD) between segmented axial slices of a 3DUS image while maintaining the segmentation reliability. We, for the first time, investigated the effect of ISD on the reproducibility of MAB and LIB segmentations. The intra-observer reproducibility of LIB and MAB segmentations at ISDs of 1 mm and 2 mm was not statistically significantly different, whereas the reproducibility at ISD = 3 mm was statistically lower. Therefore, we conclude that segmentation with an ISD of 2 mm provides sufficient reliability for CNN training. We further proposed training the CNN by the baseline images of the entire cohort of patients for automatic segmentation of the follow-up images acquired for the same cohort. We validated that segmentation with this time-based partitioning approach is more accurate than that produced by patient-based partitioning, especially at the carotid bifurcation. This study forms the basis for an efficient, reproducible, and accurate 3DUS workflow for serial monitoring of carotid atherosclerosis useful in risk stratification of cardiovascular events and in evaluating the efficacy of new treatments.

A Dual-Stream Centerline-Guided Network for Segmentation of the Common and Internal Carotid Arteries From 3D Ultrasound Images

Segmentation of the carotid section encompassing the common carotid artery (CCA), the bifurcation and the internal carotid artery (ICA) from three-dimensional ultrasound (3DUS) is required to measure the vessel wall volume (VWV) and localized vessel-wall-plus-plaque thickness (VWT), shown to be sensitive to treatment effect. We proposed an approach to combine a centerline extraction network (CHG-Net) and a dual-stream centerline-guided network (DSCG-Net) to segment the lumen-intima (LIB) and media-adventitia boundaries (MAB) from 3DUS images. Correct arterial location is essential for successful segmentation of the carotid section encompassing the bifurcation. We addressed this challenge by using the arterial centerline to enhance the localization accuracy of the segmentation network. The CHG-Net was developed to generate a heatmap indicating high probability regions for the centerline location, which was then integrated with the 3DUS image by the DSCG-Net to generate the MAB and LIB. The DSCG-Net includes a scale-based and a spatial attention mechanism to fuse multi-level features extracted by the encoder, and a centerline heatmap reconstruction side-branch connected to the end of the encoder to increase the generalization ability of the network. Experiments involving 224 3DUS volumes produce a Dice similarity coefficient (DSC) of 95.8±1.9% and 92.3±5.4% for CCA MAB and LIB, respectively, and 93.2±4.4% and 89.0±10.0% for ICA MAB and LIB, respectively. Our approach outperformed four state-of-the-art 3D CNN models, even after their performances were boosted by centerline guidance. The efficiency afforded by the framework would allow it to be incorporated into the clinical workflow for improved quantification of plaque change.

Quantification of Local Vessel Wall and Plaque Volume Change for Assessment of Effects of Therapies on Carotid Atherosclerosis Based on 3-D Ultrasound Imaging

We developed a new method to measure the voxel-based vessel-wall-plus-plaque volume (VWV). In addition to quantifying local thickness change as in the previously introduced vessel-wall-plus-plaque thickness (VWT) metric, voxel-based VWV further considers the circumferential change associated with vascular remodeling. Three-dimensional ultrasound images were acquired at baseline and 1 y afterward. The vessel wall region was divided into small voxels with the voxel-based VWV change (ΔVVol%) computed by taking the percentage volume difference between corresponding voxels in the baseline and follow-up images. A 3-D carotid atlas was developed to allow visualization of the local thickness and circumferential change patterns in the pomegranate versus the placebo groups. A new patient-based biomarker was obtained by computing the mean ΔVVol% over the entire 3-D map for each patient (ΔVVol%¯). ΔVVol%¯ detected a significant difference between patients randomized to pomegranate juice/extract and placebo groups (p = 0.0002). The number of patients required by ΔVVol%¯ to establish statistical significance was approximately a third of that required by the local VWT biomarker. The increased sensitivity afforded by the proposed biomarker improves the cost-effectiveness of clinical studies evaluating new anti-atherosclerotic treatments.

Recent advances in vascular ultrasound imaging technology and their clinical implications

In this paper recent advances in vascular ultrasound imaging technology are discussed, including three-dimensional ultrasound (3DUS), contrast-enhanced ultrasound (CEUS) and strain- (SE) and shear-wave-elastography (SWE). 3DUS imaging allows visualisation of the actual 3D anatomy and more recently of flow, and assessment of geometrical, morphological and mechanical features in the carotid artery and the aorta. CEUS involves the use of microbubble contrast agents to estimate sensitive blood flow and neovascularisation (formation of new microvessels). Recent developments include the implementation of computerised tools for automated analysis and quantification of CEUS images, and the possibility to measure blood flow velocity in the aorta. SE, which yields anatomical maps of tissue strain, is increasingly being used to investigate the vulnerability of the carotid plaque, but is also promising for the coronary artery and the aorta. SWE relies on the generation of a shear wave by remote acoustic palpation and its acquisition by ultrafast imaging, and is useful for measuring arterial stiffness. Such advances in vascular ultrasound technology, with appropriate validation in clinical trials, could positively change current management of patients with vascular disease, and improve stratification of cardiovascular risk.

Three-dimensional ultrasound assessment of effects of therapies on carotid atherosclerosis using vessel wall thickness maps

We present a new method for assessing the effects of therapies on atherosclerosis, by measuring the weighted average of carotid vessel-wall-plus-plaque thickness change (ΔVWT¯_Weighted) in 120 patients randomized to pomegranate juice/extract versus placebo. Three-dimensional ultrasound images were acquired at baseline and one year after. Three-dimensional VWT maps were reconstructed and then projected onto a carotid template to obtain two-dimensional VWT maps. Anatomic correspondence on the two-dimensional VWT maps was optimized to reduce misalignment for the same subject and across subjects. A weight was computed at each point on the two-dimensional VWT map to highlight anatomic locations likely to exhibit plaque progression/regression, resulting in ΔVWT¯_Weighted for each subject. The weighted average of VWT-Change measured from the two-dimensional VWT maps with correspondence alignment (ΔVWT¯_Weighted,MDL) detected a significant difference between the pomegranate and placebo groups (P = 0.008). This method improves the cost-effectiveness of proof-of-concept studies involving new therapies for atherosclerosis.

Segmentation of common and internal carotid arteries from 3D ultrasound images based on adaptive triple loss

PURPOSE

Vessel wall volume (VWV) and localized vessel-wall-plus-plaque thickness (VWT) measured from three-dimensional (3D) ultrasound (US) carotid images are sensitive to anti-atherosclerotic effects of medical/dietary treatments. VWV and VWT measurements require the lumen-intima (LIB) and media-adventitia boundaries (MAB) at the common and internal carotid arteries (CCA and ICA). However, most existing segmentation techniques were capable of segmenting the CCA only. An approach capable of segmenting the MAB and LIB from the CCA and ICA was required to accelerate VWV and VWT quantification.

METHODS

Segmentation for CCA and ICA was performed independently using the proposed two-channel U-Net, which was driven by a novel loss function known as the adaptive triple Dice loss (ADTL) function. The training set was augmented by interpolating manual segmentation along the longitudinal direction, thereby taking continuity of the artery into account. A test-time augmentation (TTA) approach was applied, in which segmentation was performed three times based on the input axial images and its flipped versions; the final segmentation was generated by pixel-wise majority voting.

RESULTS

Experiments involving 224 3DUS volumes produce a Dice similarity coefficient (DSC) of 95.1% ± 4.1% and 91.6% ± 6.6% for the MAB and LIB, in the CCA, respectively, and 94.2% ± 3.3% and 89.0% ± 8.1% for the MAB and LIB, in the ICA, respectively. TTA and ATDL independently contributed to a statistically significant improvement to all boundaries except the LIB in ICA.

CONCLUSIONS

The proposed two-channel U-Net with ADTL and TTA can segment the CCA and ICA accurately and efficiently from the 3DUS volume. Our approach has the potential to accelerate the transition of 3DUS measurements of carotid atherosclerosis to clinical research.

Segmentation of 3D ultrasound carotid vessel wall using U-Net and segmentation average network

Segmentation of carotid vessel wall is required in vessel wall volume (VWV) and local vessel-wall-plus-plaque thickness (VWT) quantification of the carotid artery. Manual segmentation of the vessel wall is time-consuming and prone to interobserver variability. In this paper, we proposed a convolutional neural network (CNN) to segment the common carotid artery (CCA) from 3D carotid ultrasound images. The proposed CNN involves three U-Nets that segmented the 3D ultrasound (3DUS) images in the axial, lateral and frontal orientations. The segmentation maps generated by three U-Nets were consolidated by a novel segmentation average network (SAN) we proposed in this paper. The experimental results show that the proposed CNN improved the segmentation accuracies. Compared to only using U-Net alone, the proposed CNN improved the Dice similarity coefficient (DSC) for vessel wall segmentation from 64.8% to 67.5%, the sensitivity from 63.8% to 70.5%, and the area under receiver operator characteristic curve (AUC) from 0.89 to 0.94.

Area-Preserving Mapping of 3D Carotid Ultrasound Images Using Density-Equalizing Reference Map

Atherosclerotic plaques are focal and tend to occur at arterial bends and bifurcations. To quantitatively monitor the local changes in the carotid vessel-wall-plus-plaque thickness (VWT) and compare the VWT distributions for different patients or for the same patients at different ultrasound scanning sessions, a mapping technique is required to adjust for the geometric variability of different carotid artery models. In this work, we propose a novel method called density-equalizing reference map (DERM) for mapping 3D carotid surfaces to a standardized 2D carotid template, with an emphasis on preserving the local geometry of the carotid surface by minimizing the local area distortion. The initial map was generated by a previously described arc-length scaling (ALS) mapping method, which projects a 3D carotid surface onto a 2D non-convex L-shaped domain. A smooth and area-preserving flattened map was subsequently constructed by deforming the ALS map using the proposed algorithm that combines the density-equalizing map and the reference map techniques. This combination allows, for the first time, one-to-one mapping from a 3D surface to a standardized non-convex planar domain in an area-preserving manner. Evaluations using 20 carotid surface models show that the proposed method reduced the area distortion of the flattening maps by over 80% as compared to the ALS mapping method. The proposed method is capable of improving the accuracy of area estimation for plaque regions without compromising inter-scan reproducibility.

Deep learning-based carotid media-adventitia and lumen-intima boundary segmentation from three-dimensional ultrasound images

Purpose

Quantification of carotid plaques has been shown to be important for assessing as well as monitoring the progression and regression of carotid atherosclerosis. Various metrics have been proposed and methods of measurements ranging from manual tracing to automated segmentations have also been investigated. Of those metrics, quantification of carotid plaques by measuring vessel‐wall‐volume (VWV) using the segmented media‐adventitia (MAB) and lumen‐intima (LIB) boundaries has been shown to be sensitive to temporal changes in carotid plaque burden. Thus, semi‐automatic MAB and LIB segmentation methods are required to help generate VWV measurements with high accuracy and less user interaction.

Methods

In this paper, we propose a semiautomatic segmentation method based on deep learning to segment the MAB and LIB from carotid three‐dimensional ultrasound (3DUS) images. For the MAB segmentation, we convert the segmentation problem to a pixel‐by‐pixel classification problem. A dynamic convolutional neural network (Dynamic CNN) is proposed to classify the patches generated by sliding a window along the norm line of the initial contour where the CNN model is fine‐tuned dynamically in each test task. The LIB is segmented by applying a region‐of‐interest of carotid images to a U‐Net model, which allows the network to be trained end‐to‐end for pixel‐wise classification.

Results

A total of 144 3DUS images were used in this development, and a threefold cross‐validation technique was used for evaluation of the proposed algorithm. The proposed algorithm‐generated accuracy was significantly higher than the previous methods but with less user interactions. Comparing the algorithm segmentation results with manual segmentations by an expert showed that the average Dice similarity coefficients (DSC) were 96.46 ± 2.22% and 92.84 ± 4.46% for the MAB and LIB, respectively, while only an average of 34 s (vs 1.13, 2.8 and 4.4 min in previous methods) was required to segment a 3DUS image. The interobserver experiment indicated that the DSC was 96.14 ± 1.87% between algorithm‐generated MAB contours of two observers' initialization.

Conclusions

Our results showed that the proposed carotid plaque segmentation method obtains high accuracy and repeatability with less user interactions, suggesting that the method could be used in clinical practice to measure VWV and monitor the progression and regression of carotid plaques.

Breast lesion classification based on supersonic shear-wave elastography and automated lesion segmentation from B-mode ultrasound images

Supersonic shear-wave elastography (SWE) has emerged as a useful imaging modality for breast lesion assessment. Regions of interest (ROIs) were required to be specified for extracting features that characterize malignancy of lesions. Although analyses have been performed in small rectangular ROIs identified manually by expert observers, the results were subject to observer variability and the analysis of small ROIs would potentially miss out important features available in other parts of the lesion. Recent investigations extracted features from the entire lesion segmented by B-mode ultrasound images either manually or semi-automatically, but lesion delineation using existing techniques is time-consuming and prone to variability as intensive user interactions are required. In addition, rich diagnostic features were available along the rim surrounding the lesion. The width of the rim analyzed was subjectively and empirically determined by expert observers in previous studies after intensive visual study on the images, which is time-consuming and susceptible to observer variability. This paper describes an analysis pipeline to segment and classify lesions efficiently. The lesion boundary was first initialized and then deformed based on energy fields generated by the dyadic wavelet transform. Features of the SWE images were extracted from inside and outside of a lesion for different widths of the surrounding rim. Then, feature selection was performed followed by the Support Vector Machine (SVM) classification. This strategy obviates the empirical and time-consuming selection of the surrounding rim width before the analysis. The pipeline was evaluated on 137 lesions. Feature selection was performed 20 times using different sets of 14 lesions (7 malignant, 7 benign). Leave-one-out SVM classification was performed in each of the 20 experiments with a mean sensitivity, specificity and accuracy of 95.1%, 94.6% and 94.8% respectively. The pipeline took an average of 20 s to process a lesion. The fact that this efficient pipeline generated classification accuracy superior to that of existing algorithms suggests that improved efficiency did not compromise classification accuracy. The ability to streamline the quantitative assessment of SWE images will potentially accelerate the adoption of the combined use of ultrasound and elastography in clinical practice.