Automated Segmentation of Microvessels in Intravascular OCT Images Using Deep Learning

A Robust Blood Vessel Segmentation Technique for Angiographic Images Employing Multi-Scale Filtering Approach

Background: This study focuses on the critical task of blood vessel segmentation in medical image analysis, essential for diagnosing cardiovascular diseases and enabling effective treatment planning. Although deep learning architectures often produce very high segmentation results in medical images, coronary computed tomography angiography (CTA) images are more challenging than invasive coronary angiography (ICA) images due to noise and the complexity of vessel structures. Methods: Classical architectures for medical images, such as U-Net, achieve only moderate accuracy, with an average Dice score of 0.722. Results: This study introduces Morpho-U-Net, an enhanced U-Net architecture that integrates advanced morphological operations, including Gaussian blurring, thresholding, and morphological opening/closing, to improve vascular integrity, reduce noise, and achieve a higher Dice score of 0.9108, a precision of 0.9341, and a recall of 0.8872. These enhancements demonstrate superior robustness to noise and intricate vessel geometries. Conclusions: This pre-processing filter effectively reduces noise by grouping neighboring pixels with similar intensity values, allowing the model to focus on relevant anatomical structures, thus outperforming traditional methods in handling the challenges posed by CTA images.

Optical coherence tomography-enabled classification of the human venoatrial junction

Significance

Radiofrequency ablation to treat atrial fibrillation (AF) involves isolating the pulmonary vein from the left atria to prevent AF from occurring. However, creating ablation lesions within the pulmonary veins can cause adverse complications.

Aim

We propose automated classification algorithms to classify optical coherence tomography (OCT) volumes of human venoatrial junctions.

Approach

A dataset of comprehensive OCT volumes of 26 venoatrial junctions was used for this study. Texture, statistical, and optical features were extracted from OCT patches. Patches were classified as a left atrium or pulmonary vein using random forest (RF), logistic regression (LR), and convolutional neural networks (CNNs). The features were inputs into the RF and LR classifiers. The inputs to the CNNs included: (1) patches and (2) an ensemble of patches and patch-derived features.

Results

Utilizing a sevenfold cross-validation, the patch-only CNN balances sensitivity and specificity best, with an area under the receiver operating characteristic (AUROC) curve of 0.84 ± 0.109 across the test sets. RF is more sensitive than LR, with an AUROC curve of 0.78 ± 0.102 .

Conclusions

Cardiac tissues can be identified in benchtop OCT images by automated analysis. Extending this analysis to data obtained in vivo is required to tune automated analysis further. Performing this classification in vivo could aid doctors in identifying substrates of interest and treating AF.

Harmonizing Unets: Attention Fusion module in cascaded-Unets for low-quality OCT image fluid segmentation

Optical coherence tomography (OCT) is widely used for its high resolution. Accurate OCT image segmentation can significantly improve the diagnosis and treatment of retinal diseases such as Diabetic Macular Edema (DME). However, in resource-limited regions, portable devices with low-quality output are more frequently used, severely affecting the performance of segmentation. To address this issue, we propose a novel methodology in this paper, including a dedicated pre-processing pipeline and an end-to-end double U-shaped cascaded architecture, H-Unets. In addition, an Adaptive Attention Fusion (AAF) module is elaborately designed to improve the segmentation performance of H-Unets. To demonstrate the effectiveness of our method, we conduct a bunch of ablation and comparative studies on three open-source datasets. The experimental results show the validity of the pre-processing pipeline and H-Unets, achieving the highest Dice score of 90.60%±0.87% among popular methods in a relatively small model size.

Plaque Characteristics Derived from Intravascular Optical Coherence Tomography That Predict Cardiovascular Death

This study aimed to investigate whether plaque characteristics derived from intravascular optical coherence tomography (IVOCT) could predict a long-term cardiovascular (CV) death. This study was a single-center, retrospective study on 104 patients who had undergone IVOCT-guided percutaneous coronary intervention. Plaque characterization was performed using Optical Coherence TOmography PlaqUe and Stent (OCTOPUS) software developed by our group. A total of 31 plaque features, including lesion length, lumen, calcium, fibrous cap (FC), and vulnerable plaque features (e.g., microchannel), were computed from the baseline IVOCT images. The discriminatory power for predicting CV death was determined using univariate/multivariate logistic regressions. Of 104 patients, CV death was identified in 24 patients (23.1%). Univariate logistic regression revealed that lesion length, calcium angle, calcium thickness, FC angle, FC area, and FC surface area were significantly associated with CV death (p < 0.05). In the multivariate logistic analysis, only the FC surface area (OR 2.38, CI 0.98-5.83, p < 0.05) was identified as a significant determinant for CV death, highlighting the importance of the 3D lesion analysis. The AUC of FC surface area for predicting CV death was 0.851 (95% CI 0.800-0.927, p < 0.05). Patients with CV death had distinct plaque characteristics (i.e., large FC surface area) in IVOCT. Studies such as this one might someday lead to recommendations for pharmaceutical and interventional approaches.

PlaqueNet: deep learning enabled coronary artery plaque segmentation from coronary computed tomography angiography

Cardiovascular disease, primarily caused by atherosclerotic plaque formation, is a significant health concern. The early detection of these plaques is crucial for targeted therapies and reducing the risk of cardiovascular diseases. This study presents PlaqueNet, a solution for segmenting coronary artery plaques from coronary computed tomography angiography (CCTA) images. For feature extraction, the advanced residual net module was utilized, which integrates a deepwise residual optimization module into network branches, enhances feature extraction capabilities, avoiding information loss, and addresses gradient issues during training. To improve segmentation accuracy, a depthwise atrous spatial pyramid pooling based on bicubic efficient channel attention (DASPP-BICECA) module is introduced. The BICECA component amplifies the local feature sensitivity, whereas the DASPP component expands the network's information-gathering scope, resulting in elevated segmentation accuracy. Additionally, BINet, a module for joint network loss evaluation, is proposed. It optimizes the segmentation model without affecting the segmentation results. When combined with the DASPP-BICECA module, BINet enhances overall efficiency. The CCTA segmentation algorithm proposed in this study outperformed the other three comparative algorithms, achieving an intersection over Union of 87.37%, Dice of 93.26%, accuracy of 93.12%, mean intersection over Union of 93.68%, mean Dice of 96.63%, and mean pixel accuracy value of 96.55%.

Deep learning segmentation of fibrous cap in intravascular optical coherence tomography images

Thin-cap fibroatheroma (TCFA) is a prominent risk factor for plaque rupture. Intravascular optical coherence tomography (IVOCT) enables identification of fibrous cap (FC), measurement of FC thicknesses, and assessment of plaque vulnerability. We developed a fully-automated deep learning method for FC segmentation. This study included 32,531 images across 227 pullbacks from two registries (TRANSFORM-OCT and UHCMC). Images were semi-automatically labeled using our OCTOPUS with expert editing using established guidelines. We employed preprocessing including guidewire shadow detection, lumen segmentation, pixel-shifting, and Gaussian filtering on raw IVOCT (r,θ) images. Data were augmented in a natural way by changing θ in spiral acquisitions and by changing intensity and noise values. We used a modified SegResNet and comparison networks to segment FCs. We employed transfer learning from our existing much larger, fully-labeled calcification IVOCT dataset to reduce deep-learning training. Postprocessing with a morphological operation enhanced segmentation performance. Overall, our method consistently delivered better FC segmentation results (Dice: 0.837 ± 0.012) than other deep-learning methods. Transfer learning reduced training time by 84% and reduced the need for more training samples. Our method showed a high level of generalizability, evidenced by highly-consistent segmentations across five-fold cross-validation (sensitivity: 85.0 ± 0.3%, Dice: 0.846 ± 0.011) and the held-out test (sensitivity: 84.9%, Dice: 0.816) sets. In addition, we found excellent agreement of FC thickness with ground truth (2.95 ± 20.73 µm), giving clinically insignificant bias. There was excellent reproducibility in pre- and post-stenting pullbacks (average FC angle: 200.9 ± 128.0°/202.0 ± 121.1°). Our fully automated, deep-learning FC segmentation method demonstrated excellent performance, generalizability, and reproducibility on multi-center datasets. It will be useful for multiple research purposes and potentially for planning stent deployments that avoid placing a stent edge over an FC.

QOCT-Net: A Physics-Informed Neural Network for Intravascular Optical Coherence Tomography Attenuation Imaging

Intravascular optical coherence tomography (IVOCT) provides high-resolution, depth-resolved images of coronary arterial microstructure by acquiring backscattered light. Quantitative attenuation imaging is important for accurate characterization of tissue components and identification of vulnerable plaques. In this work, we proposed a deep learning method for IVOCT attenuation imaging based on the multiple scattering model of light transport. A physics-informed deep network named Quantitative OCT Network (QOCT-Net) was designed to recover pixel-level optical attenuation coefficients directly from standard IVOCT B-scan images. The network was trained and tested on simulation and in vivo datasets. Results showed superior attenuation coefficient estimates both visually and based on quantitative image metrics. The structural similarity, energy error depth and peak signal-to-noise ratio are improved by at least 7%, 5% and 12.4%, respectively, compared with the state-of-the-art non-learning methods. This method potentially enables high-precision quantitative imaging for tissue characterization and vulnerable plaque identification.

Pericoronary Adipose Tissue Radiomics from Coronary Computed Tomography Angiography Identifies Vulnerable Plaques

Pericoronary adipose tissue (PCAT) features on Computed Tomography (CT) have been shown to reflect local inflammation and increased cardiovascular risk. Our goal was to determine whether PCAT radiomics extracted from coronary CT angiography (CCTA) images are associated with intravascular optical coherence tomography (IVOCT)-identified vulnerable-plaque characteristics (e.g., microchannels (MC) and thin-cap fibroatheroma (TCFA)). The CCTA and IVOCT images of 30 lesions from 25 patients were registered. The vessels with vulnerable plaques were identified from the registered IVOCT images. The PCAT-radiomics features were extracted from the CCTA images for the lesion region of interest (PCAT-LOI) and the entire vessel (PCAT-Vessel). We extracted 1356 radiomic features, including intensity (first-order), shape, and texture features. The features were reduced using standard approaches (e.g., high feature correlation). Using stratified three-fold cross-validation with 1000 repeats, we determined the ability of PCAT-radiomics features from CCTA to predict IVOCT vulnerable-plaque characteristics. In the identification of TCFA lesions, the PCAT-LOI and PCAT-Vessel radiomics models performed comparably (Area Under the Curve (AUC) ± standard deviation 0.78 ± 0.13, 0.77 ± 0.14). For the identification of MC lesions, the PCAT-Vessel radiomics model (0.89 ± 0.09) was moderately better associated than the PCAT-LOI model (0.83 ± 0.12). In addition, both the PCAT-LOI and the PCAT-Vessel radiomics model identified coronary vessels thought to be highly vulnerable to a similar standard (i.e., both TCFA and MC; 0.88 ± 0.10, 0.91 ± 0.09). The most favorable radiomic features tended to be those describing the texture and size of the PCAT. The application of PCAT radiomics can identify coronary vessels with TCFA or MC, consistent with IVOCT. Furthermore, the use of CCTA radiomics may improve risk stratification by noninvasively detecting vulnerable-plaque characteristics that are only visible with IVOCT.

Pericoronary adipose tissue radiomics from coronary CT angiography identifies vulnerable plaques characteristics in intravascular OCT

Pericoronary adipose tissue (PCAT) features on CT have been shown to reflect local inflammation, and signals increased cardiovascular risk. Our goal was to determine if PCAT radiomics extracted from coronary CT angiography (CCTA) images are associated with intravascular optical coherence tomography (IVOCT)-identified vulnerable plaque characteristics (e.g., microchannels [MC] and thin-cap fibroatheroma [TCFA]). CCTA and IVOCT images of 30 lesions from 25 patients were registered. Vessels with vulnerable plaques were identified from the registered IVOCT images. PCAT radiomics features were extracted from CCTA images for the lesion region of interest (PCAT-LOI) and the entire vessel (PCAT-Vessel). We extracted 1356 radiomics features, including intensity (first-order), shape, and texture features. Features were reduced using standard approaches (e.g., high feature correlation). Using stratified three-fold cross-validation with 1000 repeats, we determined the ability of PCAT radiomics features from CCTA to predict IVOCT vulnerable plaque characteristics. In identification of TCFA lesions, PCAT-LOI and PCAT-Vessel radiomics models performed comparably (AUC±standard deviation 0.78±0.13, 0.77±0.14). For identification of MC lesions, PCAT-Vessel radiomics model (0.89±0.09) was moderately better associated than that of PCAT-LOI model (0.83±0.12). Both PCAT-LOI and PCAT-Vessel radiomics models also similarly identified coronary vessels thought to be highly vulnerable (i.e., both TCFA and MC) (0.88±0.10, 0.91±0.09). Favorable radiomics features tended to be those describing texture and size of PCAT. PCAT radiomics can identify coronary vessels with TCFA or MC, consistent with IVOCT. CCTA radiomics may improve risk stratification by noninvasively detecting vulnerable plaque characteristics that are only visible with IVOCT.

Neoatherosclerosis prediction using plaque markers in intravascular optical coherence tomography images

Introduction

In-stent neoatherosclerosis has emerged as a crucial factor in post-stent complications including late in-stent restenosis and very late stent thrombosis. In this study, we investigated the ability of quantitative plaque characteristics from intravascular optical coherence tomography (IVOCT) images taken just prior to stent implantation to predict neoatherosclerosis after implantation.

Methods

This was a sub-study of the TRiple Assessment of Neointima Stent FOrmation to Reabsorbable polyMer with Optical Coherence Tomography (TRANSFORM-OCT) trial. Images were obtained before and 18 months after stent implantation. Final analysis included images of 180 lesions from 90 patients; each patient had images of two lesions in different coronary arteries. A total of 17 IVOCT plaque features, including lesion length, lumen (e.g., area and diameter); calcium (e.g., angle and thickness); and fibrous cap (FC) features (e.g., thickness, surface area, and burden), were automatically extracted from the baseline IVOCT images before stenting using dedicated software developed by our group (OCTOPUS). The predictive value of baseline IVOCT plaque features for neoatherosclerosis development after stent implantation was assessed using univariate/multivariate logistic regression and receiver operating characteristic (ROC) analyses.

Results

Follow-up IVOCT identified stents with (n = 19) and without (n = 161) neoatherosclerosis. Greater lesion length and maximum calcium angle and features related to FC were associated with a higher prevalence of neoatherosclerosis after stent implantation (p < 0.05). Hierarchical clustering identified six clusters with the best prediction p-values. In univariate logistic regression analysis, maximum calcium angle, minimum calcium thickness, maximum FC angle, maximum FC area, FC surface area, and FC burden were significant predictors of neoatherosclerosis. Lesion length and features related to the lumen were not significantly different between the two groups. In multivariate logistic regression analysis, only larger FC surface area was strongly associated with neoatherosclerosis (odds ratio 1.38, 95% confidence interval [CI] 1.05-1.80, p < 0.05). The area under the ROC curve was 0.901 (95% CI 0.859-0.946, p < 0.05) for FC surface area.

Conclusion

Post-stent neoatherosclerosis can be predicted by quantitative IVOCT imaging of plaque characteristics prior to stent implantation. Our findings highlight the additional clinical benefits of utilizing IVOCT imaging in the catheterization laboratory to inform treatment decision-making and improve outcomes.

Automated analysis of fibrous cap in intravascular optical coherence tomography images of coronary arteries

Thin-cap fibroatheroma (TCFA) and plaque rupture have been recognized as the most frequent risk factor for thrombosis and acute coronary syndrome. Intravascular optical coherence tomography (IVOCT) can identify TCFA and assess cap thickness, which provides an opportunity to assess plaque vulnerability. We developed an automated method that can detect lipidous plaque and assess fibrous cap thickness in IVOCT images. This study analyzed a total of 4360 IVOCT image frames of 77 lesions among 41 patients. Expert cardiologists manually labeled lipidous plaque based on established criteria. To improve segmentation performance, preprocessing included lumen segmentation, pixel-shifting, and noise filtering on the raw polar (r, θ) IVOCT images. We used the DeepLab-v3 plus deep learning model to classify lipidous plaque pixels. After lipid detection, we automatically detected the outer border of the fibrous cap using a special dynamic programming algorithm and assessed the cap thickness. Our method provided excellent discriminability of lipid plaque with a sensitivity of 85.8% and A-line Dice coefficient of 0.837. By comparing lipid angle measurements between two analysts following editing of our automated software, we found good agreement by Bland-Altman analysis (difference 6.7° ± 17°; mean ~ 196°). Our method accurately detected the fibrous cap from the detected lipid plaque. Automated analysis required a significant modification for only 5.5% frames. Furthermore, our method showed a good agreement of fibrous cap thickness between two analysts with Bland-Altman analysis (4.2 ± 14.6 µm; mean ~ 175 µm), indicating little bias between users and good reproducibility of the measurement. We developed a fully automated method for fibrous cap quantification in IVOCT images, resulting in good agreement with determinations by analysts. The method has great potential to enable highly automated, repeatable, and comprehensive evaluations of TCFAs.

Robust Medical Image Colorization with Spatial Mask-Guided Generative Adversarial Network

Color medical images provide better visualization and diagnostic information for doctors during clinical procedures than grayscale medical images. Although generative adversarial network-based image colorization approaches have shown promising results, in these methods, adversarial training is applied to the whole image without considering the appearance conflicts between the foreground objects and the background contents, resulting in generating various artifacts. To remedy this issue, we propose a fully automatic spatial mask-guided colorization with generative adversarial network (SMCGAN) framework for medical image colorization. It generates colorized images with fewer artifacts by introducing spatial masks, which encourage the network to focus on the colorization of the foreground regions instead of the whole image. Specifically, we propose a novel spatial mask-guided method by introducing an auxiliary foreground segmentation branch combined with the main colorization branch to obtain the spatial masks. The spatial masks are then used to generate masked colorized images where most background contents are filtered out. Moreover, two discriminators are utilized for the generated colorized images and masked generated colorized images, respectively, to assist the model in focusing on the colorization of foreground regions. We validate our proposed framework on two publicly available datasets, including the Visible Human Project (VHP) dataset and the prostate dataset from NCI-ISBI 2013 challenge. The experimental results demonstrate that SMCGAN outperforms the state-of-the-art GAN-based image colorization approaches with an average improvement of 8.48% in the PSNR metric. The proposed SMCGAN can also generate colorized medical images with fewer artifacts.