Automatic stent struts detection in optical coherence tomography based on a multiple attention convolutional model

Objective.Intravascular optical coherence tomography is a useful tool to assess stent adherence and dilation, thus guiding percutaneous coronary intervention and minimizing the risk of surgery. However, each pull-back OCT images may contain thousands of stent struts, which are tiny and dense, making manual stent labeling slow and costly for medical resources.Approach. This paper proposed a multiple attention convolutional model for automatic stent struts detection of OCT images. Multiple attention mechanisms were utilized to strengthen the feature extraction and feature fusion capabilities. In addition, to precisely detect tiny stent struts, the model integrated multiple anchor frames to predict targets in the output.Main results. The model was trained in 4625 frames OCT images of 37 patients and tested in 1156 frames OCT images of 9 patients, and achieved a precision of 0.9790 and a recall of 0.9541, which were significantly better than mainstream convolutional models. In terms of detection speed, the model achieved 25.2 ms per image. OCT images from different collection systems, collection times, and challenging scenarios were experimentally tested, and the model demonstrated stable robustness, achieving precision and recall higher than 0.9630. Meanwhile, clear 3D construction of the stent was achieved.Significance. In conclusion, the proposed model solves the problems of slow manual analysis and occupying a large amount of medical manpower resources. It enhances the detection efficiency of tiny and dense stent struts, thus facilitating the application of OCT quantitative analysis in real clinical scenarios.

Neoatherosclerosis prediction using plaque markers in intravascular optical coherence tomography images

Introduction

In-stent neoatherosclerosis has emerged as a crucial factor in post-stent complications including late in-stent restenosis and very late stent thrombosis. In this study, we investigated the ability of quantitative plaque characteristics from intravascular optical coherence tomography (IVOCT) images taken just prior to stent implantation to predict neoatherosclerosis after implantation.

Methods

This was a sub-study of the TRiple Assessment of Neointima Stent FOrmation to Reabsorbable polyMer with Optical Coherence Tomography (TRANSFORM-OCT) trial. Images were obtained before and 18 months after stent implantation. Final analysis included images of 180 lesions from 90 patients; each patient had images of two lesions in different coronary arteries. A total of 17 IVOCT plaque features, including lesion length, lumen (e.g., area and diameter); calcium (e.g., angle and thickness); and fibrous cap (FC) features (e.g., thickness, surface area, and burden), were automatically extracted from the baseline IVOCT images before stenting using dedicated software developed by our group (OCTOPUS). The predictive value of baseline IVOCT plaque features for neoatherosclerosis development after stent implantation was assessed using univariate/multivariate logistic regression and receiver operating characteristic (ROC) analyses.

Results

Follow-up IVOCT identified stents with (n = 19) and without (n = 161) neoatherosclerosis. Greater lesion length and maximum calcium angle and features related to FC were associated with a higher prevalence of neoatherosclerosis after stent implantation (p < 0.05). Hierarchical clustering identified six clusters with the best prediction p-values. In univariate logistic regression analysis, maximum calcium angle, minimum calcium thickness, maximum FC angle, maximum FC area, FC surface area, and FC burden were significant predictors of neoatherosclerosis. Lesion length and features related to the lumen were not significantly different between the two groups. In multivariate logistic regression analysis, only larger FC surface area was strongly associated with neoatherosclerosis (odds ratio 1.38, 95% confidence interval [CI] 1.05-1.80, p < 0.05). The area under the ROC curve was 0.901 (95% CI 0.859-0.946, p < 0.05) for FC surface area.

Conclusion

Post-stent neoatherosclerosis can be predicted by quantitative IVOCT imaging of plaque characteristics prior to stent implantation. Our findings highlight the additional clinical benefits of utilizing IVOCT imaging in the catheterization laboratory to inform treatment decision-making and improve outcomes.

Rapid lipid-laden plaque identification in intravascular optical coherence tomography imaging based on time-series deep learning

SIGNIFICANCE

Coronary heart disease has the highest rate of death and morbidity in the Western world. Atherosclerosis is an asymptomatic condition that is considered the primary cause of cardiovascular diseases. The accumulation of low-density lipoprotein triggers an inflammatory process in focal areas of arteries, which leads to the formation of plaques. Lipid-laden plaques containing a necrotic core may eventually rupture, causing heart attack and stroke. Lately, intravascular optical coherence tomography (IV-OCT) imaging has been used for plaque assessment. The interpretation of the IV-OCT images is performed visually, which is burdensome and requires highly trained physicians for accurate plaque identification.

AIM

Our study aims to provide high throughput lipid-laden plaque identification that can assist in vivo imaging by offering faster screening and guided decision making during percutaneous coronary interventions.

APPROACH

An A-line-wise classification methodology based on time-series deep learning is presented to fulfill this aim. The classifier was trained and validated with a database consisting of IV-OCT images of 98 artery sections. A trained physician with expertise in the analysis of IV-OCT imaging provided the visual evaluation of the database that was used as ground truth for training and validation.

RESULTS

This method showed an accuracy, sensitivity, and specificity of 89.6%, 83.6%, and 91.1%, respectively. This deep learning methodology has the potential to increase the speed of lipid-laden plaques identification to provide a high throughput of more than 100 B-scans/s.

CONCLUSIONS

These encouraging results suggest that this method will allow for high throughput video-rate atherosclerotic plaque assessment through automated tissue characterization for in vivo imaging by providing faster screening to assist in guided decision making during percutaneous coronary interventions.

Recent Trends in Artificial Intelligence-Assisted Coronary Atherosclerotic Plaque Characterization

Coronary artery disease is a major cause of morbidity and mortality worldwide. Its underlying histopathology is the atherosclerotic plaque, which comprises lipid, fibrous and—when chronic—calcium components. Intravascular ultrasound (IVUS) and intravascular optical coherence tomography (IVOCT) performed during invasive coronary angiography are reference standards for characterizing the atherosclerotic plaque. Fine image spatial resolution attainable with contemporary coronary computed tomographic angiography (CCTA) has enabled noninvasive plaque assessment, including identifying features associated with vulnerable plaques known to presage acute coronary events. Manual interpretation of IVUS, IVOCT and CCTA images demands scarce physician expertise and high time cost. This has motivated recent research into and development of artificial intelligence (AI)-assisted methods for image processing, feature extraction, plaque identification and characterization. We performed parallel searches of the medical and technical literature from 1995 to 2021 focusing respectively on human plaque characterization using various imaging modalities and the use of AI-assisted computer aided diagnosis (CAD) to detect and classify atherosclerotic plaques, including their composition and the presence of high-risk features denoting vulnerable plaques. A total of 122 publications were selected for evaluation and the analysis was summarized in terms of data sources, methods—machine versus deep learning—and performance metrics. Trends in AI-assisted plaque characterization are detailed and prospective research challenges discussed. Future directions for the development of accurate and efficient CAD systems to characterize plaque noninvasively using CCTA are proposed.

Shape prior generation and geodesic active contour interactive iterating algorithm (SPACIAL): fully automatic segmentation for 3D lumen in intravascular optical coherence tomography images

PURPOSE

Fully automatic lumen segmentation in intravascular optical coherence tomography (OCT) images can assist physicians in quickly estimating the health status of vessels. However, OCT images are usually degraded by residual blood, catheter walls, guide wire artifacts, etc., which significantly reduce the quality of segmentation. To achieve accurate lumen segmentation in low-quality images, we propose a novel segmentation algorithm named SPACIAL: Shape Prior generation and geodesic Active Contour Interactive iterAting aLgorithm, which is guided by an adaptively generated shape prior.

METHODS

In this framework, the active contour evolves under the guidance of shape prior, while the shape prior is automatically and adaptively generated based on the active contour. The active contour and the shape prior interactively iterate each other, which can generate the adaptive shape prior and consequently lead to accurate segmentation results. In addition, a fast algorithm is introduced to accelerate the segmentation in 3D images.

RESULTS

The validity of the model is verified in 3240 images from 12 OCT pullbacks. The experimental results show satisfactory segmentation accuracy and time efficiency: the average Dice coefficient of SPACIAL is 93.6(2.4)%, and 5.7 times faster than that of the classical level set method.

CONCLUSION

The proposed SPACIAL can quickly and efficiently perform accurate lumen segmentation on low quality OCT images, which is of great importance to cardiovascular disease diagnosis . The SPACIAL method shows great potential in clinical applications.

Automated classification of coronary plaque calcification in OCT pullbacks with 3D deep neural networks

SIGNIFICANCE

Detection and characterization of coronary atherosclerotic plaques often need reviews of a large number of optical coherence tomography (OCT) imaging slices to make a clinical decision. However, it is a challenge to manually review all the slices and consider the interrelationship between adjacent slices.

APPROACH

Inspired by the recent success of deep convolutional network on the classification of medical images, we proposed a ResNet-3D network for classification of coronary plaque calcification in OCT pullbacks. The ResNet-3D network was initialized with a trained ResNet-50 network and a three-dimensional convolution filter filled with zeros padding and non-zeros padding with a convolutional filter. To retrain ResNet-50, we used a dataset of ∼4860 OCT images, derived by 18 entire pullbacks from different patients. In addition, we investigated a two-phase training method to address the data imbalance. For an improved performance, we evaluated different input sizes for the ResNet-3D network, such as 3, 5, and 7 OCT slices. Furthermore, we integrated all ResNet-3D results by majority voting.

RESULTS

A comparative analysis proved the effectiveness of the proposed ResNet-3D networks against ResNet-2D network in the OCT dataset. The classification performance (F1-scores  =  94  %   for non-zeros padding and F1-score  =  96  %   for zeros padding) demonstrated the potential of convolutional neural networks (CNNs) in classifying plaque calcification.

CONCLUSIONS

This work may provide a foundation for further work in extending the CNN to voxel segmentation, which may lead to a supportive diagnostic tool for assessment of coronary plaque vulnerability.

Coronary calcification segmentation in intravascular OCT images using deep learning: application to calcification scoring

Major calcifications are of great concern when performing percutaneous coronary interventions because they inhibit proper stent deployment. We created a comprehensive software to segment calcifications in intravascular optical coherence tomography (IVOCT) images and to calculate their impact using the stent-deployment calcification score, as reported by Fujino et al. We segmented the vascular lumen and calcifications using the pretrained SegNet, convolutional neural network, which was refined for our task. We cleaned segmentation results using conditional random field processing. We evaluated the method on manually annotated IVOCT volumes of interest (VOIs) without lesions and with calcifications, lipidous, or mixed lesions. The dataset included 48 VOIs taken from 34 clinical pullbacks, giving a total of 2640 in vivo images. Annotations were determined from consensus between two expert analysts. Keeping VOIs intact, we performed 10-fold cross-validation over all data. Following segmentation noise cleaning, we obtained sensitivities of 0.85 ± 0.04 , 0.99 ± 0.01 , and 0.97 ± 0.01 for calcified, lumen, and other tissue classes, respectively. From segmented regions, we automatically determined calcification depth, angle, and thickness attributes. Bland-Altman analysis suggested strong correlation between manually and automatically obtained lumen and calcification attributes. Agreement between manually and automatically obtained stent-deployment calcification scores was good (four of five lesions gave exact agreement). Results are encouraging and suggest our classification approach could be applied clinically for assessment and treatment planning of coronary calcification lesions.

Automated A-line coronary plaque classification of intravascular optical coherence tomography images using handcrafted features and large datasets

We developed machine learning methods to identify fibrolipidic and fibrocalcific A-lines in intravascular optical coherence tomography (IVOCT) images using a comprehensive set of handcrafted features. We incorporated features developed in previous studies (e.g., optical attenuation and A-line peaks). In addition, we included vascular lumen morphology and three-dimensional (3-D) digital edge and texture features. Classification methods were developed using expansive datasets (∼7000  images), consisting of both clinical in-vivo images and an ex-vivo dataset, which was validated using 3-D cryo-imaging/histology. Conditional random field was used to perform 3-D classification noise cleaning of classification results. We tested various multiclass approaches, classifiers, and feature selection schemes and found that a three-class support vector machine with minimal-redundancy-maximal-relevance feature selection gave the best performance. We found that inclusion of our morphological and 3-D features improved overall classification accuracy. On a held-out test set consisting of >1700 images, we obtained an overall accuracy of 81.58%, with the following (sensitivity/specificity) for each class: other (81.43/89.59), fibrolipidic (94.48/87.32), and fibrocalcific (74.82/95.28). The en-face views of classification results showed that automated classification easily captured the preponderance of a disease segment (e.g., a calcified segment had large regions of fibrocalcific classifications). Finally, we demonstrated proof-of-concept for streamlining A-line classification output with existing fibrolipidic and fibrocalcific boundary segmentation methods, to enable fully automated plaque quantification. The results suggest that our classification approach is a viable step toward fully automated IVOCT plaque classification and segmentation for live-time treatment planning and for offline assessment of drug and biologic therapeutics.

Optical Coherence Tomography of Spontaneous Basilar Artery Dissection in a Patient With Acute Ischemic Stroke

The diagnosis of intracranial arterial dissection (IAD) may be challenging and multimodal imaging techniques are often needed to confirm the diagnosis. Previous studies have based their criteria for diagnosis of IAD on conventional angiography, computed tomography, or magnetic resonance imaging. We report a case with acute ischemic stroke due to spontaneous basilar artery dissection in which intravascular optical coherence tomography (OCT) was used to show features of IAD. A 59-years-old woman presented with symptoms of acute ischemic stroke. Thrombosis related to basilar artery (BA) stenosis was assumed on conventional angiography; however, no clot was retrieved after mechanical thrombectomy (MT) and a restored BA caliber was observed after a rescue recanalization with the detachment of a self-expanding stent was performed. Spontaneous IAD was suspected; however, angiographic findings were ambiguous for confirming IAD. The patient remained symptom-free until 18-months follow-up. At this point, angiography showed restenosis at the proximal tapered length of the stent. In vivo OCT was performed to assess the pathological changes of the restenosis and confirm the diagnosis of IAD.OCT revealed BA dissection with the presence of remnant transverse flap, double lumen and mural hematoma. Imaging at multiple levels identified intimal disruption that originated in the right vertebral artery and extended distally to the BA. The use of intravascular imaging with OCT enabled the accurate diagnosis of IAD. Care should be taken as the procedure may add additional risks to the patient. Future studies are needed to validate the safety of OCT in IAD.

Tissue characterization with depth-resolved attenuation coefficient and backscatter term in intravascular optical coherence tomography images

An important application of intravascular optical coherence tomography (IVOCT) for atherosclerotic tissue analysis is using it to estimate attenuation and backscatter coefficients. This work aims at exploring the potential of the attenuation coefficient, a proposed backscatter term, and image intensities in distinguishing different atherosclerotic tissue types with a robust implementation of depth-resolved (DR) approach. Therefore, the DR model is introduced to estimate the attenuation coefficient and further extended to estimate the backscatter-related term in IVOCT images, such that values can be estimated per pixel without predefining any delineation for the estimation. In order to exclude noisy regions with a weak signal, an automated algorithm is implemented to determine the cut-off border in IVOCT images. The attenuation coefficient, backscatter term, and the image intensity are further analyzed in regions of interest, which have been delineated referring to their pathology counterparts. Local statistical values were reported and their distributions were further compared with a two-sample t-test to evaluate the potential for distinguishing six types of tissues. Results show that the IVOCT intensity, DR attenuation coefficient, and backscatter term extracted with the reported implementation are complementary to each other on characterizing six tissue types: mixed, calcification, fibrous, lipid-rich, macrophages, and necrotic core.

Tissue characterization with depth-resolved attenuation coefficient and backscatter term in intravascular optical coherence tomography images

An important application of intravascular optical coherence tomography (IVOCT) for atherosclerotic tissue analysis is using it to estimate attenuation and backscatter coefficients. This work aims at exploring the potential of the attenuation coefficient, a proposed backscatter term, and image intensities in distinguishing different atherosclerotic tissue types with a robust implementation of depth-resolved (DR) approach. Therefore, the DR model is introduced to estimate the attenuation coefficient and further extended to estimate the backscatter-related term in IVOCT images, such that values can be estimated per pixel without predefining any delineation for the estimation. In order to exclude noisy regions with a weak signal, an automated algorithm is implemented to determine the cut-off border in IVOCT images. The attenuation coefficient, backscatter term, and the image intensity are further analyzed in regions of interest, which have been delineated referring to their pathology counterparts. Local statistical values were reported and their distributions were further compared with a two-sample t-test to evaluate the potential for distinguishing six types of tissues. Results show that the IVOCT intensity, DR attenuation coefficient, and backscatter term extracted with the reported implementation are complementary to each other on characterizing six tissue types: mixed, calcification, fibrous, lipid-rich, macrophages, and necrotic core.

Macrophages and intravascular OCT bright spots: a quantitative study

OBJECTIVES

This study hypothesized that bright spots in intravascular optical coherence tomography (IVOCT) images may originate by colocalization of plaque materials of differing indexes of refraction. To quantitatively identify bright spots, we developed an algorithm that accounts for factors including tissue depth, distance from light source, and signal-to-noise ratio. We used this algorithm to perform a bright spot analysis of IVOCT images and compared these results with histological examination of matching tissue sections.

BACKGROUND

Bright spots are thought to represent macrophages in IVOCT images, and studies of alternative etiologies have not been reported.

METHODS

Fresh human coronary arteries (n = 14 from 10 hearts) were imaged with IVOCT in a mock catheterization laboratory and then processed for histological analysis. The quantitative bright spot algorithm was applied to all images.

RESULTS

Results are reported for 1,599 IVOCT images co-registered with histology. Macrophages alone were responsible for only 23% of the bright spot-positive regions, although they were present in 57% of bright spot-positive regions (as determined by histology). Additional etiologies for bright spots included cellular fibrous tissue (8%), interfaces between calcium and fibrous tissue (10%), calcium and lipids (5%), and fibrous cap and lipid pool (3%). Additionally, we showed that large pools of macrophages in CD68(+) histology sections corresponded to dark regions in comparative IVOCT images; this is due to the fact that a pool of lipid-rich macrophages will have the same index of refraction as a pool of lipid and thus will not cause bright spots.

CONCLUSIONS

Bright spots in IVOCT images were correlated with a variety of plaque components that cause sharp changes in the index of refraction. Algorithms that incorporate these correlations may be developed to improve the identification of some types of vulnerable plaque and allow standardization of IVOCT image interpretation.

Miniature optical coherence tomography-ultrasound probe for automatically coregistered three-dimensional intracoronary imaging with real-time display

We have developed a novel miniature integrated optical coherence tomography (OCT)-intravascular ultrasound (IVUS) probe, with a 1.5-mm-long rigid part and 0.9-mm outer diameter, for real-time intracoronary imaging of atherosclerotic plaques and guiding of interventional procedures. By placing the OCT ball lens and IVUS transducer back-to-back at the same axial position, this probe can provide automatically coregistered, coaxial OCT-IVUS imaging. To demonstrate its real-time capability, three-dimensional OCT-IVUS imaging of a pig's coronary artery displaying in polar coordinates, as well as images of three major types of atherosclerotic plaques in human cadaver coronary segments, were obtained using this probe and our upgraded system. Histology validation is also presented.