Image-based cardiac phase retrieval in intravascular ultrasound sequences

CARDIAN: a novel computational approach for real-time end-diastolic frame detection in intravascular ultrasound using bidirectional attention networks

Introduction

Changes in coronary artery luminal dimensions during the cardiac cycle can impact the accurate quantification of volumetric analyses in intravascular ultrasound (IVUS) image studies. Accurate ED-frame detection is pivotal for guiding interventional decisions, optimizing therapeutic interventions, and ensuring standardized volumetric analysis in research studies. Images acquired at different phases of the cardiac cycle may also lead to inaccurate quantification of atheroma volume due to the longitudinal motion of the catheter in relation to the vessel. As IVUS images are acquired throughout the cardiac cycle, end-diastolic frames are typically identified retrospectively by human analysts to minimize motion artefacts and enable more accurate and reproducible volumetric analysis.

Methods

In this paper, a novel neural network-based approach for accurate end-diastolic frame detection in IVUS sequences is proposed, trained using electrocardiogram (ECG) signals acquired synchronously during IVUS acquisition. The framework integrates dedicated motion encoders and a bidirectional attention recurrent network (BARNet) with a temporal difference encoder to extract frame-by-frame motion features corresponding to the phases of the cardiac cycle. In addition, a spatiotemporal rotation encoder is included to capture the IVUS catheter's rotational movement with respect to the coronary artery.

Results

With a prediction tolerance range of 66.7 ms, the proposed approach was able to find 71.9%, 67.8%, and 69.9% of end-diastolic frames in the left anterior descending, left circumflex and right coronary arteries, respectively, when tested against ECG estimations. When the result was compared with two expert analysts' estimation, the approach achieved a superior performance.

Discussion

These findings indicate that the developed methodology is accurate and fully reproducible and therefore it should be preferred over experts for end-diastolic frame detection in IVUS sequences.

3D reconstruction of coronary artery bifurcations from intravascular ultrasound and angiography

Coronary bifurcation lesions represent a challenging anatomical subset, and the understanding of their 3D anatomy and plaque composition appears to play a key role in devising the optimal stenting strategy. This study proposes a new approach for the 3D reconstruction of coronary bifurcations and plaque materials by combining intravascular ultrasound (IVUS) and angiography. Three patient-specific silicone bifurcation models were 3D reconstructed and compared to micro-computed tomography (µCT) as the gold standard to test the accuracy and reproducibility of the proposed methodology. The clinical feasibility of the method was investigated in three diseased patient-specific bifurcations of varying anatomical complexity. The IVUS-based 3D reconstructed bifurcation models showed high agreement with the µCT reference models, with r2 values ranging from 0.88 to 0.99. The methodology successfully 3D reconstructed all the patient bifurcations, including plaque materials, in less than 60 min. Our proposed method is a simple, time-efficient, and user-friendly tool for accurate 3D reconstruction of coronary artery bifurcations. It can provide valuable information about bifurcation anatomy and plaque burden in the clinical setting, assisting in bifurcation stent planning and education.

A Deep Learning Method for Motion Artifact Correction in Intravascular Photoacoustic Image Sequence

In vivo application of intravascular photoacoustic (IVPA) imaging for coronary arteries is hampered by motion artifacts associated with the cardiac cycle. Gating is a common strategy to mitigate motion artifacts. However, a large amount of diagnostically valuable information might be lost due to one frame per cycle. In this work, we present a deep learning-based method for directly correcting motion artifacts in non-gated IVPA pullback sequences. The raw signal frames are classified into dynamic and static frames by clustering. Then, a neural network named Motion Artifact Correction (MAC)-Net is designed to correct motion in dynamic frames. Given the lack of the ground truth information on the underlying dynamics of coronary arteries, we trained and tested the network using a computer-generated dataset. Based on the results, it has been observed that the trained network can directly correct motion in successive frames while preserving the original structures without discarding any frames. The improvement in the visual effect of the longitudinal view has been demonstrated based on quantitative evaluation of the inter-frame dissimilarity. The comparison results validated the motion-suppression ability of our method comparable to gating and image registration-based non-learning methods, while maintaining the integrity of the pullbacks without image preprocessing. Experimental results from in vivo intravascular ultrasound and optical coherence tomography pullbacks validated the feasibility of our method in the in vivo intracoronary imaging scenario.

AwCPM-Net: A Collaborative Constraint GAN for 3D Coronary Artery Reconstruction in Intravascular Ultrasound Sequences

3D coronary artery reconstruction (3D-CAR) in intravascular ultrasound (IVUS) sequences allows quantitative analyses of vessel properties. Existing methods treat two main tasks of the 3D-CAR separately, including the cardiac phase retrieval (CPR) and the membrane border extraction (MBE). They ignore the CPR-MBE connection that could achieve mutual promotions to both tasks. In this paper, we pioneer to achieve one-step 3D-CAR via a collaborative constraint generative adversarial network (GAN) named the AwCPM-Net. The AwCPM-Net consists of a dual-task collaborative generator and a dual-task constraint discriminator. The generator combines a self-supervised CPR branch with a semi-supervised MBE branch via a warming-up connection. The discriminator promotes dual-branch predictions simultaneously. The CPR branch requires no annotations and outputs inter-frame deformation fields used for identifying cardiac phases. Deformation fields are additionally constrained by the MBE branch and the discriminator. The MBE branch predicts membrane boundaries for each frame. Two aspects assist the semi-supervised segmentation: annotation augmentation by deformation fields of the CPR branch; information exploitation on unlabeled images enabled by GAN design. Trained and tested on an IVUS dataset acquired from atherosclerosis patients, the AwCPM-Net is effective in both CPR and MBE tasks, superior to state-of-the-art IVUS CPR or MBE methods. Hence, the AwCPM-Net reconstructs reliable 3D artery anatomy in the IVUS modality.

Suppression of motion artifacts in intravascular photoacoustic image sequences

Intravascular photoacoustic (IVPA) imaging is an image-based imaging modality for the assessment of atherosclerotic plaques. Successful application of IVPA for in vivo coronary arterial imaging requires one overcomes the challenge of motion artifacts associated with the cardiac cycle. We propose a method for correcting artifacts owing to cardiac motion, which are observed in sequential IVPA images acquired by the continuous pullback of the imaging catheter. This method groups raw photoacoustic signals into subsets corresponding to similar phases in the cardiac cycles. Thereafter, the sequential images are reconstructed, by representing the initial pressure distribution on the vascular cross-sections based on the clustered frames of signals by time reversal. Results of simulation data demonstrate the efficacy of this method in suppressing motion artifacts. Qualitative and quantitative evaluations of the method indicate an enhancement of the image quality. Comparison results reveal that this method is computationally efficient in motion correction compared with the image-based gating.

A deep learning methodology for the automated detection of end-diastolic frames in intravascular ultrasound images

Coronary luminal dimensions change during the cardiac cycle. However, contemporary volumetric intravascular ultrasound (IVUS) analysis is performed in non-gated images as existing methods to acquire gated or to retrospectively gate IVUS images have failed to dominate in research. We developed a novel deep learning (DL)-methodology for end-diastolic frame detection in IVUS and compared its efficacy against expert analysts and a previously established methodology using electrocardiographic (ECG)-estimations as reference standard. Near-infrared spectroscopy-IVUS (NIRS-IVUS) data were prospectively acquired from 20 coronary arteries and co-registered with the concurrent ECG-signal to identify end-diastolic frames. A DL-methodology which takes advantage of changes in intensity of corresponding pixels in consecutive NIRS-IVUS frames and consists of a network model designed in a bidirectional gated-recurrent-unit (Bi-GRU) structure was trained to detect end-diastolic frames. The efficacy of the DL-methodology in identifying end-diastolic frames was compared with two expert analysts and a conventional image-based (CIB)-methodology that relies on detecting vessel movement to estimate phases of the cardiac cycle. A window of ± 100 ms from the ECG estimations was used to define accurate end-diastolic frames detection. The ECG-signal identified 3,167 end-diastolic frames. The mean difference between DL and ECG estimations was 3 ± 112 ms while the mean differences between the 1st-analyst and ECG, 2nd-analyst and ECG and CIB-methodology and ECG were 86 ± 192 ms, 78 ± 183 ms and 59 ± 207 ms, respectively. The DL-methodology was able to accurately detect 80.4%, while the two analysts and the CIB-methodology detected 39.0%, 43.4% and 42.8% of end-diastolic frames, respectively (P < 0.05). The DL-methodology can identify NIRS-IVUS end-diastolic frames accurately and should be preferred over expert analysts and CIB-methodologies, which have limited efficacy.

Image-Based Gating of Intravascular Ultrasound Sequences Using the Phase Information of Dual-Tree Complex Wavelet Transform Coefficients

Intravascular ultrasound (IVUS) is a widely used interventional imaging technique for the assessment of atherosclerosis plaque. Due to pulsatile heart motions, transverse and longitudinal motions are observed during in vivo pullbacks of IVUS sequences. These motion artifacts can mislead the volume-based data retrieved from IVUS studies and hinder the visualization of the vessel condition. To overcome this problem, a new fully automatic image-based gating algorithm was proposed in the current study. We utilized the phase information of the dual-tree complex wavelet transform (DT-CWT) coefficients to detect the motion of edge-like structures. For each IVUS sequence, first, six motion signals were detected by analyzing the phase of DT-CWT coefficients in six different directions. Then, the three best motion signals were selected by analyzing the frequency properties of each signal. Subsequently, these extracted signals were filtered using a modified Butterworth band-pass filter and the gated sequence was formed by using a combination of them. The proposed method was compared to four state-of-the-art methods and its frequency spectrum had more accurate characteristics in the cardiac frequency. In addition, the gated sequence extracted by the proposed method had the highest similarity to the extracted gated sequence by the physician.

Deep motion tracking from multiview angiographic image sequences for synchronization of cardiac phases

In the diagnosis and interventional treatment of coronary artery disease, the 3D[Formula: see text]time reconstruction of the coronary artery on the basis of x-ray angiographic image sequences can provide dynamic structural information. The synchronization of cardiac phases in the sequences is essential for minimizing the influence of cardiorespiratory motion and realizing precise 3D[Formula: see text]time reconstruction. Key points are initially extracted from the first image of a sequence. Matching grid points between consecutive images in the sequence are extracted by a multi-layer matching strategy. Then deep motion tracking (DMT) of key points is achieved by local deformation based on the neighboring grid points of key points. The local deformation is optimized by the Random sample consensus (RANSAC) algorithm. Then, a simple harmonic motion (SHM) model is utilized to distinguish cardiac motion from other motion sources (e.g. respiratory, patient movement, etc). Next, the signal which is composed of cardiac motions is filtered by a band-pass filter to reconstruct the cardiac phases. Finally, the synchronization of cardiac phases from different imaging angles is realized by a piece-wise linear transformation. The proposed method was evaluated using clinical x-ray angiographic image sequences from 13 patients. [Formula: see text] matching points can be accurately computed by the DMT method. The mean peak temporal distance (MPTD) between the reconstructed cardiac phases and the electrocardiograph signal is [Formula: see text] s. The correlation between the cardiac phases of the same patient is over [Formula: see text]. Compared with three other state-of-the-art methods, the proposed method accurately reconstructs and synchronizes the cardiac phases from different sequences of the same patient. The proposed DMT method is robust and highly effective in synchronizing cardiac phases of angiographic image sequences captured from different imaging angles.

Five multiresolution-based calcium volume measurement techniques from coronary IVUS videos: A comparative approach

BACKGROUND AND OBJECTIVE

Fast intravascular ultrasound (IVUS) video processing is required for calcium volume computation during the planning phase of percutaneous coronary interventional (PCI) procedures. Nonlinear multiresolution techniques are generally applied to improve the processing time by down-sampling the video frames.

METHODS

This paper presents four different segmentation methods for calcium volume measurement, namely Threshold-based, Fuzzy c-Means (FCM), K-means, and Hidden Markov Random Field (HMRF) embedded with five different kinds of multiresolution techniques (bilinear, bicubic, wavelet, Lanczos, and Gaussian pyramid). This leads to 20 different kinds of combinations. IVUS image data sets consisting of 38,760 IVUS frames taken from 19 patients were collected using 40 MHz IVUS catheter (Atlantis® SR Pro, Boston Scientific®, pullback speed of 0.5 mm/sec.). The performance of these 20 systems is compared with and without multiresolution using the following metrics: (a) computational time; (b) calcium volume; (c) image quality degradation ratio; and (d) quality assessment ratio.

RESULTS

Among the four segmentation methods embedded with five kinds of multiresolution techniques, FCM segmentation combined with wavelet-based multiresolution gave the best performance. FCM and wavelet experienced the highest percentage mean improvement in computational time of 77.15% and 74.07%, respectively. Wavelet interpolation experiences the highest mean precision-of-merit (PoM) of 94.06 ± 3.64% and 81.34 ± 16.29% as compared to other multiresolution techniques for volume level and frame level respectively. Wavelet multiresolution technique also experiences the highest Jaccard Index and Dice Similarity of 0.7 and 0.8, respectively. Multiresolution is a nonlinear operation which introduces bias and thus degrades the image. The proposed system also provides a bias correction approach to enrich the system, giving a better mean calcium volume similarity for all the multiresolution-based segmentation methods. After including the bias correction, bicubic interpolation gives the largest increase in mean calcium volume similarity of 4.13% compared to the rest of the multiresolution techniques. The system is automated and can be adapted in clinical settings.

CONCLUSIONS

We demonstrated the time improvement in calcium volume computation without compromising the quality of IVUS image. Among the 20 different combinations of multiresolution with calcium volume segmentation methods, the FCM embedded with wavelet-based multiresolution gave the best performance.