Recent Advances in Machine Learning Applied to Ultrasound Imaging

Machine learning (ML) methods are pervading an increasing number of fields of application because of their capacity to effectively solve a wide variety of challenging problems. The employment of ML techniques in ultrasound imaging applications started several years ago but the scientific interest in this issue has increased exponentially in the last few years. The present work reviews the most recent (2019 onwards) implementations of machine learning techniques for two of the most popular ultrasound imaging fields, medical diagnostics and non-destructive evaluation. The former, which covers the major part of the review, was analyzed by classifying studies according to the human organ investigated and the methodology (e.g., detection, segmentation, and/or classification) adopted, while for the latter, some solutions to the detection/classification of material defects or particular patterns are reported. Finally, the main merits of machine learning that emerged from the study analysis are summarized and discussed.

Using Convolutional Neural Networks for Classification of Bifurcation Regions in IVOCT Images

Optical Coherence Tomography (OCT) technology enabled the experts to analyze coronary lesions from high-resolution intravascular images. Studies have shown the relationship between bifurcation regions and a higher occurrence of wall thickening and lesions in these areas. Some level of automation could benefit experts, since examining pullback frames is a laborious and time-consuming task. Although Convolutional Neural Networks (CNN) have shown promising results in classification tasks of medical images, we did not identify the use of CNN's in IVOCT images to classify bifurcation regions in the literature. In this work, we evaluated a CNN architecture in the bifurcation classification task trained with IVOCT images from 9 pullbacks from 9 different patients. We used data augmentation to balance the dataset, due to the low amount of bifurcation-labeled frames. Our classification results are comparable to other works in the literature, presenting better result in AUC (99.70%).

Lumen-intima and media-adventitia segmentation in IVUS images using supervised classifications of arterial layers and morphological structures

BACKGROUND

Intravascular ultrasound (IVUS) provides axial grey-scale images of blood vessels. The large number of images require automatic analysis, specifically to identify the lumen and outer vessel wall. However, the high amount of noise, the presence of artifacts and anatomical structures, such as bifurcations, calcifications and fibrotic plaques, usually hinder the proper automatic segmentation of the vessel wall.

METHODS

Lumen, media, adventitia and surrounding tissues are automatically detected using Support Vector Machines (SVMs). The classification performance of the SVMs vary according to the kind of structure present within each region of the image. Random Forest (RF) is used to detect different morphological structures and to modify the initial layer classification depending on the detected structure. The resulting classification maps are fed into a segmentation method based on deformable contours to detect lumen-intima (LI) and media-adventitia (MA) interfaces.

RESULTS

The modifications in the layer classifications according to the presence of structures proved to be effective improving LI and MA segmentations. The proposed method reaches a Jaccard Measure (JM) of 0.88 ± 0.08 for LI segmentation, compared with 0.88 ± 0.05 of a semiautomatic method. When looking at MA, our method reaches a JM of 0.84 ± 0.09, and outperforms previous automatic methods in terms of HD, with 0.51mm ± 0.30.

CONCLUSIONS

A simple modification to the arterial layer classification produces results that match and improve state-of-the-art fully-automatic segmentation methods for LI and MA in 20MHz IVUS images. For LI segmentation, the proposed automatic method performs accurately as semi-automatic methods. For MA segmentation, our method matched the quality of state-of-the-art automatic methods described in the literature. Furthermore, our implementation is modular and open-source, allowing for future extensions and improvements.

Assessment of intracoronary stent location and extension in intravascular ultrasound sequences

PURPOSE

An intraluminal coronary stent is a metal scaffold deployed in a stenotic artery during percutaneous coronary intervention (PCI). In order to have an effective deployment, a stent should be optimally placed with regard to anatomical structures such as bifurcations and stenoses. Intravascular ultrasound (IVUS) is a catheter-based imaging technique generally used for PCI guiding and assessing the correct placement of the stent. A novel approach that automatically detects the boundaries and the position of the stent along the IVUS pullback is presented. Such a technique aims at optimizing the stent deployment.

METHODS

The method requires the identification of the stable frames of the sequence and the reliable detection of stent struts. Using these data, a measure of likelihood for a frame to contain a stent is computed. Then, a robust binary representation of the presence of the stent in the pullback is obtained applying an iterative and multiscale quantization of the signal to symbols using the Symbolic Aggregate approXimation algorithm.

RESULTS

The technique was extensively validated on a set of 103 IVUS of sequences of in vivo coronary arteries containing metallic and bioabsorbable stents acquired through an international multicentric collaboration across five clinical centers. The method was able to detect the stent position with an overall F-measure of 86.4%, a Jaccard index score of 75% and a mean distance of 2.5 mm from manually annotated stent boundaries, and in bioabsorbable stents with an overall F-measure of 88.6%, a Jaccard score of 77.7 and a mean distance of 1.5 mm from manually annotated stent boundaries. Additionally, a map indicating the distance between the lumen and the stent along the pullback is created in order to show the angular sectors of the sequence in which the malapposition is present.

CONCLUSIONS

Results obtained comparing the automatic results vs the manual annotation of two observers shows that the method approaches the interobserver variability. Similar performances are obtained on both metallic and bioabsorbable stents, showing the flexibility and robustness of the method.

A Novel Synchronous Micro Motor for Intravascular Ultrasound Imaging

OBJECTIVE

Intravascular ultrasound (IVUS) is an important method for evaluating lumen dimensions and guiding intervention. However, the current IVUS catheter using a proximal motor and flexible drive shaft is easily rotated at an unstable speed when it passes through along bending vessel. One approach to solve this problem is to develop a catheter driven by a distal motor.

METHODS

This paper presents a rotation device incorporating a high-frequency transducer as an attempt to facilitate this approach. A novel micro distal synchronous micro motor with 3.7 mm length and 1.2 mm outer diameter was proposed as an actuator for the IVUS catheter. A 0.5 mm × 0.5 mm Pb(Mg_1/3Nb_2/3)O₃-PbTiO₃ single crystal 1-3 composite single-element transducer was designed and manufactured. The probe is fixed to the front end of the catheter. The 45° reflector, which is opposite to the probe, was used to steer ultrasound to the tissue.

RESULTS

The results showed that the maximum torque and rotation speed of the motor were 2.79 μNm and 275 revolutions per second, respectively, at a driving current of 0.34 A. The maximum angular error was 7° at 0.13 A and 30 Hz. The center frequency and -6 dB fractional bandwidth of single element were 34 MHz and 72%, respectively. At the center frequency, the two-way insertion loss was 14 dB.

CONCLUSION

The integrated distal motor IVUS catheter, with small dimensions, a good torque, speed stability, and good ultrasound imaging performance, has tremendous potential in blood vessel imaging.

SIGNIFICANCE

The novel structure of the catheter could facilitate endoluminal sonography, reducing risks of the clinical diagnosis.

Assessment of image features for vessel wall segmentation in intravascular ultrasound images

BACKGROUND

Intravascular ultrasound (IVUS) provides axial greyscale images, allowing the assessment of the vessel wall and the surrounding tissues. Several studies have described automatic segmentation of the luminal boundary and the media-adventitia interface by means of different image features.

PURPOSE

The aim of the present study is to evaluate the capability of some of the most relevant state-of-the-art image features for segmenting IVUS images. The study is focused on Volcano 20 MHz frames not containing plaque or containing fibrotic plaques, and, in principle, it could not be applied to frames containing shadows, calcified plaques, bifurcations and side vessels.

METHODS

Several image filters, textural descriptors, edge detectors, noise and spatial measures were taken into account. The assessment is based on classification techniques previously used for IVUS segmentation, assigning to each pixel a continuous likelihood value obtained using support vector machines (SVMs). To retrieve relevant features, sequential feature selection was performed guided by the area under the precision-recall curve (AUC-PR).

RESULTS

Subsets of relevant image features for lumen, plaque and surrounding tissues characterization were obtained, and SVMs trained with these features were able to accurately identify those regions. The experimental results were evaluated with respect to ground truth segmentations from a publicly available dataset, reaching values of AUC-PR up to 0.97 and Jaccard index close to 0.85.

CONCLUSION

Noise-reduction filters and Haralick's textural features denoted their relevance to identify lumen and background. Laws' textural features, local binary patterns, Gabor filters and edge detectors had less relevance in the selection process.

Simultaneous Registration of Location and Orientation in Intravascular Ultrasound Pullbacks Pairs Via 3D Graph-Based Optimization

A novel method is reported for simultaneous registration of location (axial direction) and orientation (circumferential direction) of two intravascular ultrasound (IVUS) pullbacks of the same vessel taken at different times. Monitoring plaque progression or regression (e.g., during lipid treatment) is of high clinical relevance. Our method uses a 3D graph optimization approach, in which the cost function jointly reflects similarity of plaque morphology and plaque/perivascular image appearance. Graph arcs incorporate prior information about temporal correspondence of the two IVUS sequences and limited angular twisting between consecutive IVUS images. Additionally, our approach automatically identifies starting and ending frame pairs in the two IVUS pullbacks. Validation of our method was performed in 29 pairs of IVUS baseline/follow-up pullback sequences consisting of 8 622 IVUS image frames in total. In comparison to manual registration by three experts, the average location and orientation registration errors ranged from 0.72 mm to 0.79 mm and from 7.3(°) to 9.3(°), respectively, all close to the inter-observer variability with no difference being statistically significant (p = NS). Rotation angles determined by our automated approach and expert observers showed high correlation (r(2) of 0.97 to 0.98) and agreed closely (mutual bias between the automated method and expert observers ranged from -1.57(°) to 0.15(°)). Compared with state-of-the-art approaches, the new method offers lower errors in both location and orientation registration. Our method offers highly automated and accurate IVUS pullback registration and can be employed in IVUS-based studies of coronary disease progression, enabling more focal studies of coronary plaque development and transition of vulnerability.

A bifurcation identifier for IV-OCT using orthogonal least squares and supervised machine learning

Intravascular optical coherence tomography (IV-OCT) is an in-vivo imaging modality based on the intravascular introduction of a catheter which provides a view of the inner wall of blood vessels with a spatial resolution of 10-20 μm. Recent studies in IV-OCT have demonstrated the importance of the bifurcation regions. Therefore, the development of an automated tool to classify hundreds of coronary OCT frames as bifurcation or nonbifurcation can be an important step to improve automated methods for atherosclerotic plaques quantification, stent analysis and co-registration between different modalities. This paper describes a fully automated method to identify IV-OCT frames in bifurcation regions. The method is divided into lumen detection; feature extraction; and classification, providing a lumen area quantification, geometrical features of the cross-sectional lumen and labeled slices. This classification method is a combination of supervised machine learning algorithms and feature selection using orthogonal least squares methods. Training and tests were performed in sets with a maximum of 1460 human coronary OCT frames. The lumen segmentation achieved a mean difference of lumen area of 0.11 mm(2) compared with manual segmentation, and the AdaBoost classifier presented the best result reaching a F-measure score of 97.5% using 104 features.

Automated detection framework of the calcified plaque with acoustic shadowing in IVUS images

Intravascular Ultrasound (IVUS) is one ultrasonic imaging technology to acquire vascular cross-sectional images for the visualization of the inner vessel structure. This technique has been widely used for the diagnosis and treatment of coronary artery diseases. The detection of the calcified plaque with acoustic shadowing in IVUS images plays a vital role in the quantitative analysis of atheromatous plaques. The conventional method of the calcium detection is manual drawing by the doctors. However, it is very time-consuming, and with high inter-observer and intra-observer variability between different doctors. Therefore, the computer-aided detection of the calcified plaque is highly desired. In this paper, an automated method is proposed to detect the calcified plaque with acoustic shadowing in IVUS images by the Rayleigh mixture model, the Markov random field, the graph searching method and the prior knowledge about the calcified plaque. The performance of our method was evaluated over 996 in-vivo IVUS images acquired from eight patients, and the detected calcified plaques are compared with manually detected calcified plaques by one cardiology doctor. The experimental results are quantitatively analyzed separately by three evaluation methods, the test of the sensitivity and specificity, the linear regression and the Bland-Altman analysis. The first method is used to evaluate the ability to distinguish between IVUS images with and without the calcified plaque, and the latter two methods can respectively measure the correlation and the agreement between our results and manual drawing results for locating the calcified plaque in the IVUS image. High sensitivity (94.68%) and specificity (95.82%), good correlation and agreement (>96.82% results fall within the 95% confidence interval in the Student t-test) demonstrate the effectiveness of the proposed method in the detection of the calcified plaque with acoustic shadowing in IVUS images.

Framework to co-register longitudinal virtual histology-intravascular ultrasound data in the circumferential direction

Considerable efforts have been directed at identifying prognostic markers for rapidly progressing coronary atherosclerotic lesions that may advance into a high-risk (vulnerable) state. Intravascular ultrasound (IVUS) has become a valuable clinical tool to study the natural history of coronary artery disease (CAD). While prospectively IVUS studies have provided tremendous insight on CAD progression, and its association with independent markers (e.g., wall shear stress), they are limited by the inability to examine the focal association between spatially heterogeneous variables (in both circumferential and axial directions). Herein, we present a framework to automatically co-register longitudinal (in-time) virtual histology-intravascular ultrasound (VH-IVUS) imaging data in the circumferential direction (i.e., rotate follow-up image so circumferential basis coincides with corresponding baseline image). Multivariate normalized cross correlation was performed on paired images (n = 636) from five patients using three independent VH-IVUS defined parameters: artery thickness, VH-IVUS defined plaque constituents, and VH-IVUS perivascular imaging data. Results exhibited high correlation between co-registration rotation angles determined automatically versus manually by an expert reader ( r(2) = 0.90). Furthermore, no significant difference between automatic and manual co-registration angles was observed ( 91.31 ±1.04(°) and 91.07 ±1.04(°), respectively; p = 0.48) and Bland-Altman analysis yielded excellent agreement ( bias = 0.24(°), 95% CI +/- 16.33(°)). In conclusion, we have developed, verified, and validated an algorithm that automatically co-registers VH-IVUS imaging data that will allow for the focal examination of CAD progression.

Automatic non-rigid temporal alignment of intravascular ultrasound sequences: method and quantitative validation

Clinical studies on atherosclerosis regression/progression performed by intravascular ultrasound analysis would benefit from accurate alignment of sequences of the same patient before and after clinical interventions and at follow-up. In this article, a methodology for automatic alignment of intravascular ultrasound sequences based on the dynamic time warping technique is proposed. The non-rigid alignment is adapted to the specific task by applying it to multidimensional signals describing the morphologic content of the vessel. Moreover, dynamic time warping is embedded into a framework comprising a strategy to address partial overlapping between acquisitions and a term that regularizes non-physiologic temporal compression/expansion of the sequences. Extensive validation is performed on both synthetic and in vivo data. The proposed method reaches alignment errors of approximately 0.43 mm for pairs of sequences acquired during the same intervention phase and 0.77 mm for pairs of sequences acquired at successive intervention stages.