Performance comparison of rigid and affine models for motion estimation using ultrasound radio-frequency signals

Reformulating Optical Flow to Solve Image-Based Inverse Problems and Quantify Uncertainty

From meteorology to medical imaging and cell mechanics, many scientific domains use inverse problems (IPs) to extract physical measurements from image movement. To this end, motion estimation methods such as optical flow (OF) pre-process images into motion data to feed the IP, which then inverts for the measurements through a physical model. However, this combined OFIP pipeline exacerbates the ill-posedness inherent to each technique, propagating errors and preventing uncertainty quantification. We introduce a Bayesian PDE-constrained framework that transforms visual information directly into physical measurements in the context of probability distributions. The posterior mean is a constrained IP that tracks brightness while satisfying the physical model, thereby translating the aperture problem from the motion to the underlying physics; whereas the posterior covariance derives measurement error out of image noise. As we illustrate with traction force microscopy, our approach offers several advantages: more accurate reconstructions; unprecedented flexibility in experiment design (e.g., arbitrary boundary conditions); and the exclusivity of measurement error, central to empirical science, yet still unavailable under the OFIP strategy.

Incorporating Gradient Similarity for Robust Time Delay Estimation in Ultrasound Elastography

Energy-based ultrasound elastography techniques minimize a regularized cost function consisting of data and continuity terms to obtain local displacement estimates based on the local time-delay estimation (TDE) between radio frequency (RF) frames. The data term associated with the existing techniques takes only the amplitude similarity into account and hence is not sufficiently robust to the outlier samples present in the RF frames under consideration. This drawback creates noticeable artifacts in the strain image. To resolve this issue, we propose to formulate the data function as a linear combination of the amplitude and gradient similarity constraints. We estimate the adaptive weight concerning each similarity term following an iterative scheme. Finally, we optimize the nonlinear cost function in an efficient manner to convert the problem to a sparse system of linear equations which are solved for millions of variables. We call our technique rGLUE: robust data term in GLobal Ultrasound Elastography. rGLUE has been validated using simulation, phantom, in vivo liver, and breast datasets. In all our experiments, rGLUE substantially outperforms the recent elastography methods both visually and quantitatively. For simulated, phantom, and in vivo datasets, respectively, rGLUE achieves 107%, 18%, and 23% improvements of signal-to-noise ratio (SNR) and 61%, 19%, and 25% improvements of contrast-to-noise ratio (CNR) over global ultrasound elastography (GLUE), a recently published elastography algorithm.

Combining First- and Second-Order Continuity Constraints in Ultrasound Elastography

Ultrasound elastography is a prominent noninvasive medical imaging technique that estimates tissue elastic properties to detect abnormalities in an organ. A common approximation to tissue elastic modulus is tissue strain induced after mechanical stimulation. To compute tissue strain, ultrasound radio frequency (RF) data can be processed using energy-based algorithms. These algorithms suffer from ill-posedness to tackle. A continuity constraint along with the data amplitude similarity is imposed to obtain a unique solution to the time-delay estimation (TDE) problem. Existing energy-based methods exploit the first-order spatial derivative of the displacement field to construct a regularizer. This first-order regularization scheme alone is not fully consistent with the mechanics of tissue deformation while perturbed with an external force. As a consequence, state-of-the-art techniques suffer from two crucial drawbacks. First, the strain map is not sufficiently smooth in uniform tissue regions. Second, the edges of the hard or soft inclusions are not well-defined in the image. Herein, we address these issues by formulating a novel regularizer taking both first- and second-order derivatives of the displacement field into account. The second-order constraint, which is the principal novelty of this work, contributes both to background continuity and edge sharpness by suppressing spurious noisy edges and enhancing strong boundaries. We name the proposed technique: Second-Order Ultrasound eLastography (SOUL). Comparative assessment of qualitative and quantitative results shows that SOUL substantially outperforms three recently developed TDE algorithms called Hybrid, GLUE, and MPWC-Net++. SOUL yields 27.72%, 62.56%, and 81.37% improvements of the signal-to-noise ratio (SNR) and 72.35%, 54.03%, and 65.17% improvements of the contrast-to-noise ratio (CNR) over GLUE with data pertaining to simulation, phantom, and in vivo tissue, respectively. The SOUL code can be downloaded from code.sonography.ai.

Displacement Estimation in Ultrasound Elastography Using Pyramidal Convolutional Neural Network

In this article, two novel deep learning methods are proposed for displacement estimation in ultrasound elastography (USE). Although convolutional neural networks (CNNs) have been very successful for displacement estimation in computer vision, they have been rarely used for USE. One of the main limitations is that the radio frequency (RF) ultrasound data, which is crucial for precise displacement estimation, has vastly different frequency characteristics compared with images in computer vision. Top-rank CNN methods used in computer vision applications are mostly based on a multilevel strategy, which estimates finer resolution based on coarser ones. This strategy does not work well for RF data due to its large high-frequency content. To mitigate the problem, we propose modified pyramid warping and cost volume network (MPWC-Net) and RFMPWC-Net, both based on PWC-Net, to exploit information in RF data by employing two different strategies. We obtained promising results using networks trained only on computer vision images. In the next step, we constructed a large ultrasound simulation database and proposed a new loss function to fine-tune the network to improve its performance. The proposed networks and well-known optical flow networks as well as state-of-the-art elastography methods are evaluated using simulation, phantom, and in vivo data. Our two proposed networks substantially outperform current deep learning methods in terms of contrast-to-noise ratio (CNR) and strain ratio (SR). Also, the proposed methods perform similar to the state-of-the-art elastography methods in terms of CNR and have better SR by substantially reducing the underestimation bias.

3D normalized cross-correlation for estimation of the displacement field in ultrasound elastography

This paper introduces a novel technique to estimate tissue displacement in quasi-static elastography. A major challenge in elastography is estimation of displacement (also referred to time-delay estimation) between pre-compressed and post-compressed ultrasound data. Maximizing normalized cross correlation (NCC) of ultrasound radio-frequency (RF) data of the pre- and post-compressed images is a popular technique for strain estimation due to its simplicity and computational efficiency. Several papers have been published to increase the accuracy and quality of displacement estimation based on NCC. All of these methods use 2D spatial windows in RF data to estimate NCC, wherein displacement is assumed to be constant within each window. In this work, we extend this assumption along the third dimension. Two approaches are proposed to get third dimension. In the first approach, we use temporal domain to exploit neighboring samples in both spatial and temporal directions. Considering temporal information is important since traditional and ultrafast ultrasound machines are, respectively, capable of imaging at more than 30 frame per second (fps) and 1000 fps. Another approach is to use time-delayed pre-beam formed data (channel data) instead of RF data. In this method information of all channels that are recorded as pre-beam formed data of each RF line will be considered as 3^rd dimension. We call these methods as spatial temporal normalized cross correlation (STNCC) and channel data normalized cross correlation (CNCC) and show that they substantially outperforms NCC using simulation, phantom and in-vivo experiments. Given substantial improvements of results in addition to the relative simplicity of implementing STNCC and CNCC, the proposed approaches can potentially have a large impact in both academic and commercial work on ultrasound elastography.

Hierarchical Motion Estimation With Bayesian Regularization in Cardiac Elastography: Simulation and In Vivo Validation

Cardiac elastography (CE) is an ultrasound-based technique utilizing radio-frequency (RF) signals for assessing global and regional myocardial function. In this work, a complete strain estimation pipeline for incorporating a Bayesian regularization-based hierarchical block-matching algorithm, with Lagrangian motion description and myocardial polar strain estimation is presented. The proposed regularization approach is validated using finite-element analysis (FEA) simulations of a canine cardiac deformation model that is incorporated into an ultrasound simulation program. Interframe displacements are initially estimated using a hierarchical motion estimation framework. Incremental displacements are then accumulated under a Lagrangian description of cardiac motion from end-diastole (ED) to end-systole (ES). In-plane Lagrangian finite strain tensors are then derived from the accumulated displacements. Cartesian to cardiac coordinate transformation is utilized to calculate radial and longitudinal strains for ease of interpretation. Benefits of regularization are demonstrated by comparing the same hierarchical block-matching algorithm with and without regularization. Application of Bayesian regularization in the canine FEA model provided improved ES radial and longitudinal strain estimation with statistically significant ( ) error reduction of 48.88% and 50.16%, respectively. Bayesian regularization also improved the quality of temporal radial and longitudinal strain curves with error reductions of 78.38% and 86.67% ( ), respectively. Qualitative and quantitative improvements were also visualized for in vivo results on a healthy murine model after Bayesian regularization. Radial strain elastographic signal-to-noise ratio (SNR_e) increased from 3.83 to 4.76 dB, while longitudinal strain SNR_e increased from 2.29 to 4.58 dB with regularization.

2D ultrasound imaging based intra-fraction respiratory motion tracking for abdominal radiation therapy using machine learning

We have previously developed a robotic ultrasound imaging system for motion monitoring in abdominal radiation therapy. Owing to the slow speed of ultrasound image processing, our previous system could only track abdominal motions under breath-hold. To overcome this limitation, a novel 2D-based image processing method for tracking intra-fraction respiratory motion is proposed. Fifty-seven different anatomical features acquired from 27 sets of 2D ultrasound sequences were used in this study. Three 2D ultrasound sequences were acquired with the robotic ultrasound system from three healthy volunteers. The remaining datasets were provided by the 2015 MICCAI Challenge on Liver Ultrasound Tracking. All datasets were preprocessed to extract the feature point, and a patient-specific motion pattern was extracted by principal component analysis and slow feature analysis (SFA). The tracking finds the most similar frame (or indexed frame) by a k-dimensional-tree-based nearest neighbor search for estimating the tracked object location. A template image was updated dynamically through the indexed frame to perform a fast template matching (TM) within a learned smaller search region on the incoming frame. The mean tracking error between manually annotated landmarks and the location extracted from the indexed training frame is 1.80  ±  1.42 mm. Adding a fast TM procedure within a small search region reduces the mean tracking error to 1.14  ±  1.16 mm. The tracking time per frame is 15 ms, which is well below the frame acquisition time. Furthermore, the anatomical reproducibility was measured by analyzing the location's anatomical landmark relative to the probe; the position-controlled probe has better reproducibility and yields a smaller mean error across all three volunteer cases, compared to the force-controlled probe (2.69 versus 11.20 mm in the superior-inferior direction and 1.19 versus 8.21 mm in the anterior-posterior direction). Our method reduces the processing time for tracking respiratory motion significantly, which can reduce the delivery uncertainty.

Global Ultrasound Elastography in Spatial and Temporal Domains

In this paper, a novel computationally efficient quasi-static ultrasound elastography technique is introduced by optimizing an energy function. Unlike conventional elastography techniques, three radio frequency (RF) frames are considered to devise a nonlinear cost function consisting of data intensity similarity term, spatial regularization terms and, most importantly, temporal continuity terms. We optimize the aforesaid cost function efficiently to obtain the time-delay estimation (TDE) of all samples between the first two and last two frames of ultrasound images simultaneously, and spatially differentiate the TDE to generate axial strain map. A novelty in our spatial and temporal regularizations is that they adaptively change based on the data, which leads to substantial improvements in TDE. We handle the computational complexity resulting from incorporation of all samples from all three frames by converting our optimization problem to a sparse linear system of equations. Consideration of both spatial and temporal continuity makes the algorithm more robust to signal decorrelation than the previous algorithms. We name the proposed method GUEST: Global Ultrasound Elastography in Spatial and Temporal directions. We validated our technique with simulation, experimental phantom, and in vivo liver data and compared the results with two recently proposed TDE methods. In all the experiments, GUEST substantially outperforms other techniques in terms of signal-to-noise ratio (SNR), contrast-to-noise ratio (CNR), and strain ratio (SR) of the strain images.

Spatial Angular Compounding With Affine-Model-Based Optical Flow for Improvement of Motion Estimation

Tissue motion estimation is an essential step for ultrasound elastography. Our previous study has shown that the affine-model-based optical flow (OF) method outperforms the normalized cross-correlation-based block matching (BM) method in motion estimation. However, the quality of lateral estimation using OF is still low due to inherent limitation of ultrasound imaging. BM-based spatial angular compounding (SAC) has been developed to obtain better motion estimation. In this paper, OF-based SAC (OF-SAC) is proposed to further improve the performance of lateral (and axial) estimation, and it is compared with BM-based SAC (BM-SAC). Plane wave as well as focused wave is transmitted in both simulations and phantom experiments on a linear array. In order to compare the performance quantitatively, the root-mean-square error (RMSE) of axial/lateral displacement and strain, and signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) of axial/lateral strain are used as the evaluation criteria in the simulations. In the phantom experiments, the SNR and CNR are used to assess the quality of axial/lateral strain. The results show that for both OF and BM, SAC improves the performance of motion estimation, regardless of using plane or focused wave transmission. More importantly, OF-SAC is shown to outperform BM-SAC with lower RMSE, higher SNR, and higher CNR. In addition, preliminary in vivo experiments on the carotid artery of a healthy human subject also prove the superiority of OF-SAC. These results suggest that OF-SAC is preferred for both axial and lateral motion estimation to BM-SAC.

Interoperator Reproducibility of Carotid Elastography for Identification of Vulnerable Atherosclerotic Plaques

Ultrasound-based carotid elastography has been developed to evaluate the vulnerability of carotid atherosclerotic plaques. The aim of this study was to investigate the in vivo interoperator reproducibility of carotid elastography for the identification of vulnerable plaques, with high-resolution magnetic resonance imaging (MRI) as reference. Ultrasound radio-frequency data of 45 carotid arteries (including 53 plaques) from 32 volunteers were acquired separately by two experienced operators in the longitudinal view and then were used to estimate the interframe axial strain rate (ASR) with a two-step optical flow method. The maximum 99th percentile of absolute ASR of all plaques in a carotid artery was used as the elastographic index. MRI scanning was also performed on each volunteer to identify the vulnerable plaque. The results showed no systematic bias in the Bland-Altman plot and an intraclass correlation coefficient of 0.66 between the two operators. In addition, no statistical significance was found between the receiver operating characteristic (ROC) curves from the two operators ( ), and their areas under the ROC curves were 0.83 and 0.77, respectively. Using the mean measurements of the two operators as the classification criterion, a sensitivity of 71.4%, a specificity of 87.1%, and an accuracy of 82.2% were obtained with a cutoff value of 1.37 [Formula: see text]. This study validates the interoperator reproducibility of ultrasound-based carotid elastography for identifying vulnerable carotid plaques.

A Systematic Investigation of Lateral Estimation Using Various Interpolation Approaches in Conventional Ultrasound Imaging

Accurate lateral displacement and strain estimation is critical for some applications of elasticity imaging. Typically, motion estimation in the lateral direction is challenging because of low sampling frequency and lack of phase information in conventional ultrasound imaging. Several approaches have been proposed to improve the performance of lateral estimation, such as lateral interpolation on the radio frequency (RF) signals (Interp_RF), lateral interpolation on the cross-correlation function (Interp_CCF), and lateral interpolation on both the RF signals and cross-correlation function (Interp_Both). In this paper, the estimation performances of the above-mentioned three approaches are compared systematically in simulations and phantom experiments. In the simulations, the root-mean-square error (RMSE) of axial/lateral displacement and strain is utilized to assess the accuracy of motion estimation. In the phantom experiments, the displacement quality metric (DQM), defined as the normalized cross-correlation between the motion-compensated reference frame and the comparison frame, and the contrast-to-noise ratio (CNR) of axial/lateral strain are used as the evaluation criteria. The results show that the three approaches have similar performance in axial estimation. For lateral estimation, if the line density of ultrasound imaging is relatively high (i.e., >4.2 lines/mm), Interp_CCF is comparable to Interp_Both, and Interp_RF performs the worst. However, if the line density is relatively low (i.e., <2.8 lines/mm), Interp_Both performs the best as indicated by the lowest RMSEs or highest DQMs and CNRs in lateral estimation. The trend is consistent at different window sizes, applied strains, and sonographic signal-to-noise ratios (>20 dB). Besides, Interp_Both with a small interpolation factor (e.g., 3-5) is found to obtain the best tradeoff between the estimation accuracy and the computational cost, and thus is suggested for lateral motion estimation in the case of a low line density (i.e., <2.8 lines/mm).

Estimation of Strain Elastography from Ultrasound Radio-Frequency Data by Utilizing Analytic Gradient of the Similarity Metric

Most strain imaging techniques follow a pipeline strategy: in the first step, tissue displacement is estimated from radio-frequency (RF) frames, and in the second step, a spatial derivative operation is applied. There are two main issues that arise from this framework. First, the gradient operation amplifies noise, and therefore, smoothing techniques have to be adopted. Second, strain estimation does not exploit the original RF data. It rather relies solely on the noisy displacement field. In this paper, a novel technique is proposed that utilizes both the displacement field and the RF frames to accurately obtain the strain estimates. The normalized cross correlation (NCC) metric between two corresponding windows around the samples of the pre- and post-compressed images is employed to generate a dissimilarity measurement. The derivative of NCC with respect to the strain is analytically derived using the chain rule. This allows an efficient minimization of the dissimilarity metric with respect to the strain using the gradient descent optimization technique. The effectiveness of the proposed method is investigated through simulation data, phantom experiments, and in vivo patient data. The experimental results show that exploiting the information in RF data significantly improves the strain estimates.

Non-Invasive Identification of Vulnerable Atherosclerotic Plaques Using Texture Analysis in Ultrasound Carotid Elastography: An In Vivo Feasibility Study Validated by Magnetic Resonance Imaging

The aims of this study were to quantify the textural information of strain rate images in ultrasound carotid elastography and evaluate the feasibility of using the textural features in discriminating stable and vulnerable plaques with magnetic resonance imaging as an in vivo reference. Ultrasound radiofrequency data were acquired in 80 carotid plaques from 52 patients, mainly in the longitudinal imaging view, and axial strain rate images were estimated with an ultrasound carotid elastography technique based on an optical flow algorithm. Four textural features of strain rate images-contrast, homogeneity, correlation and angular second moment-were derived based on the gray-level co-occurrence matrix in plaque regions to quantify the deformation distribution pattern. Conventional elastographic indices based on the magnitude of the absolute strain rate, such as the maximum, mean, median, standard deviation and 99th percentile of the axial strain rate, were also obtained for comparison. Composition measurement with magnetic resonance imaging identified 30 plaques as vulnerable and the other 50 as stable. The four textural features, as well as the magnitude of strain rate images, significantly differed between the two groups of plaques. The best performing features for plaque classification were found to be the contrast and 99th percentile of the absolute strain rate, with a comparative area under the receiver operating characteristic curve of 0.81; a slightly higher maximum accuracy of plaque classification can be achieved by the textural feature of contrast (83.8% vs. 81.3%). The results indicate that the use of texture analysis in plaque classification is feasible and that larger local deformations and higher level of complexity in deformation patterns (associated with the elastic or stiffness heterogeneity of plaque tissues) are more likely to occur in vulnerable plaques.

Clinical viability of carotid plaque strain estimation using B-mode ultrasound image sequences

It is estimated that approximately 30% of ischemic strokes are caused by rupture of plaque in the carotid artery. Development of techniques focusing on identifying plaques that are vulnerable to rupture is thus indispensable for stroke prevention. Recent studies have demonstrated that motion analysis of plaques from B-mode and RF ultrasound (US) image sequences can be used to estimate plaque strain. However, viability of these methods in a clinical setting, with variable acquisition protocols, has not been demonstrated yet. In this paper, we explore the viability of estimating plaque strain from B-mode US images of asymptomatic patients, acquired in a real clinical setting with different acquisition settings, frame rates, and operators. Our proposed strain measures, shear strain rate entropy and variance, combined with the recently reported maximum absolute shear strain rate, show that the plaques fall into two distinct clusters. Moreover, these clusters show good correlations with plaque echolucency and echogenicity. We conclude that B-mode US imaging is a viable tool for characterizing plaque dynamics in clinical environments. In future studies, we plan to implement this method on multi-center studies for longitudinal monitoring of plaque.

Systematic Performance Evaluation of a Cross-Correlation-Based Ultrasound Strain Imaging Method

Estimation of tissue motion in the lateral direction remains a major challenge in 2-D ultrasound strain imaging (USI). Although various methodologies have been proposed to improve the accuracy of estimation of in-plane displacements and strains, the fundamental limitations of 2-D USI and how to choose optimal algorithmic parameters in various tissue deformation paradigms to retrieve the full strain tensor of acceptable accuracy are scattered throughout the literature. Thus, this study attempts to provide a systematic investigation of a 2-D cross-correlation-based USI method in a theoretical framework. Our previously developed cross-correlation-based USI method was revisited, and additional estimation strategies were incorporated to improve in-plane displacement and strain estimation. The performance of the presented method using different matching kernel sizes (axial: from 1λ to 14λ, where λ = wavelength; lateral: from 1 to 13 pitches) and two data formats (radiofrequency and envelope) in various kinematic scenarios (normal, shear or hybrid deformation) was investigated using Field II simulations, in which coherent plane wave compounding with 64 steered angles was realized. For radiofrequency-based USI, smaller axial and larger lateral kernel sizes were preferred in scenarios with normal strains, whereas larger kernel sizes along the shearing direction and smaller ones orthogonal to the shearing direction were more suitable in scenarios with shear strains. For envelope-based USI, in contrast, the kernel size requirement was relatively relaxed. A compromise between optimal kernel sizes and estimation accuracy of various strain components was required in complex kinematic scenarios. These practical strategies for accurate motion estimation using 2-D cross-correlation-based USI were further tested in a tissue-mimicking phantom under quasi-static compression and in a preliminary in vivo examination of a normal human median nerve at the wrist during active finger motion.

Ultrasound-Based Carotid Elastography for Detection of Vulnerable Atherosclerotic Plaques Validated by Magnetic Resonance Imaging

Ultrasound-based carotid elastography has been developed to estimate the mechanical properties of atherosclerotic plaques. The objective of this study was to evaluate the in vivo capability of carotid elastography in vulnerable plaque detection using high-resolution magnetic resonance imaging as reference. Ultrasound radiofrequency data of 46 carotid plaques from 29 patients (74 ± 5 y old) were acquired and inter-frame axial strain was estimated with an optical flow method. The maximum value of absolute strain rate for each plaque was derived as an indicator for plaque classification. Magnetic resonance imaging of carotid arteries was performed on the same patients to classify the plaques into stable and vulnerable groups for carotid elastography validation. The maximum value of absolute strain rate was found to be significantly higher in vulnerable plaques (2.15 ± 0.79 s(-1), n = 27) than in stable plaques (1.21 ± 0.37 s(-1), n = 19) (p < 0.0001). Receiver operating characteristic curve analysis was performed, and the area under the curve was 0.848. Therefore, the in vivo capability of carotid elastography to detect vulnerable plaques, validated by magnetic resonance imaging, was proven, revealing the potential of carotid elastography as an important tool in atherosclerosis assessment and stroke prevention.