Neural-network-based Motion Tracking for Breast Ultrasound Strain Elastography: An Initial Assessment of Performance and Feasibility

Teacher-student guided knowledge distillation for unsupervised convolutional neural network-based speckle tracking in ultrasound strain elastography

Accurate and efficient motion estimation is a crucial component of real-time ultrasound elastography (USE). However, obtaining radiofrequency ultrasound (RF) data in clinical practice can be challenging. In contrast, although B-mode (BM) data is readily available, elastographic data derived from BM data results in sub-optimal elastographic images. Furthermore, existing conventional ultrasound devices (e.g., portable devices) cannot provide elastography modes, which has become a significant obstacle to the widespread use of traditional ultrasound devices. To address the challenges above, we developed a teacher-student guided knowledge distillation for an unsupervised convolutional neural network (TSGUPWC-Net) to improve the accuracy of BM motion estimation by employing a well-established convolutional neural network (CNN) named modified pyramid warping and cost volume network (MPWC-Net). A pre-trained teacher model based on RF is utilized to guide the training of a student model using BM data. Innovations outlined below include employing spatial attention transfer at intermediate layers to enhance the guidance effect of the model. The loss function consists of smoothness of the displacement field, knowledge distillation loss, and intermediate layer loss. We evaluated our method on simulated data, phantoms, and in vivo ultrasound data. The results indicate that our method has higher signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) values in axial strain estimation than the model trained on BM. The model is unsupervised and requires no ground truth labels during training, making it highly promising for motion estimation applications.

Alternating direction method of multipliers for displacement estimation in ultrasound strain elastography

BACKGROUND

Ultrasound strain imaging, which delineates mechanical properties to detect tissue abnormalities, involves estimating the time delay between two radio-frequency (RF) frames collected before and after tissue deformation. The existing regularized optimization-based time-delay estimation (TDE) techniques suffer from at least one of the following drawbacks: (1) The regularizer is not aligned with the tissue deformation physics due to taking only the first-order displacement derivative into account; (2) TheL 2 $L2$ -norm of the displacement derivatives, which oversmooths the estimated time-delay, is utilized as the regularizer; (3) The modulus function defined mathematically should be approximated by a smooth function to facilitate the optimization ofL 1 $L1$ -norm.

PURPOSE

Our purpose is to develop a novel TDE technique that resolves the aforementioned shortcomings of the existing algorithms.

METHODS

Herein, we propose employing the alternating direction method of multipliers (ADMM) for optimizing a novel cost function consisting ofL 2 $L2$ -norm data fidelity term andL 1 $L1$ -norm first- and second-order spatial continuity terms. ADMM empowers the proposed algorithm to use different techniques for optimizing different parts of the cost function and obtain high-contrast strain images with smooth backgrounds and sharp boundaries. We name our technique ADMM for totaL variaTion RegUlarIzation in ultrasound STrain imaging (ALTRUIST). ALTRUIST's efficacy is quantified using absolute error (AE), Structural SIMilarity (SSIM), signal-to-noise ratio (SNR), contrast-to-noise ratio (CNR), and strain ratio (SR) with respect to GLUE, OVERWIND, andL 1 $L1$ -SOUL, three recently published energy-based techniques, and UMEN-Net, a state-of-the-art deep learning-based algorithm. Analysis of variance (ANOVA)-led multiple comparison tests and paired t $t$ -tests at5 % $5\%$ overall significance level were conducted to assess the statistical significance of our findings. The Bonferroni correction was taken into account in all statistical tests. Two simulated layer phantoms, three simulated resolution phantoms, one hard-inclusion simulated phantom, one multi-inclusion simulated phantom, one experimental breast phantom, and three in vivo liver cancer datasets have been used for validation experiments. We have published the ALTRUIST code at http://code.sonography.ai.

RESULTS

ALTRUIST substantially outperforms the four state-of-the-art benchmarks in all validation experiments, both qualitatively and quantitatively. ALTRUIST yields up to573 % ∗ ${573\%}^{*}$ ,41 % ∗ ${41\%}^{*}$ , and51 % ∗ ${51\%}^{*}$ SNR improvements and443 % ∗ ${443\%}^{*}$ ,53 % ∗ ${53\%}^{*}$ , and15 % ∗ ${15\%}^{*}$ CNR improvements overL 1 $L1$ -SOUL, its closest competitor, for simulated, phantom, and in vivo liver cancer datasets, respectively, where the asterisk (*) indicates statistical significance. In addition, ANOVA-led multiple comparison tests and paired t $t$ -tests indicate that ALTRUIST generally achieves statistically significant improvements over GLUE, UMEN-Net, OVERWIND, andL 1 $L1$ -SOUL in terms of AE, SSIM map, SNR, and CNR.

CONCLUSIONS

A novel ultrasonic displacement tracking algorithm named ALTRUIST has been developed. The principal novelty of ALTRUIST is incorporating ADMM for optimizing anL 1 $L1$ -norm regularization-based cost function. ALTRUIST exhibits promising performance in simulation, phantom, and in vivo experiments.

Ultrasound Viscoelastography by Acoustic Radiation Force: A State-of-the-Art Review

Ultrasound elastography (USE) is a promising tool for tissue characterization as several diseases result in alterations of tissue structure and composition, which manifest as changes in tissue mechanical properties. By imaging the tissue response to an applied mechanical excitation, USE mimics the manual palpation performed by clinicians to sense the tissue elasticity for diagnostic purposes. Next to elasticity, viscosity has recently been investigated as an additional, relevant, diagnostic biomarker. Moreover, since biological tissues are inherently viscoelastic, accounting for viscosity in the tissue characterization process enhances the accuracy of the elasticity estimation. Recently, methods exploiting different acquisition and processing techniques have been proposed to perform ultrasound viscoelastography. After introducing the physics describing viscoelasticity, a comprehensive overview of the currently available USE acquisition techniques is provided, followed by a structured review of the existing viscoelasticity estimators classified according to the employed processing technique. These estimators are further reviewed from a clinical usage perspective, and current outstanding challenges are discussed.

Unsupervised deep learning-based displacement estimation for vascular elasticity imaging applications

Objective. Arterial wall stiffness can provide valuable information on the proper function of the cardiovascular system. Ultrasound elasticity imaging techniques have shown great promise as a low-cost and non-invasive tool to enable localized maps of arterial wall stiffness. Such techniques rely upon motion detection algorithms that provide arterial wall displacement estimation.Approach. In this study, we propose an unsupervised deep learning-based approach, originally proposed for image registration, in order to enable improved quality arterial wall displacement estimation at high temporal and spatial resolutions. The performance of the proposed network was assessed through phantom experiments, where various models were trained by using ultrasound RF signals, or B-mode images, as well as different loss functions.Main results. Using the mean square error (MSE) for the training process provided the highest signal-to-noise ratio when training on the B-modes images (30.36 ± 1.14 dB) and highest contrast-to-noise ratio when training on the RF signals (32.84 ± 1.89 dB). In addition, training the model on RF signals demonstrated the capability of providing accurate localized pulse wave velocity (PWV) maps, with a mean relative error (MREPWV) of 3.32 ± 1.80% and anR2 of 0.97 ± 0.03. Finally, the developed model was tested in human common carotid arteriesin vivo, providing accurate tracking of the distension pulse wave propagation, with an MREPWV= 3.86 ± 2.69% andR2 = 0.95 ± 0.03.Significance. In conclusion, a novel displacement estimation approach was presented, showing promise in improving vascular elasticity imaging techniques.

Lateral Strain Imaging Using Self-Supervised and Physically Inspired Constraints in Unsupervised Regularized Elastography

Convolutional Neural Networks (CNN) have shown promising results for displacement estimation in UltraSound Elastography (USE). Many modifications have been proposed to improve the displacement estimation of CNNs for USE in the axial direction. However, the lateral strain, which is essential in several downstream tasks such as the inverse problem of elasticity imaging, remains a challenge. The lateral strain estimation is complicated since the motion and the sampling frequency in this direction are substantially lower than the axial one, and a lack of carrier signal in this direction. In computer vision applications, the axial and the lateral motions are independent. In contrast, the tissue motion pattern in USE is governed by laws of physics which link the axial and lateral displacements. In this paper, inspired by Hooke's law, we, first propose Physically Inspired ConsTraint for Unsupervised Regularized Elastography (PICTURE), where we impose a constraint on the Effective Poisson's ratio (EPR) to improve the lateral strain estimation. In the next step, we propose self-supervised PICTURE (sPICTURE) to further enhance the strain image estimation. Extensive experiments on simulation, experimental phantom and in vivo data demonstrate that the proposed methods estimate accurate axial and lateral strain maps.

Convolutional Neural Network-Based Speckle Tracking for Ultrasound Strain Elastography: An Unsupervised Learning Approach

Accurate and computationally efficient motion estimation is a critical component of real-time ultrasound strain elastography (USE). With the advent of deep-learning neural network models, a growing body of work has explored supervised convolutional neural network (CNN)-based optical flow in the framework of USE. However, the above-said supervised learning was often done using simulated ultrasound data. The research community has questioned whether simulated ultrasound data containing simple motion can train deep-learning CNN models that can reliably track complex in vivo speckle motion. In parallel with other research groups' efforts, this study developed an unsupervised motion estimation neural network (UMEN-Net) for USE by adapting a well-established CNN model named PWC-Net. Our network's input is a pair of predeformation and postdeformation radio frequency (RF) echo signals. The proposed network outputs both axial and lateral displacement fields. The loss function consists of a correlation between the predeformation signal and the motion-compensated postcompression signal, smoothness of the displacement fields, and tissue incompressibility. Notably, an innovative correlation method known as the globally optimized correspondence (GOCor) volumes module developed by Truong et al. was used to replace the original Corr module to enhance our evaluation of signal correlation. The proposed CNN model was tested using simulated, phantom, and in vivo ultrasound data containing biologically confirmed breast lesions. Its performance was compared against other state-of-the-art methods, including two deep-learning-based tracking methods (MPWC-Net++ and ReUSENet) and two conventional tracking methods (GLUE and BRGMT-LPF). In summary, compared with the four known methods mentioned above, our unsupervised CNN model not only obtained higher signal-to-noise ratios (SNRs) and contrast-to-noise ratios (CNRs) for axial strain estimates but also improved the quality of the lateral strain estimates.

Shear-Wave Particle-Velocity Estimation and Enhancement Using a Multi-Resolution Convolutional Neural Network

OBJECTIVE

Tissue mechanical properties are valuable markers for tissue characterization, aiding in the detection and staging of pathologies. Shear wave elastography (SWE) offers a quantitative assessment of tissue mechanical characteristics based on the SW propagation profile, which is derived from the SW particle motion. Improving the signal-to-noise ratio (SNR) of the SW particle motion would directly enhance the accuracy of the material property estimates such as elasticity or viscosity.

METHODS

In this paper, we present a 3-D multi-resolution convolutional neural network (MRCNN) to perform improved estimation of the SW particle velocity V_z. Additionally, we propose a novel approach to generate training data from real acquisitions, providing high SNR ground truth target data, one-to-one paired to inputs that are corrupted with real-world noise and disturbances.

DISCUSSION

By testing the network on in vitro data acquired from a commercial breast elastography phantom, we show that the MRCNN outperforms Loupas' autocorrelation algorithm with an improved SNR of 4.47 dB for the V_z signals, a two-fold decrease in the standard deviation of the downstream elasticity estimates, and a two-fold increase in the contrast-to-noise ratio of the elasticity maps. The generalizability of the network was further demonstrated with a set of ex vivo porcine liver data.

CONCLUSION

The proposed MRCNN outperforms the standard autocorrelation method, in particular in low SNR regimes.

Vascular wall motion detection models based on long short-term memory in plane-wave-based ultrasound imaging

Objective.Vascular wall motion can be used to diagnose cardiovascular diseases. In this study, long short-term memory (LSTM) neural networks were used to track vascular wall motion in plane-wave-based ultrasound imaging.Approach.The proposed LSTM and convolutional LSTM (ConvLSTM) models were trained using ultrasound data from simulations and tested experimentally using a tissue-mimicking vascular phantom and anin vivostudy using a carotid artery. The performance of the models in the simulation was evaluated using the mean square error from axial and lateral motions and compared with the cross-correlation (XCorr) method. Statistical analysis was performed using the Bland-Altman plot, Pearson correlation coefficient, and linear regression in comparison with the manually annotated ground truth.Main results.For thein vivodata, the median error and 95% limit of agreement from the Bland-Altman analysis were (0.01, 0.13), (0.02, 0.19), and (0.03, 0.18), the Pearson correlation coefficients were 0.97, 0.94, and 0.94, respectively, and the linear equations were 0.89x+ 0.02, 0.84x+ 0.03, and 0.88x+ 0.03 from linear regression for the ConvLSTM model, LSTM model, and XCorr method, respectively. In the longitudinal and transverse views of the carotid artery, the LSTM-based models outperformed the XCorr method. Overall, the ConvLSTM model was superior to the LSTM model and XCorr method.Significance.This study demonstrated that vascular wall motion can be tracked accurately and precisely using plane-wave-based ultrasound imaging and the proposed LSTM-based models.

Motion Estimation by Deep Learning in 2D Echocardiography: Synthetic Dataset and Validation

Motion estimation in echocardiography plays an important role in the characterization of cardiac function, allowing the computation of myocardial deformation indices. However, there exist limitations in clinical practice, particularly with regard to the accuracy and robustness of measurements extracted from images. We therefore propose a novel deep learning solution for motion estimation in echocardiography. Our network corresponds to a modified version of PWC-Net which achieves high performance on ultrasound sequences. In parallel, we designed a novel simulation pipeline allowing the generation of a large amount of realistic B-mode sequences. These synthetic data, together with strategies during training and inference, were used to improve the performance of our deep learning solution, which achieved an average endpoint error of 0.07 ± 0.06 mm per frame and 1.20 ± 0.67 mm between ED and ES on our simulated dataset. The performance of our method was further investigated on 30 patients from a publicly available clinical dataset acquired from a GE system. The method showed promise by achieving a mean absolute error of the global longitudinal strain of 2.5 ± 2.1% and a correlation of 0.77 compared to GLS derived from manual segmentation, much better than one of the most efficient methods in the state-of-the-art (namely the FFT-Xcorr block-matching method). We finally evaluated our method on an auxiliary dataset including 30 patients from another center and acquired with a different system. Comparable results were achieved, illustrating the ability of our method to maintain high performance regardless of the echocardiographic data processed.

Deep learning in ultrasound elastography imaging: A review

It is known that changes in the mechanical properties of tissues are associated with the onset and progression of certain diseases. Ultrasound elastography is a technique to characterize tissue stiffness using ultrasound imaging either by measuring tissue strain using quasi-static elastography or natural organ pulsation elastography, or by tracing a propagated shear wave induced by a source or a natural vibration using dynamic elastography. In recent years, deep learning has begun to emerge in ultrasound elastography research. In this review, several common deep learning frameworks in the computer vision community, such as multilayer perceptron, convolutional neural network, and recurrent neural network are described. Then, recent advances in ultrasound elastography using such deep learning techniques are revisited in terms of algorithm development and clinical diagnosis. Finally, the current challenges and future developments of deep learning in ultrasound elastography are prospected. This article is protected by copyright. All rights reserved.

Unsupervised Convolutional Neural Network for Motion Estimation in Ultrasound Elastography

High-quality motion estimation is essential for ultrasound elastography (USE). Traditional motion estimation algorithms based on speckle tracking such as normalized cross correlation (NCC) or regularization such as global ultrasound elastography (GLUE) are time-consuming. In order to reduce the computational cost and ensure the accuracy of motion estimation, many convolutional neural networks have been introduced into USE. Most of these networks such as radio-frequency modified pyramid, warping and cost volume network (RFMPWC-Net) are supervised and need many ground truths as labels in network training. However, the ground truths are laborious to collect for USE. In this study, we proposed a MaskFlownet-based unsupervised convolutional neural network (MF-UCNN) for fast and high-quality motion estimation in USE. The inputs to MF-UCNN are the concatenation of RF, envelope, and B-mode images before and after deformation, while the outputs are the axial and lateral displacement fields. The similarity between the predeformed image and the warped image (i.e., the postdeformed image compensated by the estimated displacement fields) and the smoothness of the estimated displacement fields were incorporated in the loss function. The network was compared with modified pyramid, warping and cost volume network (MPWC-Net)++, RFMPWC-Net, GLUE, and NCC. Results of simulations, breast phantom, and in vivo experiments show that MF-UCNN obtains higher signal-to-noise ratio (SNR) and higher contrast-to-noise ratio (CNR). MF-UCNN achieves high-quality motion estimation with significantly reduced computation time. It is unsupervised and does not need any ground truths as labels in the training, and, thus, has great potential for motion estimation in USE.

Therapeutic Ultrasound Effects on Human Induced Pluripotent Stem Cell Cardiomyocytes Measured Optically and with Spectral Ultrasound

To the best of our knowledge, therapeutic ultrasound (TUS) is thus far an unexplored means of delivering mechanical stimulation to cardiomyocyte cultures, which is necessary to engineer a more mature cardiomyocyte phenotype in vitro. Spectral ultrasound (SUS) may provide a way to non-invasively, non-disruptively and inexpensively monitor growth and change in cell cultures over long periods. Compared with other measurement methods, SUS as an acoustic measurement tool will not be affected by an acoustic therapy, unlike electrical measurement methods, in which motion caused by acoustic therapy can affect measurements. Further SUS has the potential to provide functional as well as morphological information in cell cultures. Human induced pluripotent stem cell cardiomyocytes (iPS-CMs) were imaged with calcium fluorescence microscopy while TUS was being applied. TUS was applied at 600 kHz and 1, 3.4 and 6 W/cm² for a continuous 1 s pulse. Measures of the instantaneous beat frequency, repolarization rate and calcium spike amplitude were calculated from the fluorescence data. At 600 kHz, TUS at 1 and 6 W/cm² had significant effects on the shortening of both the repolarization rate and instantaneous beat rate of the iPS-CMs (p < 0.05), while TUS at 3.4 and 6 W/cm² had significant effects on the shortening of the calcium spike amplitude (p < 0.05). Three SUS measures and one gray-level measure were captured from the iPS-CM monolayers while they were simultaneously being imaged with calcium-labeled confocal microscopy. The gray-level measure performed the best of all SUS measures; however, it was not reliable enough to produce a consistent determination of the beat rate of the cell. Finally, SUS measures were captured using three different transducers while simultaneously applying TUS. A center-of-mass (COM) measure calculated from the wavelet transform scalogram of the time-averaged radiofrequency data revealed that SUS was able to detect a change in the frequency content of the reflected ultrasound at 1 and 6 W/cm² before and after ultrasound application (p < 0.05), showing promise for the ability of SUS to measure changes in the beating behavior of iPS-CMs. Overall, SUS is promising as a method for constant monitoring of dynamic cell and tissue culture and growth.

Incorporating Gradient Similarity for Robust Time Delay Estimation in Ultrasound Elastography

Energy-based ultrasound elastography techniques minimize a regularized cost function consisting of data and continuity terms to obtain local displacement estimates based on the local time-delay estimation (TDE) between radio frequency (RF) frames. The data term associated with the existing techniques takes only the amplitude similarity into account and hence is not sufficiently robust to the outlier samples present in the RF frames under consideration. This drawback creates noticeable artifacts in the strain image. To resolve this issue, we propose to formulate the data function as a linear combination of the amplitude and gradient similarity constraints. We estimate the adaptive weight concerning each similarity term following an iterative scheme. Finally, we optimize the nonlinear cost function in an efficient manner to convert the problem to a sparse system of linear equations which are solved for millions of variables. We call our technique rGLUE: robust data term in GLobal Ultrasound Elastography. rGLUE has been validated using simulation, phantom, in vivo liver, and breast datasets. In all our experiments, rGLUE substantially outperforms the recent elastography methods both visually and quantitatively. For simulated, phantom, and in vivo datasets, respectively, rGLUE achieves 107%, 18%, and 23% improvements of signal-to-noise ratio (SNR) and 61%, 19%, and 25% improvements of contrast-to-noise ratio (CNR) over global ultrasound elastography (GLUE), a recently published elastography algorithm.

Bi-Directional Semi-Supervised Training of Convolutional Neural Networks for Ultrasound Elastography Displacement Estimation

The performance of ultrasound elastography (USE) heavily depends on the accuracy of displacement estimation. Recently, convolutional neural networks (CNNs) have shown promising performance in optical flow estimation and have been adopted for USE displacement estimation. Networks trained on computer vision images are not optimized for USE displacement estimation since there is a large gap between the computer vision images and the high-frequency radio frequency (RF) ultrasound data. Many researchers tried to adopt the optical flow CNNs to USE by applying transfer learning to improve the performance of CNNs for USE. However, the ground-truth displacement in real ultrasound data is unknown, and simulated data exhibit a domain shift compared to the real data and are also computationally expensive to generate. To resolve this issue, semisupervised methods have been proposed in which the networks pretrained on computer vision images are fine-tuned using real ultrasound data. In this article, we employ a semisupervised method by exploiting the first- and second-order derivatives of the displacement field for regularization. We also modify the network structure to estimate both forward and backward displacements and propose to use consistency between the forward and backward strains as an additional regularizer to further enhance the performance. We validate our method using several experimental phantom and in vivo data. We also show that the network fine-tuned by our proposed method using experimental phantom data performs well on in vivo data similar to the network fine-tuned on in vivo data. Our results also show that the proposed method outperforms current deep learning methods and is comparable to computationally expensive optimization-based algorithms.

An unsupervised learning approach to ultrasound strain elastography with spatio-temporal consistency

Quasi-static ultrasound elastography (USE) is an imaging modality that measures deformation (i.e. strain) of tissue in response to an applied mechanical force. In USE, the strain modulus is traditionally obtained by deriving the displacement field estimated between a pair of radio-frequency data. In this work we propose a recurrent network architecture with convolutional long-short-term memory decoder blocks to improve displacement estimation and spatio-temporal continuity between time series ultrasound frames. The network is trained in an unsupervised way, by optimising a similarity metric between the reference and compressed image. Our training loss is also composed of a regularisation term that preserves displacement continuity by directly optimising the strain smoothness, and a temporal continuity term that enforces consistency between successive strain predictions. In addition, we propose an open-access in vivo database for quasi-static USE, which consists of radio-frequency data sequences captured on the arm of a human volunteer. Our results from numerical simulation and in vivo data suggest that our recurrent neural network can account for larger deformations, as compared with two other feed-forward neural networks. In all experiments, our recurrent network outperformed the state-of-the-art for both learning-based and optimisation-based methods, in terms of elastographic signal-to-noise ratio, strain consistency, and image similarity. Finally, our open-source code provides a 3D-slicer visualisation module that can be used to process ultrasound RF frames in real-time, at a rate of up to 20 frames per second, using a standard GPU.

A robust time delay estimation method for ultrasonic echo signals and elastography

Modern medicine cannot ignore the significance of elastography in diagnosis and treatment plans. Despite improvements in accuracy and spatial resolution of elastograms, robustness against noise remains a neglected attribute. A method that can perform in a satisfactory manner under noisy conditions may prove useful for various elastography methods. Here, we propose a method based on eigenvalue decomposition (EVD). In this method, the estimated time delay is defined as the index of the maximum element in the eigenvector that corresponds to the minimum eigenvalue in the covariance matrix of the received signal. Moreover, the implementation of the least-squares (LS) solution and the lower-upper (LU) decomposition contributes to improving the speed of computation and the accuracy of the estimation under low signal-to-noise ratio (SNR) conditions. To assess the performance of the proposed algorithm, it is evaluated along with generalized cross-correlation (GCC) and EVD. The simulation results clearly confirm that the jitter variance achieved in the proposed algorithm outperforms GCC and EVD in the proximity of the Cramer-Rau lower band. Moreover, our algorithm provides satisfactory performance in terms of variance and bias against sub-sample delay at low SNRS. According to the experimental results, the calculated values of the elastographic signal-to-noise ratio (SNRe) and the elastographic contrast-to-noise ratio (CNRe) of the proposed algorithm were 16.7 and 20.09, respectively, clearly better than the values of the other two methods. Furthermore, the proposed algorithm offers less execution time (about 30% of GCC), with a computational complexity equal to GCC and better than EVD.

Deep Convolutional Neural Networks for Displacement Estimation in ARFI Imaging

Ultrasound elasticity imaging in soft tissue with acoustic radiation force requires the estimation of displacements, typically on the order of several microns, from serially acquired raw data A-lines. In this work, we implement a fully convolutional neural network (CNN) for ultrasound displacement estimation. We present a novel method for generating ultrasound training data, in which synthetic 3-D displacement volumes with a combination of randomly seeded ellipsoids are created and used to displace scatterers, from which simulated ultrasonic imaging is performed using Field II. Network performance was tested on these virtual displacement volumes, as well as an experimental ARFI phantom data set and a human in vivo prostate ARFI data set. In the simulated data, the proposed neural network performed comparably to Loupas's algorithm, a conventional phase-based displacement estimation algorithm; the rms error was [Formula: see text] for the CNN and 0.73 [Formula: see text] for Loupas. Similarly, in the phantom data, the contrast-to-noise ratio (CNR) of a stiff inclusion was 2.27 for the CNN-estimated image and 2.21 for the Loupas-estimated image. Applying the trained network to in vivo data enabled the visualization of prostate cancer and prostate anatomy. The proposed training method provided 26 000 training cases, which allowed robust network training. The CNN had a computation time that was comparable to Loupas's algorithm; further refinements to the network architecture may provide an improvement in the computation time. We conclude that deep neural network-based displacement estimation from ultrasonic data is feasible, providing comparable performance with respect to both accuracy and speed compared to current standard time-delay estimation approaches.

Fast Strain Estimation and Frame Selection in Ultrasound Elastography Using Machine Learning

Ultrasound elastography aims to determine the mechanical properties of the tissue by monitoring tissue deformation due to internal or external forces. Tissue deformations are estimated from ultrasound radio frequency (RF) signals and are often referred to as time delay estimation (TDE). Given two RF frames I₁ and I₂ , we can compute a displacement image, which shows the change in the position of each sample in I₁ to a new position in I₂ . Two important challenges in TDE include high computational complexity and the difficulty in choosing suitable RF frames. Selecting suitable frames is of high importance because many pairs of RF frames either do not have acceptable deformation for extracting informative strain images or are decorrelated and deformation cannot be reliably estimated. Herein, we introduce a method that learns 12 displacement modes in quasi-static elastography by performing principal component analysis (PCA) on displacement fields of a large training database. In the inference stage, we use dynamic programming (DP) to compute an initial displacement estimate of around 1% of the samples and then decompose this sparse displacement into a linear combination of the 12 displacement modes. Our method assumes that the displacement of the whole image could also be described by this linear combination of principal components. We then use the GLobal Ultrasound Elastography (GLUE) method to fine-tune the result yielding the exact displacement image. Our method, which we call PCA-GLUE, is more than 10× faster than DP in calculating the initial displacement map while giving the same result. This is due to converting the problem of estimating millions of variables in DP into a much simpler problem of only 12 unknown weights of the principal components. Our second contribution in this article is determining the suitability of the frame pair I₁ and I₂ for strain estimation, which we achieve by using the weight vector that we calculated for PCA-GLUE as an input to a multilayer perceptron (MLP) classifier. We validate PCA-GLUE using simulation, phantom, and in vivo data. Our classifier takes only 1.5 ms during the testing phase and has an F1-measure of more than 92% when tested on 1430 instances collected from both phantom and in vivo data sets.

Real-time and High Quality Ultrasound Elastography Using Convolutional Neural Network by Incorporating Analytic Signal

Convolutional Neural Networks (CNN) have been extensively used for many computer vision applications including optical flow estimation. Although CNNs have been very successful in optical flow problem, they have been rarely used for displacement estimation in Ultrasound Elastography (USE) due to vast differences between ultrasound data and computer vision images. In USE, a main goal is to obtain the strain image which is the derivative of the axial displacement in axial direction; therefore, a very accurate displacement estimation is required. Radio Frequency (RF) data is needed to obtain accurate displacement estimation. RF data contains high frequency contents which cannot be downsampled without significant loss of information, in contrast to computer vision images. We propose a novel technique to utilize LiteFlowNet for USE. For the first time, we incorporate analytic signal to improve the quality of the displacement estimation. We show that this network with the designed inputs is more suitable for USE compared to more complex networks such as FlowNet2. The network is adopted to our application and it is compared with FlowNet2 and a state-of-the-art elastography method (GLUE). The results show that this network performs well and comparable to GLUE. Furthermore, not only this network is faster and has lower memory footprint compared to FlowNet2, but also it obtains higher quality strain images which makes it suitable for portable and real-time elastography devices.

Accurate and Precise Time-Delay Estimation for Ultrasound Elastography With Prebeamformed Channel Data

Free-hand palpation ultrasound elastography is a noninvasive approach for detecting pathological alteration in tissue. In this method, the tissue is compressed by a handheld probe and displacement of each sample is estimated, a process which is also known as time-delay estimation (TDE). Even with the simplifying assumption that ignores out of plane motion, TDE is an ill-posed problem requiring estimation of axial and lateral displacements for each sample from its intensity. A well-known class of methods for making elastography a well-posed problem is regularized optimization-based methods, which imposes smoothness regularization in the associated cost function. In this article, we propose to utilize channel data that have been compensated for time gain and time delay (introduced by transmission) instead of postbeamformed radio frequency (RF) data in the optimization problem. We name our proposed method Channel data for GLobal Ultrasound Elastography (CGLUE). We analytically derive bias and variances of TDE as functions of data noise for CGLUE and Global Ultrasound Elastography (GLUE) and use the Cauchy-Schwarz inequality to prove that CGLUE provides a TDE with lower bias and variance error. To further illustrate the improved performance of CGLUE, the results of simulation, experimental phantom, and ex-vivo experiments are presented.