Building a virtual simulation platform for quasistatic breast ultrasound elastography using open source software: A preliminary investigation

Investigating pulse-echo sound speed estimation in breast ultrasound with deep learning

Ultrasound is an adjunct tool to mammography that can quickly and safely aid physicians in diagnosing breast abnormalities. Clinical ultrasound often assumes a constant sound speed to form diagnostic B-mode images. However, the components of breast tissue, such as glandular tissue, fat, and lesions, differ in sound speed. Given a constant sound speed assumption, these differences can degrade the quality of reconstructed images via phase aberration. Sound speed images can be a powerful tool for improving image quality and identifying diseases if properly estimated. To this end, we propose a supervised deep-learning approach for sound speed estimation from analytic ultrasound signals. We develop a large-scale simulated ultrasound dataset that generates representative breast tissue samples by modeling breast gland, skin, and lesions with varying echogenicity and sound speed. We adopt a fully convolutional neural network architecture trained on a simulated dataset to produce an estimated sound speed map. The simulated tissue is interrogated with a plane wave transmit sequence, and the complex-value reconstructed images are used as input for the convolutional network. The network is trained on the sound speed distribution map of the simulated data, and the trained model can estimate sound speed given reconstructed pulse-echo signals. We further incorporate thermal noise augmentation during training to enhance model robustness to artifacts found in real ultrasound data. To highlight the ability of our model to provide accurate sound speed estimations, we evaluate it on simulated, phantom, and in-vivo breast ultrasound data.

Convolutional Neural Network-Based Speckle Tracking for Ultrasound Strain Elastography: An Unsupervised Learning Approach

Accurate and computationally efficient motion estimation is a critical component of real-time ultrasound strain elastography (USE). With the advent of deep-learning neural network models, a growing body of work has explored supervised convolutional neural network (CNN)-based optical flow in the framework of USE. However, the above-said supervised learning was often done using simulated ultrasound data. The research community has questioned whether simulated ultrasound data containing simple motion can train deep-learning CNN models that can reliably track complex in vivo speckle motion. In parallel with other research groups' efforts, this study developed an unsupervised motion estimation neural network (UMEN-Net) for USE by adapting a well-established CNN model named PWC-Net. Our network's input is a pair of predeformation and postdeformation radio frequency (RF) echo signals. The proposed network outputs both axial and lateral displacement fields. The loss function consists of a correlation between the predeformation signal and the motion-compensated postcompression signal, smoothness of the displacement fields, and tissue incompressibility. Notably, an innovative correlation method known as the globally optimized correspondence (GOCor) volumes module developed by Truong et al. was used to replace the original Corr module to enhance our evaluation of signal correlation. The proposed CNN model was tested using simulated, phantom, and in vivo ultrasound data containing biologically confirmed breast lesions. Its performance was compared against other state-of-the-art methods, including two deep-learning-based tracking methods (MPWC-Net++ and ReUSENet) and two conventional tracking methods (GLUE and BRGMT-LPF). In summary, compared with the four known methods mentioned above, our unsupervised CNN model not only obtained higher signal-to-noise ratios (SNRs) and contrast-to-noise ratios (CNRs) for axial strain estimates but also improved the quality of the lateral strain estimates.

Characterizing Mechanical Properties of Soft Tissues Using Non-contact Displacement Measurements: How Should We Assess the Uncertainty?

Techniques aimed at the non-invasive characterization of soft tissues according to elastic properties are rapidly evolving. Virtual touch-based elastographic methods including acoustic radiation force imaging (ARFI) and optical elastography measure the peak axial displacement (PD) and time-to-peak-displacement (TTP) of tissue in response to a localized force. These measurements have been used clinically to differentiate tissues, albeit with mixed results. However, to date, the reason has not been fully understood. In this study, we apply a novel modeling approach to explore the mechanistic link between simplistic displacement measurements and tissue viscoelasticity in the application of virtual touch-based elastographic methods to staging chronic liver disease (CLD). To our knowledge, such a study has not been reported in the literature. Specifically, a numerical screening study was first conducted to identify factors that most strongly determine PD and TTP. Response surface experimental designs were then applied to these factors to produce meta-models of expected PD and TTP probability density functions (PDFs) as functions of identified factors. Results from the screening study suggest that both PD and TTP measurements are primarily influenced by three factors: the initial Young's modulus of the tissue, the first viscoelastic Prony series time constant, and pre-compression applied during acquisition. To investigate the implications of these results, stochastic inputs for these three factors associated were used to determine a robust response surface. The identified response surface methodology can be used to determine optimal cutoff values for PD and TTP that could be used in order to stage chronic liver disease.

An Improved Region-Growing Motion Tracking Method Using More Prior Information for 3-D Ultrasound Elastography

Three-dimensional (3-D) ultrasound elastography can provide 3-D tissue stiffness information that may be used during clinical diagnoses. In the framework of strain elastography, motion tracking plays an important role. In this study, an improved 3-D region-growing motion tracking (RGMT) algorithm based on a concept of exterior boundary points was developed. In principle, the proposed method first determines displacement at some seed points by strictly checking the local correlation and continuity in the neighborhood of those seeds. Subsequent displacement estimation is then conducted from these initial seeds to obtain displacements associated with other locations. This RGMT algorithm is designed to use more known information-including displacements and correlation values of all known-displacement neighboring points-to estimate the displacement of an unknown-displacement point, whereas previous RGMT methods employed information from only one such point. The algorithm was tested on 3-D ultrasound volumetric data acquired from a simulation, a tissue-mimicking phantom, and five human subjects. Motion-compensated cross correlations (MCCCs), strain contrast, and displacement Laplacian values (representing smoothness of an estimated displacement field) were calculated and used to evaluate the merits of the proposed RGMT method. Compared with a previously published RGMT method, the results show that the proposed RGMT method can provide smaller displacement errors and smoother displacements and improve strain contrast while maintaining reasonably high MCCC values, indicating good motion tracking quality. The proposed method is also computationally more efficient. In summary, our preliminary results demonstrated that the proposed RGMT algorithm is capable of obtaining high-quality 3-D strain elastographic data using modified clinical equipment.

Neural-network-based Motion Tracking for Breast Ultrasound Strain Elastography: An Initial Assessment of Performance and Feasibility

Accurate tracking of tissue motion is critically important for several ultrasound elastography methods. In this study, we investigate the feasibility of using three published convolution neural network (CNN) models built for optical flow (hereafter referred to as CNN-based tracking) by the computer vision community for breast ultrasound strain elastography. Elastographic datasets produced by finite element and ultrasound simulations were used to retrain three published CNN models: FlowNet-CSS, PWC-Net, and LiteFlowNet. After retraining, the three improved CNN models were evaluated using computer-simulated and tissue-mimicking phantoms, and in vivo breast ultrasound data. CNN-based tracking results were compared with two published two-dimensional (2D) speckle tracking methods: coupled tracking and GLobal Ultrasound Elastography (GLUE) methods. Our preliminary data showed that, based on the Wilcoxon rank-sum tests, the improvements due to retraining were statistically significant (p < 0.05) for all three CNN models. We also found that the PWC-Net model was the best neural network model for data investigated, and its overall performance was on par with the coupled tracking method. CNR values estimated from in vivo axial and lateral strain elastograms showed that the GLUE algorithm outperformed both the retrained PWC-Net model and the coupled tracking method, though the GLUE algorithm exhibited some biases. The PWC-Net model was also able to achieve approximately 45 frames/second for 2D speckle tracking data investigated.

Modeling Uncertainty of Strain Ratio Measurements in Ultrasound Breast Strain Elastography: A Factorial Experiment

Strain elastography (SE) is a technique in which images of localized tissue strains are used to detect the relative stiffness of tissues. The application of SE in differentiating malignant breast lesions from benign ones has been broadly investigated. The strain ratio (SR) between the background and the breast tumor has been used and its results have been mixed. Due to the complex nature of tissue elasticity and how it relates to the strain fields measured in SE, the exact reason is not known. In this study, we apply a novel design-of-experiments-based metamodeling approach to mechanical simulation of SE in the human breast. To our knowledge, such a study has not been reported in the ultrasound SE literature. More specifically, we first conduct a screening study to identify the biomechanical factors/simulation inputs that most strongly determine SR. We then apply a response surface experimental design to these factors to produce a metamodel of SR as a function of said factors. Results from the screening study suggest that the SR measurements are primarily influenced by three factors: the initial shear modulus of the lesion, the elastic nonlinearity of the lesion, and the precompression applied during acquisition. In order to investigate the implications of these results, stochastic inputs for these three factors associated with the malignant and benign cases were applied to the resulting response surface. The resulting optimal cutoffs, sensitivity, and specificity were generally in line with a majority (>60%) of 19 clinical trials in the literature.

Combining Total Variation Regularization with Window-Based Time Delay Estimation in Ultrasound Elastography

A major challenge of free-hand palpation ultrasound elastography (USE) is estimating the displacement of RF samples between pre- and post-compressed RF data. The problem of displacement estimation is ill-posed since the displacement of one sample by itself cannot be uniquely calculated. To resolve this problem, two categories of methods have emerged. The first category assumes that the displacement of samples within a small window surrounding the reference sample is constant. The second class imposes smoothness regularization and optimizes an energy function. Herein, we propose a novel method that combines both approaches, and as such, is more robust to noise. The second contribution of this work is the introduction of the L1 norm as the regularization term in our cost function, which is often referred to as the total variation (TV) regularization. Compared to previous work that used the L2 norm regularization, optimization of the new cost function is more challenging. However, the advantages of using the L1 norm are twofold. First, it leads to substantial improvement in the sharpness of displacement estimates. Second, to optimize the cost function with the L1 norm regularization, we use an iterative method that further increases the robustness. We name our proposed method tOtal Variation Regularization and WINDow-based time delay estimation (OVERWIND) and show that it is robust to signal decorrelation and generates sharp displacement and strain maps for simulated, experimental phantom and in-vivo data. In particular, OVERWIND improves strain contrast-to-noise ratio (CNR) by 27.26%, 144.05%, and 49.90% on average in simulation, phantom, and in-vivo data, respectively, compared to our recent Global Ultrasound Elastography (GLUE) method.

A REAL-TIME MEDICAL ULTRASOUND SIMULATOR BASED ON A GENERATIVE ADVERSARIAL NETWORK MODEL

This paper presents an artificial intelligence-based ultrasound simulator suitable for medical simulation and clinical training. Particularly, we propose a machine learning approach to realistically simulate ultrasound images based on generative adversarial networks (GANs). Using B-mode ultrasound images simulated by a known ultrasound simulator, Field II, an "image-to-image" ultrasound simulator was trained. Then, through evaluations, we found that the GAN-based simulator can generate B-mode images following Rayleigh scattering. Our preliminary study demonstrated that ultrasound B-mode images from anatomies inferred from magnetic resonance imaging (MRI) data were feasible. While some image blurring was observed, ultrasound B- mode images obtained were both visually and quantitatively comparable to those obtained using the Field II simulator. It is also important to note that the GAN-based ultrasound simulator was computationally efficient and could achieve a frame rate of 15 frames/second using a regular laptop computer. In the future, the proposed GAN-based simulator will be used to synthesize more realistic looking ultrasound images with artifacts such as shadowing.

4D cardiac electromechanical activation imaging

Cardiac abnormalities, a major cause of morbidity and mortality, affect millions of people worldwide. Despite the urgent clinical need for early diagnosis, there is currently no noninvasive technique that can infer to the electrical function of the whole heart in 3D and thereby localize abnormalities at the point of care. Here we present a new method for noninvasive 4D mapping of the cardiac electromechanical activity in a single heartbeat for heart disease characterization such as arrhythmia and infarction. Our novel technique captures the 3D activation wave of the heart in vivo using high volume-rate (500 volumes per second) ultrasound with a 32 × 32 matrix array. Electromechanical activation maps are first presented in a normal and infarcted cardiac model in silico and in canine heart during pacing and re-entrant ventricular tachycardia in vivo. Noninvasive 4D electromechanical activation mapping in a healthy volunteer and a heart failure patient are also determined. The technique described herein allows for direct, simultaneous and noninvasive visualization of electromechanical activation in 3D, which provides complementary information on myocardial viability and/or abnormality to clinical imaging.

A two-dimensional (2D) systems biology-based discrete liver tissue model: A simulation study with implications for ultrasound elastography of liver fibrosis

Continuum tissue models that were often used to simulate or analyze the mechanical properties of tissues being imaged may not be biologically realistic. Our primary objective was to establish the feasibility of using systems biology to construct biologically relevant tissue models linking tissue structure, composition and architecture to the ultrasound measurements directly. The first application was designated to model fibrotic liver tissues. The proposed liver tissue model leveraged established histopathology knowledge of fibrotic liver tissues. Particularly, rules of systems biology derived from molecular histopathology were first implemented into an agent-based software platform SPARK to reflect progressions of liver fibrosis with/without steatosis. Then, microscopic compositions of tissues (e.g. cellular components) were converted to computing grids (at the 50-100 μm scale) for wave simulations using an open-source K-Wave. To verify the physical soundness of the proposed model, virtual wave speed measurements (i.e. shear wave speed [SWS] and the speed of sound [SOS]) were performed. Our initial results demonstrated that the simulated SWS values increased with the progression of liver fibrosis (from 1.5 m/s [Fibrosis stage 1] to 4 m/s [Fibrosis stage 4]). Similarly, the simulated SOS values were within the range of clinical data (from 1575 m/s [Fibrosis stage 0-3] to 1594 m/s [Fibrosis stage 4]). In summary, we found that those systems biology simulated fibrotic liver tissues with and without steatosis can reflect spatial characteristics of relevant histology. Also, their mechanical characteristics (i.e. shear/compressional wave speed) were in good agreement with data reported in the clinical literature.

Acoustic Radiation Force Based Ultrasound Elasticity Imaging for Biomedical Applications

Pathological changes in biological tissue are related to the changes in mechanical properties of biological tissue. Conventional medical screening tools such as ultrasound, magnetic resonance imaging or computed tomography have failed to produce the elastic properties of biological tissues directly. Ultrasound elasticity imaging (UEI) has been proposed as a promising imaging tool to map the elastic parameters of soft tissues for the clinical diagnosis of various diseases include prostate, liver, breast, and thyroid gland. Existing UEI-based approaches can be classified into three groups: internal physiologic excitation, external excitation, and acoustic radiation force (ARF) excitation methods. Among these methods, ARF has become one of the most popular techniques for the clinical diagnosis and treatment of disease. This paper provides comprehensive information on the recently developed ARF-based UEI techniques and instruments for biomedical applications. The mechanical properties of soft tissue, ARF and displacement estimation methods, working principle and implementation instruments for each ARF-based UEI method are discussed.

Antibody with Infinite Affinity for In Vivo Tracking of Genetically Engineered Lymphocytes

There remains an urgent need for the noninvasive tracking of transfused chimeric antigen receptor (CAR) T cells to determine their biodistribution, viability, expansion, and antitumor functionality. DOTA antibody reporter 1 (DAbR1) comprises a single-chain fragment of the antilanthanoid-DOTA antibody 2D12.5/G54C fused to the human CD4-transmembrane domain and binds irreversibly to lanthanoid (S)-2-(4-acrylamidobenzyl)-DOTA (AABD). The aim of this study was to investigate whether DAbR1 can be expressed on lymphocytes and used as a reporter gene as well as a suicide gene for therapy of immune-related adverse effects. Methods: DAbR1 was subcloned together with green fluorescent protein into an SFG-retroviral vector and used to transduce CD3/CD28-activated primary human T cells and second-generation 1928z (CAR) T cells. Cell surface expression of DAbR1 was confirmed by cell uptake studies with radiolabeled AABD. In addition, the feasibility of imaging of DAbR1-positive T cells in vivo after intravenous injection of ⁸⁶Y/¹⁷⁷Lu-AABD was studied and radiation doses determined. Results: A panel of DAbR1-expressing T cells and CAR T cells exhibited greater than 8-fold increased uptake of ⁸⁶Y-AABD in vitro when compared with nontransduced cells. Imaging studies showed ⁸⁶Y-AABD was retained by DAbR1-positive T cells while it continuously cleared from normal tissues, allowing for in vivo tracking of intravenously administered CAR T cells. Normal-organ dose estimates were favorable for repeated PET/CT studies. Selective T cell ablation in vivo with ¹⁷⁷Lu-AABD seems feasible for clustered T-cell populations. Conclusion: We have demonstrated for the first time that T cells can be modified with DAbR1, enabling their in vivo tracking via PET and SPECT. The favorable biodistribution and high image contrast observed warrant further studies of this new reporter gene.

A Normalized Shear Deformation Indicator for Ultrasound Strain Elastography in Breast Tissues: An In Vivo Feasibility Study

The shear deformation under loads contains useful information for distinguishing benign breast lesions from malignant ones. In this study, we proposed a normalized shear deformation indicator (NSDI) that was derived from the concept of principal strains. Since the NSDI requires both high-quality axial and lateral (parallel and perpendicular to the beam, resp.) displacement estimates, a strategy combining high-quality speckle tracking with signal “denoising” was employed. Both techniques were previously published by our group. Finite element (FE) models were used to identify possible causes for elevated NSDI values in and around breast lesions, followed by an analysis of ultrasound data acquired from 26 biopsy-confirmed in vivo breast lesions. We found that, theoretically, the elevated NSDI values could be attributed to two factors: significantly hardened tissue stiffness and increasing heterogeneity. The analysis of in vivo data showed that the proposed NSDI values were higher (p < 0.05) among malignant cancers as compared to those measured from benign ones. In conclusion, our preliminary results demonstrated that the calculation of NSDI value is feasible and NSDI could add value to breast lesion differentiation with current clinical equipment as a postprocessing tool.

Influence of Tissue Microstructure on Shear Wave Speed Measurements in Plane Shear Wave Elastography: A Computational Study in Lossless Fibrotic Liver Media

Shear wave elastography (SWE) has been used to measure viscoelastic properties for characterization of fibrotic livers. In this technique, external mechanical vibrations or acoustic radiation forces are first transmitted to the tissue being imaged to induce shear waves. Ultrasonically measured displacement/velocity is then utilized to obtain elastographic measurements related to shear wave propagation. Using an open-source wave simulator, k-Wave, we conducted a case study of the relationship between plane shear wave measurements and the microstructure of fibrotic liver tissues. Particularly, three different virtual tissue models (i.e., a histology-based model, a statistics-based model, and a simple inclusion model) were used to represent underlying microstructures of fibrotic liver tissues. We found underlying microstructures affected the estimated mean group shear wave speed (SWS) under the plane shear wave assumption by as much as 56%. Also, the elastic shear wave scattering resulted in frequency-dependent attenuation coefficients and introduced changes in the estimated group SWS. Similarly, the slope of group SWS changes with respect to the excitation frequency differed as much as 78% among three models investigated. This new finding may motivate further studies examining how elastic scattering may contribute to frequency-dependent shear wave dispersion and attenuation in biological tissues.

An analysis of intrinsic variations of low-frequency shear wave speed in a stochastic tissue model: the first application for staging liver fibrosis

Shear wave elastography is increasingly being used to non-invasively stage liver fibrosis by measuring shear wave speed (SWS). This study quantitatively investigates intrinsic variations among SWS measurements obtained from heterogeneous media such as fibrotic livers. More specifically, it aims to demonstrate that intrinsic variations in SWS measurements, in general, follow a non-Gaussian distribution and are related to the heterogeneous nature of the medium being measured. Using the principle of maximum entropy (ME), our primary objective is to derive a probability density function (PDF) of the SWS distribution in conjunction with a lossless stochastic tissue model. Our secondary objective is to evaluate the performance of the proposed PDF using Monte Carlo (MC)-simulated shear wave (SW) data against three other commonly used PDFs. Based on statistical evaluation criteria, initial results showed that the derived PDF fits better to MC-simulated SWS data than the other three PDFs. It was also found that SW fronts stabilized after a short (compared with the SW wavelength) travel distance in lossless media. Furthermore, in lossless media, the distance required to stabilize the SW propagation was not correlated to the SW wavelength at the low frequencies investigated (i.e. 50, 100 and 150 Hz). Examination of the MC simulation data suggests that elastic (shear) wave scattering became more pronounced when the volume fraction of hard inclusions increased from 10 to 30%. In conclusion, using the principle of ME, we theoretically demonstrated for the first time that SWS measurements in this model follow a non-Gaussian distribution. Preliminary data indicated that the proposed PDF can quantitatively represent intrinsic variations in SWS measurements simulated using a two-phase random medium model. The advantages of the proposed PDF are its physically meaningful parameters and solid theoretical basis.

Building an open-source simulation platform of acoustic radiation force-based breast elastography

Ultrasound-based elastography including strain elastography, acoustic radiation force impulse (ARFI) imaging, point shear wave elastography and supersonic shear imaging (SSI) have been used to differentiate breast tumors among other clinical applications. The objective of this study is to extend a previously published virtual simulation platform built for ultrasound quasi-static breast elastography toward acoustic radiation force-based breast elastography. Consequently, the extended virtual breast elastography simulation platform can be used to validate image pixels with known underlying soft tissue properties (i.e. 'ground truth') in complex, heterogeneous media, enhancing confidence in elastographic image interpretations. The proposed virtual breast elastography system inherited four key components from the previously published virtual simulation platform: an ultrasound simulator (Field II), a mesh generator (Tetgen), a finite element solver (FEBio) and a visualization and data processing package (VTK). Using a simple message passing mechanism, functionalities have now been extended to acoustic radiation force-based elastography simulations. Examples involving three different numerical breast models with increasing complexity-one uniform model, one simple inclusion model and one virtual complex breast model derived from magnetic resonance imaging data, were used to demonstrate capabilities of this extended virtual platform. Overall, simulation results were compared with the published results. In the uniform model, the estimated shear wave speed (SWS) values were within 4% compared to the predetermined SWS values. In the simple inclusion and the complex breast models, SWS values of all hard inclusions in soft backgrounds were slightly underestimated, similar to what has been reported. The elastic contrast values and visual observation show that ARFI images have higher spatial resolution, while SSI images can provide higher inclusion-to-background contrast. In summary, our initial results were consistent with our expectations and what have been reported in the literature. The proposed (open-source) simulation platform can serve as a single gateway to perform many elastographic simulations in a transparent manner, thereby promoting collaborative developments.

Virtual Breast Quasi-static Elastography (VBQE)

Viscoelasticity Imaging (VEI) has been proposed to measure relaxation time constants for characterization of in vivo breast lesions. In this technique, an external compression force on the tissue being imaged is maintained for a fixed period of time to induce strain creep. A sequence of ultrasound echo signals is then utilized to generate time-resolved strain measurements. Relaxation time constants can be obtained by fitting local time-resolved strain measurements to a viscoelastic tissue model (e.g., a modified Kevin-Voigt model). In this study, our primary objective is to quantitatively evaluate the contrast transfer efficiency (CTE) of VEI, which contains useful information regarding image interpretations. Using an open-source simulator for virtual breast quasi-static elastography (VBQE), we conducted a case study of contrast transfer efficiency of VEI. In multiple three-dimensional (3D) numerical breast phantoms containing various degrees of heterogeneity, finite element (FE) simulations in conjunction with quasi-linear viscoelastic constitutive tissue models were performed to mimic data acquisition of VEI under freehand scanning. Our results suggested that there were losses in CTE, and the losses could be as high as -18 dB. FE results also qualitatively corroborated clinical observations, for example, artifacts around tissue interfaces.