Deep Learning of Spatiotemporal Filtering for Fast Super-Resolution Ultrasound Imaging

Super-Resolution Ultrasound Imaging for Monitoring the Therapeutic Efficacy of a Vascular Disrupting Agent in an Animal Model of Breast Cancer

OBJECTIVE

Evaluate the use of super-resolution ultrasound (SRUS) imaging for the early detection of tumor response to treatment using a vascular-disrupting agent (VDA).

METHODS

A population of 28 female nude athymic mice (Charles River Laboratories) were implanted with human breast cancer cells (MDA-MB-231, ATCC) in the mammary fat pad and allowed to grow. Ultrasound imaging was performed using a Vevo 3100 scanner (FUJIFILM VisualSonics Inc) equipped with the MX250 linear array transducer immediately before and after receiving bolus injections of a microbubble (MB) contrast agent (Definity, Lantheus Medical Imaging) via the tail vein. Following baseline ultrasound imaging, VDA drug (combretastatin A4 phosphate, CA4P, Sigma Aldrich) or control saline was injected via the placed catheter. After 4 or 24 hours, repeat ultrasound imaging along the same tumor cross-section occurred. Direct intratumoral pressure measurements were obtained using a calibrated sensor. All raw ultrasound data were saved for offline processing and SRUS image reconstruction using custom MATLAB software (MathWorks Inc). From a region encompassing the tumor space and the entire postprocessed ultrasound image sequence, time MB count (TMC) curves were generated in addition to traditional SRUS maps reflecting MB enumeration at each pixel location. Peak enhancement (PE) and wash-in rate (WIR) were extracted from these TMC curves. At termination, intratumoral microvessel density (MVD) was quantified using tomato lectin labeling of patent blood vessels.

RESULTS

SRUS images exhibited a clear difference between control and treated tumors. While there was no difference in any group parameters at baseline (0 hour, P > .09), both SRUS-derived PE and WIR measurements in tumors treated with VDA exhibited significant decreases by 4 (P = .03 and P = .05, respectively) and 24 hours (P = .02 and P = .01, respectively), but not in control group tumors (P > .22). Similarly, SRUS derived microvascular maps were not different at baseline (P = .81), but measures of vessel density were lower in treated tumors at both 4 and 24 hours (P < .04). An inverse relationship between intratumoral pressure and both PE and WIR parameters were found in control tumors (R2 > .09, P < .03).

CONCLUSION

SRUS imaging is a new modality for assessing tumor response to treatment using a VDA.

A Convolutional Neural Network for Beamforming and Image Reconstruction in Passive Cavitation Imaging

Convolutional neural networks (CNNs), initially developed for image processing applications, have recently received significant attention within the field of medical ultrasound imaging. In this study, passive cavitation imaging/mapping (PCI/PAM), which is used to map cavitation sources based on the correlation of signals across an array of receivers, is evaluated. Traditional reconstruction techniques in PCI, such as delay-and-sum, yield high spatial resolution at the cost of a substantial computational time. This results from the resource-intensive process of determining sensor weights for individual pixels in these methodologies. Consequently, the use of conventional algorithms for image reconstruction does not meet the speed requirements that are essential for real-time monitoring. Here, we show that a three-dimensional (3D) convolutional network can learn the image reconstruction algorithm for a 16×16 element matrix probe with a receive frequency ranging from 256 kHz up to 1.0 MHz. The network was trained and evaluated using simulated data representing point sources, resulting in the successful reconstruction of volumetric images with high sensitivity, especially for single isolated sources (100% in the test set). As the number of simultaneous sources increased, the network's ability to detect weaker intensity sources diminished, although it always correctly identified the main lobe. Notably, however, network inference was remarkably fast, completing the task in approximately 178 s for a dataset comprising 650 frames of 413 volume images with signal duration of 20μs. This processing speed is roughly thirty times faster than a parallelized implementation of the traditional time exposure acoustics algorithm on the same GPU device. This would open a new door for PCI application in the real-time monitoring of ultrasound ablation.

Deep learning for fast denoising filtering in ultrasound localization microscopy

Objective.Addition of a denoising filter step in ultrasound localization microscopy (ULM) has been shown to effectively reduce the error localizations of microbubbles (MBs) and achieve resolution improvement for super-resolution ultrasound (SR-US) imaging. However, previous image-denoising methods (e.g. block-matching 3D, BM3D) requires long data processing times, making ULM only able to be processed offline. This work introduces a new way to reduce data processing time through deep learning.Approach.In this study, we propose deep learning (DL) denoising based on contrastive semi-supervised network (CS-Net). The neural network is mainly trained with simulated MBs data to extract MB signals from noise. And the performances of CS-Net denoising are evaluated in bothin vitroflow phantom experiment andin vivoexperiment of New Zealand rabbit tumor.Main results.Forin vitroflow phantom experiment, the signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) of single microbubble image are 26.91 dB and 4.01 dB, repectively. Forin vivoanimal experiment , the SNR and CNR were 12.29 dB and 6.06 dB. In addition, single microvessel of 24μm and two microvessels separated by 46μm could be clearly displayed. Most importantly,, the CS-Net denoising speeds forin vitroandin vivoexperiments were 0.041 s frame-1and 0.062 s frame-1, respectively.Significance.DL denoising based on CS-Net can improve the resolution of SR-US as well as reducing denoising time, thereby making further contributions to the clinical real-time imaging of ULM.

Super-resolution of 2D ultrasound images and videos

This paper proposes a novel deep-learning framework for super-resolution ultrasound images and videos in terms of spatial resolution and line reconstruction. To this end, we up-sample the acquired low-resolution image through a vision-based interpolation method; then, we train a learning-based model to improve the quality of the up-sampling. We qualitatively and quantitatively test our model on different anatomical districts (e.g., cardiac, obstetric) images and with different up-sampling resolutions (i.e., 2X, 4X). Our method improves the PSNR median value with respect to SOTA methods of [Formula: see text] on obstetric 2X raw images, [Formula: see text] on cardiac 2X raw images, and [Formula: see text] on abdominal raw 4X images; it also improves the number of pixels with a low prediction error of [Formula: see text] on obstetric 4X raw images, [Formula: see text] on cardiac 4X raw images, and [Formula: see text] on abdominal 4X raw images. The proposed method is then applied to the spatial super-resolution of 2D videos, by optimising the sampling of lines acquired by the probe in terms of the acquisition frequency. Our method specialises trained networks to predict the high-resolution target through the design of the network architecture and the loss function, taking into account the anatomical district and the up-sampling factor and exploiting a large ultrasound data set. The use of deep learning on large data sets overcomes the limitations of vision-based algorithms that are general and do not encode the characteristics of the data. Furthermore, the data set can be enriched with images selected by medical experts to further specialise the individual networks. Through learning and high-performance computing, the proposed super-resolution is specialised to different anatomical districts by training multiple networks. Furthermore, the computational demand is shifted to centralised hardware resources with a real-time execution of the network's prediction on local devices.

Applications of Deep Learning Algorithms to Ultrasound Imaging Analysis in Preclinical Studies on In Vivo Animals

BACKGROUND AND AIM

Ultrasound (US) imaging is increasingly preferred over other more invasive modalities in preclinical studies using animal models. However, this technique has some limitations, mainly related to operator dependence. To overcome some of the current drawbacks, sophisticated data processing models are proposed, in particular artificial intelligence models based on deep learning (DL) networks. This systematic review aims to overview the application of DL algorithms in assisting US analysis of images acquired in in vivo preclinical studies on animal models.

METHODS

A literature search was conducted using the Scopus and PubMed databases. Studies published from January 2012 to November 2022 that developed DL models on US images acquired in preclinical/animal experimental scenarios were eligible for inclusion. This review was conducted according to PRISMA guidelines.

RESULTS

Fifty-six studies were enrolled and classified into five groups based on the anatomical district in which the DL models were used. Sixteen studies focused on the cardiovascular system and fourteen on the abdominal organs. Five studies applied DL networks to images of the musculoskeletal system and eight investigations involved the brain. Thirteen papers, grouped under a miscellaneous category, proposed heterogeneous applications adopting DL systems. Our analysis also highlighted that murine models were the most common animals used in in vivo studies applying DL to US imaging.

CONCLUSION

DL techniques show great potential in terms of US images acquired in preclinical studies using animal models. However, in this scenario, these techniques are still in their early stages, and there is room for improvement, such as sample sizes, data preprocessing, and model interpretability.

Assessment of Transarterial Chemoembolization Using Super-resolution Ultrasound Imaging and a Rat Model of Hepatocellular Carcinoma

OBJECTIVE

Hepatocellular carcinoma (HCC) is a highly prevalent form of liver cancer diagnosed annually in 600,000 people worldwide. A common treatment is transarterial chemoembolization (TACE), which interrupts the blood supply of oxygen and nutrients to the tumor mass. The need for repeat TACE treatments may be assessed in the weeks after therapy with contrast-enhanced ultrasound (CEUS) imaging. Although the spatial resolution of traditional CEUS has been restricted by the diffraction limit of ultrasound (US), this physical barrier has been overcome by a recent innovation known as super-resolution US (SRUS) imaging. In short, SRUS enhances the visible details of smaller microvascular structures on the 10 to 100 µm scale, which unlocks a host of new clinical opportunities for US.

METHODS

In this study, a rat model of orthotopic HCC is introduced and TACE treatment response (to a doxorubicin-lipiodol emulsion) is assessed using longitudinal SRUS and magnetic resonance imaging (MRI) performed at 0, 7 and 14 d. Animals were euthanized at 14 d for histological analysis of excised tumor tissue and determination of TACE response, that is, control, partial response or complete response. CEUS imaging was performed using a pre-clinical US system (Vevo 3100, FUJIFILM VisualSonics Inc.) equipped with an MX201 linear array transducer. After administration of a microbubble contrast agent (Definity, Lantheus Medical Imaging), a series of CEUS images were collected at each tissue cross-section as the transducer was mechanically stepped at 100 μm increments. SRUS images were formed at each spatial position, and a microvascular density metric was calculated. Microscale computed tomography (microCT, OI/CT, MILabs) was used to confirm TACE procedure success, and tumor size was monitored using a small animal MRI system (BioSpec 3T, Bruker Corp.).

RESULTS

Although there were no differences at baseline (p > 0.15), both microvascular density levels and tumor size measures from the complete responder cases at 14 d were considerably lower and smaller, respectively, than those in the partial responder or control group animals. Histological analysis revealed tumor-to-necrosis levels of 8.4%, 51.1% and 100%, for the control, partial responder and complete responder groups, respectively (p < 0.005).

CONCLUSION

SRUS imaging is a promising modality for assessing early changes in microvascular networks in response to tissue perfusion-altering interventions such as TACE treatment of HCC.

Generalized mathematical framework for contrast-enhanced ultrasound imaging with pulse inversion spectral deconvolution

A generalized mathematical framework for performing contrast-enhanced ultrasound (CEUS) imaging is introduced. Termed pulse inversion spectral deconvolution (PISD), this CEUS approach is founded on Gaussian derivative functions (GDFs). PISD pulses are used to form two inverted pulse sequences, which are then used to filter backscattered ultrasound (US) data for isolation of the nonlinear (NL) microbubble (MB) signal component. An US scanner equipped with a linear array transducer was used for data acquisition. With a vascular flow phantom perfused with MBs, data was collected using PISD and NL-based CEUS imaging. The role of wide-beam transmit aperture size (32 or 64 elements) was also evaluated using an US pulse frequency of 6.25 MHz. Image enhancement was quantified by a contrast-to-noise ratio (CNR). Preliminary in vivo data was collected in the hindlimb and kidney of healthy rats. Overall, in vitro wide-beam CEUS imaging using an aperture size of 64 elements yielded improved CNR values. Specifically, PISD-based CEUS imaging produced CNR values of 37.3 dB. For comparison, CNR values obtained using B-mode US or NL approaches were 2.1 and 12.1 dB, respectively. In vivo results demonstrated that PISD-based CEUS imaging improved vascular visualization compared to the NL imaging strategy.

Laser ultrasonic imaging of complex defects with full-matrix capture and deep-learning extraction

Phased array-based full-matrix ultrasonic imaging has been the golden standard for the non-destructive evaluation of critical components. However, the piezoelectric phased array cannot be applied in hazardous environments and online monitoring due to its couplant requirement. The laser ultrasonic technique can readily address these challenging tasks via fully non-contact inspection, but low detection sensitivity and complicated wave mode conversion hamper its practical applications. The laser-induced full-matrix ultrasonic imaging of complex defects was displayed in this study. Full matrix data acquisition and deep learning method were adapted to the laser ultrasonic technique to overcome the existing challenges. For proof-of-concept demonstrations, simulations and experiments were conducted on an aluminum sample with representative defects. Numerical and experimental results showed good agreement, revealing the excellent imaging performance of proposed method. Compared with the total focusing method based on ray-trace model, the deep learning method could create superior images with additional quantitative information through end-to-end networks, which use the hierarchical features and generate details from all the relevant imaging and physical characteristics information. The proposed method may help assess defect formation and development at the early stage in a hazardous environment and understand the in-situ manufacturing process due to its couplant-free nature.

Ultrasound Signal Processing: From Models to Deep Learning

Medical ultrasound imaging relies heavily on high-quality signal processing to provide reliable and interpretable image reconstructions. Conventionally, reconstruction algorithms have been derived from physical principles. These algorithms rely on assumptions and approximations of the underlying measurement model, limiting image quality in settings where these assumptions break down. Conversely, more sophisticated solutions based on statistical modeling or careful parameter tuning or derived from increased model complexity can be sensitive to different environments. Recently, deep learning-based methods, which are optimized in a data-driven fashion, have gained popularity. These model-agnostic techniques often rely on generic model structures and require vast training data to converge to a robust solution. A relatively new paradigm combines the power of the two: leveraging data-driven deep learning and exploiting domain knowledge. These model-based solutions yield high robustness and require fewer parameters and training data than conventional neural networks. In this work we provide an overview of these techniques from the recent literature and discuss a wide variety of ultrasound applications. We aim to inspire the reader to perform further research in this area and to address the opportunities within the field of ultrasound signal processing. We conclude with a future perspective on model-based deep learning techniques for medical ultrasound.

Three-Dimensional Ultrasound Localization Microscopy with Bipartite Graph-Based Microbubble Pairing and Kalman-Filtering-Based Tracking on a 256-Channel Verasonics Ultrasound System with a 32 × 32 Matrix Array

Three-dimensional (3D) ultrasound localization microscopy (ULM) using a 2-D matrix probe and microbubbles (MBs) has been recently proposed to visualize microvasculature beyond the ultrasound diffraction limit in three spatial dimensions. However, 3D ULM suffers from several limitations: (1) high system complexity due to numerous channel counts, (2) complex MB flow dynamics in 3D, and (3) extremely long acquisition time. To reduce the system complexity while maintaining high image quality, we used a sub-aperture process to reduce received channel counts. To address the second issue, a 3D bipartite graph-based method with Kalman filtering-based tracking was used in this study for MB tracking. An MB separation approach was incorporated to separate high concentration MB data into multiple, sparser MB datasets, allowing better MB localization and tracking for a limited acquisition time. The proposed method was first validated in a flow channel phantom, showing improved spatial resolutions compared with the contrasted enhanced power Doppler image. Then the proposed method was evaluated with an in vivo chicken embryo brain dataset. Results showed that the reconstructed 3D super-resolution image achieved a spatial resolution of around 52 μm (smaller than the wavelength of around 200 μm). Microvessels that cannot be resolved clearly using localization only, can be well identified with the tailored 3D pairing and tracking algorithms. To sum up, the feasibility of the 3D ULM is shown, indicating the great possibility in clinical applications.

LVNet: Lightweight Model for Left Ventricle Segmentation for Short Axis Views in Echocardiographic Imaging

Lightweight segmentation models are becoming more popular for fast diagnosis on small and low cost medical imaging devices. This study focuses on the segmentation of the left ventricle (LV) in cardiac ultrasound (US) images. A new lightweight model [LV network (LVNet)] is proposed for segmentation, which gives the benefits of requiring fewer parameters but with improved segmentation performance in terms of Dice score (DS). The proposed model is compared with state-of-the-art methods, such as UNet, MiniNetV2, and fully convolutional dense dilated network (FCdDN). The model proposed comes with a post-processing pipeline that further enhances the segmentation results. In general, the training is done directly using the segmentation mask as the output and the US image as the input of the model. A new strategy for segmentation is also introduced in addition to the direct training method used. Compared with the UNet model, an improvement in DS performance as high as 5% for segmentation with papillary (WP) muscles was found, while showcasing an improvement of 18.5% when the papillary muscles are excluded. The model proposed requires only 5% of the memory required by a UNet model. LVNet achieves a better trade-off between the number of parameters and its segmentation performance as compared with other conventional models. The developed codes are available at https://github.com/navchetanawasthi/Left_Ventricle_Segmentation.

Deep Learning-Based Microbubble Localization for Ultrasound Localization Microscopy

Ultrasound localization microscopy (ULM) is an emerging vascular imaging technique that overcomes the resolution-penetration compromise of ultrasound imaging. Accurate and robust microbubble (MB) localization is essential for successful ULM. In this study, we present a deep learning (DL)-based localization technique that uses both Field-II simulation and in vivo chicken embryo chorioallantoic membrane (CAM) data for training. Both radio frequency (RF) and in-phase and quadrature (IQ) data were tested in this study. The simulation experiment shows that the proposed DL-based localization was able to reduce both missing MB localization rate and MB localization error. In general, RF data showed better performance than IQ. For the in vivo CAM study with high MB concentration, DL-based localization was able to reduce the vessel MB saturation time by more than 50% compared to conventional localization. In addition, we propose a DL-based framework for real-time visualization of the high-resolution microvasculature. The findings of this article support the use of DL for more robust and faster MB localization, especially under high MB concentrations. The results indicate that further improvement could be achieved by incorporating temporal information of the MB data.

Evaluation of Nonlinear Contrast Pulse Sequencing for Use in Super-Resolution Ultrasound Imaging

The use of super-resolution ultrasound (SR-US) imaging greatly improves visualization of microvascular structures, but clinical adoption is limited by long imaging times. This method depends on detecting and localizing isolated microbubbles (MBs), forcing the use of a dilute contrast agent concentration. Contrast-enhanced ultrasound (CEUS) image acquisition times as long as minutes arise as the localization of thousands of MBs are acquired to form a complete SR-US image. In this article, we explore the use of nonlinear CEUS strategies using nonlinear fundamental contrast pulse sequencing (CPS) to increase the contrast-to-tissue ratio (CTR) and compare MB detection effectiveness to linear B-mode CEUS imaging. The CPS compositions of amplitude modulation (AM), pulse inversion (PI), and a combination of the two (AMPI) were studied. A simulation study combined the Rayleigh-Plesset-Marmottant (RPM) model of MB characteristics and a nonlinear tissue model using the k-Wave toolbox for MATLAB (MathWorks Inc., Natick, MA, USA). Validation was conducted using an in vitro flow phantom and in vivo in the rat hind-limb. Imaging was performed with a programmable US scanner (Vantage 256, Verasonics Inc., Kirkland, WA, USA) and customized to transmit a set of basis US pulses from which both B-mode US (frame rate (FR) of 800 Hz) and multiple nonlinear CPS compositions (FR of 200 Hz) could be assessed from identical in vitro and in vivo datasets using a near simultaneous method. The simulations suggest that MB characteristics, such as diameter and motion, help to predict which US imaging strategy will enhance MB detection. The in vitro and in vivo US imaging studies revealed that different subpopulations of polydisperse MB contrast agents were detected by linear imaging and by each different nonlinear CPS composition. The most effective single imaging strategy at a 200-Hz FR was found to be B-mode US imaging. However, a combination of B-mode US imaging with a nonlinear CPS imaging strategy was more effective in detecting MBs in vivo at all depths and was shown to shorten image acquisition time by an average of 33.3%-76.7% when combining one or more CPS sequences.

Faster super-resolution ultrasound imaging with a deep learning model for tissue decluttering and contrast agent localization

Super-resolution ultrasound (SR-US) imaging allows visualization of microvascular structures as small as tens of micrometers in diameter. However, use in the clinical setting has been impeded in part by ultrasound (US) acquisition times exceeding a breath-hold and by the need for extensive offline computation. Deep learning techniques have been shown to be effective in modeling the two more computationally intensive steps of microbubble (MB) contrast agent detection and localization. Performance gains by deep networks over conventional methods are more than two orders of magnitude and in addition the networks can localize overlapping MBs. The ability to separate overlapping MBs allows use of higher contrast agent concentrations and reduces US image acquisition time. Herein we propose a fully convolutional neural network (CNN) architecture to perform the operations of MB detection as well as localization in a single model. Termed SRUSnet, the network is based on the MobileNetV3 architecture modified for 3-D input data, minimal convergence time, and high-resolution data output using a flexible regression head. Also, we propose to combine linear B-mode US imaging and nonlinear contrast pulse sequencing (CPS) which has been shown to increase MB detection and further reduce the US image acquisition time. The network was trained within silicodata and tested onin vitrodata from a tissue-mimicking flow phantom, and onin vivodata from the rat hind limb (N = 3). Images were collected with a programmable US system (Vantage 256, Verasonics Inc., Kirkland, WA) using an L11-4v linear array transducer. The network exceeded 99.9% detection accuracy onin silicodata. The average localization accuracy was smaller than the resolution of a pixel (i.e.λ/8). The average processing time on a Nvidia GeForce 2080Ti GPU was 64.5 ms for a 128 × 128-pixel image.

In Vivo Motion Correction in Super-Resolution Imaging of Rat Kidneys

Super-resolution (SR) imaging has the potential of visualizing the microvasculature down to the 10- [Formula: see text] level, but motion induced by breathing, heartbeats, and muscle contractions are often significantly above this level. This article, therefore, introduces a method for estimating tissue motion and compensating for this. The processing pipeline is described and validated using Field II simulations of an artificial kidney. In vivo measurements were conducted using a modified bk5000 research scanner (BK Medical, Herlev, Denmark) with a BK 9009 linear array probe employing a pulse amplitude modulation scheme. The left kidney of ten Sprague-Dawley rats was scanned during open laparotomy. A 1:10 diluted SonoVue contrast agent (Bracco, Milan, Italy) was injected through a jugular vein catheter at 100 [Formula: see text]/min. Motion was estimated using speckle tracking and decomposed into contributions from the heartbeats, breathing, and residual motion. The estimated peak motions and their precisions were: heart: axial- [Formula: see text] and lateral- [Formula: see text], breathing: axial- [Formula: see text] and lateral- [Formula: see text], and residual: axial-30 [Formula: see text] and lateral-90 [Formula: see text]. The motion corrected microbubble tracks yielded SR images of both bubble density and blood vector velocity. The estimation was, thus, sufficiently precise to correct shifts down to the 10- [Formula: see text] capillary level. Similar results were found in the other kidney measurements with a restoration of resolution for the small vessels demonstrating that motion correction in 2-D can enhance SR imaging quality.

Multiscale computational modeling of cancer growth using features derived from microCT images

Advances in medical imaging technologies now allow noninvasive image acquisition from individual patients at high spatiotemporal resolutions. A relatively new effort of predictive oncology is to develop a paradigm for forecasting the future status of an individual tumor given initial conditions and an appropriate mathematical model. The objective of this study was to introduce a comprehensive multiscale computational method to predict cancer and microvascular network growth patterns. A rectangular lattice-based model was designed so different evolutionary scenarios could be simulated and for predicting the impact of diffusible factors on tumor morphology and size. Further, the model allows prediction-based simulation of cell and microvascular behavior. Like a single cell, each agent is fully realized within the model and interactions are governed in part by machine learning methods. This multiscale computational model was developed and incorporated input information from in vivo microscale computed tomography (microCT) images acquired from breast cancer-bearing mice. It was found that as the difference between expansion of the cancer cell population and microvascular network increases, cells undergo proliferation and migration with a greater probability compared to other phenotypes. Overall, multiscale computational model agreed with both theoretical expectations and experimental findings (microCT images) not used during model training.

3-D Super-Resolution Ultrasound Imaging for Monitoring Early Changes in Breast Cancer after Treatment with a Vascular-Disrupting Agent

The purpose of this research project was to evaluate the use of 3-dimensional (3-D) super-resolution ultrasound (SR-US) imaging to assess any early changes in breast cancer after treatment with a vascular-disrupting agent (VDA). A Vevo 3100 ultrasound system (FUJIFILM VisualSonics Inc) equipped with an MX 201 transducer was used for image acquisition. A total of 2.5 × 107 microbubbles (MBs) were injected into the tail vein of anesthetized breast cancer-bearing mice using repeat bolus injections every 5 min. A total of 10 stacks of ultrasound images were collected as the transducer was mechanically moved across the tumor at 0.6 mm intervals yielding a 6-mm thick volume. At each tumor location, a stack contained 1 × 104 frames of ultrasound data that were acquired at 463 frames/sec and stored as in-phase/quadrature (IQ) format. After motion correction, each temporal stack of ultrasound images was processed separately for clutter signal removal, which was followed by MB localization and enumeration before generation of the final SR-US image. After reconstruction of the 3-D SR-US volume dataset, the tumor microvasculature was enhanced using a multiscale vessel enhancement filter. Vessels from the resultant microvascular network were then segmented using an adaptive thresholding method. Finally, mean microvascular density (MVD) measurements from each tumor volume were computed as a summarizing statistic. While no differences were found between baseline SR-US image-derived measures of MVD (p = 0.76), these same measurements were significantly lower at 24 h after VDA treatment (p < 0.001). Overall, 3-D SR-US imaging detected early tumor changes following treatment with a vascular-targeted drug.

A Deep Learning-Based Automatic First-Arrival Picking Method for Ultrasound Sound-Speed Tomography

Ultrasound sound-speed tomography (USST) has shown great prospects for breast cancer diagnosis due to its advantages of nonradiation, low cost, 3-D breast images, and quantitative indicators. However, the reconstruction quality of USST is highly dependent on the first-arrival picking of the transmission wave. Traditional first-arrival picking methods have low accuracy and noise robustness. To improve the accuracy and robustness, we introduced a self-attention mechanism into the bidirectional long short-term memory (BLSTM) network and proposed the self-attention BLSTM (SAT-BLSTM) network. The proposed method predicts the probability of the first-arrival time and selects the time with maximum probability. A numerical simulation and prototype experiment were conducted. In the numerical simulation, the proposed SAT-BLSTM showed the best results. For signal-to-noise ratios (SNRs) of 50, 30, and 15 dB, the mean absolute errors (MAEs) were 48, 49, and 76 ns, respectively. The BLSTM had the second-best results, with MAEs of 55, 56, and 85 ns, respectively. The MAEs of the Akaike information criterion (AIC) method were 57, 296, and 489 ns, respectively. In the prototype experiment, the MAEs of the SAT-BLSTM, the BLSTM, and the AIC were 94, 111, and 410 ns, respectively.

Ultrasound Scatterer Density Classification Using Convolutional Neural Networks and Patch Statistics

Quantitative ultrasound (QUS) can reveal crucial information on tissue properties, such as scatterer density. If the scatterer density per resolution cell is above or below 10, the tissue is considered as fully developed speckle (FDS) or underdeveloped speckle (UDS), respectively. Conventionally, the scatterer density has been classified using estimated statistical parameters of the amplitude of backscattered echoes. However, if the patch size is small, the estimation is not accurate. These parameters are also highly dependent on imaging settings. In this article, we adapt convolutional neural network (CNN) architectures for QUS and train them using simulation data. We further improve the network's performance by utilizing patch statistics as additional input channels. Inspired by deep supervision and multitask learning, we propose a second method to exploit patch statistics. We evaluate the networks using simulation data and experimental phantoms. We also compare our proposed methods with different classic and deep learning models and demonstrate their superior performance in the classification of tissues with different scatterer density values. The results also show that we are able to classify scatterer density in different imaging parameters with no need for a reference phantom. This work demonstrates the potential of CNNs in classifying scatterer density in ultrasound images.

A Deep Learning Framework for Spatiotemporal Ultrasound Localization Microscopy

Ultrasound Localization Microscopy (ULM) can resolve the microvascular bed down to a few micrometers. To achieve such performance, microbubble contrast agents must perfuse the entire microvascular network. Microbubbles are then located individually and tracked over time to sample individual vessels, typically over hundreds of thousands of images. To overcome the fundamental limit of diffraction and achieve a dense reconstruction of the network, low microbubble concentrations must be used, which leads to acquisitions lasting several minutes. Conventional processing pipelines are currently unable to deal with interference from multiple nearby microbubbles, further reducing achievable concentrations. This work overcomes this problem by proposing a Deep Learning approach to recover dense vascular networks from ultrasound acquisitions with high microbubble concentrations. A realistic mouse brain microvascular network, segmented from 2-photon microscopy, was used to train a three-dimensional convolutional neural network (CNN) based on a V-net architecture. Ultrasound data sets from multiple microbubbles flowing through the microvascular network were simulated and used as ground truth to train the 3D CNN to track microbubbles. The 3D-CNN approach was validated in silico using a subset of the data and in vivo in a rat brain. In silico, the CNN reconstructed vascular networks with higher precision (81%) than a conventional ULM framework (70%). In vivo, the CNN could resolve micro vessels as small as 10 μ m with an improvement in resolution when compared against a conventional approach.

Three-dimensional visualization and improved quantification with super-resolution ultrasound imaging - validation framework for analysis of microvascular morphology using a chicken embryo model

The purpose of this study was to improve the morphological analysis of microvascular networks depicted in three-dimensional (3D) super-resolution ultrasound (SR-US) images. This was supported by qualitative and quantitative validation by comparison to matched brightfield microscopy and traditional B-mode ultrasound (US) images. Contrast-enhanced US (CEUS) images were collected using a preclinical US scanner (Vevo 3100, FUJIFILM VisualSonics Inc.) equipped with an MX250 linear array transducer. CEUS imaging was performed after administration of a microbubble (MB) contrast agent into the vitelline network of a developing chicken embryo. Volume data was collected by mechanically scanning the US transducer throughout a tissue volume-of-interest in 90μm step increments. CEUS images were collected at each increment and stored as in-phase/quadrature data (2000 frames at 152 frames per sec). SR-US images were created for each cross-sectional plane using established data processing methods. All SR-US images were then used to reconstruct a final 3D volume for vessel diameter (VD) quantification and for surface rendering. VD quantification from the 3D SR-US data exhibited an average error of 6.1% ± 6.0% when compared with matched brightfield microscopy images, whereas measurements from B-mode US images had an average error of 77.1% ± 68.9%. Volume and surface renderings in 3D space enabled qualitative validation and improved visualization of small vessels below the axial resolution of the US system. Overall, 3D SR-US image reconstructions depicted the microvascular network of the developing chicken embryos. Improved visualization of isolated vessels and quantification of microvascular morphology from SR-US images achieved a considerably greater accuracy compared to B-mode US measurements.

Current Development and Applications of Super-Resolution Ultrasound Imaging

Abnormal changes of the microvasculature are reported to be key evidence of the development of several critical diseases, including cancer, progressive kidney disease, and atherosclerotic plaque. Super-resolution ultrasound imaging is an emerging technology that can identify the microvasculature noninvasively, with unprecedented spatial resolution beyond the acoustic diffraction limit. Therefore, it is a promising approach for diagnosing and monitoring the development of diseases. In this review, we introduce current super-resolution ultrasound imaging approaches and their preclinical applications on different animals and disease models. Future directions and challenges to overcome for clinical translations are also discussed.

Three-dimensional super-resolution ultrasound imaging of chicken embryos - A validation framework for analysis of microvascular morphology

The purpose of this present study was to improve the quantification of microvascular networks depicted in three-dimensional (3D) super-resolution ultrasound (SR-US) images and compare results with matched brightfield microscopy and B-mode ultrasound (US) images. Standard contrast-enhanced US (CEUS) images were collected using a high-frequency US scanner (Vevo 3100, FUJIFILM VisualSonics Inc) equipped with an MX250 linear array transducer. Using a developing chicken embryo as our model system, US imaging was performed after administration of a custom microbubble (MB) contrast agent. Guided by stereo microscopy, MBs were introduced into a perfused blood vessel by microinjection with a glass capillary needle. Volume data was collected by mechanically scanning the US transducer throughout a tissue volume-of-interest (VOI) in 90 μm step increments. CEUS images were collected at each increment and stored as in-phase/quadrature (IQ) data (N = 2000 at 152 frames per sec). SR-US images were created for each cross-sectional plane using established data processing methods, and all were then used to form a final 3D volume for subsequent quantification of morphological features. Vessel diameter quantifications from 3D SR-US data exhibited an average error of 1.9% when compared with microscopy images, whereas measures from B-mode US images had an average error of 75.3%. Overall, 3D SR-US images clearly depicted the microvascular network of the developing chicken embryo and measurements of microvascular morphology achieved better accuracy compared to traditional B-mode US.

Three-dimensional evaluation of microvascular networks using contrast-enhanced ultrasound and microbubble tracking

Evaluating tumor microvascular networks with use of contrast-enhanced ultrasound (CEUS) imaging and one-dimensional (1D) linear array transducers have inherit limitations as tumors exist in volume space. The use of a mechanical sweep allows users to overcome this limitation. To that end, we have developed a new method by which a 1D linear array transducer can be mechanically scanned over a region-of-interest to capture a volume of data allowing for the evaluation of microvasculature structures in 3D space. After intravascular injection of a microbubble (MB) contrast agent into a developing chicken embryo, a sequence of CEUS images were acquired using a Vevo 3100 scanner (VisualSonics Inc) and taken at multiple tissue cross-sections. The CEUS images were processed with a singular value filter (SVF) to help remove any clutter signal. MB localization was performed, and frame-to-frame MB movement was analyzed to produce spatial maps depicting blood flow and velocity at each tissue cross-section. Reconstruction of all images allowed visualization of microvascular networks and blood velocity distribution in volume space.