Nondestructive Detection of Targeted Microbubbles Using Dual-Mode Data and Deep Learning for Real-Time Ultrasound Molecular Imaging

Ultrasound wavelet spectra enable direct tissue recognition and full-color visualization

Traditional brightness-mode ultrasound imaging is primarily constrained by the low specificity among tissues and the inconsistency among sonographers. The major cause is the imaging method that represents the amplitude of echoes as brightness and ignores other detailed information, leaving sonographers to interpret based on organ contours that depend highly on specific imaging planes. Other ultrasound imaging modalities, color Doppler imaging or shear wave elastography, overlay motion or stiffness information to brightness-mode images. However, tissue-specific scattering properties and spectral patterns remain unknown in ultrasound imaging. Here we demonstrate that the distribution (size and average distance) of scattering particles leads to characteristic wavelet spectral patterns, which enables tissue recognition and high-contrast ultrasound imaging. Ultrasonic wavelet spectra from similar particle distributions tend to cluster in the eigenspace according to principal component analysis, whereas those with different distributions tend to be distinguishable from one another. For each distribution, a few wavelet spectra are unique and act as a fingerprint to recognize the corresponding tissue. Illumination of specific tissues and organs with designated colors according to the recognition results yields high-contrast ultrasound imaging. The fully-colorized tissue-specific ultrasound imaging potentially simplifies the interpretation and promotes consistency among sonographers, or even enables the applicability for non-professionals.

Phase Aberration Correction for In Vivo Ultrasound Localization Microscopy Using a Spatiotemporal Complex-Valued Neural Network

Ultrasound Localization Microscopy (ULM) can map microvessels at a resolution of a few micrometers ( [Formula: see text]). Transcranial ULM remains challenging in presence of aberrations caused by the skull, which lead to localization errors. Herein, we propose a deep learning approach based on recently introduced complex-valued convolutional neural networks (CV-CNNs) to retrieve the aberration function, which can then be used to form enhanced images using standard delay-and-sum beamforming. CV-CNNs were selected as they can apply time delays through multiplication with in-phase quadrature input data. Predicting the aberration function rather than corrected images also confers enhanced explainability to the network. In addition, 3D spatiotemporal convolutions were used for the network to leverage entire microbubble tracks. For training and validation, we used an anatomically and hemodynamically realistic mouse brain microvascular network model to simulate the flow of microbubbles in presence of aberration. The proposed CV-CNN performance was compared to the coherence-based method by using microbubble tracks. We then confirmed the capability of the proposed network to generalize to transcranial in vivo data in the mouse brain (n=3). Vascular reconstructions using a locally predicted aberration function included additional and sharper vessels. The CV-CNN was more robust than the coherence-based method and could perform aberration correction in a 6-month-old mouse. After correction, we measured a resolution of [Formula: see text] for younger mice, representing an improvement of 25.8%, while the resolution was improved by 13.9% for the 6-month-old mouse. This work leads to different applications for complex-valued convolutions in biomedical imaging and strategies to perform transcranial ULM.

A Convolutional Neural Network for Beamforming and Image Reconstruction in Passive Cavitation Imaging

Convolutional neural networks (CNNs), initially developed for image processing applications, have recently received significant attention within the field of medical ultrasound imaging. In this study, passive cavitation imaging/mapping (PCI/PAM), which is used to map cavitation sources based on the correlation of signals across an array of receivers, is evaluated. Traditional reconstruction techniques in PCI, such as delay-and-sum, yield high spatial resolution at the cost of a substantial computational time. This results from the resource-intensive process of determining sensor weights for individual pixels in these methodologies. Consequently, the use of conventional algorithms for image reconstruction does not meet the speed requirements that are essential for real-time monitoring. Here, we show that a three-dimensional (3D) convolutional network can learn the image reconstruction algorithm for a 16×16 element matrix probe with a receive frequency ranging from 256 kHz up to 1.0 MHz. The network was trained and evaluated using simulated data representing point sources, resulting in the successful reconstruction of volumetric images with high sensitivity, especially for single isolated sources (100% in the test set). As the number of simultaneous sources increased, the network's ability to detect weaker intensity sources diminished, although it always correctly identified the main lobe. Notably, however, network inference was remarkably fast, completing the task in approximately 178 s for a dataset comprising 650 frames of 413 volume images with signal duration of 20μs. This processing speed is roughly thirty times faster than a parallelized implementation of the traditional time exposure acoustics algorithm on the same GPU device. This would open a new door for PCI application in the real-time monitoring of ultrasound ablation.

3-D Ultrafast Shear Wave Absolute Vibro-Elastography Using a Matrix Array Transducer

Real-time ultrasound imaging plays an important role in ultrasound-guided interventions. The 3-D imaging provides more spatial information compared to conventional 2-D frames by considering the volumes of data. One of the main bottlenecks of 3-D imaging is the long data acquisition time, which reduces practicality and can introduce artifacts from unwanted patient or sonographer motion. This article introduces the first shear wave absolute vibro-elastography (S-WAVE) method with real-time volumetric acquisition using a matrix array transducer. In S-WAVE, an external vibration source generates mechanical vibrations inside the tissue. The tissue motion is then estimated and used in solving a wave equation inverse problem to provide the tissue elasticity. A matrix array transducer is used with a Verasonics ultrasound machine and a frame rate of 2000 volumes/s to acquire 100 radio frequency (RF) volumes in 0.05 s. Using plane wave (PW) and compounded diverging wave (CDW) imaging methods, we estimate axial, lateral, and elevational displacements over 3-D volumes. The curl of the displacements is used with local frequency estimation to estimate elasticity in the acquired volumes. Ultrafast acquisition extends substantially the possible S-WAVE excitation frequency range, now up to 800 Hz, enabling new tissue modeling and characterization. The method was validated on three homogeneous liver fibrosis phantoms and on four different inclusions within a heterogeneous phantom. The homogeneous phantom results show less than 8% (PW) and 5% (CDW) difference between the manufacturer values and the corresponding estimated values over a frequency range of 80-800 Hz. The estimated elasticity values for the heterogeneous phantom at 400-Hz excitation frequency show the average errors of 9% (PW) and 6% (CDW) compared to the provided average values by magnetic resonance elastography (MRE). Furthermore, both imaging methods were able to detect the inclusions within the elasticity volumes. An ex vivo study on a bovine liver sample shows less than 11% (PW) and 9% (CDW) difference between the estimated elasticity ranges by the proposed method and the elasticity ranges provided by MRE and acoustic radiation force impulse (ARFI).

Applications of Deep Learning Algorithms to Ultrasound Imaging Analysis in Preclinical Studies on In Vivo Animals

BACKGROUND AND AIM

Ultrasound (US) imaging is increasingly preferred over other more invasive modalities in preclinical studies using animal models. However, this technique has some limitations, mainly related to operator dependence. To overcome some of the current drawbacks, sophisticated data processing models are proposed, in particular artificial intelligence models based on deep learning (DL) networks. This systematic review aims to overview the application of DL algorithms in assisting US analysis of images acquired in in vivo preclinical studies on animal models.

METHODS

A literature search was conducted using the Scopus and PubMed databases. Studies published from January 2012 to November 2022 that developed DL models on US images acquired in preclinical/animal experimental scenarios were eligible for inclusion. This review was conducted according to PRISMA guidelines.

RESULTS

Fifty-six studies were enrolled and classified into five groups based on the anatomical district in which the DL models were used. Sixteen studies focused on the cardiovascular system and fourteen on the abdominal organs. Five studies applied DL networks to images of the musculoskeletal system and eight investigations involved the brain. Thirteen papers, grouped under a miscellaneous category, proposed heterogeneous applications adopting DL systems. Our analysis also highlighted that murine models were the most common animals used in in vivo studies applying DL to US imaging.

CONCLUSION

DL techniques show great potential in terms of US images acquired in preclinical studies using animal models. However, in this scenario, these techniques are still in their early stages, and there is room for improvement, such as sample sizes, data preprocessing, and model interpretability.

A Single-Shot Harmonic Imaging Approach Utilizing Deep Learning for Medical Ultrasound

Tissue Harmonic Imaging (THI) is an invaluable tool in clinical ultrasound owing to its enhanced contrast resolution and reduced reverberation clutter in comparison to fundamental mode imaging. However, harmonic content separation based on high pass filtering suffers from potential contrast degradation or lower axial resolution due to spectral leakage. Whereas nonlinear multi-pulse harmonic imaging schemes, such as amplitude modulation and pulse inversion, suffer from a reduced framerate and comparatively higher motion artifacts due to the necessity of at least two pulse echo acquisitions. To address this problem, we propose a deep-learning-based single-shot harmonic imaging technique capable of generating comparable image quality to pulse amplitude modulation methods, yet at a higher framerate and with fewer motion artifacts. Specifically, an asymmetric convolutional encoder-decoder structure is designed to estimate the combination of the echoes resulting from the half-amplitude transmissions using the echo produced from the full amplitude transmission as input. The echoes were acquired with the checkerboard amplitude modulation technique for training. The model was evaluated across various targets and samples to illustrate generalizability as well as the possibility and impact of transfer learning. Furthermore, for possible interpretability of the network, we investigate if the latent space of the encoder holds information on the nonlinearity parameter of the medium. We demonstrate the ability of the proposed approach to generate harmonic images with a single firing that are comparable to those from a multi-pulse acquisition.

Acoustic Molecular Imaging Beyond the Diffraction Limit In Vivo

Ultrasound molecular imaging (USMI) is a technique used to noninvasively estimate the distribution of molecular markers in vivo by imaging microbubble contrast agents (MCAs) that have been modified to target receptors of interest on the vascular endothelium. USMI is especially relevant for preclinical and clinical cancer research and has been used to predict tumor malignancy and response to treatment. In the last decade, methods that improve the resolution of contrast-enhanced ultrasound by an order of magnitude and allow researchers to noninvasively image individual capillaries have emerged. However, these approaches do not translate directly to molecular imaging. In this work, we demonstrate super-resolution visualization of biomarker expression in vivo using superharmonic ultrasound imaging (SpHI) with dual-frequency transducers, targeted contrast agents, and localization microscopy processing. We validate and optimize the proposed method in vitro using concurrent optical and ultrasound microscopy and a microvessel phantom. With the same technique, we perform a proof-of-concept experiment in vivo in a rat fibrosarcoma model and create maps of biomarker expression co-registered with images of microvasculature. From these images, we measure a resolution of 23 μm, a nearly fivefold improvement in resolution compared to previous diffraction-limited molecular imaging studies.

Deep-fUS: A Deep Learning Platform for Functional Ultrasound Imaging of the Brain Using Sparse Data

Functional ultrasound (fUS) is a rapidly emerging modality that enables whole-brain imaging of neural activity in awake and mobile rodents. To achieve sufficient blood flow sensitivity in the brain microvasculature, fUS relies on long ultrasound data acquisitions at high frame rates, posing high demands on the sampling and processing hardware. Here we develop an image reconstruction method based on deep learning that significantly reduces the amount of data necessary while retaining imaging performance. We trained convolutional neural networks to learn the power Doppler reconstruction function from sparse sequences of ultrasound data with compression factors of up to 95%. High-quality images from in vivo acquisitions in rats were used for training and performance evaluation. We demonstrate that time series of power Doppler images can be reconstructed with sufficient accuracy to detect the small changes in cerebral blood volume (~10%) characteristic of task-evoked cortical activation, even though the network was not formally trained to reconstruct such image series. The proposed platform may facilitate the development of this neuroimaging modality in any setting where dedicated hardware is not available or in clinical scanners.

Upstream Machine Learning in Radiology

Machine learning (ML) and Artificial intelligence (AI) has the potential to dramatically improve radiology practice at multiple stages of the imaging pipeline. Most of the attention has been garnered by applications focused on improving the end of the pipeline: image interpretation. However, this article reviews how AI/ML can be applied to improve upstream components of the imaging pipeline, including exam modality selection, hardware design, exam protocol selection, data acquisition, image reconstruction, and image processing. A breadth of applications and their potential for impact is shown across multiple imaging modalities, including ultrasound, computed tomography, and MRI.

Dynamic Filtering of Adherent and Non-adherent Microbubble Signals Using Singular Value Thresholding and Normalized Singular Spectrum Area Techniques

Ultrasound molecular imaging techniques rely on the separation and identification of three types of signals: static tissue, adherent microbubbles and non-adherent microbubbles. In this study, the image filtering techniques of singular value thresholding (SVT) and normalized singular spectrum area (NSSA) were combined to isolate and identify vascular endothelial growth factor receptor 2-targeted microbubbles in a mouse hindlimb tumor model (n = 24). By use of a Verasonics Vantage 256 imaging system with an L12-5 transducer, a custom-programmed pulse inversion sequence employing synthetic aperture virtual source element imaging was used to collect contrast images of mouse tumors perfused with microbubbles. SVT was used to suppress static tissue signals by 9.6 dB while retaining adherent and non-adherent microbubble signals. NSSA was used to classify microbubble signals as adherent or non-adherent with high accuracy (receiver operating characteristic area under the curve [ROC AUC] = 0.97), matching the classification performance of differential targeted enhancement. The combined SVT + NSSA filtering method also outperformed differential targeted enhancement in differentiating MB signals from all other signals (ROC AUC = 0.89) without necessitating destruction of the contrast agent. The results from this study indicate that SVT and NSSA can be used to automatically segment and classify contrast signals. This filtering method with potential real-time capability could be used in future diagnostic settings to improve workflow and speed the clinical uptake of ultrasound molecular imaging techniques.

Phospholipid-coated targeted microbubbles for ultrasound molecular imaging and therapy

Phospholipid-coated microbubbles are ultrasound contrast agents that, when functionalized, adhere to specific biomarkers on cells. In this concise review, we highlight recent developments in strategies for targeting the microbubbles and their use for ultrasound molecular imaging (UMI) and therapy. Recently developed novel targeting strategies include magnetic functionalization, triple targeting, and the use of several new ligands. UMI is a powerful technique for studying disease progression, diagnostic imaging, and monitoring of therapeutic responses. Targeted microbubbles (tMBs) have been used for the treatment of cardiovascular diseases and cancer, with therapeutics either coadministered or loaded onto the tMBs. Regardless of which disease was treated, the use of tMBs always resulted in a better therapeutic outcome than non-tMBs when compared in vitro or in vivo.

New Span-PEG-composited Fe3O4-CNT as a multifunctional ultrasound contrast agent for inflammation and thrombotic niduses

By attaching ferroferric oxide (Fe3O4) to drug-carrying carbon nanotubes (CNTs), we generated a new Span-PEG composite with Fe3O4-CNT multifunctional microbubbles for inflammation and thrombus niduses. The Fe3O4-CNT magnetic targeting complex was prepared by in situ synthesis, and then acetylsalicylic acid (ASA) and gentamicin (GM) were loaded onto the Fe3O4-CNT complex by physical methods to produce Fe3O4-CNT-ASA and Fe3O4-CNT-GM complexes, respectively. Span-PEG-composited Fe3O4-CNT-ASA or Fe3O4-CNT-GM microbubbles were synthesized with Span and PEG as the membrane materials by the acoustic cavitation method. The obtained composite microbubbles were smooth, hollow spheres with an average particle size of 425 nm. The ASA and GM loading rates in Span-PEG-composited Fe3O4-CNT-ASA and Fe3O4-CNT-GM microbubbles were 1.12% and 19.05%, respectively. Span-PEG-composited Fe3O4-CNT-ASA microbubbles inhibited thrombosis and demonstrated an anticoagulation effect in vitro. Additionally, Span-PEG-composited Fe3O4-CNT-ASA microbubbles showed significantly enhanced ultrasound imaging of rabbit abdominal aorta and extended the signal time under the action of an external magnetic field. Thus, Span-PEG-composited Fe3O4-CNT-GM microbubbles inhibited Escherichia coli and Staphylococcus aureus, enhanced the ultrasound imaging of rabbit abdominal uterus and had better stability and fluidity.