Wireless and sensorless 3D ultrasound imaging

Long-term Dependency for 3D Reconstruction of Freehand Ultrasound Without External Tracker

OBJECTIVE

Reconstructing freehand ultrasound in 3D without any external tracker has been a long-standing challenge in ultrasound-assisted procedures. We aim to define new ways of parameterising long-term dependencies, and evaluate the performance.

METHODS

First, long-term dependency is encoded by transformation positions within a frame sequence. This is achieved by combining a sequence model with a multi-transformation prediction. Second, two dependency factors are proposed, anatomical image content and scanning protocol, for contributing towards accurate reconstruction. Each factor is quantified experimentally by reducing respective training variances.

RESULTS

1) The added long-term dependency up to 400 frames at 20 frames per second (fps) indeed improved reconstruction, with an up to 82.4% lowered accumulated error, compared with the baseline performance. The improvement was found to be dependent on sequence length, transformation interval and scanning protocol and, unexpectedly, not on the use of recurrent networks with long-short term modules; 2) Decreasing either anatomical or protocol variance in training led to poorer reconstruction accuracy. Interestingly, greater performance was gained from representative protocol patterns, than from representative anatomical features.

CONCLUSION

The proposed algorithm uses hyperparameter tuning to effectively utilise long-term dependency. The proposed dependency factors are of practical significance in collecting diverse training data, regulating scanning protocols and developing efficient networks.

SIGNIFICANCE

The proposed new methodology with publicly available volunteer data and code for parametersing the long-term dependency, experimentally shown to be valid sources of performance improvement, which could potentially lead to better model development and practical optimisation of the reconstruction application.

Organic–Inorganic Hybrid Perovskite Materials for Ultrasonic Transducer in Medical Diagnosis

The ultrasonic transducer is considered the most important component of ultrasound medical instruments, and its key active layer is generally fabricated by piezoelectric materials, such as BaTiO3, Pb (Zn, Ti)O3, PVDF, etc. As the star material, perovskite photovoltaic materials (organic and inorganic halide perovskite materials, such as CH3NH3PbI3, CsPbI3, etc.) have great potential to be widely used in solar cells, LEDs, detectors, and photoelectric and piezoelectric detectors due to their outstanding photoelectric and piezoelectric effects. Herein, we firstly discussed the research progress of commonly used piezoelectric materials and the corresponding piezoelectric effects, the current key scientific status, as well as the current application status in the field of ultrasound medicine. Then, we further explored the current progress of perovskite materials used in piezoelectric-effect devices, and their research difficulties. Finally, we designed an ideal ultrasonic transducer fabricated by perovskite photovoltaic materials and considered the future application prospects of organic and inorganic halide perovskite material in the field of ultrasound.

Tattoo tomography: Freehand 3D photoacoustic image reconstruction with an optical pattern

Purpose

Photoacoustic tomography (PAT) is a novel imaging technique that can spatially resolve both morphological and functional tissue properties, such as vessel topology and tissue oxygenation. While this capacity makes PAT a promising modality for the diagnosis, treatment, and follow-up of various diseases, a current drawback is the limited field of view provided by the conventionally applied 2D probes.

Methods

In this paper, we present a novel approach to 3D reconstruction of PAT data (Tattoo tomography) that does not require an external tracking system and can smoothly be integrated into clinical workflows. It is based on an optical pattern placed on the region of interest prior to image acquisition. This pattern is designed in a way that a single tomographic image of it enables the recovery of the probe pose relative to the coordinate system of the pattern, which serves as a global coordinate system for image compounding.

Results

To investigate the feasibility of Tattoo tomography, we assessed the quality of 3D image reconstruction with experimental phantom data and in vivo forearm data. The results obtained with our prototype indicate that the Tattoo method enables the accurate and precise 3D reconstruction of PAT data and may be better suited for this task than the baseline method using optical tracking.

Conclusions

In contrast to previous approaches to 3D ultrasound (US) or PAT reconstruction, the Tattoo approach neither requires complex external hardware nor training data acquired for a specific application. It could thus become a valuable tool for clinical freehand PAT.

Fast Acoustic Steering Via Tilting Electromechanical Reflectors (FASTER): A Novel Method for High Volume Rate 3-D Ultrasound Imaging

The 3-D ultrasound imaging is essential for a wide range of clinical applications in diagnostic and interventional cardiology, radiology, and obstetrics for prenatal imaging. 3-D ultrasound imaging is also pivotal for advancing technical developments of emerging imaging technologies, such as elastography, blood flow imaging, functional ultrasound (fUS), and super-resolution microvessel imaging. At present, however, existing 3-D ultrasound imaging methods suffer from low imaging volume rate, suboptimal imaging quality, and high costs associated with 2-D ultrasound transducers. Here, we report a novel 3-D ultrasound imaging technique, fast acoustic steering via tilting electromechanical reflectors (FASTER), which provides both high imaging quality and fast imaging speed while at low cost. Capitalizing upon unique water immersible and fast-tilting microfabricated mirror to scan ultrafast plane waves in the elevational direction, FASTER is capable of high volume rate, large field-of-view (FOV) 3-D imaging with conventional 1-D transducers. In this article, we introduce the fundamental concepts of FASTER and present a series of calibration and validation studies for FASTER 3-D imaging. In a wire phantom and tissue-mimicking phantom study, we demonstrated that FASTER was capable of providing spatially accurate 3-D images with a 500-Hz imaging volume rate and an imaging FOV with a range of 48° (20 mm at 25-mm depth) in the elevational direction. We also showed that FASTER had comparable imaging quality with conventional mechanical translation-based 3-D imaging. The principles and results presented in this study establish the technical foundation for the new paradigm of high volume rate 3-D ultrasound imaging based on ultrafast plane waves and fast-tilting, water-immersible microfabricated mirrors.

A Single-Shot Region-Adaptive Network for Myotendinous Junction Segmentation in Muscular Ultrasound Images

Tracking the myotendinous junction (MTJ) in consecutive ultrasound images is crucial for understanding the mechanics and pathological conditions of the muscle-tendon unit. However, the lack of reliable and efficient identification of MTJ due to poor image quality and boundary ambiguity restricts its application in motion analysis. In recent years, with the rapid development of deep learning, the region-based convolution neural network (RCNN) has shown great potential in the field of simultaneous objection detection and instance segmentation in medical images. This article proposes a region-adaptive network (RAN) to localize MTJ region and to segment it in a single shot. Our model learns about the salient information of MTJ with the help of a composite architecture. Herein, a region-based multitask learning network explores the region containing MTJ, while a parallel end-to-end U-shaped path extracts the MTJ structure from the adaptively selected region for combating data imbalance and boundary ambiguity. By demonstrating the ultrasound images of the gastrocnemius, we showed that the RAN achieves superior segmentation performance when compared with the state-of-the-art Mask RCNN method with an average Dice score of 80.1%. Our proposed method is robust and reliable for advanced muscle and tendon function examinations obtained by ultrasound imaging.

Compact and Wireless Freehand 3D Ultrasound Real-time Spine Imaging System: A pilot study

The 3D ultrasound reconstruction technology has led to a rapid development of ultrasound spine imaging in recent decades. However, the current imaging apparatus is bulky and not portable. The objective of this study is to develop a new compact and wireless system to offer the real-time visualized spine images during data acquisition. A portable and WI-FI based ultrasound scanner and a compact EM tracking system were assembled to acquire ultrasound transverse frames with location information which could be reconstructed into 3D spine image volume in real-time. The validation was implemented on the 2D coronal images of vertebra phantoms, and the in vivo data acquisition and reconstruction were demonstrated on volunteers. The result showed that the new system could provide reconstructed spine images in real time and the average errors of the reconstructed images were about 1mm (approximate to image pixel size).

Receive/Transmit Aperture Selection for 3D Ultrasound Imaging with a 2D Matrix Transducer

Recently, we realized a prototype matrix transducer consisting of 48 rows of 80 elements on top of a tiled set of Application Specific Integrated Circuits (ASICs) implementing a row-level control connecting one transmit/receive channel to an arbitrary subset of elements per row. A fully sampled array data acquisition is implemented by a column-by-column (CBC) imaging scheme (80 transmit-receive shots) which achieves 250 volumes/second (V/s) at a pulse repetition frequency of 20 kHz. However, for several clinical applications such as carotid pulse wave imaging (CPWI), a volume rate of 1000 per second is needed. This allows only 20 transmit-receive shots per 3D image. In this study, we propose a shifting aperture scheme and investigate the effects of receive/transmit aperture size and aperture shifting step in the elevation direction. The row-level circuit is used to interconnect elements of a receive aperture in the elevation (row) direction. An angular weighting method is used to suppress the grating lobes caused by the enlargement of the effective elevation pitch of the array, as a result of element interconnection in the elevation direction. The effective aperture size, level of grating lobes, and resolution/sidelobes are used to select suitable reception/transmission parameters. Based on our assessment, the proposed imaging sequence is a full transmission (all 80 elements excited at the same time), a receive aperture size of 5 and an aperture shifting step of 3. Numerical results obtained at depths of 10, 15, and 20 mm show that, compared to the fully sampled array, the 1000 V/s is achieved at the expense of, on average, about two times wider point spread function and 4 dB higher clutter level. The resulting grating lobes were at −27 dB. The proposed imaging sequence can be used for carotid pulse wave imaging to generate an informative 3D arterial stiffness map, for cardiovascular disease assessment.

Probe Sector Matching for Freehand 3D Ultrasound Reconstruction

A 3D ultrasound image reconstruction technique, named probe sector matching (PSM), is proposed in this paper for a freehand linear array ultrasound probe equipped with multiple sensors, providing the position and attitude of the transducer and the pressure between the transducer and the target surface. The proposed PSM method includes three main steps. First, the imaging target and the working range of the probe are set to be the center and the radius of the imaging field of view, respectively. To reconstruct a 3D volume, the positions of all necessary probe sectors are pre-calculated inversely to form a sector database. Second, 2D cross-section probe sectors with the corresponding optical positioning, attitude and pressure information are collected when the ultrasound probe is moving around the imaging target. Last, an improved 3D Hough transform is used to match the plane of the current probe sector to the existing sector images in the sector database. After all pre-calculated probe sectors are acquired and matched into the 3D space defined by the sector database, a 3D ultrasound reconstruction is completed. The PSM is validated through two experiments: a virtual simulation using a numerical model and a lab experiment using a real physical model. The experimental results show that the PSM effectively reduces the errors caused by changes in the target position due to the uneven surface pressure or the inhomogeneity of the transmission media. We conclude that the PSM proposed in this study may help to design a lightweight, inexpensive and flexible ultrasound device with accurate 3D imaging capacity.

Novel Similarity Metric for Image-Based Out-Of-Plane Motion Estimation in 3D Ultrasound

Over the past decade, Freehand 3D Ultrasound(US) reconstruction using only image information has become a widely researched topic because it eliminates the need for an external tracking system and provides real-time volumetric information. But most of the state-of-art methods are inhibited by their inability to find a simple and robust similarity metric that could learn and estimate the spatial transformation between two US slices in a US sweep. In this work, we propose a novel similarity metric (TexSimAR), which computes the similarity value between two consecutive US images by correlating the parametric representation of the image-texture instead of the image itself. The purpose of this approach is to capture and compare the dynamics in the texture characteristics of two US images. We modelled these dynamics using a parametrical auto-regressive (AR) model. Experiments were performed on forearm datasets of three subjects. For every pair of consecutive US slices, we computed our TexSimAR similarity value and out-of-plane transformation from the ground truth to train a Support Vector Machine (SVM) based regression model, which was then used to predict the out-of-plane transformation with the similarity value as input. The proposed method shows promising results with predictions better than state-of-the-art methods even with 1/8^th of training data compared to other methods in the literature.

A hybrid camera- and ultrasound-based approach for needle localization and tracking using a 3D motorized curvilinear ultrasound probe

Three-dimensional (3D) motorized curvilinear ultrasound probes provide an effective, low-cost tool to guide needle interventions, but localizing and tracking the needle in 3D ultrasound volumes is often challenging. In this study, a new method is introduced to localize and track the needle using 3D motorized curvilinear ultrasound probes. In particular, a low-cost camera mounted on the probe is employed to estimate the needle axis. The camera-estimated axis is used to identify a volume of interest (VOI) in the ultrasound volume that enables high needle visibility. This VOI is analyzed using local phase analysis and the random sample consensus algorithm to refine the camera-estimated needle axis. The needle tip is determined by searching the localized needle axis using a probabilistic approach. Dynamic needle tracking in a sequence of 3D ultrasound volumes is enabled by iteratively applying a Kalman filter to estimate the VOI that includes the needle in the successive ultrasound volume and limiting the localization analysis to this VOI. A series of ex vivo animal experiments are conducted to evaluate the accuracy of needle localization and tracking. The results show that the proposed method can localize the needle in individual ultrasound volumes with maximum error rates of 0.7 mm for the needle axis, 1.7° for the needle angle, and 1.2 mm for the needle tip. Moreover, the proposed method can track the needle in a sequence of ultrasound volumes with maximum error rates of 1.0 mm for the needle axis, 2.0° for the needle angle, and 1.7 mm for the needle tip. These results suggest the feasibility of applying the proposed method to localize and track the needle using 3D motorized curvilinear ultrasound probes.

3D freehand ultrasound without external tracking using deep learning

This work aims at creating 3D freehand ultrasound reconstructions from 2D probes with image-based tracking, therefore not requiring expensive or cumbersome external tracking hardware. Existing model-based approaches such as speckle decorrelation only partially capture the underlying complexity of ultrasound image formation, thus producing reconstruction accuracies incompatible with current clinical requirements. Here, we introduce an alternative approach that relies on a statistical analysis rather than physical models, and use a convolutional neural network (CNN) to directly estimate the motion of successive ultrasound frames in an end-to-end fashion. We demonstrate how this technique is related to prior approaches, and derive how to further improve its predictive capabilities by incorporating additional information such as data from inertial measurement units (IMU). This novel method is thoroughly evaluated and analyzed on a dataset of 800 in vivo ultrasound sweeps, yielding unprecedentedly accurate reconstructions with a median normalized drift of 5.2%. Even on long sweeps exceeding 20 cm with complex trajectories, this allows to obtain length measurements with median errors of 3.4%, hence paving the way toward translation into clinical routine.

2.5-D Extended Field-of-View Ultrasound

Recently, the growing emphasis on medical ultrasound (US) has led to a rapid development of US extended field-of-view (EFOV) techniques. US EFOV techniques can be classified into three categories: 2-D US EFOV, 3-D US, and 3-D US EFOV. In this paper, we propose a novel EFOV method called 2.5-D US EFOV that combines both the advantages of the 2-D US EFOV and the 3-D US by generating a panorama on a curved image plane guided by a curved scanning trajectory of the US probe. In 2.5-D US EFOV, the real-time position and orientation of the US image plane can be recorded via an electromagnetic spatial sensor attached to the probe. The scanning direction is not necessarily straight and can be curved according to the regions of interest (ROI). To form the curved panorama, an image cutting method is proposed. Finally, the curved panorama is rendered in a 3-D space using a surface rendering based on a texture mapping technique. This allows 3-D measurements of lines and angles. Phantom experiments demonstrated that 2.5-D US EFOV images could show anatomical structures of ROI accurately and rapidly. The overall average errors for the distance and angle measurements are -0.097 ± 0.128 cm (-1% ± 1.2%) and 1.50° ± 1.60° (1.9% ± 2%), respectively. A typical extended US image can be reconstructed from 321 B-scans images within 3 s. The satisfying quantitative result on the spinal tissues of a scoliosis subject demonstrates that our system has potential applications in the assessment of musculoskeletal issues.

Fully Automatic Three-Dimensional Ultrasound Imaging Based on Conventional B-Scan

Robotic ultrasound systems have turned into clinical use over the past few decades, increasing precision and quality of medical operations. In this paper, we propose a fully automatic scanning system for three-dimensional (3-D) ultrasound imaging. A depth camera was first used to obtain the depth data and color data of the tissue surface. Based on the depth image, the 3-D contour of the tissue was rendered and the scan path of ultrasound probe was automatically planned. Following the scan path, a 3-D translating device drove the probe to move on the tissue surface. Simultaneously, the B-scans and their positional information were recorded for subsequent volume reconstruction. In order to stop the scanning process when the pressure on the skin exceeded a preset threshold, two force sensors were attached to the front side of the probe for force measurement. In vitro and in vivo experiments were conducted for assessing the performance of the proposed system. Quantitative results show that the error of volume measurement was less than 1%, indicating that the system is capable of automatic ultrasound scanning and 3-D imaging. It is expected that the proposed system can be well used in clinical practices.

Automatic Myotendinous Junction Tracking in Ultrasound Images with Phase-Based Segmentation

Displacement of the myotendinous junction (MTJ) obtained by ultrasound imaging is crucial to quantify the interactive length changes of muscles and tendons for understanding the mechanics and pathological conditions of the muscle-tendon unit during motion. However, the lack of a reliable automatic measurement method restricts its application in human motion analysis. This paper presents an automated measurement of MTJ displacement using prior knowledge on tendinous tissues and MTJ, precluding the influence of nontendinous components on the estimation of MTJ displacement. It is based on the perception of tendinous features from musculoskeletal ultrasound images using Radon transform and thresholding methods, with information about the symmetric measures obtained from phase congruency. The displacement of MTJ is achieved by tracking manually marked points on tendinous tissues with the Lucas-Kanade optical flow algorithm applied over the segmented MTJ region. The performance of this method was evaluated on ultrasound images of the gastrocnemius obtained from 10 healthy subjects (26.0 ± 2.9 years of age). Waveform similarity between the manual and automatic measurements was assessed by calculating the overall similarity with the coefficient of multiple correlation (CMC). In vivo experiments demonstrated that MTJ tracking with the proposed method (CMC = 0.97 ± 0.02) was more consistent with the manual measurements than existing optical flow tracking methods (CMC = 0.79 ± 0.11). This study demonstrated that the proposed method was robust to the interference of nontendinous components, resulting in a more reliable measurement of MTJ displacement, which may facilitate further research and applications related to the architectural change of muscles and tendons.

Nonlocal total variation based on symmetric Kullback-Leibler divergence for the ultrasound image despeckling

Background

Ultrasound imaging is safer than other imaging modalities, because it is noninvasive and nonradiative. Speckle noise degrades the quality of ultrasound images and has negative effects on visual perception and diagnostic operations.

Methods

In this paper, a nonlocal total variation (NLTV) method for ultrasonic speckle reduction is proposed. A spatiogram similarity measurement is introduced for the similarity calculation between image patches. It is based on symmetric Kullback-Leibler (KL) divergence and signal-dependent speckle model for log-compressed ultrasound images. Each patch is regarded as a spatiogram, and the spatial distribution of each bin of the spatiogram is regarded as a weighted Gamma distribution. The similarity between the corresponding bins of the two spatiograms is computed by the symmetric KL divergence. The Split-Bregman fast algorithm is then used to solve the adapted NLTV object function. Kolmogorov-Smirnov (KS) test is performed on synthetic noisy images and real ultrasound images.

Results

We validate our method on synthetic noisy images and clinical ultrasound images. Three measures are adopted for the quantitative evaluation of the despeckling performance: the signal-to-noise ratio (SNR), structural similarity index (SSIM), and natural image quality evaluator (NIQE). For synthetic noisy images, when the noise level increases, the proposed algorithm achieves slightly higher SNRS than that of the other two algorithms, and the SSIMS yielded by the proposed algorithm is obviously higher than that of the other two algorithms. For liver, IVUS and 3DUS images, the NIQE values are 8.25, 6.42 and 9.01, all of which are higher than that of the other two algorithms.

Conclusions

The results of the experiments over synthetic and real ultrasound images demonstrate that the proposed method outperforms current state-of-the-art despeckling methods with respect to speckle reduction and tissue texture preservation.

A Review on Real-Time 3D Ultrasound Imaging Technology

Real-time three-dimensional (3D) ultrasound (US) has attracted much more attention in medical researches because it provides interactive feedback to help clinicians acquire high-quality images as well as timely spatial information of the scanned area and hence is necessary in intraoperative ultrasound examinations. Plenty of publications have been declared to complete the real-time or near real-time visualization of 3D ultrasound using volumetric probes or the routinely used two-dimensional (2D) probes. So far, a review on how to design an interactive system with appropriate processing algorithms remains missing, resulting in the lack of systematic understanding of the relevant technology. In this article, previous and the latest work on designing a real-time or near real-time 3D ultrasound imaging system are reviewed. Specifically, the data acquisition techniques, reconstruction algorithms, volume rendering methods, and clinical applications are presented. Moreover, the advantages and disadvantages of state-of-the-art approaches are discussed in detail.