Linear tracking for 3-D medical ultrasound imaging

Six-Degree-of-Freedom Freehand 3D Ultrasound: A Low-Cost Computer Vision-Based Approach for Orthopedic Applications

In orthopedics, X-rays and computed tomography (CT) scans play pivotal roles in diagnosing and treating bone pathologies. Machine bulkiness and the emission of ionizing radiation remain the main problems associated with these techniques. The accessibility and low risks related to ultrasound handling make it a popular 2D imaging method. Indeed, 3D ultrasound assembles 2D slices into a 3D volume. This study aimed to implement a probe-tracking method for 6 DoF 3D ultrasound. The proposed method involves a dodecahedron with ArUco markers attached, enabling computer vision tracking of the ultrasound probe's position and orientation. The algorithm focuses on the data acquisition phase but covers the basic reconstruction required for data generation and analysis. In the best case, the analysis revealed an average error norm of 2.858 mm with a standard deviation norm of 5.534 mm compared to an infrared optical tracking system used as a reference. This study demonstrates the feasibility of performing volumetric imaging without ionizing radiation or bulky systems. This marker-based approach shows promise for enhancing orthopedic imaging, providing a more accessible imaging modality for helping clinicians to diagnose pathologies regarding complex joints, such as the shoulder, replacing standard infrared tracking systems known to suffer from marker occlusion problems.

A System for Reproducible 3D Ultrasound Measurements of Skeletal Muscles

In 3D freehand ultrasound imaging, operator dependent variations in applied forces and movements can lead to errors in the reconstructed images. In this paper, we introduce an automated 3D ultrasound system, which enables acquisitions with controlled movement trajectories by using motors, which electrically move the probe. Due to integrated encoders there is no need of position sensors. An included force control mechanism ensures a constant contact force to the skin. We conducted 8 trials with the automated 3D ultrasound system on 2 different phantoms with 3 force settings and 10 trials on a human tibialis anterior muscle with 2 force settings. For comparison, we also conducted 8 freehand 3D ultrasound scans from 2 operators (4 force settings) on one phantom and 10 with one operator on the tibialis anterior muscle. Both freehand and automated trials showed small errors in volume and length computations of the reconstructions, however the freehand trials showed larger standard deviations. We also computed the thickness of the phantom and the tibialis anterior muscle. We found significant differences in force settings for the operators and higher coefficients of variation for the freehand trials. Overall, the automated 3D ultrasound system shows a high accuracy in reconstruction. Due to the smaller coefficients of variation, the automated 3D ultrasound system enables more reproducible ultrasound examinations than the freehand scanning. Therefore, the automated 3D ultrasound system is a reliable tool for 3D investigations of skeletal muscle.

Comparison of image quality of 3D ultrasound: motorized acquisition versus freehand navigated acquisition, a phantom study

PURPOSE

Intra-operative assessment of resection margins during oncological surgery is a field that needs improvement. Ultrasound (US) shows the potential to fulfill this need, but this imaging technique is highly operator-dependent. A 3D US image of the whole specimen may remedy the operator dependence. This study aims to compare and evaluate the image quality of 3D US between freehand acquisition (FA) and motorized acquisition (MA).

METHODS

Multiple 3D US volumes of a commercial phantom were acquired in motorized and freehand fashion. FA images were collected with electromagnetic navigation. An integrated algorithm reconstructed the FA images. MA images were stacked into a 3D volume. The image quality is evaluated following the metrics: contrast resolution, axial and elevation resolution, axial and elevation distance calibration, stability, inter-operator variability, and intra-operator variability. A linear mixed model determined statistical differences between FA and MA for these metrics.

RESULTS

The MA results in a statistically significant lower error of axial distance calibration (p < 0.0001) and higher stability (p < 0.0001) than FA. On the other hand, the FA has a better elevation resolution (p < 0.003) than the MA.

CONCLUSION

MA results in better image quality of 3D US than the FA method based on axial distance calibration, stability, and variability. This study suggests acquiring 3D US volumes for intra-operative ex vivo margin assessment in a motorized fashion.

Recent Advances in Tracking Devices for Biomedical Ultrasound Imaging Applications

With the rapid advancement of tracking technologies, the applications of tracking systems in ultrasound imaging have expanded across a wide range of fields. In this review article, we discuss the basic tracking principles, system components, performance analyses, as well as the main sources of error for popular tracking technologies that are utilized in ultrasound imaging. In light of the growing demand for object tracking, this article explores both the potential and challenges associated with different tracking technologies applied to various ultrasound imaging applications, including freehand 3D ultrasound imaging, ultrasound image fusion, ultrasound-guided intervention and treatment. Recent development in tracking technology has led to increased accuracy and intuitiveness of ultrasound imaging and navigation with less reliance on operator skills, thereby benefiting the medical diagnosis and treatment. Although commercially available tracking systems are capable of achieving sub-millimeter resolution for positional tracking and sub-degree resolution for orientational tracking, such systems are subject to a number of disadvantages, including high costs and time-consuming calibration procedures. While some emerging tracking technologies are still in the research stage, their potentials have been demonstrated in terms of the compactness, light weight, and easy integration with existing standard or portable ultrasound machines.

[Establishment of a 3D ultrasound imaging system based on pulse-triggered image acquisition]

OBJECTIVE

To establish a 3D ultrasound imaging system based on pulse-triggered image acquisition using the linear probe on the Verasonics^TM vantage 128 platform and evaluate its performance in scanning standard phantom and human carotid artery.

OBJECTIVE

The 3D ultrasound imaging system included 3 modules for probe motion control, image acquisition and storage, and 3D image reconstruction and display. To improve the precision of image acquisition, we used fixed frequency pulses to control the external trigger function combined with mechanical scanning. Voxel-based 3D reconstruction was used for image reconstruction and display. The user interface was designed to allow direct operations of the platform. We carried out scanning tests of standard ultrasound phantom and human carotid artery to evaluate the performance of this imaging system.

OBJECTIVE

We successfully constructed a 3D ultrasound imaging system based on pulse-triggered image acquisition. The results of standard phantom and human carotid scanning tests showed that each module of the system was fully functional. The self-designed user interface of this ultrasound imaging system allowed full control of the system functions for original image acquisition, 3D image reconstruction, and display of cross-sections in 3 different views.

OBJECTIVE

This 3D ultrasound imaging system achieves high-quality 3D ultrasound imaging and provides the basis for further study and clinical application of 3D ultrasound imaging.

Improved 3D Catheter Shape Estimation Using Ultrasound Imaging for Endovascular Navigation: A Further Study

OBJECTIVE

Two-dimensional fluoroscopy is the standard guidance imaging method for closed endovascular intervention. However, two-dimensional fluoroscopy lacks depth perception for the intervention catheter and causes radiation exposure for both surgeons and patients. In this paper, we extend our previous study and develop the improved three-dimensional (3D) catheter shape estimation using ultrasound imaging. In addition, we perform further quantitative evaluations of endovascular navigation.

METHOD

First, the catheter tracking accuracy in ultrasound images is improved by adjusting the state vector and adding direction information. Then, the 3D catheter points from the catheter tracking are further optimized based on the 3D catheter shape optimization with a high-quality sample set. Finally, the estimated 3D catheter shapes from ultrasound images are overlaid with preoperative 3D tissue structures for the intuitive endovascular navigation.

RESULTS

the tracking accuracy of the catheter increased by 24.39%, and the accuracy of the catheter shape optimization step also increased by approximately 17.34% compared with our previous study. Furthermore, the overall error of catheter shape estimation was further validated in the catheter intervention experiment of in vitro cardiovascular tissue and in a vivo swine, and the errors were 2.13 mm and 3.37 mm, respectively.

CONCLUSION

Experimental results demonstrate that the improved catheter shape estimation using ultrasound imaging is accurate and appropriate for endovascular navigation.

SIGNIFICANCE

Improved navigation reduces the radiation risk because it decreases use of X-ray imaging. In addition, this navigation method can also provide accurate 3D catheter shape information for endovascular surgery.

Visual Object Tracking by Hierarchical Attention Siamese Network

Visual tracking addresses the problem of localizing an arbitrary target in video according to the annotated bounding box. In this article, we present a novel tracking method by introducing the attention mechanism into the Siamese network to increase its matching discrimination. We propose a new way to compute attention weights to improve matching performance by a sub-Siamese network [Attention Net (A-Net)], which locates attentive parts for solving the searching problem. In addition, features in higher layers can preserve more semantic information while features in lower layers preserve more location information. Thus, in order to solve the tracking failure cases by the higher layer features, we fully utilize location and semantic information by multilevel features and propose a new way to fuse multiscale response maps from each layer to obtain a more accurate position estimation of the object. We further propose a hierarchical attention Siamese network by combining the attention weights and multilayer integration for tracking. Our method is implemented with a pretrained network which can outperform most well-trained Siamese trackers even without any fine-tuning and online updating. The comparison results with the state-of-the-art methods on popular tracking benchmarks show that our method achieves better performance. Our source code and results will be available at https://github.com/shenjianbing/HASN.

In Vivo Measurement of the Human Lumbar Spine Using Magnetic Resonance Imaging to Ultrasound Registration

OBJECTIVE

This study aimed to refine a magnetic resonance imaging (MRI)-ultrasound registration (ie, alignment) technique to make noninvasive, nonionizing, 3-dimensional measurement of the lumbar segmental motion in vivo.

METHODS

Five healthy participants participated in this validation study. We scanned the lumbar region of each participant 5 times using an ultrasound probe while he or she kept a prone lying posture on a plinth. Participant-specific models of L1-L5 were constructed from magnetic resonance (MR) images and aligned with the 3-dimensional ultrasound dataset of each scan using 4 variants of MRI-ultrasound registration approach (simplified intensity-based registration [1] with and [2] without including the transverse processes and their surrounding soft tissues [denoted as TP complex]; and hierarchical intensity-based registration [3] with and [4] without including the TP complex). The robustness and precision of these registration approaches were compared.

RESULTS

Although all registration approaches converged to a similar solution, excluding the TP complex improved the percentage of successful registration from 92% to 100%. There was no significant difference in the precision among the 4 MRI-ultrasound registration variants. For the simplified intensity-based registration without including the TP complex, average precision at each degree of freedom was 1.33° (flexion-extension), 2.48° (lateral bending), 1.32° (axial rotation), 2.15 mm (left/right), 1.08 mm (anterior-posterior), and 1.16 (superior-inferior), respectively.

CONCLUSION

Given that using simplified intensity-based MRI-ultrasound registration can substantially streamline the registration process and excluding the TP complex would improve the robustness of the registration, we conclude that this combination is the method of choice for in vivo human applications.

2.5-D Extended Field-of-View Ultrasound

Recently, the growing emphasis on medical ultrasound (US) has led to a rapid development of US extended field-of-view (EFOV) techniques. US EFOV techniques can be classified into three categories: 2-D US EFOV, 3-D US, and 3-D US EFOV. In this paper, we propose a novel EFOV method called 2.5-D US EFOV that combines both the advantages of the 2-D US EFOV and the 3-D US by generating a panorama on a curved image plane guided by a curved scanning trajectory of the US probe. In 2.5-D US EFOV, the real-time position and orientation of the US image plane can be recorded via an electromagnetic spatial sensor attached to the probe. The scanning direction is not necessarily straight and can be curved according to the regions of interest (ROI). To form the curved panorama, an image cutting method is proposed. Finally, the curved panorama is rendered in a 3-D space using a surface rendering based on a texture mapping technique. This allows 3-D measurements of lines and angles. Phantom experiments demonstrated that 2.5-D US EFOV images could show anatomical structures of ROI accurately and rapidly. The overall average errors for the distance and angle measurements are -0.097 ± 0.128 cm (-1% ± 1.2%) and 1.50° ± 1.60° (1.9% ± 2%), respectively. A typical extended US image can be reconstructed from 321 B-scans images within 3 s. The satisfying quantitative result on the spinal tissues of a scoliosis subject demonstrates that our system has potential applications in the assessment of musculoskeletal issues.

Fully Automatic Three-Dimensional Ultrasound Imaging Based on Conventional B-Scan

Robotic ultrasound systems have turned into clinical use over the past few decades, increasing precision and quality of medical operations. In this paper, we propose a fully automatic scanning system for three-dimensional (3-D) ultrasound imaging. A depth camera was first used to obtain the depth data and color data of the tissue surface. Based on the depth image, the 3-D contour of the tissue was rendered and the scan path of ultrasound probe was automatically planned. Following the scan path, a 3-D translating device drove the probe to move on the tissue surface. Simultaneously, the B-scans and their positional information were recorded for subsequent volume reconstruction. In order to stop the scanning process when the pressure on the skin exceeded a preset threshold, two force sensors were attached to the front side of the probe for force measurement. In vitro and in vivo experiments were conducted for assessing the performance of the proposed system. Quantitative results show that the error of volume measurement was less than 1%, indicating that the system is capable of automatic ultrasound scanning and 3-D imaging. It is expected that the proposed system can be well used in clinical practices.

Multiobject Tracking by Submodular Optimization

In this paper, we propose a new multiobject visual tracking algorithm by submodular optimization. The proposed algorithm is composed of two main stages. At the first stage, a new selecting strategy of tracklets is proposed to cope with occlusion problem. We generate low-level tracklets using overlap criteria and min-cost flow, respectively, and then integrate them into a candidate tracklets set. In the second stage, we formulate the multiobject tracking problem as the submodular maximization problem subject to related constraints. The submodular function selects the correct tracklets from the candidate set of tracklets to form the object trajectory. Then, we design a connecting process which connects the corresponding trajectories to overcome the occlusion problem. Experimental results demonstrate the effectiveness of our tracking algorithm. Our source code is available at https://github.com/shenjianbing/submodulartrack.

Freehand 3-D Ultrasound Imaging: A Systematic Review

Two-dimensional ultrasound (US) imaging has been successfully used in clinical applications as a low-cost, portable and non-invasive image modality for more than three decades. Recent advances in computer science and technology illustrate the promise of the 3-D US modality as a medical imaging technique that is comparable to other prevalent modalities and that overcomes certain drawbacks of 2-D US. This systematic review covers freehand 3-D US imaging between 1970 and 2017, highlighting the current trends in research fields, the research methods, the main limitations, the leading researchers, standard assessment criteria and clinical applications. Freehand 3-D US systems are more prevalent in the academic environment, whereas in clinical applications and industrial research, most studies have focused on 3-D US transducers and improvement of hardware performance. This topic is still an interesting active area for researchers, and there remain many unsolved problems to be addressed.

An Acoustic Tracking Approach for Medical Ultrasound Image Simulator

Ultrasound examinations are a standard procedure in the clinical diagnosis of many diseases. However, the efficacy of an ultrasound examination is highly dependent on the skill and experience of the operator, which has prompted proposals for ultrasound simulation systems to facilitate training and education in hospitals and medical schools. The key technology of the medical ultrasound simulation system is the probe tracking method that is used to determine the position and inclination angle of the sham probe, since this information is used to display the ultrasound images in real time. This study investigated a novel acoustic tracking approach for an ultrasound simulation system that exhibits high sensitivity and is cost-effective. Five air-coupled ultrasound elements are arranged as a 1D array in front of a sham probe for transmitting the acoustic signals, and a 5 × 5 2D array of receiving elements is used to receive the acoustic signals from the moving transmitting elements. Since the patterns of the received signals can differ for different positions and angles of the moving probe, the probe can be tracked precisely by the acoustic tracking approach. After the probe position has been determined by the system, the corresponding ultrasound image is immediately displayed on the screen. The system performance was verified by scanning three different subjects as image databases: a simple commercial phantom, a complicated self-made phantom, and a porcine heart. The experimental results indicated that the tracking and angle accuracies of the presented acoustic tracking approach were 0.7 mm and 0.5°, respectively. The performance of the acoustic tracking approach is compared with those of other tracking technologies.

Hand-Eye Calibration in Visually-Guided Robot Grinding

Visually-guided robot grinding is a novel and promising automation technique for blade manufacturing. One common problem encountered in robot grinding is hand-eye calibration, which establishes the pose relationship between the end effector (hand) and the scanning sensor (eye). This paper proposes a new calibration approach for robot belt grinding. The main contribution of this paper is its consideration of both joint parameter errors and pose parameter errors in a hand-eye calibration equation. The objective function of the hand-eye calibration is built and solved, from which 30 compensated values (corresponding to 24 joint parameters and six pose parameters) are easily calculated in a closed solution. The proposed approach is economic and simple because only a criterion sphere is used to calculate the calibration parameters, avoiding the need for an expensive and complicated tracking process using a laser tracker. The effectiveness of this method is verified using a calibration experiment and a blade grinding experiment. The code used in this approach is attached in the Appendix.

Development of a Wireless and Near Real-Time 3D Ultrasound Strain Imaging System

Ultrasound elastography is an important medical imaging tool for characterization of lesions. In this paper, we present a wireless and near real-time 3D ultrasound strain imaging system. It uses a 3D translating device to control a commercial linear ultrasound transducer to collect pre-compression and post-compression radio-frequency (RF) echo signal frames. The RF frames are wirelessly transferred to a high-performance server via a local area network (LAN). A dynamic programming strain estimation algorithm is implemented with the compute unified device architecture (CUDA) on the graphic processing unit (GPU) in the server to calculate the strain image after receiving a pre-compression RF frame and a post-compression RF frame at the same position. Each strain image is inserted into a strain volume which can be rendered in near real-time. We take full advantage of the translating device to precisely control the probe movement and compression. The GPU-based parallel computing techniques are designed to reduce the computation time. Phantom and in vivo experimental results demonstrate that our system can generate strain volumes with good quality and display an incrementally reconstructed volume image in near real-time.

Accurate ultrasound imaging based on range point migration method for the depiction of fetal surface

PURPOSE

The purpose of this study is to evaluate the performance of a modified range point migration (RPM) method using a semi-broad transmit beam for fetal surface imaging.

METHODS

The conventional RPM method depicts accurate images of target surfaces by estimating the reflection point on a target surface from the path length of plural transmit-and-receive element combinations. However, the conventional RPM method depicts false images when echoes from different targets are received simultaneously. For the elimination of false images in the employment of the RPM method, we propose a modified RPM method with a semi-broad transmit beam to decrease the number of targets in each measurement region.

RESULTS

The modified RPM method depicted two acrylic cylinders of 2 cm in diameter with a root-mean-square error (RMSE) of 0.062 mm, where the RMSE of the migration method was 0.145 mm. The modified RPM method also succeeded in depicting a 7-month fetal phantom with a RMSE of 0.058 mm relative to a 3D image acquired using optical measurement.

CONCLUSION

This study shows the potential of the modified RPM method in achieving accurate surface imaging of multiple targets using a semi-broad beam, indicating that the method is suitable for fetal surface imaging.

3-D ultrasonic strain imaging based on a linear scanning system

This paper introduces a 3-D strain imaging method based on a freehand linear scanning mode. We designed a linear sliding track with a position sensor and a height-adjustable holder to constrain the movement of an ultrasound probe in a freehand manner. When moving the probe along the sliding track, the corresponding positional measures for the probe are transmitted via a wireless communication module based on Bluetooth in real time. In a single examination, the probe is scanned in two sweeps in which the height of the probe is adjusted by the holder to collect the pre- and postcompression radio-frequency echoes, respectively. To generate a 3-D strain image, a volume cubic in which the voxels denote relative strains for tissues is defined according to the range of the two sweeps. With respect to the post-compression frames, several slices in the volume are determined and the pre-compression frames are re-sampled to precisely correspond to the post-compression frames. Thereby, a strain estimation method based on minimizing a cost function using dynamic programming is used to obtain the 2-D strain image for each pair of frames from the re-sampled pre-compression sweep and the post-compression sweep, respectively. A software system is developed for volume reconstruction, visualization, and measurement of the 3-D strain images. The experimental results show that high-quality 3-D strain images of phantom and human tissues can be generated by the proposed method, indicating that the proposed system can be applied for real clinical applications (e.g., musculoskeletal assessments).