1
|
Guo H, Chao H, Xu S, Wood BJ, Wang J, Yan P. Ultrasound Volume Reconstruction From Freehand Scans Without Tracking. IEEE Trans Biomed Eng 2023; 70:970-979. [PMID: 36103448 PMCID: PMC10011008 DOI: 10.1109/tbme.2022.3206596] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/25/2023]
Abstract
Transrectal ultrasound is commonly used for guiding prostate cancer biopsy, where 3D ultrasound volume reconstruction is often desired. Current methods for 3D reconstruction from freehand ultrasound scans require external tracking devices to provide spatial information of an ultrasound transducer. This paper presents a novel deep learning approach for sensorless ultrasound volume reconstruction, which efficiently exploits content correspondence between ultrasound frames to reconstruct 3D volumes without external tracking. The underlying deep learning model, deep contextual-contrastive network (DC 2-Net), utilizes self-attention to focus on the speckle-rich areas to estimate spatial movement and then minimizes a margin ranking loss for contrastive feature learning. A case-wise correlation loss over the entire input video helps further smooth the estimated trajectory. We train and validate DC 2-Net on two independent datasets, one containing 619 transrectal scans and the other having 100 transperineal scans. Our proposed approach attained superior performance compared with other methods, with a drift rate of 9.64 % and a prostate Dice of 0.89. The promising results demonstrate the capability of deep neural networks for universal ultrasound volume reconstruction from freehand 2D ultrasound scans without tracking information.
Collapse
|
2
|
Benjamin A, Chen M, Li Q, Chen L, Dong Y, Carrascal CA, Xie H, Samir AE, Anthony BW. Renal Volume Estimation Using Freehand Ultrasound Scans: An Ex Vivo Demonstration. ULTRASOUND IN MEDICINE & BIOLOGY 2020; 46:1769-1782. [PMID: 32376189 DOI: 10.1016/j.ultrasmedbio.2020.03.006] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/05/2019] [Revised: 02/27/2020] [Accepted: 03/10/2020] [Indexed: 06/11/2023]
Abstract
Renal volume has the potential to serve as a robust biomarker for tracking the onset and progression of renal diseases and also for quantifying renal function. We propose a method to estimate renal volumes using freehand ultrasound scans at the point of care. A conventional ultrasound probe was augmented with an Intel RealSense D435 i camera. Visual inertial simultaneous localization and mapping was used to localize the probe in free space. The acquired 2-D ultrasound images, segmented by trained clinicians, were combined with the estimated poses of the probe to yield accurate volumes. The method was tested on two ex vivo sheep kidneys embedded in gelatin phantoms. Four different scanning protocols were tested: transverse linear, transverse fan, longitudinal linear and longitudinal fan. The estimated renal volumes were compared with those obtained using the water displacement method, the ellipsoidal method and computed tomography imaging. The water displacement method yielded mean volumes of 66.00 and 66.20 mL for kidneys 1 and 2, respectively (ground truth). Freehand ultrasound scans produced mean volumes of 64.08 mL (2.90% error) and 65.25 mL (1.40% error); the ellipsoidal method yielded volumes of 57.49 mL (12.90% error) and 60.15 mL (9.13% error); and computed tomography yielded a volume of 63.00 mL (4.54% error).
Collapse
Affiliation(s)
- Alex Benjamin
- Device Realization and Computational Instrumentation Laboratory, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA
| | - Melinda Chen
- Device Realization and Computational Instrumentation Laboratory, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA
| | - Qian Li
- Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, USA
| | - Lei Chen
- Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, USA
| | - Yi Dong
- Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, USA
| | | | - Hua Xie
- Philips Research North America, Cambridge, Massachusetts, USA
| | - Anthony E Samir
- Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, USA
| | - Brian W Anthony
- Device Realization and Computational Instrumentation Laboratory, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA.
| |
Collapse
|
3
|
Prevost R, Salehi M, Jagoda S, Kumar N, Sprung J, Ladikos A, Bauer R, Zettinig O, Wein W. 3D freehand ultrasound without external tracking using deep learning. Med Image Anal 2018; 48:187-202. [PMID: 29936399 DOI: 10.1016/j.media.2018.06.003] [Citation(s) in RCA: 40] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2018] [Revised: 06/05/2018] [Accepted: 06/06/2018] [Indexed: 11/18/2022]
Abstract
This work aims at creating 3D freehand ultrasound reconstructions from 2D probes with image-based tracking, therefore not requiring expensive or cumbersome external tracking hardware. Existing model-based approaches such as speckle decorrelation only partially capture the underlying complexity of ultrasound image formation, thus producing reconstruction accuracies incompatible with current clinical requirements. Here, we introduce an alternative approach that relies on a statistical analysis rather than physical models, and use a convolutional neural network (CNN) to directly estimate the motion of successive ultrasound frames in an end-to-end fashion. We demonstrate how this technique is related to prior approaches, and derive how to further improve its predictive capabilities by incorporating additional information such as data from inertial measurement units (IMU). This novel method is thoroughly evaluated and analyzed on a dataset of 800 in vivo ultrasound sweeps, yielding unprecedentedly accurate reconstructions with a median normalized drift of 5.2%. Even on long sweeps exceeding 20 cm with complex trajectories, this allows to obtain length measurements with median errors of 3.4%, hence paving the way toward translation into clinical routine.
Collapse
Affiliation(s)
| | - Mehrdad Salehi
- ImFusion GmbH, Agnes-Pockels-Bogen 1, Munich, Germany; Computer Aided Medical Procedures (CAMP), TU Munich, Munich, Germany
| | - Simon Jagoda
- ImFusion GmbH, Agnes-Pockels-Bogen 1, Munich, Germany
| | - Navneet Kumar
- ImFusion GmbH, Agnes-Pockels-Bogen 1, Munich, Germany
| | | | | | | | | | - Wolfgang Wein
- ImFusion GmbH, Agnes-Pockels-Bogen 1, Munich, Germany
| |
Collapse
|
4
|
Martins NSDF, Carneiro LT, Dantas HDM, Esperança C, Marroquim RG, Oliveira LFD, Machado JC. Generation of 3D ultrasound biomicroscopic images: technique validation and in vivo volumetric imaging of rat lateral gastrocnemius. ACTA ACUST UNITED AC 2015. [DOI: 10.1590/1517-3151.0209] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/31/2023]
Affiliation(s)
| | | | | | | | | | | | - João Carlos Machado
- Universidade Federal do Rio de Janeiro, Brasil; Universidade Federal do Rio de Janeiro, Brasil
| |
Collapse
|
5
|
Afsham N, Najafi M, Abolmaesumi P, Rohling R. A Generalized Correlation-Based Model for Out-of-Plane Motion Estimation in Freehand Ultrasound. IEEE TRANSACTIONS ON MEDICAL IMAGING 2014; 33:186-199. [PMID: 24108710 DOI: 10.1109/tmi.2013.2283969] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/02/2023]
Abstract
A big challenge in sensorless image-based ultrasound tracking is in the out-of-plane motion estimation. The correlation value of a specific model of speckle known as fully developed speckle (FDS) can be used to estimate the out-of-plane displacement. In real tissue, this kind of pattern is rare and the deviation of speckle pattern from the ideal FDS model diminishes the accuracy of the out-of-plane motion estimation. In this paper a new method for estimation of the out-of-plane motion is proposed. Firstly a closed-form mathematical derivation is provided for the correlation of two RF echo signal patches at different positions. A linear regression model of the ultrasound beam profile is proposed to account for the spatial variability of the ultrasound beam and enhance the accuracy of out-of-plane motion estimation in real tissue. The statistical model of speckle used here is based on the Rician-Inverse Gaussian (RiIG) stochastic process of the speckle formation, which can be considered as a generalized form of the K-distribution with richer parametrization. In this work, for the first time the second-order statistics of the RIG model is used for speckle tracking. This statistical model allows for derivation of a closed-form formulation for the correlation coefficient based on the statistical parameters of every patch. Since the effect of coherency is considered in the RiIG model, it increases the reliability of the out-of-plane motion estimation. The flexibility of the proposed method enables almost any patch through the whole image to be used for the purpose of displacement estimation. The method has been evaluated both on ex vivo and in vivo tissues in various experiments including out-of-plane rotation (tilt, yaw) and free-hand imaging. The overall outcome demonstrates the potential of the proposed method for in vivo tissues.
Collapse
|
6
|
Hsiao YH, Huang YL, Kuo SJ, Liang WM, Chen ST, Chen DR. Characterization of benign and malignant solid breast masses in harmonic 3D power Doppler imaging. Eur J Radiol 2009; 71:89-95. [DOI: 10.1016/j.ejrad.2008.03.024] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2008] [Accepted: 03/27/2008] [Indexed: 11/26/2022]
|
7
|
Real-time tissue tracking with B-mode ultrasound using speckle and visual servoing. MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION : MICCAI ... INTERNATIONAL CONFERENCE ON MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION 2008. [PMID: 18044546 DOI: 10.1007/978-3-540-75759-7_1] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register]
Abstract
We present a method for real-time tracking of moving soft tissue with B-mode ultrasound (US). The method makes use of the speckle information contained in the US images to estimate the in-plane and out-of-plane motion of a fixed target relative to the ultrasound scan plane. The motion information is then used as closed-loop feedback to a robot which corrects for the target motion. The concept is demonstrated for translation motions in an experimental setup consisting of an ultrasound speckle phantom, a robot for simulating tissue motion, and a robot that performs motion stabilization from US images. This concept shows promise for US-guided procedures that require real-time motion tracking and compensation.
Collapse
|
8
|
Garson CD, Li Y, Hossack JA. Free-hand ultrasound scanning approaches for volume quantification of the mouse heart Left ventricle. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2007; 54:966-77. [PMID: 17523561 DOI: 10.1109/tuffc.2007.342] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/15/2023]
Abstract
Two approaches for free-hand motion tracking that enable volumetric quantification of the murine heart were investigated. One approach used an instrumented, multijointed articulated arm attached to a 14 MHz ultrasound transducer array. A second approach used an E-beam transducer--a modified linear transducer array containing a main imaging array adjacent to three perpendicular tracking arrays. Motion between successive B-mode image frames was computed by tracking image speckle in each tracking array. Both tracking systems produced accurate results in a phantom validation study (4.50% error and 3.75% error for estimates derived using the articulated arm and E-beam tracking techniques, respectively). The tracking approaches also were tested in vivo on three mice. Results were compared to values obtained by mounting each mouse on a micromanipulator, adjusting its position by 0.5-mm increments, and acquiring B-mode images using a high-resolution ultrasound scanner. Left ventricular end diastolic volume (LVEDV) estimates differed from values obtained using the high-resolution scanner by a mean error of 18.2% and 2.60% for eight scans conducted on each of two mice using the articulated arm, and a mean error of 13.6%, 6.53%, and 12.58% for eight scans conducted on each of three mice using the E-beam.
Collapse
Affiliation(s)
- Christopher D Garson
- University of Virginia, Department of Biomedical Engineering, Charlottesville, VA, USA
| | | | | |
Collapse
|
9
|
Krupa A, Fichtinger G, Hager GD. Full Motion Tracking in Ultrasound Using Image Speckle Information and Visual Servoing. ACTA ACUST UNITED AC 2007. [DOI: 10.1109/robot.2007.363688] [Citation(s) in RCA: 24] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
10
|
Laporte C, Arbel T. Probabilistic speckle decorrelation for 3D ultrasound. MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION : MICCAI ... INTERNATIONAL CONFERENCE ON MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION 2007; 10:925-932. [PMID: 18051147 DOI: 10.1007/978-3-540-75757-3_112] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/25/2023]
Abstract
Recent developments in freehand 3D ultrasound (US) have shown how image registration and speckle decorrelation methods can be used for 3D reconstruction instead of relying on a tracking device. Estimating elevational separation between untracked US images using speckle decorrelation is error prone due to the uncertainty that plagues the correlation measurements. In this paper, using maximum entropy estimation methods, the uncertainty is directly modeled from the calibration data normally used to estimate an average decorrelation curve. Multiple correlation measurements can then be fused within a maximum likelihood estimation framework in order to reduce the drift in elevational pose estimation over large image sequences. The approach is shown to be effective through empirical results on simulated and phantom US data.
Collapse
Affiliation(s)
- Catherine Laporte
- Centre for Intelligent Machines, McGill University, Montréal, Canada.
| | | |
Collapse
|
11
|
Gee AH, James Housden R, Hassenpflug P, Treece GM, Prager RW. Sensorless freehand 3D ultrasound in real tissue: Speckle decorrelation without fully developed speckle. Med Image Anal 2006; 10:137-49. [PMID: 16143560 DOI: 10.1016/j.media.2005.08.001] [Citation(s) in RCA: 41] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2005] [Accepted: 07/28/2005] [Indexed: 11/29/2022]
Abstract
It has previously been demonstrated that freehand 3D ultrasound can be acquired without a position sensor by measuring the elevational speckle decorrelation from frame to frame. However, this requires that the B-scans contain significant amounts of fully developed speckle. In this paper, we show that this condition is rarely satisfied in scans of real tissue, which instead exhibit fairly ubiquitous coherent scattering. By examining the axial and lateral correlation functions, we propose an heuristic technique to quantify the amount of coherency at each point in the B-scans. This leads to an adapted elevational decorrelation scheme which allows for the coherent scattering. Using the adapted scheme, we demonstrate markedly improved reconstructions of animal tissue in vitro.
Collapse
Affiliation(s)
- Andrew H Gee
- University of Cambridge, Department of Engineering, Trumpington Street, Cambridge, Cambs CB2 1PZ, UK.
| | | | | | | | | |
Collapse
|