1
|
Su PC, Tan SY, Liu Z, Yeh WC. A Mixed-Heuristic Quantum-Inspired Simplified Swarm Optimization Algorithm for scheduling of real-time tasks in the multiprocessor system. Appl Soft Comput 2022. [DOI: 10.1016/j.asoc.2022.109807] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
|
2
|
Lediju Bell MA, Shubert J. Photoacoustic-based visual servoing of a needle tip. Sci Rep 2018; 8:15519. [PMID: 30341371 PMCID: PMC6195562 DOI: 10.1038/s41598-018-33931-9] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2018] [Accepted: 10/08/2018] [Indexed: 12/15/2022] Open
Abstract
In intraoperative settings, the presence of acoustic clutter and reflection artifacts from metallic surgical tools often reduces the effectiveness of ultrasound imaging and complicates the localization of surgical tool tips. We propose an alternative approach for tool tracking and navigation in these challenging acoustic environments by augmenting ultrasound systems with a light source (to perform photoacoustic imaging) and a robot (to autonomously and robustly follow a surgical tool regardless of the tissue medium). The robotically controlled ultrasound probe continuously visualizes the location of the tool tip by segmenting and tracking photoacoustic signals generated from an optical fiber inside the tool. System validation in the presence of fat, muscle, brain, skull, and liver tissue with and without the presence of an additional clutter layer resulted in mean signal tracking errors <2 mm, mean probe centering errors <1 mm, and successful recovery from ultrasound perturbations, representing either patient motion or switching from photoacoustic images to ultrasound images to search for a target of interest. A detailed analysis of channel SNR in controlled experiments with and without significant acoustic clutter revealed that the detection of a needle tip is possible with photoacoustic imaging, particularly in cases where ultrasound imaging traditionally fails. Results show promise for guiding surgeries and procedures in acoustically challenging environments with this novel robotic and photoacoustic system combination.
Collapse
Affiliation(s)
- Muyinatu A Lediju Bell
- Johns Hopkins University, Department of Electrical and Computer Engineering, Baltimore, MD, 21218, USA. .,Johns Hopkins University, Department of Biomedical Engineering, Baltimore, MD, 21218, USA. .,Johns Hopkins University, Department of Computer Science, Baltimore, MD, 21218, USA.
| | - Joshua Shubert
- Johns Hopkins University, Department of Electrical and Computer Engineering, Baltimore, MD, 21218, USA
| |
Collapse
|
3
|
Nawapun K, Phithakwatchara N, Jaingam S, Viboonchart S, Mongkolchat N, Wataganara T. Advanced ultrasound for prenatal interventions. Ultrasonography 2018; 37:200-210. [PMID: 29852543 PMCID: PMC6044223 DOI: 10.14366/usg.18011] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2018] [Accepted: 05/02/2018] [Indexed: 12/29/2022] Open
Abstract
Ultrasound is an integral part of prenatal interventions. Doppler studies and 3-dimensional ultrasound (3DUS) are frequently used to determine whether fetal surgery is required. The operator's experience remains crucial for reducing procedure-related morbidity. Real-time 3DUS (or 4DUS) can simultaneously display the needle tip in three orthogonal planes, providing reassurance that no fetal parts are in the path. In experienced hands, 4DUS guidance may not be more effective than B-mode, but its value for less-experienced operators remains to be determined. Recent developments in needle, shunt, and video endoscopic technologies may compliment the use of image-guided in utero procedures. Future developments of higher-dimensional transducers and image software may improve the utility of ultrasound for invasive obstetric interventions.
Collapse
Affiliation(s)
- Katika Nawapun
- Division of Maternal-Fetal Medicine, Department of Obstetrics and Gynecology, Faculty of Medicine, Siriraj Hospital, Bangkok, Thailand
| | - Nisarat Phithakwatchara
- Division of Maternal-Fetal Medicine, Department of Obstetrics and Gynecology, Faculty of Medicine, Siriraj Hospital, Bangkok, Thailand
| | - Suparat Jaingam
- Division of Maternal-Fetal Medicine, Department of Obstetrics and Gynecology, Faculty of Medicine, Siriraj Hospital, Bangkok, Thailand
| | - Sommai Viboonchart
- Division of Maternal-Fetal Medicine, Department of Obstetrics and Gynecology, Faculty of Medicine, Siriraj Hospital, Bangkok, Thailand
| | - Nadda Mongkolchat
- Division of Maternal-Fetal Medicine, Department of Obstetrics and Gynecology, Faculty of Medicine, Siriraj Hospital, Bangkok, Thailand
| | - Tuangsit Wataganara
- Division of Maternal-Fetal Medicine, Department of Obstetrics and Gynecology, Faculty of Medicine, Siriraj Hospital, Bangkok, Thailand
| |
Collapse
|
4
|
Pourtaherian A, Scholten HJ, Kusters L, Zinger S, Mihajlovic N, Kolen AF, Zuo F, Ng GC, Korsten HHM, de With PHN. Medical Instrument Detection in 3-Dimensional Ultrasound Data Volumes. IEEE TRANSACTIONS ON MEDICAL IMAGING 2017; 36:1664-1675. [PMID: 28410101 DOI: 10.1109/tmi.2017.2692302] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
Ultrasound-guided medical interventions are broadly applied in diagnostics and therapy, e.g., regional anesthesia or ablation. A guided intervention using 2-D ultrasound is challenging due to the poor instrument visibility, limited field of view, and the multi-fold coordination of the medical instrument and ultrasound plane. Recent 3-D ultrasound transducers can improve the quality of the image-guided intervention if an automated detection of the needle is used. In this paper, we present a novel method for detecting medical instruments in 3-D ultrasound data that is solely based on image processing techniques and validated on various ex vivo and in vivo data sets. In the proposed procedure, the physician is placing the 3-D transducer at the desired position, and the image processing will automatically detect the best instrument view, so that the physician can entirely focus on the intervention. Our method is based on the classification of instrument voxels using volumetric structure directions and robust approximation of the primary tool axis. A novel normalization method is proposed for the shape and intensity consistency of instruments to improve the detection. Moreover, a novel 3-D Gabor wavelet transformation is introduced and optimally designed for revealing the instrument voxels in the volume, while remaining generic to several medical instruments and transducer types. Experiments on diverse data sets, including in vivo data from patients, show that for a given transducer and an instrument type, high detection accuracies are achieved with position errors smaller than the instrument diameter in the 0.5-1.5-mm range on average.
Collapse
|
5
|
González SJ, Mooney B, Lin HY, Zhao X, Kiluk JV, Khakpour N, Laronga C, Lee MC. 2-D and 3-D Ultrasound for Tumor Volume Analysis: A Prospective Study. ULTRASOUND IN MEDICINE & BIOLOGY 2017; 43:775-781. [PMID: 28187928 DOI: 10.1016/j.ultrasmedbio.2016.12.009] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/03/2016] [Revised: 12/12/2016] [Accepted: 12/14/2016] [Indexed: 06/06/2023]
Abstract
Ultrasound (US) allows real-time tumor assessment. We evaluated the volumetric limits of 2-D and 3-D US, compared with magnetic resonance imaging (MRI), with a prospective institutional review board-approved clinical evaluation of US-to-MRI volumetric correlation. US images of pre- and post-neoadjuvant breast cancers were obtained. Volume discrepancy was evaluated with the non-parametric Wilcoxon signed-rank test. Expected inter-observer variability <14% was evaluated as relative paired difference (RPD); clinical relevance was gauged with the volumetric standard error of the mean (SEM). For 42 patients, 133 of 170 US examinations were evaluable. For tumors ≤20 cm3, both highly correlated to MRI with RPD within inter-observer variability and Pearson's correlation up to 0.86 (0.80 before and 0.86 after neoadjuvant chemotherapy, respectively). Lesions 20-40 cm3 had US-to-MRI discrepancy within inter-observer variability for 2-D (RPD: 13%), but not 3-D (RPD: 27%) US (SEM: 1.47 cm3 for 2-D, SEM: 2.28 cm3 for 3-D), suggesting clinical utility. Tumors >40 cm3 correlated poorly. Tumor volumes ≤20 cm3 exhibited a good correlation to MRI. Studies of clinical applications are warranted.
Collapse
Affiliation(s)
- Segundo J González
- Comprehensive Breast Program, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida, USA
| | - Blaise Mooney
- Diagnostic Breast Imaging, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida, USA
| | - Hui-Yi Lin
- Biostatistics and Bioinformatics, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida, USA
| | - Xiuhua Zhao
- Biostatistics and Bioinformatics, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida, USA
| | - John V Kiluk
- Comprehensive Breast Program, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida, USA
| | - Nazanin Khakpour
- Comprehensive Breast Program, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida, USA
| | - Christine Laronga
- Comprehensive Breast Program, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida, USA
| | - M Catherine Lee
- Comprehensive Breast Program, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida, USA.
| |
Collapse
|
6
|
Mura M, Parrini S, Ciuti G, Ferrari V, Freschi C, Ferrari M, Dario P, Menciassi A. A computer-assisted robotic platform for vascular procedures exploiting 3D US-based tracking. Comput Assist Surg (Abingdon) 2016; 21:63-79. [DOI: 10.1080/24699322.2016.1185467] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022] Open
Affiliation(s)
- Marco Mura
- The BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy
| | | | - Gastone Ciuti
- The BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy
| | | | | | | | - Paolo Dario
- The BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy
| | | |
Collapse
|
7
|
Tully S, Choset H. A Filtering Approach for Image-Guided Surgery With a Highly Articulated Surgical Snake Robot. IEEE Trans Biomed Eng 2015; 63:392-402. [PMID: 26241966 DOI: 10.1109/tbme.2015.2461531] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
GOAL The objective of this paper is to introduce a probabilistic filtering approach to estimate the pose and internal shape of a highly flexible surgical snake robot during minimally invasive surgery. METHODS Our approach renders a depiction of the robot that is registered to preoperatively reconstructed organ models to produce a 3-D visualization that can be used for surgical feedback. Our filtering method estimates the robot shape using an extended Kalman filter that fuses magnetic tracker data with kinematic models that define the motion of the robot. Using Lie derivative analysis, we show that this estimation problem is observable, and thus, the shape and configuration of the robot can be successfully recovered with a sufficient number of magnetic tracker measurements. RESULTS We validate this study with benchtop and in-vivo image-guidance experiments in which the surgical robot was driven along the epicardial surface of a porcine heart. CONCLUSION This paper introduces a filtering approach for shape estimation that can be used for image guidance during minimally invasive surgery. SIGNIFICANCE The methods being introduced in this paper enable informative image guidance for highly articulated surgical robots, which benefits the advancement of robotic surgery.
Collapse
|
8
|
Häcker A, Dinter D, Michel MS, Alken P. High-intensity focused ultrasound as a treatment option in renal cell carcinoma. Expert Rev Anticancer Ther 2014; 5:1053-9. [PMID: 16336096 DOI: 10.1586/14737140.5.6.1053] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Due to the widespread use of modern imaging modalities, small renal masses are discovered incidentally at increasing rates. Advances in minimally invasive technologies have changed the treatment options for renal cell carcinoma. High-intensity focused ultrasound aims to completely ablate renal tumors in a noninvasive manner. Experimental studies have demonstrated principle feasibility and safety of the technology. However, clinical studies on renal cell carcinoma are very limited and no substantial oncologic results are available to date. Major technical improvements are mandatory to enable high-intensity focused ultrasound as an effective treatment option for patients with small renal masses.
Collapse
Affiliation(s)
- Axel Häcker
- Department of Urology, University Hospital Mannheim, Theodor-Kutzer-Ufer 1-3, 68135 Mannheim, Germany.
| | | | | | | |
Collapse
|
9
|
Uherčík M, Kybic J, Zhao Y, Cachard C, Liebgott H. Line filtering for surgical tool localization in 3D ultrasound images. Comput Biol Med 2013; 43:2036-45. [DOI: 10.1016/j.compbiomed.2013.09.020] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2013] [Revised: 09/22/2013] [Accepted: 09/25/2013] [Indexed: 10/26/2022]
|
10
|
|
11
|
GPU-accelerated 3D mipmap for real-time visualization of ultrasound volume data. Comput Biol Med 2013; 43:1382-9. [DOI: 10.1016/j.compbiomed.2013.07.014] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2013] [Revised: 07/12/2013] [Accepted: 07/13/2013] [Indexed: 11/19/2022]
|
12
|
Ren H, Dupont PE. Tubular structure enhancement for surgical instrument detection in 3D ultrasound. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2012; 2011:7203-6. [PMID: 22256000 DOI: 10.1109/iembs.2011.6091820] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Three-dimensional ultrasound has been an effective imaging modality for diagnostics and is now an emerging modality for image-guided minimally-invasive interventions since it enables visualization of both instruments and tissue. Challenges to ultrasound-guided interventions arise, however, due to the low signal-to-noise ratio and the imaging artifacts created by the interventional instruments. Metallic instruments, in particular, are strong scatters and so produce a variety of artifacts. For many interventions, the manual or robotic instrument is comprised of a long curved tubular structure with specialized tooling at its tip. Toward the goal of developing a surgical navigation system, this paper proposes an image processing algorithm for enhancing the tubular structure of imaged instruments while also reducing imaging artifacts. Experiments are presented to evaluate the effectiveness of the approach in the context of robotic instruments whose shape comprises a smooth curve along their length.
Collapse
Affiliation(s)
- Hongliang Ren
- Department of Cardiac Surgery, Children’s Hospital Boston, Harvard MedicalSchool, 300 Longwood Ave, Boston, MA 02115, USA.
| | | |
Collapse
|
13
|
Vasilyev NV, Dupont PE, del Nido PJ. Robotics and imaging in congenital heart surgery. Future Cardiol 2012; 8:285-96. [PMID: 22413986 DOI: 10.2217/fca.12.20] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022] Open
Abstract
The initial success seen in adult cardiac surgery with the application of available robotic systems has not been realized as broadly in pediatric cardiac surgery. The main obstacles include extended set-up time and complexity of the procedures, as well as the large size of the instruments with respect to the size of the child. Moreover, while the main advantage of robotic systems is the ability to minimize incision size, for intracardiac repairs, cardiopulmonary bypass is still required. Catheter-based interventions, on the other hand, have expanded rapidly in both application as well as the complexity of procedures and lesions being treated. However, despite the development of sophisticated devices, robotic systems to aid catheter procedures have not been commonly applied in children. In this article, we describe new catheter-like robotic delivery platforms, which facilitate safe navigation and enable complex repairs, such as tissue approximation and fixation, and tissue removal, inside the beating heart. Additional features including the tracking of rapidly moving tissue targets and novel imaging approaches are described, along with a discussion of future prospects for steerable robotic systems.
Collapse
|
14
|
Brattain LJ, Vasilyev NV, Howe RD. Enabling 3D Ultrasound Procedure Guidance through Enhanced Visualization. INFORMATION PROCESSING IN COMPUTER-ASSISTED INTERVENTIONS : THIRD INTERNATIONAL CONFERENCE, IPCAI 2012, PISA, ITALY, JUNE 27, 2012 PROCEEDINGS. IPCAI (CONFERENCE) (3RD : 2012 : PISA, ITALY) 2012; 7330:115-124. [PMID: 29862385 PMCID: PMC5983382 DOI: 10.1007/978-3-642-30618-1_12] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Real-time 3D ultrasound (3DUS) imaging offers improved spatial orientation information relative to 2D ultrasound. However, in order to improve its efficacy in guiding minimally invasive intra-cardiac procedures where real-time visual feedback of an instrument tip location is crucial, 3DUS volume visualization alone is inadequate. This paper presents a set of enhanced visualization functionalities that are able to track the tip of an instrument in slice views at real-time. User study with in vitro porcine heart indicates a speedup of over 30% in task completion time.
Collapse
Affiliation(s)
- Laura J Brattain
- Harvard School of Engineering and Applied Sciences, Cambridge, MA USA 02138
- MIT Lincoln Laboratory, 244 Wood St., Lexington, MA USA 02420
| | - Nikolay V Vasilyev
- Department of Cardiac Surgery, Children's Hospital Boston, Boston, MA USA 02115
| | - Robert D Howe
- Harvard School of Engineering and Applied Sciences, Cambridge, MA USA 02138
| |
Collapse
|
15
|
Ren H, Dupont PE. Tubular Enhanced Geodesic Active Contours for Continuum Robot Detection using 3D Ultrasound. IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION : ICRA : [PROCEEDINGS]. IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION 2012. [PMID: 24231880 DOI: 10.1109/icra.2012.6225033] [Citation(s) in RCA: 26] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Three dimensional ultrasound is a promising imaging modality for minimally invasive robotic surgery. As the robots are typically metallic, they interact strongly with the sound waves in ways that are not modeled by the ultrasound system's signal processing algorithms. Consequently, they produce substantial imaging artifacts that can make image guidance difficult, even for experienced surgeons. This paper introduces a new approach for detecting curved continuum robots in 3D ultrasound images. The proposed approach combines geodesic active contours with a speed function that is based on enhancing the "tubularity" of the continuum robot. In particular, it takes advantage of the known robot diameter along its length. It also takes advantage of the fact that the robot surface facing the ultrasound probe provides the most accurate image. This method, termed Tubular Enhanced Geodesic Active Contours (TEGAC), is demonstrated through ex vivo intracardiac experiments to offer superior performance compared to conventional active contours.
Collapse
Affiliation(s)
- Hongliang Ren
- Department of Cardiovascular Surgery, Children's Hospital Boston, Harvard Medical School, 300 Longwood Ave., Boston, MA, 02115 USA
| | | |
Collapse
|
16
|
Stoll J, Ren H, Dupont PE. Passive markers for tracking surgical instruments in real-time 3-D ultrasound imaging. IEEE TRANSACTIONS ON MEDICAL IMAGING 2012; 31:563-575. [PMID: 22042148 PMCID: PMC3638948 DOI: 10.1109/tmi.2011.2173586] [Citation(s) in RCA: 20] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/31/2023]
Abstract
A family of passive echogenic markers is presented by which the position and orientation of a surgical instrument can be determined in a 3-D ultrasound volume, using simple image processing. Markers are attached near the distal end of the instrument so that they appear in the ultrasound volume along with the instrument tip. They are detected and measured within the ultrasound image, thus requiring no external tracking device. This approach facilitates imaging instruments and tissue simultaneously in ultrasound-guided interventions. Marker-based estimates of instrument pose can be used in augmented reality displays or for image-based servoing. Design principles for marker shapes are presented that ensure imaging system and measurement uniqueness constraints are met. An error analysis is included that can be used to guide marker design and which also establishes a lower bound on measurement uncertainty. Finally, examples of marker measurement and tracking algorithms are presented along with experimental validation of the concepts.
Collapse
Affiliation(s)
- Jeffrey Stoll
- Ultrasound Division, Siemens Healthcare, Mountain View, CA 94043, USA.
| | | | | |
Collapse
|
17
|
Koolwal AB, Barbagli F, Carlson CR, Liang DH. A fast slam approach to freehand 3-d ultrasound reconstruction for catheter ablation guidance in the left atrium. ULTRASOUND IN MEDICINE & BIOLOGY 2011; 37:2037-2054. [PMID: 22014856 DOI: 10.1016/j.ultrasmedbio.2011.08.007] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/28/2010] [Revised: 07/31/2011] [Accepted: 08/09/2011] [Indexed: 05/31/2023]
Abstract
We present a method for real-time, freehand 3D ultrasound (3D-US) reconstruction of moving anatomy, with specific application towards guiding the catheter ablation procedure in the left atrium. Using an intracardiac echo (ICE) catheter with a pose (position/orientation) sensor mounted to its tip, we continually mosaic 2D-ICE images of a left atrium phantom model to form a 3D-US volume. Our mosaicing strategy employs a probabilistic framework based on simultaneous localization and mapping (SLAM), a technique commonly used in mobile robotics for creating maps of unexplored environments. The measured ICE catheter tip pose provides an initial estimate for compounding 2D-ICE image data into the 3D-US volume. However, we simultaneously consider the overlap-consistency shared between 2D-ICE images and the 3D-US volume, computing a "corrected" tip pose if need be to ensure spatially-consistent reconstruction. This allows us to compensate for anatomic movement and sensor drift that would otherwise cause motion artifacts in the 3D-US volume. Our approach incorporates 2D-ICE data immediately after acquisition, allowing us to continuously update the registration parameters linking sensor coordinates to 3D-US coordinates. This, in turn, enables real-time localization and display of sensorized therapeutic catheters within the 3D-US volume for facilitating procedural guidance.
Collapse
Affiliation(s)
- Aditya B Koolwal
- Department of Mechanical Engineering, Stanford University, Stanford, CA, USA.
| | | | | | | |
Collapse
|
18
|
Ren H, Vasilyev NV, Dupont PE. Detection of Curved Robots using 3D Ultrasound. PROCEEDINGS OF THE ... IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS. IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS 2011; 2011:2083-2089. [PMID: 22229110 PMCID: PMC3252206 DOI: 10.1109/iros.2011.6094915] [Citation(s) in RCA: 23] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Three-dimensional ultrasound can be an effective imaging modality for image-guided interventions since it enables visualization of both the instruments and the tissue. For robotic applications, its realtime frame rates create the potential for image-based instrument tracking and servoing. These capabilities can enable improved instrument visualization, compensation for tissue motion as well as surgical task automation. Continuum robots, whose shape comprises a smooth curve along their length, are well suited for minimally invasive procedures. Existing techniques for ultrasound tracking, however, are limited to straight, laparoscopic-type instruments and thus are not applicable to continuum robot tracking. Toward the goal of developing tracking algorithms for continuum robots, this paper presents a method for detecting a robot comprised of a single constant curvature in a 3D ultrasound volume. Computational efficiency is achieved by decomposing the six-dimensional circle estimation problem into two sequential three-dimensional estimation problems. Simulation and experiment are used to evaluate the proposed method.
Collapse
Affiliation(s)
- Hongliang Ren
- Hongliang Ren, Ph.D., , Nikolay V. Vasilyev, M.D., and Pierre Dupont, Ph.D., are with the Department of Cardiac Surgery, Children's Hospital Boston, Harvard Medical School, 300 Longwood Ave., Boston, MA, 02115 USA
| | - Nikolay V. Vasilyev
- Hongliang Ren, Ph.D., , Nikolay V. Vasilyev, M.D., and Pierre Dupont, Ph.D., are with the Department of Cardiac Surgery, Children's Hospital Boston, Harvard Medical School, 300 Longwood Ave., Boston, MA, 02115 USA
| | - Pierre E. Dupont
- Hongliang Ren, Ph.D., , Nikolay V. Vasilyev, M.D., and Pierre Dupont, Ph.D., are with the Department of Cardiac Surgery, Children's Hospital Boston, Harvard Medical School, 300 Longwood Ave., Boston, MA, 02115 USA
| |
Collapse
|
19
|
Brattain LJ, Howe RD. Real-time 4D ultrasound mosaicing and visualization. MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION : MICCAI ... INTERNATIONAL CONFERENCE ON MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION 2011; 14:105-12. [PMID: 22003606 PMCID: PMC3248050 DOI: 10.1007/978-3-642-23623-5_14] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
Intra-cardiac 3D ultrasound imaging has enabled new minimally invasive procedures. Its narrow field of view, however, limits its efficacy in guiding beating heart procedures where geometrically complex and spatially extended moving anatomic structures are often involved. In this paper, we present a system that performs electrocardiograph gated 4D mosaicing and visualization of 3DUS volumes. Real-time operation is enabled by GPU implementation. The method is validated on phantom and porcine heart data.
Collapse
Affiliation(s)
- Laura J Brattain
- Harvard School of Engineering and Applied Sciences, Cambrige, MA 02138
- MIT Lincoln Laboratory, 244 Wood St., Lexington, MA 02420
| | - Robert D Howe
- Harvard School of Engineering and Applied Sciences, Cambrige, MA 02138
| |
Collapse
|
20
|
Advantages and limitations of intraoperative 3D ultrasound in neurosurgery. Technical note. ACTA NEUROCHIRURGICA. SUPPLEMENT 2011; 109:191-6. [PMID: 20960342 DOI: 10.1007/978-3-211-99651-5_30] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/20/2023]
Abstract
Three-dimensional ultrasound (US) technology is supposed to help combat some of the orientation difficulties inherent to two-dimensional US. Contemporary navigation solutions combine reconstructed 3D US images with common navigation images and support orientation. New real-time 3D US (without neuronavigation) is more time effective, but whether it further assists in orientation remains to be determined. An integrated US system (IGSonic, VectorVision, BrainLAB, Munich Germany) and a non-integrated system with real-time 3D US (iU22, Philips, Bothell, USA) were recently compared in neurosurgical procedures in our group. The reconstructed navigation view was time-consuming, but images were displayed in familiar planes (e.g., axial, sagittal, coronal). Further potential applications of US angiography and pure US navigation are possible. Real-time 3D images were displayed without the need for an additional acquisition and reconstruction process, but spatial orientation remained challenging in this preliminary testing phase. Reconstructed 3D US navigation appears to be superior with respect to spatial orientation, and the technique can be combined with other imaging data. However, the potential of real-time 3D US imaging is promising.
Collapse
|
21
|
Yuen SG, Vasilyev NV, del Nido PJ, Howe RD. Robotic tissue tracking for beating heart mitral valve surgery. Med Image Anal 2010; 17:1236-42. [PMID: 23973122 DOI: 10.1016/j.media.2010.06.007] [Citation(s) in RCA: 21] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2009] [Revised: 06/16/2010] [Accepted: 06/21/2010] [Indexed: 11/29/2022]
Abstract
The rapid motion of the heart presents a significant challenge to the surgeon during intracardiac beating heart procedures. We present a 3D ultrasound-guided motion compensation system that assists the surgeon by synchronizing instrument motion with the heart. The system utilizes the fact that certain intracardiac structures, like the mitral valve annulus, have trajectories that are largely constrained to translation along one axis. This allows the development of a real-time 3D ultrasound tissue tracker that we integrate with a 1 degree-of-freedom (DOF) actuated surgical instrument and predictive filter to devise a motion tracking system adapted to mitral valve annuloplasty. In vivo experiments demonstrate that the system provides highly accurate tracking (1.0 mm error) with 70% less error than manual tracking attempts.
Collapse
Affiliation(s)
- Shelten G Yuen
- Harvard School of Engineering and Applied Sciences, 29 Oxford Street, Cambridge, MA 02138, USA
| | | | | | | |
Collapse
|
22
|
Uhercík M, Kybic J, Liebgott H, Cachard C. Model fitting using RANSAC for surgical tool localization in 3-D ultrasound images. IEEE Trans Biomed Eng 2010; 57:1907-16. [PMID: 20483680 DOI: 10.1109/tbme.2010.2046416] [Citation(s) in RCA: 51] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
Ultrasound guidance is used for many surgical interventions such as biopsy and electrode insertion. We present a method to localize a thin surgical tool such as a biopsy needle or a microelectrode in a 3-D ultrasound image. The proposed method starts with thresholding and model fitting using random sample consensus for robust localization of the axis. Subsequent local optimization refines its position. Two different tool image models are presented: one is simple and fast and the second uses learned a priori information about the tool's voxel intensities and the background. Finally, the tip of the tool is localized by finding an intensity drop along the axis. The simulation study shows that our algorithm can localize the tool at nearly real-time speed, even using a MATLAB implementation, with accuracy better than 1 mm. In an experimental comparison with several alternative localization methods, our method appears to be the fastest and the most robust one. We also show the results on real 3-D ultrasound data from a PVA cryogel phantom, turkey breast, and breast biopsy.
Collapse
Affiliation(s)
- Marián Uhercík
- Department of Cybernetics, Faculty of Electrical Engineering, Center for Machine Perception, Czech Technical University in Prague, Prague 16627, Czech Republic.
| | | | | | | |
Collapse
|
23
|
Live augmented reality: a new visualization method for laparoscopic surgery using continuous volumetric computed tomography. Surg Endosc 2010; 24:1976-85. [DOI: 10.1007/s00464-010-0890-8] [Citation(s) in RCA: 46] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2009] [Accepted: 12/27/2009] [Indexed: 10/19/2022]
|
24
|
Talib H, Rajamani K, Kowal J, Nolte LP, Styner M, Ballester MAG. A comparison study assessing the feasibility of ultrasound-initialized deformable bone models. ACTA ACUST UNITED AC 2010; 10:293-9. [PMID: 16410231 DOI: 10.3109/10929080500379390] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
Abstract
This article presents a feasibility and evaluation study for using 2D ultrasound in conjunction with our statistical deformable bone model within the scope of computer-assisted surgery. The final aim is to provide the surgeon with enhanced 3D visualization for surgical navigation in orthopedic surgery without the need for preoperative CT or MRI scans. We unified our earlier work to combine several automatic methods for statistical bone shape prediction and ultrasound segmentation and calibration to provide the intended rapid and accurate visualization. We compared the use of a tracked digitizing pointer and ultrasound for acquiring landmarks and bone surface points for the estimation of two cast proximal femurs.
Collapse
Affiliation(s)
- Haydar Talib
- Institute for Surgical Technology and Biomechanics, MEM Research Center, University of Bern, Bern, Switzerland.
| | | | | | | | | | | |
Collapse
|
25
|
Congenital absence of posteromedial papillary muscle and anterior mitral leaflet chordae: the use of three-dimensional echocardiography and approach in complex pediatric mitral valve disease. J Thorac Cardiovasc Surg 2009; 139:e75-7. [PMID: 19660364 DOI: 10.1016/j.jtcvs.2009.03.006] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/16/2008] [Revised: 02/16/2009] [Accepted: 03/07/2009] [Indexed: 11/24/2022]
|
26
|
Brij Koolwal A, Barbagli F, Carlson C, Liang D. An Ultrasound-based Localization Algorithm for Catheter Ablation Guidance in the Left Atrium. Int J Rob Res 2009. [DOI: 10.1177/0278364909105332] [Citation(s) in RCA: 24] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
We present a method for catheter localization in the left atrium based on the unscented particle filter (UPF), a Monte Carlo method employed in stochastic state estimation. Using an intracardiac echo (ICE) ultrasound catheter, we acquire ultrasound images of the atrium from multiple configurations and iteratively determine the catheter’s pose with respect to anatomy. At each time step, the catheter’s change in pose is determined using either a six-degree-of-freedom electromagnetic pose sensor or a robotic guide catheter whose kinematics serve as a pseudo-pose measurement. Sensor and kinematic model uncertainties are explicitly considered when computing the localization estimate. Acquired ultrasound images are compared with simulated ultrasound images based on segmented computed tomography (CT) or magnetic resonance (MR) data of the left atrium. The results of these comparisons are used to refine the localization estimate. After considering less than 30 seconds’ worth of ICE data, our algorithm converges to an accurate pose estimate. Furthermore, our algorithm is robust to sensor drift and kinematic model errors, as well as gradual, unmodeled movements in the anatomy. Such problems typically complicate traditional image-based localization.
Collapse
Affiliation(s)
- Aditya Brij Koolwal
- Department of Mechanical Engineering, Stanford University, Stanford, CA 94305-9025, USA,
| | - Federico Barbagli
- Department of Computer Science, Stanford University, Stanford, CA 94305-9025, USA, Hansen Medical, Inc. 800 East Middlefield Road, Mountain View, CA 94043, USA
| | - Christopher Carlson
- Hansen Medical, Inc. 800 East Middlefield Road, Mountain View, CA 94043, USA
| | - David Liang
- Department of Cardiovascular Medicine, Stanford School of Medicine, Falk Cardiovascular Research Center, 300 Pasteur Drive, Stanford, CA 94305, USA
| |
Collapse
|
27
|
A probabilistic framework for freehand 3D ultrasound reconstruction applied to catheter ablation guidance in the left atrium. Int J Comput Assist Radiol Surg 2009; 4:425-37. [DOI: 10.1007/s11548-009-0354-6] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2009] [Accepted: 04/26/2009] [Indexed: 10/20/2022]
|
28
|
Yuen SG, Kettler DT, Novotny PM, Plowes RD, Howe RD. Robotic Motion Compensation for Beating Heart Intracardiac Surgery. Int J Rob Res 2009; 28:1355-1372. [PMID: 20436927 DOI: 10.1177/0278364909104065] [Citation(s) in RCA: 108] [Impact Index Per Article: 7.2] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
3D ultrasound imaging has enabled minimally invasive, beating heart intracardiac procedures. However, rapid heart motion poses a serious challenge to the surgeon that is compounded by significant time delays and noise in 3D ultrasound. This paper investigates the concept of using a one-degree-of-freedom motion compensation system to synchronize with tissue motions that may be approximated by 1D motion models. We characterize the motion of the mitral valve annulus and show that it is well approximated by a 1D model. The subsequent development of a motion compensation instrument (MCI) is described, as well as an extended Kalman filter (EKF) that compensates for system delays. The benefits and robustness of motion compensation are tested in user trials under a series of non-ideal tracking conditions. Results indicate that the MCI provides an approximately 50% increase in dexterity and 50% decrease in force when compared with a solid tool, but is sensitive to time delays. We demonstrate that the use of the EKF for delay compensation restores performance, even in situations of high heart rate variability. The resulting system is tested in an in vitro 3D ultrasound-guided servoing task, yielding accurate tracking (1.15 mm root mean square) in the presence of noisy, time-delayed 3D ultrasound measurements.
Collapse
Affiliation(s)
- Shelten G Yuen
- School of Engineering and Applied Sciences, Harvard University, Cambridge, MA 02138, USA
| | | | | | | | | |
Collapse
|
29
|
Berrang TS, Truong PT, Popescu C, Drever L, Kader HA, Hilts ML, Mitchell T, Soh SY, Sands L, Silver S, Olivotto IA. 3D Ultrasound Can Contribute to Planning CT to Define the Target for Partial Breast Radiotherapy. Int J Radiat Oncol Biol Phys 2009; 73:375-83. [PMID: 18692322 DOI: 10.1016/j.ijrobp.2008.04.041] [Citation(s) in RCA: 43] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2008] [Revised: 04/15/2008] [Accepted: 04/17/2008] [Indexed: 11/16/2022]
Affiliation(s)
- Tanya S Berrang
- Radiation Therapy Program, British Columbia Cancer Agency, Vancouver Island Centre, Victoria, BC, Canada
| | | | | | | | | | | | | | | | | | | | | |
Collapse
|
30
|
|
31
|
Salgo IS. 3D echocardiographic visualization for intracardiac beating heart surgery and intervention. Semin Thorac Cardiovasc Surg 2008; 19:325-9. [PMID: 18395632 DOI: 10.1053/j.semtcvs.2007.12.007] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/04/2007] [Indexed: 11/11/2022]
Abstract
Three-dimensional echocardiography has emerged as an essential tool for visualizing cardiac anatomy and for making more accurate measurements of cardiac structure and function. Recently, improvements in 3D beam-forming and transducer technologies have allowed higher resolution imaging from a transesophageal echocardiographic probe. This is creating new avenues for real-time visualization of intracardiac procedures without the need for cardiopulmonary bypass or opening the beating heart. Evolutions in visualization will allow a wider array of reparative procedures to be performed minimally invasively within a beating heart.
Collapse
Affiliation(s)
- Ivan S Salgo
- Cardiovascular Investigations, Ultrasound Research and Development, Philips Healthcare, Andover, MA 01810, USA.
| |
Collapse
|
32
|
Barva M, Uhercik M, Mari JM, Kybic J, Duhamel JR, Liebgott H, Hlavac V, Cachard C. Parallel integral projection transform for straight electrode localization in 3-D ultrasound images. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2008; 55:1559-1569. [PMID: 18986947 DOI: 10.1109/tuffc.2008.833] [Citation(s) in RCA: 25] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/27/2023]
Abstract
In surgical practice, small metallic instruments are frequently used to perform various tasks inside the human body. We address the problem of their accurate localization in the tissue. Recent experiments using medical ultrasound have shown that this modality is suitable for real-time visualization of anatomical structures as well as the position of surgical instruments. We propose an image-processing algorithm that permits automatic estimation of the position of a line-segment-shaped object. This method was applied to the localization of a thin metallic electrode in biological tissue. We show that the electrode axis can be found through maximizing the parallel integral projection transform that is a form of the Radon transform. To accelerate this step, hierarchical mesh-grid algorithm is implemented. Once the axis position is known, localization of the electrode tip is performed. The method was tested on simulated images, on ultrasound images of a tissue mimicking phantom containing a metallic electrode, and on real ultrasound images from breast biopsy. The results indicate that the algorithm is robust with respect to variations in electrode position and speckle noise. Localization accuracy is of the order of hundreds of micrometers and is comparable to the ultrasound system axial resolution.
Collapse
Affiliation(s)
- M Barva
- Center for Machine Perception, Czech Tech. Univ. in Prague, Prague, Czech Republic
| | | | | | | | | | | | | | | |
Collapse
|
33
|
Vasilyev NV, Novotny PM, Martinez JF, Loyola H, Salgo IS, Howe RD, del Nido PJ. Stereoscopic vision display technology in real-time three-dimensional echocardiography-guided intracardiac beating-heart surgery. J Thorac Cardiovasc Surg 2008; 135:1334-41. [DOI: 10.1016/j.jtcvs.2007.12.045] [Citation(s) in RCA: 21] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/26/2007] [Revised: 11/16/2007] [Accepted: 12/06/2007] [Indexed: 11/25/2022]
|
34
|
Novotny PM, Kettler DT, Jordan P, Dupont PE, del Nido PJ, Howe RD. Stereo display of 3D ultrasound images for surgical robot guidance. CONFERENCE PROCEEDINGS : ... ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL CONFERENCE 2008; 2006:1509-12. [PMID: 17946048 DOI: 10.1109/iembs.2006.259486] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
The recent advent of real-time 3-D ultrasound (3DUS) imaging enables a variety of surgical procedures to be performed within the beating heart. Implementation of these procedures is hampered by the difficulty of manipulating tissue guided by the distorted, low resolution 3DUS images and the dexterity constraints imposed by the confined intracardiac space. This paper investigates the use of surgical robotics in conjunction with 3DUS to overcome these limitations. In addition, it describes the development of a graphics processor based volume Tenderer for real-time stereo visualization of the ultrasound data. Stereo displayed 3DUS was compared to ID-displayed 3DUS and endoscopic guidance with a user study. Five subjects performed in vitro surgical tasks using a surgical robot. Results indicate that subjects were able to complete surgical tasks 35 % faster with stereo-displayed 3DUS images compared to conventional two dimensional display of 3DUS.
Collapse
Affiliation(s)
- Paul M Novotny
- Division of Engineering and Applied Sciences, Harvard University, Cambridge, MA, USA
| | | | | | | | | | | |
Collapse
|
35
|
Linguraru MG, Vasilyev NV, Marx GR, Tworetzky W, Del Nido PJ, Howe RD. Fast block flow tracking of atrial septal defects in 4D echocardiography. Med Image Anal 2008; 12:397-412. [PMID: 18282783 DOI: 10.1016/j.media.2007.12.005] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2006] [Revised: 12/17/2007] [Accepted: 12/21/2007] [Indexed: 11/26/2022]
Abstract
We are working to develop beating-heart atrial septal defect (ASD) closure techniques using real-time 3D ultrasound guidance. The major image processing challenges are the low-image quality and the processing of information at high-frame rate. This paper presents comparative results for ASD tracking in time sequences of 3D volumes of cardiac ultrasound. We introduce a block flow technique, which combines the velocity computation from optical flow for an entire block with template matching. Enforcing adapted similarity constraints to both the previous and first frames ensures optimal and unique solutions. We compare the performance of the proposed algorithm with that of block matching and region-based optical flow on eight in vivo 4D datasets acquired from porcine beating-heart procedures. Results show that our technique is more stable and has higher sensitivity than both optical flow and block matching in tracking ASDs. Computing velocity at the block level, our technique tracks ASD motion at 2 frames/s, much faster than optical flow and comparable in computation cost to block matching, and shows promise for real-time (30 frames/s). We report consistent results on clinical intra-operative images and retrieve the cardiac cycle (in ungated images) from error analysis. Quantitative results are evaluated on synthetic data with maximum tracking errors of 1 voxel.
Collapse
|
36
|
Yuen SG, Kesner SB, Vasilyev NV, Del Nido PJ, Howe RD. 3D ultrasound-guided motion compensation system for beating heart mitral valve repair. MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION : MICCAI ... INTERNATIONAL CONFERENCE ON MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION 2008; 11:711-719. [PMID: 18979809 PMCID: PMC2909194 DOI: 10.1007/978-3-540-85988-8_85] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/27/2023]
Abstract
Beating heart intracardiac procedures promise significant benefits for patients, however, the fast motion of the heart poses serious challenges to surgeons. We present a new 3D ultrasound-guided motion (3DUS) compensation system that synchronizes instrument motion with the heart. The system utilizes the fact that the motion of some intracardiac structures, including the mitral valve annulus, is largely constrained to translation along one axis. This allows the development of a real-time 3DUS tissue tracker which we integrate with a 1 degree-of-freedom actuated surgical instrument, real-time 3DUS instrument tracker, and predictive filter to devise a system with synchronization accuracy of 1.8 mm RMSE. User studies involving the deployment of surgical anchors in a simulated mitral annuloplasty procedure demonstrate that the system increases success rates by over 100%. Furthermore, it enables more careful anchor deployment by reducing forces to the tissue by 50% while allowing instruments to remain in contact with the tissue for longer periods.
Collapse
Affiliation(s)
- Shelten G Yuen
- Harvard School of Engineering and Applied Sciences, Cambridge, MA, USA
| | | | | | | | | |
Collapse
|
37
|
Linguraru MG, Kabla A, Marx GR, del Nido PJ, Howe RD. Real-time tracking and shape analysis of atrial septal defects in 3D echocardiography. Acad Radiol 2007; 14:1298-309. [PMID: 17964455 DOI: 10.1016/j.acra.2007.07.011] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2006] [Revised: 06/22/2007] [Accepted: 07/13/2007] [Indexed: 10/22/2022]
Abstract
RATIONALE AND OBJECTIVES Real-time cardiac ultrasound (US) allows monitoring the heart motion during intracardiac beating heart procedures. Our application assists pediatric atrial septal defect (ASD) closure techniques using real-time 3D US guidance and rigid instruments. ASD tracking is also an important tool for facilitating systematic clinical studies of the dynamic behavior of the intra-atrial communication. One major image processing challenge is associated with the required processing of information at high frame rate, especially given the low image quality. MATERIALS AND METHODS We present an optimization scheme for a block flow technique, which combines the probability-based velocity computation for an entire block (a 3D volume centered on the ASD) with cyclic template matching. The adapted similarity imposes constraints both locally (from frame to frame) to conserve energy, and globally (from a reference template) to minimize cumulative errors. The algorithm is optimized for fast and reliable results. For tests, we use three intra-operational 4D ultrasound sequences of clinical infant beating hearts with ASD. RESULTS Computing velocity at the block level with an optimized scheme, our technique tracks ASD motion at a frequency of 60 frames/s on clinical 4D datasets. Results are stable and accurate for changes in resolution and block size. In particular, we show robust real-time tracking and preliminary segmentation results of the ASD shape, size and orientation as a function of time. CONCLUSIONS We present an optimized block flow technique for real-time tracking of ASD to assist in minimally invasive beating heart surgery. Our method proposes the standard use of references for processing repetitive data. This paper represents, to our knowledge, the first study on the dynamic morphology of ASD that takes into account the angular effect introduced by the slanted position of the intra-atrial communication with respect to the US probe.
Collapse
|
38
|
Huang J, Triedman JK, Vasilyev NV, Suematsu Y, Cleveland RO, Dupont PE. Imaging artifacts of medical instruments in ultrasound-guided interventions. JOURNAL OF ULTRASOUND IN MEDICINE : OFFICIAL JOURNAL OF THE AMERICAN INSTITUTE OF ULTRASOUND IN MEDICINE 2007; 26:1303-22. [PMID: 17901134 DOI: 10.7863/jum.2007.26.10.1303] [Citation(s) in RCA: 48] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/17/2023]
Abstract
OBJECTIVE Real-time 3-dimensional (3D) ultrasound imaging has the potential to become a dominant imaging technique for minimally invasive surgery. One barrier to its widespread use is that surgical instruments generate imaging artifacts, which can obfuscate their location, orientation, and geometry and obscure nearby tissue. The purpose of this study was to identify and describe the types of artifacts which could be produced by metallic instruments during interventions guided by 3D ultrasound imaging. METHODS Three imaging studies were performed. First, imaging artifacts from stainless steel rods were identified in vitro and acoustically characterized. Second, 3 typical minimally invasive instruments were imaged (in vitro and in vivo), and their artifacts were analyzed. The third study compared the intensity of imaging artifacts (in vitro and in vivo) from stainless steel rods with rods composed of 3 different materials and stainless steel rods with roughened and coated surfaces. RESULTS For the stainless steel rods, all observed artifacts are described and illustrated, and their physical origins are explained. Artifacts from the 3 minimally invasive instruments are characterized with the use of the artifacts observed with the rods. Finally, it is shown that artifacts can be greatly reduced through the use of alternate materials or by surface modification. CONCLUSIONS Instrument artifacts in 3D ultrasound images can be more confusing than those from the same instruments imaged in 2 dimensions. Real-time 3D ultrasound imaging can, however, be used effectively for in vivo imaging of minimally invasive instruments by using artifact mitigation techniques, including careful selection of probe and incision locations, as well as by instrument modification.
Collapse
Affiliation(s)
- Jinlan Huang
- Department of Cardiology, Division of Basic Cardiovascular Research, Children's Hospital Boston, Boston, Massachusetts, USA
| | | | | | | | | | | |
Collapse
|
39
|
Linguraru MG, Vasilyev NV, Del Nido PJ, Howe RD. Statistical segmentation of surgical instruments in 3-D ultrasound images. ULTRASOUND IN MEDICINE & BIOLOGY 2007; 33:1428-37. [PMID: 17521802 PMCID: PMC2597268 DOI: 10.1016/j.ultrasmedbio.2007.03.003] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/04/2006] [Revised: 01/31/2007] [Accepted: 03/04/2007] [Indexed: 05/15/2023]
Abstract
The recent development of real-time 3-D ultrasound (US) enables intracardiac beating-heart procedures, but the distorted appearance of surgical instruments is a major challenge to surgeons. In addition, tissue and instruments have similar gray levels in US images and the interface between instruments and tissue is poorly defined. We present an algorithm that automatically estimates instrument location in intracardiac procedures. Expert-segmented images are used to initialize the statistical distributions of blood, tissue and instruments. Voxels are labeled through an iterative expectation-maximization algorithm using information from the neighboring voxels through a smoothing kernel. Once the three classes of voxels are separated, additional neighboring information is combined with the known shape characteristics of instruments to correct for misclassifications. We analyze the major axis of segmented data through their principal components and refine the results by a watershed transform, which corrects the results at the contact between instrument and tissue. We present results on 3-D in-vitro data from a tank trial and 3-D in-vivo data from cardiac interventions on porcine beating hearts, using instruments of four types of materials. The comparison of algorithm results to expert-annotated images shows the correct segmentation and position of the instrument shaft.
Collapse
Affiliation(s)
- Marius George Linguraru
- Division of Engineering and Applied Sciences, Harvard Medical School, Harvard University, Cambridge, and Department of Cardiac Surgery, Children's Hospital, Boston, MA, USA.
| | | | | | | |
Collapse
|
40
|
Novotny PM, Stoll JA, Vasilyev NV, del Nido PJ, Dupont PE, Zickler TE, Howe RD. GPU based real-time instrument tracking with three-dimensional ultrasound. Med Image Anal 2007; 11:458-64. [PMID: 17681483 PMCID: PMC2693901 DOI: 10.1016/j.media.2007.06.009] [Citation(s) in RCA: 98] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2007] [Accepted: 06/04/2007] [Indexed: 11/20/2022]
Abstract
Real-time three-dimensional ultrasound enables new intracardiac surgical procedures, but the distorted appearance of instruments in ultrasound poses a challenge to surgeons. This paper presents a detection technique that identifies the position of the instrument within the ultrasound volume. The algorithm uses a form of the generalized Radon transform to search for long straight objects in the ultrasound image, a feature characteristic of instruments and not found in cardiac tissue. When combined with passive markers placed on the instrument shaft, the full position and orientation of the instrument is found in 3D space. This detection technique is amenable to rapid execution on the current generation of personal computer graphics processor units (GPU). Our GPU implementation detected a surgical instrument in 31 ms, sufficient for real-time tracking at the 25 volumes per second rate of the ultrasound machine. A water tank experiment found instrument orientation errors of 1.1 degrees and tip position errors of less than 1.8mm. Finally, an in vivo study demonstrated successful instrument tracking inside a beating porcine heart.
Collapse
Affiliation(s)
- Paul M Novotny
- School of Engineering and Applied Sciences, Harvard University, 29 Oxford Street, Cambridge, MA 02138, USA
| | | | | | | | | | | | | |
Collapse
|
41
|
Abstract
This article addresses the current state of the art of technology in three-dimensional echocardiography as it applies to transducer design, beam forming, display, and quantification. Because three-dimensional echocardiography encompasses many technical and clinical areas, this article reviews its strengths and limitations and concludes with an analysis of what to use when.
Collapse
Affiliation(s)
- Ivan S Salgo
- Cardiovascular Investigations, Ultrasound Research & Development, Philips Medical Systems, 3000 Minuteman Road, Andover, MA 01810, USA.
| |
Collapse
|
42
|
Hashizume M. MRI-guided laparoscopic and robotic surgery for malignancies. Int J Clin Oncol 2007; 12:94-8. [PMID: 17443276 DOI: 10.1007/s10147-007-0664-z] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2007] [Indexed: 11/25/2022]
Abstract
Endoscopic surgery has some clear benefits, but it also has some disadvantages in reducing surgeons' normal dexterity and limiting their ability to deal with difficult situations. Computer-aided surgery has been proposed to overcome some of the drawbacks of traditional minimally invasive surgery. The proposed systems make possible a secure, precise procedure with no limitations on the operator's freedom of movement. Image-guided surgery is a new technical tool in surgical oncology. Interventional magnetic resonance imaging (MRI) has entered a new stage in which computer-based techniques play an expanding role in planning, monitoring, and controlling procedures. MRI-guided surgery not only represents a technical challenge but is a transformation from conventional hand-eye coordination to interactive navigational operations. We have recently developed an MRI-guided robot-assisted interventional surgical system as well as an MRI-compatible endoscope. They allow the performance of precise image-guided interventional therapy and endoscopic surgery. MRI-guided laparoscopic surgery is now feasible for malignancies and will play an important part in the development of minimally invasive therapy.
Collapse
Affiliation(s)
- Makoto Hashizume
- Department of Advanced Medical Initiatives, Faculty of Medical Sciences, Kyushu University, 3-1-1 Maidashi, Higashi-ku, Fukuoka 812-8582, Japan.
| |
Collapse
|
43
|
Hahn ST. High-intensity Focused Ultrasound in the Solid Tumor Treatment. JOURNAL OF THE KOREAN MEDICAL ASSOCIATION 2006. [DOI: 10.5124/jkma.2006.49.8.707] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022] Open
Affiliation(s)
- Seong Tae Hahn
- Department of Diagnostic Radiology/HIFU Cancer Therapy Center, The Catholic University of Korea College of Medicine, Korea.
| |
Collapse
|
44
|
Novotny PM, Stoll JA, Vasilyev NV, del Nido PJ, Dupont PE, Howe RD. GPU Based Real-Time Instrument Tracking with Three Dimensional Ultrasound. MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION – MICCAI 2006 2006; 9:58-65. [PMID: 17354874 DOI: 10.1007/11866565_8] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
Abstract
Real-time 3D ultrasound can enable new image-guided surgical procedures, but high data rates prohibit the use of traditional tracking techniques. We present a new method based on the modified Radon transform that identifies the axis of instrument shafts as bright patterns in planar projections. Instrument rotation and tip location are then determined using fiducial markers. These techniques are amenable to rapid execution on the current generation of personal computer graphics processor units (GPU). Our GPU implementation detected a surgical instrument in 31 ms, sufficient for real-time tracking at the 26 volumes per second rate of the ultrasound machine. A water tank experiment found instrument tip position errors of less than 0.2 mm, and an in vivo study tracked an instrument inside a beating porcine heart. The tracking results showed good correspondence to the actual movements of the instrument.
Collapse
Affiliation(s)
- Paul M Novotny
- Division of Engineering and Applied Sciences, Harvard University, MA, USA
| | | | | | | | | | | |
Collapse
|
45
|
Novotny PM, Jacobsen SK, Vasilyev NV, Kettler DT, Salgo IS, Dupont PE, Del Nido PJ, Howe RD. 3D ultrasound in robotic surgery: performance evaluation with stereo displays. Int J Med Robot 2006; 2:279-85. [PMID: 17520643 DOI: 10.1002/rcs.102] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
BACKGROUND The recent advent of real-time 3D ultrasound (3DUS) imaging enables a variety of new surgical procedures. These procedures are hampered by the difficulty of manipulating tissue guided by the distorted, low-resolution 3DUS images. To lessen the effects of these limitations, we investigated stereo displays and surgical robots for 3DUS-guided procedures. METHODS By integrating real-time stereo rendering of 3DUS with the binocular display of a surgical robot, we compared stereo-displayed 3DUS with normally displayed 3DUS. To test the efficacy of stereo-displayed 3DUS, eight surgeons and eight non-surgeons performed in vitro tasks with the surgical robot. RESULTS Error rates dropped by 50% with a stereo display. In addition, subjects completed tasks faster with the stereo-displayed 3DUS as compared to normal-displayed 3DUS. A 28% decrease in task time was seen across all subjects. CONCLUSIONS The results highlight the importance of using a stereo display. By reducing errors and increasing speed, it is an important enhancement to 3DUS-guided robotics procedures.
Collapse
Affiliation(s)
- Paul M Novotny
- Division of Engineering and Applied Sciences, Harvard University, Cambridge, MA 02138, USA
| | | | | | | | | | | | | | | |
Collapse
|
46
|
Suematsu Y, Martinez JF, Wolf BK, Marx GR, Stoll JA, DuPont PE, Howe RD, Triedman JK, del Nido PJ. Three-dimensional echo-guided beating heart surgery without cardiopulmonary bypass: atrial septal defect closure in a swine model. J Thorac Cardiovasc Surg 2005; 130:1348-57. [PMID: 16256788 DOI: 10.1016/j.jtcvs.2005.06.043] [Citation(s) in RCA: 54] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/28/2005] [Revised: 06/20/2005] [Accepted: 06/28/2005] [Indexed: 11/22/2022]
Abstract
OBJECTIVE In this study, we tested 3 techniques of atrial septal defect closure under real-time 3-dimensional echocardiography guidance in a swine model. METHODS The operations were conducted under the sole guidance of a modified real-time 3-dimensional echocardiography guidance system with a x4 matrix transducer (Sonos 7500, Philips Medical Systems, Andover, Mass). Eighteen swine were anesthetized, and after median sternotomy, the echo probe was applied directly to the surface of the right atrium. To create an atrial septal defect, balloon atrial septostomy and atrial septal defect enlargement were performed. Subsequently, 3 different techniques of atrial septal defect closure were attempted: group I, direct suture closure; group II, closure of the atrial septal defect using the Amplatzer device (AGA Medical Corp, Golden Valley, Minn); and group III, patch closure of the atrial septal defect (n = 6 each). RESULTS Real-time 3-dimensional echocardiography guidance provided sufficient spatial resolution and a satisfactory frame rate to provide a "virtual surgeon's view" of the relevant anatomy during the entire procedure. All atrial septal defects were enlarged, and the mean final size was 8.5 +/- 1.8 mm. Atrial septal defect closure was successfully accomplished with all the 3 surgical techniques examined. In groups I and III, the needles (1-3 sutures) and staples (6-12 staples) penetrated the tissue and patch material consistently, whereas in group III, the Amplatzer atrial septal defect device was easily deployed. There was no incident device/staple embolization or air introduction. Neither intraoperative 2-dimensional color Doppler echocardiography nor postmortem macro-evaluation revealed any residual shunts. CONCLUSIONS Beating heart atrial septal defect closure under real-time 3-dimensional echocardiographic guidance is feasible and, unlike catheter-based devices, applicable for any type of secundum atrial septal defect.
Collapse
Affiliation(s)
- Yoshihiro Suematsu
- Department of Cardiac Surgery and Cardiology, Children's Hospital and Harvard Medical School, Boston, Mass 02115, USA
| | | | | | | | | | | | | | | | | |
Collapse
|
47
|
Abstract
Traditionally, surgery has been the only cure for many solid tumours. Technological advances have catalysed a shift from open surgery towards less invasive techniques. Laparoscopic surgery and minimally invasive techniques continue to evolve, but for decades high-intensity focused ultrasound has promised to deliver the ultimate objective - truly non-invasive tumour ablation. Only now, however, with recent improvements in imaging, has this objective finally emerged as a real clinical possibility.
Collapse
|
48
|
Bacha EAM, Hijazi ZM. Hybrid procedures in pediatric cardiac surgery. Semin Thorac Cardiovasc Surg Pediatr Card Surg Annu 2005:78-85. [PMID: 15818362 DOI: 10.1053/j.pcsu.2005.01.001] [Citation(s) in RCA: 31] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/02/2023]
Abstract
Hybrid pediatric cardiac surgery is an emerging field that combines skills and techniques used by pediatric cardiac surgeons and interventional pediatric cardiologists. This article describes the emerging indications and techniques in hybrid pediatric cardiac surgery and discusses potential future applications. It focuses on peratrial and perventricular septal defect closure, intraoperative stenting, hybrid stage I palliation for hypoplastic left heart syndrome, and percutaneous valve implantation.
Collapse
Affiliation(s)
- Emile A M Bacha
- Section of Cardiothoracic Surgery (Congenital and Pediatric Cardiac Surgery), The University of Chicago Children's Hospital, IL 60637, USA
| | | |
Collapse
|
49
|
Abstract
Advances in robotic technology and imaging systems have enabled the broad application of minimally invasive techniques in cardiac surgery, including coronary artery bypass grafting and mitral valve repair in adults. In pediatric cardiac surgery, however, current robotic systems have been used primarily to facilitate thoracoscopic pediatric procedures on extracardiac lesions, such as ligation of patent ductus and division of vascular rings. The use of smaller instruments with sophisticated robotic wrists may make it possible to perform more complex extracardiac procedures even in young infants. Additionally, future technological improvements, including incorporation of tactile feedback, instrument tracking, and intracardiac imaging (such as real-time 3-dimensional echocardiography), may enable intracardiac robotic surgery to be performed in children. This article reviews the current and potential future applications of pediatric robotic surgery and the developmental work required to enable performance of these procedures, along with an overview of the problems associated with the use of current robotic surgical systems in children.
Collapse
Affiliation(s)
- Yoshihiro Suematsu
- Department of Cardiac Surgery, Children's Hospital Boston, Harvard Medical School, 300 Longwood Avenue, Boston, Massachusetts 02115, USA
| | | |
Collapse
|
50
|
Suematsu Y, Marx GR, Stoll JA, DuPont PE, Cleveland RO, Howe RD, Triedman JK, Mihaljevic T, Mora BN, Savord BJ, Salgo IS, del Nido PJ. Three-dimensional echocardiography–guided beating-heart surgery without cardiopulmonary bypass: A feasibility study. J Thorac Cardiovasc Surg 2004; 128:579-87. [PMID: 15457159 DOI: 10.1016/j.jtcvs.2004.06.011] [Citation(s) in RCA: 31] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
BACKGROUND There is no current acceptable approach for intracardiac beating-heart interventions. We have adapted real-time 3-dimensional echocardiography with specialized instrumentation to facilitate beating-heart repair of atrial septal defects and mitral valve plasty to investigate the feasibility of real-time 3-dimensional echocardiography-guided cardiac surgery. METHODS In experiment I a modified real-time 3-dimensional echocardiography system with x4 matrix transducer was compared with 2-dimensional echocardiography in the performance of common surgical tasks. Completion times, deviation from an ideal trajectory, and an echogenic target were measured. In experiment II porcine atrial septal defects were closed with an original semiautomatic suturing device (n = 4) and with a 5-mm endoscopic stapler and a pericardial or polytetrafluoroethylene patch (n = 4). In experiment III a pulsatile porcine mitral valve model was developed, and suture placement through the anterior and posterior mitral leaflets was performed (n = 8). During all experiments, the operator was blinded to the target and operated on only with ultrasonic guidance. RESULTS In experiment I, compared with 2-dimensional echocardiographic guidance, completion times improved by 21% ( P <.01) with high-trajectory accuracy, and suture deviation was significantly smaller (2-dimensional echocardiography, 5.4 +/- 2.7 mm; 3-dimensional echocardiography, 1.7 +/- 0.7 mm; P <.05) in real-time 3-dimensional echocardiography-guided tasks. In experiments II and III in both atrial septal defect closure and mitral valve plasty, real-time 3-dimensional echocardiography provided satisfactory images and sufficient anatomic detail for suturing and patch deployment. All surgical tasks were successfully performed with accuracy. CONCLUSIONS Real-time 3-dimensional echocardiography provides adequate imaging and anatomic detail to act as a sole guide for surgical task performance. These initial experiments demonstrate the feasibility of beating-heart direct or patch closure of atrial septal defects and mitral valve plasty without cardiopulmonary bypass.
Collapse
Affiliation(s)
- Yoshihiro Suematsu
- Department of Cardiac Surgery, Children's Hospital-Boston and Brigham and Women's Hospital, Harvard Medical School, Boston, MA 02115, USA
| | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|