1
|
Sahu SK, Sozer C, Rosa B, Tamadon I, Renaud P, Menciassi A. Shape Reconstruction Processes for Interventional Application Devices: State of the Art, Progress, and Future Directions. Front Robot AI 2021; 8:758411. [PMID: 34869615 PMCID: PMC8640970 DOI: 10.3389/frobt.2021.758411] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2021] [Accepted: 10/11/2021] [Indexed: 01/02/2023] Open
Abstract
Soft and continuum robots are transforming medical interventions thanks to their flexibility, miniaturization, and multidirectional movement abilities. Although flexibility enables reaching targets in unstructured and dynamic environments, it also creates challenges for control, especially due to interactions with the anatomy. Thus, in recent years lots of efforts have been devoted for the development of shape reconstruction methods, with the advancement of different kinematic models, sensors, and imaging techniques. These methods can increase the performance of the control action as well as provide the tip position of robotic manipulators relative to the anatomy. Each method, however, has its advantages and disadvantages and can be worthwhile in different situations. For example, electromagnetic (EM) and Fiber Bragg Grating (FBG) sensor-based shape reconstruction methods can be used in small-scale robots due to their advantages thanks to miniaturization, fast response, and high sensitivity. Yet, the problem of electromagnetic interference in the case of EM sensors, and poor response to high strains in the case of FBG sensors need to be considered. To help the reader make a suitable choice, this paper presents a review of recent progress on shape reconstruction methods, based on a systematic literature search, excluding pure kinematic models. Methods are classified into two categories. First, sensor-based techniques are presented that discuss the use of various sensors such as FBG, EM, and passive stretchable sensors for reconstructing the shape of the robots. Second, imaging-based methods are discussed that utilize images from different imaging systems such as fluoroscopy, endoscopy cameras, and ultrasound for the shape reconstruction process. The applicability, benefits, and limitations of each method are discussed. Finally, the paper draws some future promising directions for the enhancement of the shape reconstruction methods by discussing open questions and alternative methods.
Collapse
Affiliation(s)
- Sujit Kumar Sahu
- The BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy
- Department of Excellence in Robotics & AI, Scuola Superiore Sant’Anna, Pisa, Italy
- ICube, CNRS, INSA Strasbourg, University of Strasbourg, Strasbourg, France
| | - Canberk Sozer
- The BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy
- Department of Excellence in Robotics & AI, Scuola Superiore Sant’Anna, Pisa, Italy
| | - Benoit Rosa
- ICube, CNRS, INSA Strasbourg, University of Strasbourg, Strasbourg, France
| | - Izadyar Tamadon
- The BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy
- Department of Excellence in Robotics & AI, Scuola Superiore Sant’Anna, Pisa, Italy
| | - Pierre Renaud
- ICube, CNRS, INSA Strasbourg, University of Strasbourg, Strasbourg, France
| | - Arianna Menciassi
- The BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy
- Department of Excellence in Robotics & AI, Scuola Superiore Sant’Anna, Pisa, Italy
| |
Collapse
|
2
|
Origami Lesion-Targeting Device for CT-Guided Interventions. J Imaging 2019; 5:jimaging5020023. [PMID: 34460471 PMCID: PMC8320902 DOI: 10.3390/jimaging5020023] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2018] [Revised: 01/19/2019] [Accepted: 01/21/2019] [Indexed: 11/17/2022] Open
Abstract
The objective of this study is to preliminarily evaluate a lesion-targeting device for CT-guided interventions. The device is created by laser cutting the structure from a sheet of medical grade paperboard, 3D printing two radiocontrast agent grids onto the surface and folding the structure into a rectangular prism with a viewing window. An abdominal imaging phantom was used to evaluate the device through CT imaging and the targeting of lesions for needle insertion. The lesion-targeting trials resulted in a mean targeting error of 2.53 mm (SD 0.59 mm, n = 30). The device is rigid enough to adequately support standard biopsy needles, and it attaches to the patient, reducing the risk of tissue laceration by needles held rigidly in place by an external manipulator. Additional advantages include adequate support for the insertion of multiple surgical tools at once for procedures such as composite ablation and the potential to guide off-axial needle insertion. The low-cost and disposability of the device make it well-suited for the minimally invasive image-guided therapy environment.
Collapse
|
3
|
Tella-Amo M, Peter L, Shakir DI, Deprest J, Stoyanov D, Iglesias JE, Vercauteren T, Ourselin S. Probabilistic visual and electromagnetic data fusion for robust drift-free sequential mosaicking: application to fetoscopy. J Med Imaging (Bellingham) 2018; 5:021217. [PMID: 29487889 PMCID: PMC5822039 DOI: 10.1117/1.jmi.5.2.021217] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2017] [Accepted: 01/23/2018] [Indexed: 11/23/2022] Open
Abstract
The most effective treatment for twin-to-twin transfusion syndrome is laser photocoagulation of the shared vascular anastomoses in the placenta. Vascular connections are extremely challenging to locate due to their caliber and the reduced field-of-view of the fetoscope. Therefore, mosaicking techniques are beneficial to expand the scene, facilitate navigation, and allow vessel photocoagulation decision-making. Local vision-based mosaicking algorithms inherently drift over time due to the use of pairwise transformations. We propose the use of an electromagnetic tracker (EMT) sensor mounted at the tip of the fetoscope to obtain camera pose measurements, which we incorporate into a probabilistic framework with frame-to-frame visual information to achieve globally consistent sequential mosaics. We parametrize the problem in terms of plane and camera poses constrained by EMT measurements to enforce global consistency while leveraging pairwise image relationships in a sequential fashion through the use of local bundle adjustment. We show that our approach is drift-free and performs similarly to state-of-the-art global alignment techniques like bundle adjustment albeit with much less computational burden. Additionally, we propose a version of bundle adjustment that uses EMT information. We demonstrate the robustness to EMT noise and loss of visual information and evaluate mosaics for synthetic, phantom-based and ex vivo datasets.
Collapse
Affiliation(s)
- Marcel Tella-Amo
- University College London, Wellcome/EPSRC Center for Interventional and Surgical Sciences, London, United Kingdom
| | - Loic Peter
- University College London, Wellcome/EPSRC Center for Interventional and Surgical Sciences, London, United Kingdom
| | - Dzhoshkun I Shakir
- University College London, Wellcome/EPSRC Center for Interventional and Surgical Sciences, London, United Kingdom
| | - Jan Deprest
- University College London, Wellcome/EPSRC Center for Interventional and Surgical Sciences, London, United Kingdom.,KU Leuven, Center for Surgical Technologies, Faculty of Medicine, Leuven, Belgium
| | - Danail Stoyanov
- University College London, Wellcome/EPSRC Center for Interventional and Surgical Sciences, London, United Kingdom
| | - Juan Eugenio Iglesias
- University College London, Translational Imaging Group, CMIC, Medical Physics, London, United Kingdom
| | - Tom Vercauteren
- University College London, Wellcome/EPSRC Center for Interventional and Surgical Sciences, London, United Kingdom.,KU Leuven, Center for Surgical Technologies, Faculty of Medicine, Leuven, Belgium
| | - Sebastien Ourselin
- University College London, Wellcome/EPSRC Center for Interventional and Surgical Sciences, London, United Kingdom
| |
Collapse
|
4
|
Cheng A, Guo X, Zhang HK, Kang HJ, Etienne-Cummings R, Boctor EM. Active phantoms: a paradigm for ultrasound calibration using phantom feedback. J Med Imaging (Bellingham) 2017; 4:035001. [PMID: 28894765 DOI: 10.1117/1.jmi.4.3.035001] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2016] [Accepted: 07/06/2017] [Indexed: 11/14/2022] Open
Abstract
In ultrasound (US)-guided medical procedures, accurate tracking of interventional tools is crucial to patient safety and clinical outcome. This requires a calibration procedure to recover the relationship between the US image and the tracking coordinate system. In literature, calibration has been performed on passive phantoms, which depend on image quality and parameters, such as frequency, depth, and beam-thickness as well as in-plane assumptions. In this work, we introduce an active phantom for US calibration. This phantom actively detects and responds to the US beams transmitted from the imaging probe. This active echo (AE) approach allows identification of the US image midplane independent of image quality. Both target localization and segmentation can be done automatically, minimizing user dependency. The AE phantom is compared with a crosswire phantom in a robotic US setup. An out-of-plane estimation US calibration method is also demonstrated through simulation and experiments to compensate for remaining elevational uncertainty. The results indicate that the AE calibration phantom can have more consistent results across experiments with varying image configurations. Automatic segmentation is also shown to have similar performance to manual segmentation.
Collapse
Affiliation(s)
- Alexis Cheng
- Johns Hopkins University, Department of Computer Science, Baltimore, Maryland, United States
| | - Xiaoyu Guo
- Johns Hopkins University, Department of Electrical and Computer Engineering, Baltimore, Maryland, United States
| | - Haichong K Zhang
- Johns Hopkins University, Department of Computer Science, Baltimore, Maryland, United States
| | - Hyun Jae Kang
- Johns Hopkins University, Department of Computer Science, Baltimore, Maryland, United States
| | - Ralph Etienne-Cummings
- Johns Hopkins University, Department of Electrical and Computer Engineering, Baltimore, Maryland, United States
| | - Emad M Boctor
- Johns Hopkins University, Department of Computer Science, Baltimore, Maryland, United States.,Johns Hopkins University, Department of Electrical and Computer Engineering, Baltimore, Maryland, United States.,Johns Hopkins University, Department of Radiology, Baltimore, Maryland, United States
| |
Collapse
|
5
|
Chen PH, Hsieh KS, Huang CC. An Acoustic Tracking Approach for Medical Ultrasound Image Simulator. J Med Biol Eng 2017; 37:944-952. [PMID: 30416414 PMCID: PMC6208925 DOI: 10.1007/s40846-017-0258-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2016] [Accepted: 12/02/2016] [Indexed: 11/16/2022]
Abstract
Ultrasound examinations are a standard procedure in the clinical diagnosis of many diseases. However, the efficacy of an ultrasound examination is highly dependent on the skill and experience of the operator, which has prompted proposals for ultrasound simulation systems to facilitate training and education in hospitals and medical schools. The key technology of the medical ultrasound simulation system is the probe tracking method that is used to determine the position and inclination angle of the sham probe, since this information is used to display the ultrasound images in real time. This study investigated a novel acoustic tracking approach for an ultrasound simulation system that exhibits high sensitivity and is cost-effective. Five air-coupled ultrasound elements are arranged as a 1D array in front of a sham probe for transmitting the acoustic signals, and a 5 × 5 2D array of receiving elements is used to receive the acoustic signals from the moving transmitting elements. Since the patterns of the received signals can differ for different positions and angles of the moving probe, the probe can be tracked precisely by the acoustic tracking approach. After the probe position has been determined by the system, the corresponding ultrasound image is immediately displayed on the screen. The system performance was verified by scanning three different subjects as image databases: a simple commercial phantom, a complicated self-made phantom, and a porcine heart. The experimental results indicated that the tracking and angle accuracies of the presented acoustic tracking approach were 0.7 mm and 0.5°, respectively. The performance of the acoustic tracking approach is compared with those of other tracking technologies.
Collapse
Affiliation(s)
- Po-Heng Chen
- 1Department of Biomedical Engineering, National Cheng Kung University, No 1, University Road, Tainan City, 70101 Taiwan
| | - Kai-Sheng Hsieh
- 2Division of Critical Care Medicine, Department of Pediatrics, Kaohsiung Chang Gung Memorial Hospital and Chang Gung University College of Medicine, Kaohsiung, 833 Taiwan
| | - Chih-Chung Huang
- 1Department of Biomedical Engineering, National Cheng Kung University, No 1, University Road, Tainan City, 70101 Taiwan
| |
Collapse
|
6
|
Active Localization and Tracking of Needle and Target in Robotic Image-Guided Intervention Systems. Auton Robots 2017; 42:83-97. [PMID: 29449761 DOI: 10.1007/s10514-017-9640-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Abstract
This paper describes a framework of algorithms for the active localization and tracking of flexible needles and targets during image-guided percutaneous interventions. The needle and target configurations are tracked by Bayesian filters employing models of the needle and target motions and measurements of the current system state obtained from an intra-operative imaging system which is controlled by an entropy-minimizing active localization algorithm. Versions of the system were built using particle and unscented Kalman filters and their performance was measured using both simulations and hardware experiments with real magnetic resonance imaging data of needle insertions into gel phantoms. Performance of the localization algorithms is given in terms of accuracy of the predictions and computational efficiency is discussed.
Collapse
|
7
|
Tully S, Choset H. A Filtering Approach for Image-Guided Surgery With a Highly Articulated Surgical Snake Robot. IEEE Trans Biomed Eng 2015; 63:392-402. [PMID: 26241966 DOI: 10.1109/tbme.2015.2461531] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
GOAL The objective of this paper is to introduce a probabilistic filtering approach to estimate the pose and internal shape of a highly flexible surgical snake robot during minimally invasive surgery. METHODS Our approach renders a depiction of the robot that is registered to preoperatively reconstructed organ models to produce a 3-D visualization that can be used for surgical feedback. Our filtering method estimates the robot shape using an extended Kalman filter that fuses magnetic tracker data with kinematic models that define the motion of the robot. Using Lie derivative analysis, we show that this estimation problem is observable, and thus, the shape and configuration of the robot can be successfully recovered with a sufficient number of magnetic tracker measurements. RESULTS We validate this study with benchtop and in-vivo image-guidance experiments in which the surgical robot was driven along the epicardial surface of a porcine heart. CONCLUSION This paper introduces a filtering approach for shape estimation that can be used for image guidance during minimally invasive surgery. SIGNIFICANCE The methods being introduced in this paper enable informative image guidance for highly articulated surgical robots, which benefits the advancement of robotic surgery.
Collapse
|
8
|
Patil V, Gupta R, San José Estépar R, Lacson R, Cheung A, Wong JM, Popp AJ, Golby A, Ogilvy C, Vosburgh KG. Smart stylet: the development and use of a bedside external ventricular drain image-guidance system. Stereotact Funct Neurosurg 2015; 93:50-8. [PMID: 25662506 DOI: 10.1159/000368906] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2014] [Accepted: 10/07/2014] [Indexed: 11/19/2022]
Abstract
BACKGROUND Placement accuracy of ventriculostomy catheters is reported in a wide and variable range. Development of an efficient image-guidance system may improve physician performance and patient safety. OBJECTIVE We evaluate the prototype of Smart Stylet, a new electromagnetic image-guidance system for use during bedside ventriculostomy. METHODS Accuracy of the Smart Stylet system was assessed. System operators were evaluated for their ability to successfully target the ipsilateral frontal horn in a phantom model. RESULTS Target registration error across 15 intracranial targets ranged from 1.3 to 4.6 mm (mean 3.1 mm). Using Smart Stylet guidance, a test operator successfully passed a ventriculostomy catheter to a shifted ipsilateral frontal horn 20/20 (100%) times from the frontal approach in a skull phantom. Without Smart Stylet guidance, the operator was successful 4/10 (40%) times from the right frontal approach and 6/10 (60%) times from the left frontal approach. In a separate experiment, resident operators were successful 2/4 (50%) times when targeting the shifted ipsilateral frontal horn with Smart Stylet guidance and 0/4 (0%) times without image guidance using a skull phantom. CONCLUSIONS Smart Stylet may improve the ability to successfully target the ventricles during frontal ventriculostomy.
Collapse
Affiliation(s)
- Vaibhav Patil
- Department of Radiology, Brigham and Women's Hospital, Boston, Mass., USA
| | | | | | | | | | | | | | | | | | | |
Collapse
|
9
|
Electromagnetic tracking in surgical and interventional environments: usability study. Int J Comput Assist Radiol Surg 2014; 10:253-62. [PMID: 25193146 DOI: 10.1007/s11548-014-1110-0] [Citation(s) in RCA: 25] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2014] [Accepted: 08/10/2014] [Indexed: 12/13/2022]
|
10
|
|
11
|
Zemirline A, Agnus V, Soler L, Mathoulin CL, Liverneaux PA, Obdeijn M. Augmented reality-based navigation system for wrist arthroscopy: feasibility. J Wrist Surg 2013; 2:294-298. [PMID: 24436832 PMCID: PMC3858680 DOI: 10.1055/s-0033-1359321] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Abstract
PURPOSE In video surgery, and more specifically in arthroscopy, one of the major problems is positioning the camera and instruments within the anatomic environment. The concept of computer-guided video surgery has already been used in ear, nose, and throat (ENT), gynecology, and even in hip arthroscopy. These systems, however, rely on optical or mechanical sensors, which turn out to be restricting and cumbersome. The aim of our study was to develop and evaluate the accuracy of a navigation system based on electromagnetic sensors in video surgery. METHODS We used an electromagnetic localization device (Aurora, Northern Digital Inc., Ontario, Canada) to track the movements in space of both the camera and the instruments. We have developed a dedicated application in the Python language, using the VTK library for the graphic display and the OpenCV library for camera calibration. RESULTS A prototype has been designed and evaluated for wrist arthroscopy. It allows display of the theoretical position of instruments onto the arthroscopic view with useful accuracy. DISCUSSION The augmented reality view represents valuable assistance when surgeons want to position the arthroscope or locate their instruments. It makes the maneuver more intuitive, increases comfort, saves time, and enhances concentration.
Collapse
Affiliation(s)
- Ahmed Zemirline
- Department of Hand Surgery, Strasbourg University Hospitals,
France
- Institut de Recherche sur les Cancers de l'Appareil Digestifs,
Strasbourg, France
| | - Vincent Agnus
- Institut de Recherche sur les Cancers de l'Appareil Digestifs,
Strasbourg, France
| | - Luc Soler
- Institut de Recherche sur les Cancers de l'Appareil Digestifs,
Strasbourg, France
| | | | | | - Miryam Obdeijn
- Department of Plastic, Reconstruction, and Hand Surgery, Academic
Medical Center, University of Amsterdam, The Netherlands
| |
Collapse
|
12
|
Zhang H, Banovac F, Lin R, Glossop N, Wood BJ, Lindisch D, Levy E, Cleary K. Electromagnetic tracking for abdominal interventions in computer aided surgery. COMPUTER AIDED SURGERY : OFFICIAL JOURNAL OF THE INTERNATIONAL SOCIETY FOR COMPUTER AIDED SURGERY 2006; 11:127-36. [PMID: 16829506 PMCID: PMC4136656 DOI: 10.3109/10929080600751399] [Citation(s) in RCA: 52] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
Abstract
Electromagnetic tracking has great potential for assisting physicians in precision placement of instruments during minimally invasive interventions in the abdomen, since electromagnetic tracking is not limited by the line-of-sight restrictions of optical tracking. A new generation of electromagnetic tracking has recently become available, with sensors small enough to be included in the tips of instruments. To fully exploit the potential of this technology, our research group has been developing a computer aided, image-guided system that uses electromagnetic tracking for visualization of the internal anatomy during abdominal interventions. As registration is a critical component in developing an accurate image-guided system, we present three registration techniques: 1) enhanced paired-point registration (time-stamp match registration and dynamic registration); 2) orientation-based registration; and 3) needle shape-based registration. Respiration compensation is another important issue, particularly in the abdomen, where respiratory motion can make precise targeting difficult. To address this problem, we propose reference tracking and affine transformation methods. Finally, we present our prototype navigation system, which integrates the registration, segmentation, path-planning and navigation functions to provide real-time image guidance in the clinical environment. The methods presented here have been tested with a respiratory phantom specially designed by our group and in swine animal studies under approved protocols. Based on these tests, we conclude that our system can provide quick and accurate localization of tracked instruments in abdominal interventions, and that it offers a user-friendly display for the physician.
Collapse
Affiliation(s)
- Hui Zhang
- Imaging Science and Information Systems (ISIS) Center, Department of Radiology, Georgetown University, Washington, DC, USA
| | - Filip Banovac
- Imaging Science and Information Systems (ISIS) Center, Department of Radiology, Georgetown University, Washington, DC, USA
| | - Ralph Lin
- Imaging Science and Information Systems (ISIS) Center, Department of Radiology, Georgetown University, Washington, DC, USA
| | | | | | - David Lindisch
- Department of Radiology, Georgetown University Hospital, Washington, DC, USA
| | - Elliot Levy
- Department of Radiology, Georgetown University Hospital, Washington, DC, USA
| | - Kevin Cleary
- Imaging Science and Information Systems (ISIS) Center, Department of Radiology, Georgetown University, Washington, DC, USA
| |
Collapse
|