1
|
Brekken R, Hofstad EF, Solberg OV, Tangen GA, Leira HO, Gruionu L, Langø T. Accuracy of instrument tip position using fiber optic shape sensing for navigated bronchoscopy. Med Eng Phys 2024; 125:104116. [PMID: 38508792 DOI: 10.1016/j.medengphy.2024.104116] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2023] [Revised: 12/21/2023] [Accepted: 02/13/2024] [Indexed: 03/22/2024]
Abstract
The purpose of this study was to evaluate the accuracy of a method for estimating the tip position of a fiber optic shape-sensing (FOSS) integrated instrument being inserted through a bronchoscope. A modified guidewire with a multicore optical fiber was inserted into the working channel of a custom-made catheter with three electromagnetic (EM) sensors. The displacement between the instruments was manually set, and a point-based method was applied to match the position of the EM sensors to corresponding points on the shape. The accuracy was evaluated in a realistic bronchial model. An additional EM sensor was used to sample the tip of the guidewire, and the absolute deviation between this position and the estimated tip position was calculated. For small displacements between the tip of the FOSS integrated tool and the catheter, the median deviation in estimated tip position was ≤5 mm. For larger displacements, deviations exceeding 10 mm were observed. The deviations increased when the shape sensor had sharp curvatures relative to more straight shapes. The method works well for clinically relevant displacements of a biopsy tool from the bronchoscope tip, and when the path to the lesion has limited curvatures. However, improvements must be made to our configuration before pursuing further clinical testing.
Collapse
Affiliation(s)
- Reidar Brekken
- SINTEF Digital, Department of Health Research, Trondheim, Norway; Norwegian National Research Centre for Minimally Invasive and Image-guided Diagnostics and Therapy, St. Olavs Hospital, Trondheim, Norway.
| | - Erlend Fagertun Hofstad
- SINTEF Digital, Department of Health Research, Trondheim, Norway; Norwegian National Research Centre for Minimally Invasive and Image-guided Diagnostics and Therapy, St. Olavs Hospital, Trondheim, Norway
| | - Ole Vegard Solberg
- SINTEF Digital, Department of Health Research, Trondheim, Norway; Norwegian National Research Centre for Minimally Invasive and Image-guided Diagnostics and Therapy, St. Olavs Hospital, Trondheim, Norway
| | - Geir Arne Tangen
- SINTEF Digital, Department of Health Research, Trondheim, Norway; Norwegian National Research Centre for Minimally Invasive and Image-guided Diagnostics and Therapy, St. Olavs Hospital, Trondheim, Norway
| | - Håkon Olav Leira
- Norwegian National Research Centre for Minimally Invasive and Image-guided Diagnostics and Therapy, St. Olavs Hospital, Trondheim, Norway; Department of Thoracic Medicine, St Olavs Hospital, Trondheim, Norway
| | - Lucian Gruionu
- Faculty of Mechanics, University of Craiova, Craiova, Romania
| | - Thomas Langø
- SINTEF Digital, Department of Health Research, Trondheim, Norway; Norwegian National Research Centre for Minimally Invasive and Image-guided Diagnostics and Therapy, St. Olavs Hospital, Trondheim, Norway
| |
Collapse
|
2
|
Yang S, Wang Y, Ai D, Geng H, Zhang D, Xiao D, Song H, Li M, Yang J. Augmented Reality Navigation System for Biliary Interventional Procedures With Dynamic Respiratory Motion Correction. IEEE Trans Biomed Eng 2024; 71:700-711. [PMID: 38241137 DOI: 10.1109/tbme.2023.3316290] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/21/2024]
Abstract
OBJECTIVE Biliary interventional procedures require physicians to track the interventional instrument tip (Tip) precisely with X-ray image. However, Tip positioning relies heavily on the physicians' experience due to the limitations of X-ray imaging and the respiratory interference, which leads to biliary damage, prolonged operation time, and increased X-ray radiation. METHODS We construct an augmented reality (AR) navigation system for biliary interventional procedures. It includes system calibration, respiratory motion correction and fusion navigation. Firstly, the magnetic and 3D computed tomography (CT) coordinates are aligned through system calibration. Secondly, a respiratory motion correction method based on manifold regularization is proposed to correct the misalignment of the two coordinates caused by respiratory motion. Thirdly, the virtual biliary, liver and Tip from CT are overlapped to the corresponding position of the patient for dynamic virtual-real fusion. RESULTS Our system is respectively evaluated and achieved an average alignment error of 0.75 ± 0.17 mm and 2.79 ± 0.46 mm on phantoms and patients. The navigation experiments conducted on phantoms achieve an average Tip positioning error of 0.98 ± 0.15 mm and an average fusion error of 1.67 ± 0.34 mm after correction. CONCLUSION Our system can automatically register the Tip to the corresponding location in CT, and dynamically overlap the 3D virtual model onto patients to provide accurate and intuitive AR navigation. SIGNIFICANCE This study demonstrates the clinical potential of our system by assisting physicians during biliary interventional procedures. Our system enables dynamic visualization of virtual model on patients, reducing the reliance on contrast agents and X-ray usage.
Collapse
|
3
|
Guo L, Nahm W. A cGAN-based network for depth estimation from bronchoscopic images. Int J Comput Assist Radiol Surg 2024; 19:33-36. [PMID: 37561354 PMCID: PMC10769989 DOI: 10.1007/s11548-023-02978-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2023] [Accepted: 05/24/2023] [Indexed: 08/11/2023]
Abstract
PURPOSE Depth estimation is the basis of 3D reconstruction of airway structure from 2D bronchoscopic scenes, which can be further used to develop a vision-based bronchoscopic navigation system. This work aims to improve the performance of depth estimation directly from bronchoscopic images by training a depth estimation network on both synthetic and real datasets. METHODS We propose a cGAN-based network Bronchoscopic-Depth-GAN (BronchoDep-GAN) to estimate depth from bronchoscopic images by translating bronchoscopic images into depth maps. The network is trained in a supervised way learning from synthetic textured bronchoscopic image-depth pairs and virtual bronchoscopic image-depth pairs, and simultaneously, also in an unsupervised way learning from unpaired real bronchoscopic images and depth maps to adapt the model to real bronchoscopic scenes. RESULTS Our method is tested on both synthetic data and real data. However, the tests on real data are only qualitative, as no ground truth is available. The results show that our network obtains better accuracy in all cases in estimating depth from bronchoscopic images compared to the well-known cGANs pix2pix. CONCLUSIONS Including virtual and real bronchoscopic images in the training phase of the depth estimation networks can improve depth estimation's performance on both synthetic and real scenes. Further validation of this work is planned on 3D clinical phantoms. Based on the depth estimation results obtained in this work, the accuracy of locating bronchoscopes with corresponding pre-operative CTs will also be evaluated in comparison with the current clinical status.
Collapse
Affiliation(s)
- Lu Guo
- Institute of Biomedical Engineering, Karlsruhe Institute of Technology, Kaiserstraße 12, 76131, Karlsruhe, Germany.
| | - Werner Nahm
- Institute of Biomedical Engineering, Karlsruhe Institute of Technology, Kaiserstraße 12, 76131, Karlsruhe, Germany
| |
Collapse
|
4
|
Guo L, Nahm W. Texture synthesis for generating realistic-looking bronchoscopic videos. Int J Comput Assist Radiol Surg 2023; 18:2287-2293. [PMID: 37162734 PMCID: PMC10632244 DOI: 10.1007/s11548-023-02874-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Accepted: 03/15/2023] [Indexed: 05/11/2023]
Abstract
PURPOSE Synthetic realistic-looking bronchoscopic videos are needed to develop and evaluate depth estimation methods as part of investigating vision-based bronchoscopic navigation system. To generate these synthetic videos under the circumstance where access to real bronchoscopic images/image sequences is limited, we need to create various realistic-looking image textures of the airway inner surface with large size using a small number of real bronchoscopic image texture patches. METHODS A generative adversarial networks-based method is applied to create realistic-looking textures of the airway inner surface by learning from a limited number of small texture patches from real bronchoscopic images. By applying a purely convolutional architecture without any fully connected layers, this method allows the production of textures with arbitrary size. RESULTS Authentic image textures of airway inner surface are created. An example of the synthesized textures and two frames of the thereby generated bronchoscopic video are shown. The necessity and sufficiency of the generated textures as image features for further depth estimation methods are demonstrated. CONCLUSIONS The method can generate textures of the airway inner surface that meet the requirements for the texture itself and for the thereby generated bronchoscopic videos, including "realistic-looking," "long-term temporal consistency," "sufficient image features for depth estimation," and "large size and variety of synthesized textures." Besides, it also shows advantages with respect to the easy accessibility to required data source. A further validation of this approach is planned by utilizing the realistic-looking bronchoscopic videos with textures generated by this method as training and test data for some depth estimation networks.
Collapse
Affiliation(s)
- Lu Guo
- Karlsruhe Institute of Technology, Kaiserstraße 12, Karlsruhe, 76131, Germany.
| | - Werner Nahm
- Karlsruhe Institute of Technology, Kaiserstraße 12, Karlsruhe, 76131, Germany
| |
Collapse
|
5
|
Fried I, Hoelscher J, Akulian JA, Pizer S, Alterovitz R. Landmark Based Bronchoscope Localization for Needle Insertion Under Respiratory Deformation. PROCEEDINGS OF THE ... IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS. IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS 2023; 2023:6593-6600. [PMID: 38947248 PMCID: PMC11214542 DOI: 10.1109/iros55552.2023.10342115] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/02/2024]
Abstract
Bronchoscopy is currently the least invasive method for definitively diagnosing lung cancer, which kills more people in the United States than any other form of cancer. Successfully diagnosing suspicious lung nodules requires accurate localization of the bronchoscope relative to a planned biopsy site in the airways. This task is challenging because the lung deforms intraoperatively due to respiratory motion, the airways lack photometric features, and the anatomy's appearance is repetitive. In this paper, we introduce a real-time camera-based method for accurately localizing a bronchoscope with respect to a planned needle insertion pose. Our approach uses deep learning and accounts for deformations and overcomes limitations of global pose estimation by estimating pose relative to anatomical landmarks. Specifically, our learned model considers airway bifurcations along the airway wall as landmarks because they are distinct geometric features that do not vary significantly with respiratory motion. We evaluate our method in a simulated dataset of lungs undergoing respiratory motion. The results show that our method generalizes across patients and localizes the bronchoscope with accuracy sufficient to access the smallest clinically-relevant nodules across all levels of respiratory deformation, even in challenging distal airways. Our method could enable physicians to perform more accurate biopsies and serve as a key building block toward accurate autonomous robotic bronchoscopy.
Collapse
Affiliation(s)
- Inbar Fried
- I. Fried, J. Hoelscher, S. Pizer, and R. Alterovitz are with the Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
- I. Fried is also with the Medical Scientist Training Program, University of North Carolina School of Medicine, Chapel Hill, NC, 27599, USA
| | - Janine Hoelscher
- I. Fried, J. Hoelscher, S. Pizer, and R. Alterovitz are with the Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Jason A Akulian
- J. A. Akulian is with the Division of Pulmonary Diseases and Critical Care Medicine at the University of North Carolina at Chapel Hill, NC 27599, USA
| | - Stephen Pizer
- I. Fried, J. Hoelscher, S. Pizer, and R. Alterovitz are with the Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Ron Alterovitz
- I. Fried, J. Hoelscher, S. Pizer, and R. Alterovitz are with the Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| |
Collapse
|
6
|
Navaei Lavasani S, Farnia P, Najafzadeh E, Saghatchi S, Samavati M, Abtahi H, Deevband M, Ahmadian A. Bronchoscope motion tracking using centerline-guided Gaussian mixture model in navigated bronchoscopy. Phys Med Biol 2021; 66:025001. [PMID: 33181494 DOI: 10.1088/1361-6560/abca07] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
Electromagnetic-based navigation bronchoscopy requires accurate and robust estimation of the bronchoscope position inside the bronchial tree. However, respiratory motion, coughing, patient movement, and airway deformation inflicted by bronchoscope significantly hinder the accuracy of intraoperative bronchoscopic localization. In this study, a real-time and automatic registration procedure was proposed to superimpose the current location of the bronchoscope to corresponding locations on a centerline extracted from bronchial computed tomography (CT) images. A centerline-guided Gaussian mixture model (CG-GMM) was introduced to register a bronchoscope's position concerning extracted centerlines. A GMM was fitted to bronchoscope positions where the orientation likelihood was chosen to assign the membership probabilities of the mixture model, which led to preserving the global and local structures. The problem was formulated and solved under the expectation maximization framework, where the feature correspondence and spatial transformation are estimated iteratively. Validation was performed on a dynamic phantom with four different respiratory motions and four human real bronchoscopy (RB) datasets. Results of the experiments conducted on the bronchial phantom showed that the average positional tracking error using the proposed approach was equal to 1.98 [Formula: see text] 0.98 mm that was reduced in comparison with independent electromagnetic tracking (EMT), iterative closest point (ICP), and coherent point drift (CPD) methods by 64%, 58%, and 53%, respectively. In the patient assessment part of the study, the average positional tracking error was 4.73 [Formula: see text] 4.76 mm and compared to ICP, and CPD methods showed 31.4% improvement of successfully tracked frames. Our approach introduces a novel method for real-time respiratory motion compensation that provides reliable guidance during bronchoscopic interventions and, thus could increase the diagnostic yield of transbronchial biopsy.
Collapse
Affiliation(s)
- Saeedeh Navaei Lavasani
- Biomedical Engineering and Medical Physics Department, School of Medicine, Shahid Beheshti University of Medical Sciences (SBMU), Tehran, Iran. Research Centre of Biomedical Technology and Robotics (RCBTR), Imam Khomeini Hospital Complex, Tehran University of Medical Sciences, Tehran, Iran
| | | | | | | | | | | | | | | |
Collapse
|
7
|
Ahmad I, Arora A, El‐Boghdadly K. Embracing the robotic revolution into anaesthetic practice. Anaesthesia 2020; 75:848-851. [DOI: 10.1111/anae.14986] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/18/2019] [Indexed: 01/22/2023]
Affiliation(s)
- I. Ahmad
- Department of Anaesthesia Guy's and St Thomas NHS Foundation Trust London UK
- King's College London UK
| | - A. Arora
- Department of Surgery Guy's and St. Thomas NHS Foundation Trust London UK
| | - K. El‐Boghdadly
- Department of Anaesthesia Guy's and St Thomas NHS Foundation Trust London UK
- King's College London UK
| |
Collapse
|
8
|
Xia W, Jin Q, Ni C, Wang Y, Gao X. Thorax x‐ray and
CT
interventional dataset for nonrigid 2D/3D image registration evaluation. Med Phys 2018; 45:5343-5351. [PMID: 30187928 DOI: 10.1002/mp.13174] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2017] [Revised: 08/20/2018] [Accepted: 08/31/2018] [Indexed: 11/11/2022] Open
Affiliation(s)
- Wei Xia
- Medical Imaging Department Suzhou Institute of Biomedical Engineering and Technology Chinese Academy of Sciences Suzhou 215163 China
| | - Qingpeng Jin
- Medical Imaging Department Suzhou Institute of Biomedical Engineering and Technology Chinese Academy of Sciences Suzhou 215163 China
- University of Chinese Academy of Sciences Beijing 100049 China
| | - Caifang Ni
- Radiology Department The First Affiliated Hospital of Soochow University Suzhou 215006 China
| | - Yanling Wang
- Radiology Department The People's Hospital of Suzhou New District Suzhou 215163 China
| | - Xin Gao
- Medical Imaging Department Suzhou Institute of Biomedical Engineering and Technology Chinese Academy of Sciences Suzhou 215163 China
| |
Collapse
|
9
|
Vining PF, Lee TM, Bizekis CS, Zervos MD. Use of electromagnetic navigational bronchoscopy in robotic pulmonary resection. J Robot Surg 2018; 12:613-616. [PMID: 29396843 DOI: 10.1007/s11701-018-0783-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2017] [Accepted: 01/28/2018] [Indexed: 12/19/2022]
Abstract
Robotic resection of pulmonary lesions has become a more common approach in the field of thoracic surgery. The greatest drawback of robotic resection is the lack of tactile feedback as compared to open approaches, making identification of intrapulmonary lesion difficult. Electromagnetic navigational bronchoscopy (navibronch) enables pre-incisional marking of pulmonary lesions for intraoperative identification. We sought to determine how effective navibronch was in our institution's robotic cases. Thirty-one patients underwent robotically assisted resection of 35 lesions with the assistance of navibronch from 7/2014 to 9/2015. Retrospective demographic and operative data were collected on these patients, and statistical analysis was conducted using ANOVA means testing, Chi-square, and non-parametric tests. The average age in this patient population was 63.7 ± 13.5 years. Eight patients (25.8%) were male. Twenty-five (80.6%) of the patients had pathology involving one lobe, with six (19.4%) in two lobes. 34 of the resections (97.1%) resulted in dye being localized to the first specimen; 34 (97.1%) were found to have the target pathology in the initial specimen. Further resection was carried out in 22 (62.9%) cases, with the final resection resulting in a segment in 2 (5.7%) and a lobe in 14 (40.0%). The mean number of lung specimens collected was 1.94 ± 0.13. The mean number of tumors in each target resection was 1.46 ± 0.66 in final pathology. Malignancy was found in 19 (54.3%) of final specimens. There were no complications related to navibronch. Navibronch is an effective technique in the identification and localization of pulmonary lesions in robotically assisted lung resections.
Collapse
Affiliation(s)
- Patrick F Vining
- Department of Cardiothoracic Surgery, New York University, 550 1st Avenue, New York, NY, 10016, USA.
| | - Timothy M Lee
- Department of Cardiothoracic Surgery, New York University, 550 1st Avenue, New York, NY, 10016, USA
| | - Costas S Bizekis
- Department of Cardiothoracic Surgery, New York University, 550 1st Avenue, New York, NY, 10016, USA
| | - Michael D Zervos
- Department of Cardiothoracic Surgery, New York University, 550 1st Avenue, New York, NY, 10016, USA
| |
Collapse
|