1
|
Fried I, Hoelscher J, Akulian JA, Pizer S, Alterovitz R. Landmark Based Bronchoscope Localization for Needle Insertion Under Respiratory Deformation. PROCEEDINGS OF THE ... IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS. IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS 2023; 2023:6593-6600. [PMID: 38947248 PMCID: PMC11214542 DOI: 10.1109/iros55552.2023.10342115] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/02/2024]
Abstract
Bronchoscopy is currently the least invasive method for definitively diagnosing lung cancer, which kills more people in the United States than any other form of cancer. Successfully diagnosing suspicious lung nodules requires accurate localization of the bronchoscope relative to a planned biopsy site in the airways. This task is challenging because the lung deforms intraoperatively due to respiratory motion, the airways lack photometric features, and the anatomy's appearance is repetitive. In this paper, we introduce a real-time camera-based method for accurately localizing a bronchoscope with respect to a planned needle insertion pose. Our approach uses deep learning and accounts for deformations and overcomes limitations of global pose estimation by estimating pose relative to anatomical landmarks. Specifically, our learned model considers airway bifurcations along the airway wall as landmarks because they are distinct geometric features that do not vary significantly with respiratory motion. We evaluate our method in a simulated dataset of lungs undergoing respiratory motion. The results show that our method generalizes across patients and localizes the bronchoscope with accuracy sufficient to access the smallest clinically-relevant nodules across all levels of respiratory deformation, even in challenging distal airways. Our method could enable physicians to perform more accurate biopsies and serve as a key building block toward accurate autonomous robotic bronchoscopy.
Collapse
Affiliation(s)
- Inbar Fried
- I. Fried, J. Hoelscher, S. Pizer, and R. Alterovitz are with the Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
- I. Fried is also with the Medical Scientist Training Program, University of North Carolina School of Medicine, Chapel Hill, NC, 27599, USA
| | - Janine Hoelscher
- I. Fried, J. Hoelscher, S. Pizer, and R. Alterovitz are with the Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Jason A Akulian
- J. A. Akulian is with the Division of Pulmonary Diseases and Critical Care Medicine at the University of North Carolina at Chapel Hill, NC 27599, USA
| | - Stephen Pizer
- I. Fried, J. Hoelscher, S. Pizer, and R. Alterovitz are with the Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Ron Alterovitz
- I. Fried, J. Hoelscher, S. Pizer, and R. Alterovitz are with the Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| |
Collapse
|
2
|
Oda M, Furukawa K, Navab N, Mori K. KST-Mixer: kinematic spatio-temporal data mixer for colon shape estimation. COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING: IMAGING & VISUALIZATION 2023. [DOI: 10.1080/21681163.2022.2151938] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Affiliation(s)
- Masahiro Oda
- Information and Communications, Nagoya University, Nagoya, Japan
- Graduate School of Informatics, Nagoya University, Nagoya, Japan
| | - Kazuhiro Furukawa
- Department of Gastroenterology and Hepatology, Nagoya University Graduate School of Medicine, Nagoya, Japan
| | - Nassir Navab
- Chair for Computer Aided Medical Procedures, Technical University of Munich, Munich, Germany
| | - Kensaku Mori
- Graduate School of Informatics, Nagoya University, Nagoya, Japan
- Research Center for Medical Bigdata, National Institute of Informatics, Tokyo, Japan
| |
Collapse
|
3
|
Wang C, Oda M, Hayashi Y, Kitasaka T, Itoh H, Honma H, Takebatake H, Mori M, Natori H, Mori K. Anatomy aware-based 2.5D bronchoscope tracking for image-guided bronchoscopic navigation. COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING: IMAGING & VISUALIZATION 2022. [DOI: 10.1080/21681163.2022.2152728] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Affiliation(s)
- Cheng Wang
- Graduate School of Informatics, Nagoya University, Nagoya, Japan
| | - Masahiro Oda
- Graduate School of Informatics, Nagoya University, Nagoya, Japan
- Information and Communications, Nagoya University, Nagoya, Japan
| | - Yuichiro Hayashi
- Graduate School of Informatics, Nagoya University, Nagoya, Japan
| | - Takayuki Kitasaka
- School of Information Science, Aichi Institute of Technology, Toyota, Japan
| | - Hayato Itoh
- Graduate School of Informatics, Nagoya University, Nagoya, Japan
| | - Hirotoshi Honma
- Medical Examination Department, Seamen’s Insurance Hokkaido Healthcare Center, Sapporo, Japan
| | - Hirotsugu Takebatake
- Department of Respiratory Medicine, Sapporo Minami-Sanjo Hospital, Sapporo, Japan
| | - Masaki Mori
- Department of Respiratory Medicine, Sapporo-Kosei General Hospital, Sapporo, Japan
| | - Hiroshi Natori
- Department of Internal Medicine, Keiwakai Nishioka Hospital, Sapporo, Japan
| | - Kensaku Mori
- Graduate School of Informatics, Nagoya University, Nagoya, Japan
- Information Technology Center, Nagoya University, Nagoya, Japan
- Research Center for Medical Bigdata, National Institute of Informatics, Tokyo, Japan
| |
Collapse
|
4
|
Banach A, King F, Masaki F, Tsukada H, Hata N. Visually Navigated Bronchoscopy using three cycle-Consistent generative adversarial network for depth estimation. Med Image Anal 2021; 73:102164. [PMID: 34314953 PMCID: PMC8453111 DOI: 10.1016/j.media.2021.102164] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2021] [Revised: 06/29/2021] [Accepted: 07/06/2021] [Indexed: 11/30/2022]
Abstract
[Background] Electromagnetically Navigated Bronchoscopy (ENB) is currently the state-of-the art diagnostic and interventional bronchoscopy. CT-to-body divergence is a critical hurdle in ENB, causing navigation error and ultimately limiting the clinical efficacy of diagnosis and treatment. In this study, Visually Navigated Bronchoscopy (VNB) is proposed to address the aforementioned issue of CT-to-body divergence. [Materials and Methods] We extended and validated an unsupervised learning method to generate a depth map directly from bronchoscopic images using a Three Cycle-Consistent Generative Adversarial Network (3cGAN) and registering the depth map to preprocedural CTs. We tested the working hypothesis that the proposed VNB can be integrated to the navigated bronchoscopic system based on 3D Slicer, and accurately register bronchoscopic images to pre-procedural CTs to navigate transbronchial biopsies. The quantitative metrics to asses the hypothesis we set was Absolute Tracking Error (ATE) of the tracking and the Target Registration Error (TRE) of the total navigation system. We validated our method on phantoms produced from the pre-procedural CTs of five patients who underwent ENB and on two ex-vivo pig lung specimens. [Results] The ATE using 3cGAN was 6.2 +/- 2.9 [mm]. The ATE of 3cGAN was statistically significantly lower than that of cGAN, particularly in the trachea and lobar bronchus (p < 0.001). The TRE of the proposed method had a range of 11.7 to 40.5 [mm]. The TRE computed by 3cGAN was statistically significantly smaller than those computed by cGAN in two of the five cases enrolled (p < 0.05). [Conclusion] VNB, using 3cGAN to generate the depth maps was technically and clinically feasible. While the accuracy of tracking by cGAN was acceptable, the TRE warrants further investigation and improvement.
Collapse
Affiliation(s)
- Artur Banach
- National Center for Image-guided Therapy, Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, Boston, MA, United States; QUT Centre for Robotics, Queensland University of Technology, Brisbane, Australia.
| | - Franklin King
- National Center for Image-guided Therapy, Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, Boston, MA, United States
| | - Fumitaro Masaki
- National Center for Image-guided Therapy, Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, Boston, MA, United States; Healthcare Optics Research Laboratory, Canon U.S.A., Cambridge, MA, United States
| | - Hisashi Tsukada
- Division of Thoracic Surgery, Department of Surgery, Brigham and Women's Hospital and Harvard Medical School, Boston, MA, United States
| | - Nobuhiko Hata
- National Center for Image-guided Therapy, Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, Boston, MA, United States
| |
Collapse
|
5
|
Depth-based branching level estimation for bronchoscopic navigation. Int J Comput Assist Radiol Surg 2021; 16:1795-1804. [PMID: 34392469 DOI: 10.1007/s11548-021-02460-8] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2020] [Accepted: 07/12/2021] [Indexed: 10/20/2022]
Abstract
PURPOSE Bronchoscopists rely on navigation systems during bronchoscopy to reduce the risk of getting lost in the complex bronchial tree-like structure and the homogeneous bronchus lumens. We propose a patient-specific branching level estimation method for bronchoscopic navigation because it is vital to identify the branches being examined in the bronchus tree during examination. METHODS We estimate the branching level by integrating the changes in the number of bronchial orifices and the camera motions among the frames. We extract the bronchial orifice regions from a depth image, which is generated using a cycle generative adversarial network (CycleGAN) from real bronchoscopic images. We calculate the number of orifice regions using the vertical and horizontal projection profiles of the depth images and obtain the camera-moving direction using the feature point-based camera motion estimation. The changes in the number of bronchial orifices are combined with the camera-moving direction to estimate the branching level. RESULTS We used three in vivo and one phantom case to train the CycleGAN model and four in vivo cases to validate the proposed method. We manually created the ground truth of the branching level. The experimental results showed that the proposed method can estimate the branching level with an average accuracy of 87.6%. The processing time per frame was about 61 ms. CONCLUSION Experimental results show that it is feasible to estimate the branching level using the number of bronchial orifices and camera-motion estimation from real bronchoscopic images.
Collapse
|
6
|
Chien JC, Lee JD, Su E, Li SH. A Bronchoscope Localization Method Using an Augmented Reality Co-Display of Real Bronchoscopy Images with a Virtual 3D Bronchial Tree Model. SENSORS (BASEL, SWITZERLAND) 2020; 20:E6997. [PMID: 33297531 PMCID: PMC7730431 DOI: 10.3390/s20236997] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/23/2020] [Revised: 12/04/2020] [Accepted: 12/06/2020] [Indexed: 11/27/2022]
Abstract
In recent years, Image-Guide Navigation Systems (IGNS) have become an important tool for various surgical operations. In the preparations for planning a surgical path, verifying the location of a lesion, etc., it is an essential tool; in operations such as bronchoscopy, which is the procedure for the inspection and retrieval of diagnostic samples for lung-related surgeries, it is even more so. The IGNS for bronchoscopy uses 2D-based images from a flexible bronchoscope to navigate through the bronchial airways in order to reach the targeted location. In this procedure, the accurate localization of the scope becomes very important, because incorrect information could potentially cause a surgeon to mistakenly direct the scope down the wrong passage. It would be a great aid for the surgeon to be able to visualize the bronchoscope images alongside the current location of the bronchoscope. For this purpose, in this paper, we propose a novel registration method to match real bronchoscopy images with virtual bronchoscope images from a 3D bronchial tree model built using computed tomography (CT) image stacks in order to obtain the current 3D position of the bronchoscope in the airways. This method is a combination of a novel position-tracking method using the current frames from the bronchoscope and the verification of the position of the real bronchoscope image against an image extracted from the 3D model using an adaptive-network-based fuzzy inference system (ANFIS)-based image matching method. Experimental results show that the proposed method performs better than the other methods used in the comparison.
Collapse
Affiliation(s)
- Jong-Chih Chien
- School of Informatics, Kainan University, Taoyuan 33857, Taiwan;
| | - Jiann-Der Lee
- Department of Electrical Engineering, Chang Gung University, Taoyuan 33302, Taiwan;
- Department of Neurosurgery, Chang Gung Memorial Hospital at Linkou, Taoyuan 33305, Taiwan
- Department of Electrical Engineering, Ming Chi University of Technology, New Taipei City 24301, Taiwan
| | - Ellen Su
- Department of Electrical Engineering, Chang Gung University, Taoyuan 33302, Taiwan;
| | - Shih-Hong Li
- Department of Thoracic Medicine, Chang Gung Memorial Hospital at LinKou, Taoyuan 33305, Taiwan;
| |
Collapse
|
7
|
Yang L, Etsuko K. Review on vision‐based tracking in surgical navigation. IET CYBER-SYSTEMS AND ROBOTICS 2020. [DOI: 10.1049/iet-csr.2020.0013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022] Open
Affiliation(s)
- Liangjing Yang
- Zhejiang University/University of Illinois at Urbana‐Champaign Institute, Zhejiang University Haining People's Republic of China
- School of Mechanical Engineering Zhejiang University Hangzhou People's Republic of China
- Department of Mechanical Science and Engineering University of Illinois at Urbana‐Champaign Urbana USA
| | - Kobayashi Etsuko
- Graduate School of Engineering The University of Tokyo Tokyo Japan
- Institute of Advanced Biomedical Engineering and Science Tokyo Women's Medical University Tokyo Japan
| |
Collapse
|
8
|
A visual SLAM-based bronchoscope tracking scheme for bronchoscopic navigation. Int J Comput Assist Radiol Surg 2020; 15:1619-1630. [DOI: 10.1007/s11548-020-02241-9] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2020] [Accepted: 07/21/2020] [Indexed: 10/23/2022]
|
9
|
Shi RB, Mirza S, Martinez D, Douglas C, Cho J, Irish JC, Jaffray DA, Weersink RA. Cost-function testing methodology for image-based registration of endoscopy to CT images in the head and neck. Phys Med Biol 2020; 65. [PMID: 32702685 DOI: 10.1088/1361-6560/aba8b3] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2020] [Accepted: 07/23/2020] [Indexed: 11/11/2022]
Abstract
One of the largest geometric uncertainties in designing radiotherapy treatment plans for squamous cell cancers of the head and neck is contouring the gross tumour volume. We have previously described a method of projecting mucosal disease contours, visible on endoscopy, to volumetrically reconstructed planning CT datasets, using electromagnetic (EM) tracking of a flexible endoscope, enabling rigid registration between endoscopic and CT images. However, to achieve better accuracy for radiotherapy planning, we propose refining this initial registration with image-based registration methods. In this paper, several types of cost functions are evaluated based on accuracy and robustness. Three phantoms and eight clinical cases are used to test each cost function, with initial registration of endoscopy to CT provided by the pose of the flexible endoscope recovered from EM tracking. Cost function classes include: cross correlation, mutual information and gradient methods. For each test case, a ground truth virtual camera pose was first defined by manual registration of anatomical features visible in both real and virtual endoscope images. A new set of evenly spaced fiducial points and a sample contour were created and projected onto the CT image to be used in assessing image registration quality. A new set of 5000 displaced poses was generated by random sampling displacements along each translational and rotational dimension. At each pose, fiducial and contour points in the real image were again projected on the CT image. The cost function, fiducial registration error and contouring error values were then calculated. While all cost functions performed well in select cases, only the normalized gradient field function consistently had registration errors less than 2 mm, which is the accuracy needed if this application of registering mucosal disease identified on optical image to CT images is to be used in the clinical practice of radiation treatment planning. (Registration: ClinicalTrials.gov NCT02704169).
Collapse
Affiliation(s)
| | - Souzan Mirza
- University of Toronto Institute of Biomaterials and Biomedical Engineering, Toronto, Ontario, CANADA
| | - Diego Martinez
- Radiation Medicine Program, Princess Margaret Hospital Cancer Centre, Toronto, Ontario, CANADA
| | - Catriona Douglas
- Surgical Oncology, University of Toronto Department of Surgery, Toronto, Ontario, CANADA
| | - John Cho
- Radiation Medicine Program, Princess Margaret Hospital Cancer Centre, Toronto, Ontario, CANADA
| | - Jonathan C Irish
- Surgical Oncology, University of Toronto Department of Surgery, Toronto, Ontario, CANADA
| | - David A Jaffray
- Radiation Medicine Program, Princess Margaret Hospital Cancer Centre, Toronto, Ontario, CANADA
| | - Robert A Weersink
- Radiation Medicine Program, Princess Margaret Hospital Cancer Centre, Toronto, Ontario, CANADA
| |
Collapse
|
10
|
A Hybrid Method for Real-Time Bronchoscope Tracking Using Contour Registration and Synchronous EMT Data. IRANIAN JOURNAL OF RADIOLOGY 2019. [DOI: 10.5812/iranjradiol.66994] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
11
|
Luo X, Mori K, Peters TM. Advanced Endoscopic Navigation: Surgical Big Data, Methodology, and Applications. Annu Rev Biomed Eng 2018; 20:221-251. [PMID: 29505729 DOI: 10.1146/annurev-bioeng-062117-120917] [Citation(s) in RCA: 33] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/29/2022]
Abstract
Interventional endoscopy (e.g., bronchoscopy, colonoscopy, laparoscopy, cystoscopy) is a widely performed procedure that involves either diagnosis of suspicious lesions or guidance for minimally invasive surgery in a variety of organs within the body cavity. Endoscopy may also be used to guide the introduction of certain items (e.g., stents) into the body. Endoscopic navigation systems seek to integrate big data with multimodal information (e.g., computed tomography, magnetic resonance images, endoscopic video sequences, ultrasound images, external trackers) relative to the patient's anatomy, control the movement of medical endoscopes and surgical tools, and guide the surgeon's actions during endoscopic interventions. Nevertheless, it remains challenging to realize the next generation of context-aware navigated endoscopy. This review presents a broad survey of various aspects of endoscopic navigation, particularly with respect to the development of endoscopic navigation techniques. First, we investigate big data with multimodal information involved in endoscopic navigation. Next, we focus on numerous methodologies used for endoscopic navigation. We then review different endoscopic procedures in clinical applications. Finally, we discuss novel techniques and promising directions for the development of endoscopic navigation.
Collapse
Affiliation(s)
- Xiongbiao Luo
- Department of Computer Science, Fujian Key Laboratory of Computing and Sensing for Smart City, Xiamen University, Xiamen 361005, China;
| | - Kensaku Mori
- Department of Intelligent Systems, Graduate School of Informatics, Nagoya University, Nagoya 464-8601, Japan;
| | - Terry M Peters
- Robarts Research Institute, Western University, London, Ontario N6A 3K7, Canada;
| |
Collapse
|
12
|
Oda M, Kondo H, Kitasaka T, Furukawa K, Miyahara R, Hirooka Y, Goto H, Navab N, Mori K. Robust colonoscope tracking method for colon deformations utilizing coarse-to-fine correspondence findings. Int J Comput Assist Radiol Surg 2016; 12:39-50. [PMID: 27431209 DOI: 10.1007/s11548-016-1456-6] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2015] [Accepted: 07/01/2016] [Indexed: 01/06/2023]
Abstract
PURPOSE Polyps found during CT colonography can be removed by colonoscopic polypectomy. A colonoscope navigation system that navigates a physician to polyp positions while performing the colonoscopic polypectomy is required. Colonoscope tracking methods are essential for implementing colonoscope navigation systems. Previous colonoscope tracking methods have failed when the colon deforms during colonoscope insertions. This paper proposes a colonoscope tracking method that is robust against colon deformations. METHOD The proposed method generates a colon centerline from a CT volume and a curved line representing the colonoscope shape (colonoscope line) by using electromagnetic sensors. We find correspondences between points on a deformed colon centerline and colonoscope line by a landmark-based coarse correspondence finding and a length-based fine correspondence finding processes. Even if the coarse correspondence finding process fails to find some correspondences, which occurs with colon deformations, the fine correspondence finding process is able to find correct correspondences by using previously recorded line lengths. RESULT Experimental results using a colon phantom showed that the proposed method finds the colonoscope tip position with tracking errors smaller than 50 mm in most trials. A physician who specializes in gastroenterology commented that tracking errors smaller than 50 mm are acceptable. This is because polyps are observable from the colonoscope camera when positions of the colonoscope tip and polyps are closer than 50 mm. CONCLUSIONS We developed a colonoscope tracking method that is robust against deformations of the colon. Because the process was designed to consider colon deformations, the proposed method can track the colonoscope tip position even if the colon deforms.
Collapse
Affiliation(s)
- Masahiro Oda
- Graduate School of Information Science, Nagoya University, Furo-cho, Chikusa-ku, Nagoya, Aichi, Japan.
| | - Hiroaki Kondo
- Graduate School of Information Science, Nagoya University, Furo-cho, Chikusa-ku, Nagoya, Aichi, Japan
| | - Takayuki Kitasaka
- School of Information Science, Aichi Institute of Technology, 1247 Yachigusa, Yagusa-cho, Toyota, Aichi, Japan
| | - Kazuhiro Furukawa
- Department of Gastroenterology and Hepatology, Nagoya University Graduate School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya, Aichi, Japan
| | - Ryoji Miyahara
- Department of Gastroenterology and Hepatology, Nagoya University Graduate School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya, Aichi, Japan
| | - Yoshiki Hirooka
- Department of Endoscopy, Nagoya University Hospital, 65 Tsurumai-cho, Showa-ku, Nagoya, Aichi, Japan
| | - Hidemi Goto
- Department of Gastroenterology and Hepatology, Nagoya University Graduate School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya, Aichi, Japan
| | - Nassir Navab
- Technische Universität München, Boltzmannstr. 3, 85748, Garching bei München, Germany
| | - Kensaku Mori
- Information and Communications, Nagoya University, Furo-cho, Chikusa-ku, Nagoya, Aichi, Japan
| |
Collapse
|
13
|
Wang J, Suenaga H, Yang L, Kobayashi E, Sakuma I. Video see-through augmented reality for oral and maxillofacial surgery. Int J Med Robot 2016; 13. [PMID: 27283505 DOI: 10.1002/rcs.1754] [Citation(s) in RCA: 50] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2016] [Revised: 03/26/2016] [Accepted: 04/29/2016] [Indexed: 11/11/2022]
Abstract
BACKGROUND Oral and maxillofacial surgery has not been benefitting from image guidance techniques owing to the limitations in image registration. METHODS A real-time markerless image registration method is proposed by integrating a shape matching method into a 2D tracking framework. The image registration is performed by matching the patient's teeth model with intraoperative video to obtain its pose. The resulting pose is used to overlay relevant models from the same CT space on the camera video for augmented reality. RESULTS The proposed system was evaluated on mandible/maxilla phantoms, a volunteer and clinical data. Experimental results show that the target overlay error is about 1 mm, and the frame rate of registration update yields 3-5 frames per second with a 4 K camera. CONCLUSIONS The significance of this work lies in its simplicity in clinical setting and the seamless integration into the current medical procedure with satisfactory response time and overlay accuracy. Copyright © 2016 John Wiley & Sons, Ltd.
Collapse
Affiliation(s)
- Junchen Wang
- School of Mechanical Engineering and Automation, Beihang University, Beijing, China.,Graduate School of Engineering, The University of Tokyo, Tokyo, Japan
| | - Hideyuki Suenaga
- Department of Oral-Maxillofacial Surgery, Dentistry and Orthodontics, The University of Tokyo Hospital, Tokyo, Japan
| | - Liangjing Yang
- Graduate School of Engineering, The University of Tokyo, Tokyo, Japan
| | - Etsuko Kobayashi
- Graduate School of Engineering, The University of Tokyo, Tokyo, Japan
| | - Ichiro Sakuma
- Graduate School of Engineering, The University of Tokyo, Tokyo, Japan
| |
Collapse
|
14
|
Luo X, Wan Y, He X, Mori K. Observation-driven adaptive differential evolution and its application to accurate and smooth bronchoscope three-dimensional motion tracking. Med Image Anal 2015; 24:282-296. [DOI: 10.1016/j.media.2015.01.002] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2014] [Revised: 01/05/2015] [Accepted: 01/08/2015] [Indexed: 10/24/2022]
|
15
|
Gwo CY, Wei CH, Li Y, Chiu NH. Reconstruction of Banknote Fragments Based on Keypoint Matching Method. J Forensic Sci 2015; 60:906-13. [PMID: 25903323 DOI: 10.1111/1556-4029.12777] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2013] [Revised: 07/23/2014] [Accepted: 08/06/2014] [Indexed: 11/29/2022]
Abstract
Banknotes may be shredded by a scrap machine, ripped up by hand, or damaged in accidents. This study proposes an image registration method for reconstruction of multiple sheets of banknotes. The proposed method first constructs different scale spaces to identify keypoints in the underlying banknote fragments. Next, the features of those keypoints are extracted to represent their local patterns around keypoints. Then, similarity is computed to find the keypoint pairs between the fragment and the reference banknote. The banknote fragments can determine the coordinate and amend the orientation. Finally, an assembly strategy is proposed to piece multiple sheets of banknote fragments together. Experimental results show that the proposed method causes, on average, a deviation of 0.12457 ± 0.12810° for each fragment while the SIFT method deviates 1.16893 ± 2.35254° on average. The proposed method not only reconstructs the banknotes but also decreases the computing cost. Furthermore, the proposed method can estimate relatively precisely the orientation of the banknote fragments to assemble.
Collapse
Affiliation(s)
- Chih-Ying Gwo
- Department of Information Management, Chien Hsin University of Science and Technology, 229 Chien-Hsin Road, Taoyuan, 320, Taiwan
| | - Chia-Hung Wei
- Department of Information Management, Chien Hsin University of Science and Technology, 229 Chien-Hsin Road, Taoyuan, 320, Taiwan.,Graduate Institute of Biomedical Informatics, Taipei Medical University, 250 Wuxing Street, Taipei City, 110, Taiwan
| | - Yue Li
- College of Software, Nankai University, 94 Weijin Road, Nankai District, 300071, China
| | - Nan-Hsing Chiu
- Department of Information Management, Chien Hsin University of Science and Technology, 229 Chien-Hsin Road, Taoyuan, 320, Taiwan
| |
Collapse
|
16
|
Shen M, Giannarou S, Yang GZ. Robust camera localisation with depth reconstruction for bronchoscopic navigation. Int J Comput Assist Radiol Surg 2015; 10:801-13. [DOI: 10.1007/s11548-015-1197-y] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2015] [Accepted: 03/25/2015] [Indexed: 12/21/2022]
|
17
|
Luo X, Wan Y, He X. Robust electromagnetically guided endoscopic procedure using enhanced particle swarm optimization for multimodal information fusion. Med Phys 2015; 42:1808-17. [PMID: 25832071 DOI: 10.1118/1.4915285] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022] Open
Abstract
PURPOSE Electromagnetically guided endoscopic procedure, which aims at accurately and robustly localizing the endoscope, involves multimodal sensory information during interventions. However, it still remains challenging in how to integrate these information for precise and stable endoscopic guidance. To tackle such a challenge, this paper proposes a new framework on the basis of an enhanced particle swarm optimization method to effectively fuse these information for accurate and continuous endoscope localization. METHODS The authors use the particle swarm optimization method, which is one of stochastic evolutionary computation algorithms, to effectively fuse the multimodal information including preoperative information (i.e., computed tomography images) as a frame of reference, endoscopic camera videos, and positional sensor measurements (i.e., electromagnetic sensor outputs). Since the evolutionary computation method usually limits its possible premature convergence and evolutionary factors, the authors introduce the current (endoscopic camera and electromagnetic sensor's) observation to boost the particle swarm optimization and also adaptively update evolutionary parameters in accordance with spatial constraints and the current observation, resulting in advantageous performance in the enhanced algorithm. RESULTS The experimental results demonstrate that the authors' proposed method provides a more accurate and robust endoscopic guidance framework than state-of-the-art methods. The average guidance accuracy of the authors' framework was about 3.0 mm and 5.6° while the previous methods show at least 3.9 mm and 7.0°. The average position and orientation smoothness of their method was 1.0 mm and 1.6°, which is significantly better than the other methods at least with (2.0 mm and 2.6°). Additionally, the average visual quality of the endoscopic guidance was improved to 0.29. CONCLUSIONS A robust electromagnetically guided endoscopy framework was proposed on the basis of an enhanced particle swarm optimization method with using the current observation information and adaptive evolutionary factors. The authors proposed framework greatly reduced the guidance errors from (4.3, 7.8) to (3.0 mm, 5.6°), compared to state-of-the-art methods.
Collapse
Affiliation(s)
- Xiongbiao Luo
- Robarts Research Institute, Western University, London, Ontario N6A 5K8, Canada
| | - Ying Wan
- School of Computing and Communications, University of Technology, Sydney, New South Wales 2007, Australia
| | - Xiangjian He
- School of Computing and Communications, University of Technology, Sydney, New South Wales 2007, Australia
| |
Collapse
|
18
|
Luo X, Mori K. A discriminative structural similarity measure and its application to video-volume registration for endoscope three-dimensional motion tracking. IEEE TRANSACTIONS ON MEDICAL IMAGING 2014; 33:1248-1261. [PMID: 24893255 DOI: 10.1109/tmi.2014.2307052] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
Endoscope 3-D motion tracking, which seeks to synchronize pre- and intra-operative images in endoscopic interventions, is usually performed as video-volume registration that optimizes the similarity between endoscopic video and pre-operative images. The tracking performance, in turn, depends significantly on whether a similarity measure can successfully characterize the difference between video sequences and volume rendering images driven by pre-operative images. The paper proposes a discriminative structural similarity measure, which uses the degradation of structural information and takes image correlation or structure, luminance, and contrast into consideration, to boost video-volume registration. By applying the proposed similarity measure to endoscope tracking, it was demonstrated to be more accurate and robust than several available similarity measures, e.g., local normalized cross correlation, normalized mutual information, modified mean square error, or normalized sum squared difference. Based on clinical data evaluation, the tracking error was reduced significantly from at least 14.6 mm to 4.5 mm. The processing time was accelerated more than 30 frames per second using graphics processing unit.
Collapse
|
19
|
Xiongbiao Luo, Mori K. Robust Endoscope Motion Estimation Via an Animated Particle Filter for Electromagnetically Navigated Endoscopy. IEEE Trans Biomed Eng 2014; 61:85-95. [DOI: 10.1109/tbme.2013.2277609] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
20
|
Gibbs JD, Graham MW, Bascom R, Cornish DC, Khare R, Higgins WE. Optimal procedure planning and guidance system for peripheral bronchoscopy. IEEE Trans Biomed Eng 2013; 61:638-57. [PMID: 24235246 DOI: 10.1109/tbme.2013.2285627] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/29/2022]
Abstract
With the development of multidetector computed-tomography (MDCT) scanners and ultrathin bronchoscopes, the use of bronchoscopy for diagnosing peripheral lung-cancer nodules is becoming a viable option. The work flow for assessing lung cancer consists of two phases: 1) 3-D MDCT analysis and 2) live bronchoscopy. Unfortunately, the yield rates for peripheral bronchoscopy have been reported to be as low as 14%, and bronchoscopy performance varies considerably between physicians. Recently, proposed image-guided systems have shown promise for assisting with peripheral bronchoscopy. Yet, MDCT-based route planning to target sites has relied on tedious error-prone techniques. In addition, route planning tends not to incorporate known anatomical, device, and procedural constraints that impact a feasible route. Finally, existing systems do not effectively integrate MDCT-derived route information into the live guidance process. We propose a system that incorporates an automatic optimal route-planning method, which integrates known route constraints. Furthermore, our system offers a natural translation of the MDCT-based route plan into the live guidance strategy via MDCT/video data fusion. An image-based study demonstrates the route-planning method's functionality. Next, we present a prospective lung-cancer patient study in which our system achieved a successful navigation rate of 91% to target sites. Furthermore, when compared to a competing commercial system, our system enabled bronchoscopy over two airways deeper into the airway-tree periphery with a sample time that was nearly 2 min shorter on average. Finally, our system's ability to almost perfectly predict the depth of a bronchoscope's navigable route in advance represents a substantial benefit of optimal route planning.
Collapse
|
21
|
Bell CS, Obstein KL, Valdastri P. Image partitioning and illumination in image-based pose detection for teleoperated flexible endoscopes. Artif Intell Med 2013; 59:185-96. [PMID: 24188575 DOI: 10.1016/j.artmed.2013.09.002] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2012] [Revised: 09/20/2013] [Accepted: 09/23/2013] [Indexed: 02/07/2023]
Abstract
OBJECTIVE Colorectal cancer is one of the leading causes of cancer-related deaths in the world, although it can be effectively treated if detected early. Teleoperated flexible endoscopes are an emerging technology to ease patient apprehension about the procedure, and subsequently increase compliance. Essential to teleoperation is robust feedback reflecting the change in pose (i.e., position and orientation) of the tip of the endoscope. The goal of this study is to first describe a novel image-based tracking system for teleoperated flexible endoscopes, and subsequently determine its viability in a clinical setting. The proposed approach leverages artificial neural networks (ANNs) to learn the mapping that links the optical flow between two sequential images to the change in the pose of the camera. Secondly, the study investigates for the first time how narrow band illumination (NBI) - today available in commercial gastrointestinal endoscopes - can be applied to enhance feature extraction, and quantify the effect of NBI and white light illumination (WLI), as well as their color information, on the strength of features extracted from the endoscopic camera stream. METHODS AND MATERIALS In order to provide the best features for the neural networks to learn the change in pose based on the image stream, we investigated two different imaging modalities - WLI and NBI - and we applied two different spatial partitions - lumen-centered and grid-based - to create descriptors used as input to the ANNs. An experiment was performed to compare the error of these four variations, measured in root mean square error (RMSE) from ground truth given by a robotic arm, to that of a commercial state-of-the-art magnetic tracker. The viability of this technique for a clinical setting was then tested using the four ANN variations, a magnetic tracker, and a commercial colonoscope. The trial was performed by an expert endoscopist (>2000 lifetime procedures) on a colonoscopy training model with porcine blood, and the RMSE of the ANN output was calculated with respect to the magnetic tracker readings. Using the image stream obtained from the commercial endoscope, the strength of features extracted was evaluated. RESULTS In the first experiment, the best ANNs resulted from grid-based partitioning under WLI (2.42mm RMSE) for position, and from lumen-centered partitioning under NBI (1.69° RMSE) for rotation. By comparison, the performance of the tracker was 2.49mm RMSE in position and 0.89° RMSE in rotation. The trial with the commercial endoscope indicated that lumen-centered partitioning was the best overall, while NBI outperformed WLI in terms of illumination modality. The performance of lumen-centered partitioning with NBI was 1.03±0.8mm RMSE in positional degrees of freedom (DOF), and 1.26±0.98° RMSE in rotational DOF, while with WLI, the performance was 1.56±1.15mm RMSE in positional DOF and 2.45±1.90° RMSE in rotational DOF. Finally, the features extracted under NBI were found to be twice as strong as those extracted under WLI, but no significance in feature strengths was observed between a grayscale version of the image, and the red, blue, and green color channels. CONCLUSIONS This work demonstrates that both WLI and NBI, combined with feature partitioning based on the anatomy of the colon, provide valid mechanisms for endoscopic camera pose estimation via image stream. Illumination provided by WLI and NBI produce ANNs with similar performance which are comparable to that of a state-of-the-art magnetic tracker. However, NBI produces features that are stronger than WLI, which enables more robust feature tracking, and better performance of the ANN in terms of accuracy. Thus, NBI with lumen-centered partitioning resulted the best approach among the different variations tested for vision-based pose estimation. The proposed approach takes advantage of components already available in commercial gastrointestinal endoscopes to provide accurate feedback about the motion of the tip of the endoscope. This solution may serve as an enabling technology for closed-loop control of teleoperated flexible endoscopes.
Collapse
Affiliation(s)
- Charreau S Bell
- Department of Mechanical Engineering, Vanderbilt University, 2301 Vanderbilt Place, PMB 351826, Nashville, TN 37235-1826, USA.
| | | | | |
Collapse
|
22
|
Luo X, Kitasaka T, Mori K. Externally navigated bronchoscopy using 2-D motion sensors: dynamic phantom validation. IEEE TRANSACTIONS ON MEDICAL IMAGING 2013; 32:1745-1764. [PMID: 23686944 DOI: 10.1109/tmi.2013.2263152] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/02/2023]
Abstract
The paper presents a new endoscope motion tracking method that is based on a novel external endoscope tracking device and our modified stochastic optimization method for boosting endoscopy navigation. We designed a novel tracking prototype where a 2-D motion sensor was introduced to directly measure the insertion-retreat linear motion and also the rotation of the endoscope. With our developed stochastic optimization method, which embeds traceable particle swarm optimization in the Condensation algorithm, a full six degrees-of-freedom endoscope pose (position and orientation) can be recovered from 2-D motion sensor measurements. Experiments were performed on a dynamic bronchial phantom with maximal simulated respiratory motion around 24.0 mm. The experimental results demonstrate that our proposed method provides a promising endoscope motion tracking approach with more effective and robust performance than several current available tracking techniques. The average tracking accuracy of the position improved from 6.5 to 3.3 mm, which further approaches the clinical requirement of 2.0 mm in practice.
Collapse
|
23
|
Merritt SA, Khare R, Bascom R, Higgins WE. Interactive CT-video registration for the continuous guidance of bronchoscopy. IEEE TRANSACTIONS ON MEDICAL IMAGING 2013; 32:1376-96. [PMID: 23508260 PMCID: PMC3911781 DOI: 10.1109/tmi.2013.2252361] [Citation(s) in RCA: 33] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/01/2023]
Abstract
Bronchoscopy is a major step in lung cancer staging. To perform bronchoscopy, the physician uses a procedure plan, derived from a patient's 3D computed-tomography (CT) chest scan, to navigate the bronchoscope through the lung airways. Unfortunately, physicians vary greatly in their ability to perform bronchoscopy. As a result, image-guided bronchoscopy systems, drawing upon the concept of CT-based virtual bronchoscopy (VB), have been proposed. These systems attempt to register the bronchoscope's live position within the chest to a CT-based virtual chest space. Recent methods, which register the bronchoscopic video to CT-based endoluminal airway renderings, show promise but do not enable continuous real-time guidance. We present a CT-video registration method inspired by computer-vision innovations in the fields of image alignment and image-based rendering. In particular, motivated by the Lucas-Kanade algorithm, we propose an inverse-compositional framework built around a gradient-based optimization procedure. We next propose an implementation of the framework suitable for image-guided bronchoscopy. Laboratory tests, involving both single frames and continuous video sequences, demonstrate the robustness and accuracy of the method. Benchmark timing tests indicate that the method can run continuously at 300 frames/s, well beyond the real-time bronchoscopic video rate of 30 frames/s. This compares extremely favorably to the ≥ 1 s/frame speeds of other methods and indicates the method's potential for real-time continuous registration. A human phantom study confirms the method's efficacy for real-time guidance in a controlled setting, and, hence, points the way toward the first interactive CT-video registration approach for image-guided bronchoscopy. Along this line, we demonstrate the method's efficacy in a complete guidance system by presenting a clinical study involving lung cancer patients.
Collapse
Affiliation(s)
| | - Rahul Khare
- Sheikh Zayed Institute at the Childrens National Medical Center, Washington, DC 20010 USA
| | - Rebecca Bascom
- Department of Medicine, Pennsylvania State Hershey Medical Center, Hershey, PA 17033 USA
| | - William E. Higgins
- Departments of Electrical Engineering, Computer Science and Engineering, and Bioengineering, Pennsylvania State University, University Park, PA 16802 USA
| |
Collapse
|
24
|
Towards automated visual flexible endoscope navigation. Surg Endosc 2013; 27:3539-47. [PMID: 23670745 DOI: 10.1007/s00464-013-3003-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2013] [Accepted: 04/26/2013] [Indexed: 10/26/2022]
Abstract
BACKGROUND The design of flexible endoscopes has not changed significantly in the past 50 years. A trend is observed towards a wider application of flexible endoscopes with an increasing role in complex intraluminal therapeutic procedures. The nonintuitive and nonergonomical steering mechanism now forms a barrier in the extension of flexible endoscope applications. Automating the navigation of endoscopes could be a solution for this problem. This paper summarizes the current state of the art in image-based navigation algorithms. The objectives are to find the most promising navigation system(s) to date and to indicate fields for further research. METHODS A systematic literature search was performed using three general search terms in two medical-technological literature databases. Papers were included according to the inclusion criteria. A total of 135 papers were analyzed. Ultimately, 26 were included. RESULTS Navigation often is based on visual information, which means steering the endoscope using the images that the endoscope produces. Two main techniques are described: lumen centralization and visual odometry. Although the research results are promising, no successful, commercially available automated flexible endoscopy system exists to date. CONCLUSIONS Automated systems that employ conventional flexible endoscopes show the most promising prospects in terms of cost and applicability. To produce such a system, the research focus should lie on finding low-cost mechatronics and technologically robust steering algorithms. Additional functionality and increased efficiency can be obtained through software development. The first priority is to find real-time, robust steering algorithms. These algorithms need to handle bubbles, motion blur, and other image artifacts without disrupting the steering process.
Collapse
|
25
|
Reichl T, Luo X, Menzel M, Hautmann H, Mori K, Navab N. Hybrid electromagnetic and image-based tracking of endoscopes with guaranteed smooth output. Int J Comput Assist Radiol Surg 2013; 8:955-65. [PMID: 23543323 DOI: 10.1007/s11548-013-0835-5] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2012] [Accepted: 03/16/2013] [Indexed: 10/27/2022]
Abstract
PURPOSE Flexible fiber-optic bronchoscopy is a widespread medical procedure for the diagnosis and treatment of lung diseases. Navigation systems are needed to track the flexible endoscope within the bronchial tree. Electromagnetic (EM) tracking is currently the only technology used clinically for this purpose. The registration between EM tracking and patient anatomy may become inaccurate due to breathing motion, so the addition of image-based tracking has been proposed as a hybrid EM-image-based system. METHODS When EM tracking is used as an initialization for image registration, small changes in the initialization may lead to different local minima and noise is amplified by hybrid tracking. The tracking output is modeled as continuous and uses splines for interpolation, thus smoothness is greatly improved. The bronchoscope pose relative to computed tomography data is interpolated using Catmull-Rom splines for position and spherical linear interpolation (SLERP) for orientation. RESULTS The hybrid method was evaluated using ground truth poses manually selected by experts, where mean inter-expert agreement was determined as 1.26 mm. Using four dynamic phantom data sets, the accuracy was 4.91 mm, which is equivalent to previous methods. Compared to state-of-art methods, inter-frame smoothness was improved from 2.77-3.72 to 1.24 mm. CONCLUSIONS Hybrid image and electromagnetic endoscope guidance provides a more realistic and physically plausible solution with significantly less jitter. This quantitative result is confirmed by visual comparison of real and virtual video, where the virtual video output is much more consistent and robust, with fewer occasions of tracking loss or unexpected movement compared with previous methods.
Collapse
Affiliation(s)
- Tobias Reichl
- Computer-Aided Medical Procedures, TUM, Munich, Germany,
| | | | | | | | | | | |
Collapse
|
26
|
Luó X, Feuerstein M, Deguchi D, Kitasaka T, Takabatake H, Mori K. Development and comparison of new hybrid motion tracking for bronchoscopic navigation. Med Image Anal 2012; 16:577-96. [DOI: 10.1016/j.media.2010.11.001] [Citation(s) in RCA: 29] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2009] [Revised: 08/10/2010] [Accepted: 11/13/2010] [Indexed: 11/29/2022]
|
27
|
Gergel I, Hering J, Tetzlaff R, Meinzer HP, Wegner I. An electromagnetic navigation system for transbronchial interventions with a novel approach to respiratory motion compensation. Med Phys 2011; 38:6742-53. [DOI: 10.1118/1.3662871] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022] Open
|
28
|
Lederman D, Lampotang S, Shamir MY. Automatic endotracheal tube position confirmation system based on image classification – A preliminary assessment. Med Eng Phys 2011; 33:1017-26. [DOI: 10.1016/j.medengphy.2011.04.006] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2010] [Revised: 03/28/2011] [Accepted: 04/12/2011] [Indexed: 10/18/2022]
|
29
|
Robust bronchoscope motion tracking using sequential Monte Carlo methods in navigated bronchoscopy: dynamic phantom and patient validation. Int J Comput Assist Radiol Surg 2011; 7:371-87. [DOI: 10.1007/s11548-011-0645-6] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2011] [Accepted: 07/06/2011] [Indexed: 10/18/2022]
|
30
|
An endotracheal intubation confirmation system based on carina image detection: a proof of concept. Med Biol Eng Comput 2010; 49:75-83. [PMID: 20878551 DOI: 10.1007/s11517-010-0680-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2010] [Accepted: 09/11/2010] [Indexed: 10/19/2022]
Abstract
In this paper, a novel system for automatic confirmation of endotracheal intubation is proposed. The system comprises a miniature CMOS sensor and electric wires attached to a rigid stylet. Video signals are continuously acquired and processed by the algorithm implemented on a PC/DSP. The system is based on detection of the carina image as an anatomical landmark of correct tube positioning and it thus utilizes direct visual cues. Detection of the carina is performed based on unsupervised clustering, using a greedy-Gaussian mixture framework. The performance of the proposed system was initially evaluated using a mannequin model. A scientific prototype was assembled and used to perform repeated intubations on the model and collect a database of video signals which were processed off-line. The videos were categoried by a medical professional into carina, upper-trachea, and esophagus. An accuracy of 100% was achieved in discriminating between the carina and other anatomical structures including esophagus and upper-trachea. As an additional validation, the system was tested using a dataset of 231 video images recorded from five human subjects during intubation. The system correctly classified 120 out of 125 non-carina images (i.e. a sensitivity of 96.0%), and 100 out of 106 carina images (i.e. a specificity 94.3%). Using a 10th-order median filter, applied on the frame-based classification results, a 100% accuracy rate was obtained.
Collapse
|
31
|
Lederman D. Endotracheal intubation confirmation based on video image classification using a parallel GMMs framework: a preliminary evaluation. Ann Biomed Eng 2010; 39:508-16. [PMID: 20878236 DOI: 10.1007/s10439-010-0172-6] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2010] [Accepted: 09/17/2010] [Indexed: 11/26/2022]
Abstract
In this paper, the problem of endotracheal intubation confirmation is addressed. Endotracheal intubation is a complex procedure which requires high skills and the use of secondary confirmation devices to ensure correct positioning of the tube. A novel confirmation approach, based on video images classification, is introduced. The approach is based on identification of specific anatomical landmarks, including esophagus, upper trachea and main bifurcation of the trachea into the two primary bronchi ("carina"), as indicators of correct or incorrect tube insertion and positioning. Classification of the images is performed using a parallel Gaussian mixture models (GMMs) framework, which is composed of several GMMs, schematically connected in parallel, where each GMM represents a different imaging angle. The performance of the proposed approach was evaluated using a dataset of cow-intubation videos and a dataset of human-intubation videos. Each one of the video images was manually (visually) classified by a medical expert into one of three categories: upper-tracheal intubation, correct (carina) intubation, and esophageal intubation. The image classification algorithm was applied off-line using a leave-one-case-out method. The results show that the system correctly classified 1517 out of 1600 (94.8%) of the cow-intubation images, and 340 out of the 358 human images (95.0%). The classification results compared favorably with a "standard" GMM approach utilizing textural based features, as well as with a state-of-the-art classification method, tested on the cow-intubation dataset.
Collapse
Affiliation(s)
- Dror Lederman
- Department of Radiology, University of Pittsburgh School of Medicine, 3362 Fifth Avenue, Pittsburgh, PA 15213, USA.
| |
Collapse
|