A multimodal image guiding system for Navigated Ultrasound Bronchoscopy (EBUS): A human feasibility study

Navigated ultrasound bronchoscopy with integrated positron emission tomography-A human feasibility study

BACKGROUND AND OBJECTIVE

Patients suspected to have lung cancer, undergo endobronchial ultrasound bronchoscopy (EBUS) for the purpose of diagnosis and staging. For presumptive curable patients, the EBUS bronchoscopy is planned based on images and data from computed tomography (CT) images and positron emission tomography (PET). Our study aimed to evaluate the feasibility of a multimodal electromagnetic navigation platform for EBUS bronchoscopy, integrating ultrasound and segmented CT, and PET scan imaging data.

METHODS

The proof-of-concept study included patients with suspected lung cancer and pathological mediastinal/hilar lymph nodes identified on both CT and PET scans. Images obtained from these two modalities were segmented to delineate target lymph nodes and then incorporated into the CustusX navigation platform. The EBUS bronchoscope was equipped with a sensor, calibrated, and affixed to a 3D printed click-on device positioned at the bronchoscope's tip. Navigation accuracy was measured postoperatively using ultrasound recordings.

RESULTS

The study enrolled three patients, all presenting with suspected mediastinal lymph node metastasis (N1-3). All PET-positive lymph nodes were displayed in the navigation platform during the EBUS procedures. In total, five distinct lymph nodes were sampled, yielding malignant cells from three nodes and lymphocytes from the remaining two. The median accuracy of the navigation system was 7.7 mm.

CONCLUSION

Our study introduces a feasible multimodal electromagnetic navigation platform that combines intraoperative ultrasound with preoperative segmented CT and PET imaging data for EBUS lymph node staging examinations. This innovative approach holds promise for enhancing the accuracy and effectiveness of EBUS procedures.

Automatic Segmentation of Mediastinal Lymph Nodes and Blood Vessels in Endobronchial Ultrasound (EBUS) Images Using Deep Learning

Endobronchial ultrasound (EBUS) is used in the minimally invasive sampling of thoracic lymph nodes. In lung cancer staging, the accurate assessment of mediastinal structures is essential but challenged by variations in anatomy, image quality, and operator-dependent image interpretation. This study aimed to automatically detect and segment mediastinal lymph nodes and blood vessels employing a novel U-Net architecture-based approach in EBUS images. A total of 1161 EBUS images from 40 patients were annotated. For training and validation, 882 images from 30 patients and 145 images from 5 patients were utilized. A separate set of 134 images was reserved for testing. For lymph node and blood vessel segmentation, the mean ± standard deviation (SD) values of the Dice similarity coefficient were 0.71 ± 0.35 and 0.76 ± 0.38, those of the precision were 0.69 ± 0.36 and 0.82 ± 0.22, those of the sensitivity were 0.71 ± 0.38 and 0.80 ± 0.25, those of the specificity were 0.98 ± 0.02 and 0.99 ± 0.01, and those of the F1 score were 0.85 ± 0.16 and 0.81 ± 0.21, respectively. The average processing and segmentation run-time per image was 55 ± 1 ms (mean ± SD). The new U-Net architecture-based approach (EBUS-AI) could automatically detect and segment mediastinal lymph nodes and blood vessels in EBUS images. The method performed well and was feasible and fast, enabling real-time automatic labeling.

Multimodal Registration for Image-Guided EBUS Bronchoscopy

The state-of-the-art procedure for examining the lymph nodes in a lung cancer patient involves using an endobronchial ultrasound (EBUS) bronchoscope. The EBUS bronchoscope integrates two modalities into one device: (1) videobronchoscopy, which gives video images of the airway walls; and (2) convex-probe EBUS, which gives 2D fan-shaped views of extraluminal structures situated outside the airways. During the procedure, the physician first employs videobronchoscopy to navigate the device through the airways. Next, upon reaching a given node’s approximate vicinity, the physician probes the airway walls using EBUS to localize the node. Due to the fact that lymph nodes lie beyond the airways, EBUS is essential for confirming a node’s location. Unfortunately, it is well-documented that EBUS is difficult to use. In addition, while new image-guided bronchoscopy systems provide effective guidance for videobronchoscopic navigation, they offer no assistance for guiding EBUS localization. We propose a method for registering a patient’s chest CT scan to live surgical EBUS views, thereby facilitating accurate image-guided EBUS bronchoscopy. The method entails an optimization process that registers CT-based virtual EBUS views to live EBUS probe views. Results using lung cancer patient data show that the method correctly registered 28/28 (100%) lymph nodes scanned by EBUS, with a mean registration time of 3.4 s. In addition, the mean position and direction errors of registered sites were 2.2 mm and 11.8∘, respectively. In addition, sensitivity studies show the method’s robustness to parameter variations. Lastly, we demonstrate the method’s use in an image-guided system designed for guiding both phases of EBUS bronchoscopy.

A personalized image-guided intervention system for peripheral lung cancer on patient-specific respiratory motion model

PURPOSE

Due to respiratory motion, precise tracking of lung nodule movement is a persistent challenge for guiding percutaneous lung biopsy during image-guided intervention. We developed an automated image-guided system incorporating effective and robust tracking algorithms to address this challenge. Accurate lung motion prediction and personalized image-guided intervention are the key technological contributions of this work.

METHODS

A patient-specific respiratory motion model is developed to predict pulmonary movements of individual patients. It is based on the relation between the artificial 4D CT and corresponding positions tracked by position sensors attached on the chest using an electromagnetic (EM) tracking system. The 4D CT image of the thorax during breathing is calculated through deformable registration of two 3D CT scans acquired at inspiratory and expiratory breath-hold. The robustness and accuracy of the image-guided intervention system were assessed on a static thorax phantom under different clinical parametric combinations.

RESULTS

Real 4D CT images of ten patients were used to evaluate the accuracy of the respiratory motion model. The mean error of the model in different breathing phases was 1.59 ± 0.66 mm. Using a static thorax phantom, we achieved an average targeting accuracy of 3.18 ± 1.2 mm across 50 independent tests with different intervention parameters. The positive results demonstrate the robustness and accuracy of our system for personalized lung cancer intervention.

CONCLUSIONS

The proposed system integrates a patient-specific respiratory motion compensation model to reduce the effect of respiratory motion during percutaneous lung biopsy and help interventional radiologists target the lesion efficiently. Our preclinical studies indicate that the image-guided system has the ability to accurately predict and track lung nodules of individual patients and has the potential for use in the diagnosis and treatment of early stage lung cancer.

Image-guided EBUS bronchoscopy system for lung-cancer staging

The staging of the central-chest lymph nodes is a major step in the management of lung-cancer patients. For this purpose, the physician uses a device that integrates videobronchoscopy and an endobronchial ultrasound (EBUS) probe. To biopsy a lymph node, the physician first uses videobronchoscopy to navigate through the airways and then invokes EBUS to localize and biopsy the node. Unfortunately, this process proves difficult for many physicians, with the choice of biopsy site found by trial and error. We present a complete image-guided EBUS bronchoscopy system tailored to lymph-node staging. The system accepts a patient’s 3D chest CT scan, an optional PET scan, and the EBUS bronchoscope’s video sources as inputs. System workflow follows two phases: (1) procedure planning and (2) image-guided EBUS bronchoscopy. Procedure planning derives airway guidance routes that facilitate optimal EBUS scanning and nodal biopsy. During the live procedure, the system’s graphical display suggests a series of device maneuvers to perform and provides multimodal visual cues for locating suitable biopsy sites. To this end, the system exploits data fusion to drive a multimodal virtual bronchoscope and other visualization tools that lead the physician through the process of device navigation and localization. A retrospective lung-cancer patient study and follow-on prospective patient study, performed within the standard clinical workflow, demonstrate the system’s feasibility and functionality. For the prospective study, 60/60 selected lymph nodes (100%) were correctly localized using the system, and 30/33 biopsied nodes (91%) gave adequate tissue samples. Also, the mean procedure time including all user interactions was 6 min 43 s All of these measures improve upon benchmarks reported for other state-of-the-art systems and current practice. Overall, the system enabled safe, efficient EBUS-based localization and biopsy of lymph nodes.

The Essential Role of Open Data and Software for the Future of Ultrasound-Based Neuronavigation

With the recent developments in machine learning and modern graphics processing units (GPUs), there is a marked shift in the way intra-operative ultrasound (iUS) images can be processed and presented during surgery. Real-time processing of images to highlight important anatomical structures combined with in-situ display, has the potential to greatly facilitate the acquisition and interpretation of iUS images when guiding an operation. In order to take full advantage of the recent advances in machine learning, large amounts of high-quality annotated training data are necessary to develop and validate the algorithms. To ensure efficient collection of a sufficient number of patient images and external validity of the models, training data should be collected at several centers by different neurosurgeons, and stored in a standard format directly compatible with the most commonly used machine learning toolkits and libraries. In this paper, we argue that such effort to collect and organize large-scale multi-center datasets should be based on common open source software and databases. We first describe the development of existing open-source ultrasound based neuronavigation systems and how these systems have contributed to enhanced neurosurgical guidance over the last 15 years. We review the impact of the large number of projects worldwide that have benefited from the publicly available datasets “Brain Images of Tumors for Evaluation” (BITE) and “Retrospective evaluation of Cerebral Tumors” (RESECT) that include MR and US data from brain tumor cases. We also describe the need for continuous data collection and how this effort can be organized through the use of a well-adapted and user-friendly open-source software platform that integrates both continually improved guidance and automated data collection functionalities.

Passive Magnetic-Flux-Concentrator Based Electromagnetic Targeting System for Endobronchoscopy

In this paper, we demonstrate an innovative electromagnetic targeting system utilizing a passive magnetic-flux-concentrator for tracking endobronchoscope used in the diagnosis process of lung cancer tumors/lesions. The system consists of a magnetic-flux emitting coil, a magnetic-flux receiving electromagnets-array, and high permeability silicon-steel sheets rolled as a collar (as the passive magnetic-flux-concentrator) fixed in a guide sheath of an endobronchoscope. The emitting coil is used to produce AC magnetic-flux, which is consequently received by the receiving electromagnets-array. Due to the electromagnetic-induction, a voltage is induced in the receiving electromagnets-array. When the endobronchoscope’s guide sheath (with the silicon-steel collar) travels between the emitting coil and the receiving electromagnets-arrays, the magnetic flux is concentrated by the silicon-steel collar and thereby the induced voltage is changed. Through analyzing the voltage–pattern change, the location of the silicon–steel collar with the guide sheath is targeted. For testing, a bronchial-tree model for training medical doctors and operators is used to test our system. According to experimental results, the system is successfully verified to be able to target the endobronchoscope in the bronchial-tree model. The targeting errors on the x-, y- and z-axes are 9 mm, 10 mm, and 5 mm, respectively.

Optimal route planning for image-guided EBUS bronchoscopy

The staging of the central-chest lymph nodes is a major lung-cancer management procedure. To perform a staging procedure, the physician first uses a patient's 3D X-ray computed-tomography (CT) chest scan to interactively plan airway routes leading to selected target lymph nodes. Next, using an integrated EBUS bronchoscope (EBUS = endobronchial ultrasound), the physician uses videobronchoscopy to navigate through the airways toward a target node's general vicinity and then invokes EBUS to localize the node for biopsy. Unfortunately, during the procedure, the physician has difficulty in translating the preplanned airway routes into safe, effective biopsy sites. We propose an automatic route-planning method for EBUS bronchoscopy that gives optimal localization of safe, effective nodal biopsy sites. To run the method, a 3D chest model is first computed from a patient's chest CT scan. Next, an optimization method derives feasible airway routes that enables maximal tissue sampling of target lymph nodes while safely avoiding major blood vessels. In a lung-cancer patient study entailing 31 nodes (long axis range: [9.0 mm, 44.5 mm]), 25/31 nodes yielded safe airway routes having an optimal tissue sample size = 8.4 mm (range: [1.0 mm, 18.6 mm]) and sample adequacy = 0.42 (range: [0.05, 0.93]). Quantitative results indicate that the method potentially enables successful biopsies in essentially 100% of selected lymph nodes versus the 70-94% success rate of other approaches. The method also potentially facilitates adequate tissue biopsies for nearly 100% of selected nodes, as opposed to the 55-77% tissue adequacy rates of standard methods. The remaining nodes did not yield a safe route within the preset safety-margin constraints, with 3 nodes never yielding a route even under the most lenient safety-margin conditions. Thus, the method not only helps determine effective airway routes and expected sample quality for nodal biopsy, but it also helps point out situations where biopsy may not be advisable. We also demonstrate the methodology in an image-guided EBUS bronchoscopy system, used successfully in live lung-cancer patient studies. During a live procedure, the method provides dynamic real-time sample size visualization in an enhanced virtual bronchoscopy viewer. In this way, the physician vividly sees the most promising biopsy sites along the airway walls as the bronchoscope moves through the airways.

Devices for image-guided lung interventions: State-of-the-art review

Lung cancer is the leading cause of cancer-related death. According to the American Cancer Society, there were an estimated 222,500 new cases of lung cancer and 155,870 deaths from lung cancer in the United States in 2017. Accurate localization in lung interventions is one of the keys to reducing the death rate from lung cancer. In this study, a total of 217 publications from 2006 to 2017 about designs of medical devices for localization in lung interventions were screened, shortlisted, and categorized by localization principle and reviewed for functionality. Each study was analyzed for engineering characteristics and clinical significance. Research regarding interventional imaging equipment, navigation systems, and surgical devices was reviewed, and both research prototypes and commercial products were discussed. Finally, the future directions and existing challenges were summarized, including real-time intra-procedure guidance, accuracy of localization, clinical application, clinical adoptability, and clinical regulatory issues.

Vessel-based rigid registration for endovascular therapy of the abdominal aorta

BACKGROUND

Combining electromagnetic tracking of instruments with preoperatively acquired images can provide detailed visualization for intraoperative guidance and reduce the need for fluoroscopy and contrast. In this study, we investigated the accuracy of a vessel-based registration method designed for matching preoperative image and electromagnetically tracked positions for endovascular therapy.

MATERIAL AND METHODS

An open-source registration method was used to match the centerline extracted from computed tomography (CT) to electromagnetically tracked positions within a vascular phantom representing the abdominal aorta with bifurcations. The target registration error (TRE) was calculated for 11 fiducials distributed over the phantom. Median and intra-quartile range (IQR) for 30 registrations was reported. TRE < 5 mm was claimed sufficient for endovascular navigation, evaluated using the Wilcoxon signed-rank test. TRE was also compared to a 3D-3D registration method based on intraoperative cone-beam CT, using the Mann-Whitney U-test.

RESULTS

The TRE was 3.75 (IQR: 3.48-3.99) mm for the centerline registration algorithm and 3.21 (IQR: 1.50-3.57) mm for the 3D-3D method (p < .001). For both methods, the TRE was significantly < 5 mm (p < .001).

CONCLUSION

The centerline registration method was feasible, with an accuracy sufficient for navigation in endovascular therapy. The centerline method avoids additional image acquisition for registration purpose only.