1
|
Zeng B, Wang H, Tao X, Shi H, Joskowicz L, Chen X. A bidirectional framework for fracture simulation and deformation-based restoration prediction in pelvic fracture surgical planning. Med Image Anal 2024; 97:103267. [PMID: 39053167 DOI: 10.1016/j.media.2024.103267] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2024] [Revised: 07/05/2024] [Accepted: 07/05/2024] [Indexed: 07/27/2024]
Abstract
Pelvic fracture is a severe trauma with life-threatening implications. Surgical reduction is essential for restoring the anatomical structure and functional integrity of the pelvis, requiring accurate preoperative planning. However, the complexity of pelvic fractures and limited data availability necessitate labor-intensive manual corrections in a clinical setting. We describe in this paper a novel bidirectional framework for automatic pelvic fracture surgical planning based on fracture simulation and structure restoration. Our fracture simulation method accounts for patient-specific pelvic structures, bone density information, and the randomness of fractures, enabling the generation of various types of fracture cases from healthy pelvises. Based on these features and on adversarial learning, we develop a novel structure restoration network to predict the deformation mapping in CT images before and after a fracture for the precise structural reconstruction of any fracture. Furthermore, a self-supervised strategy based on pelvic anatomical symmetry priors is developed to optimize the details of the restored pelvic structure. Finally, the restored pelvis is used as a template to generate a surgical reduction plan in which the fragments are repositioned in an efficient jigsaw puzzle registration manner. Extensive experiments on simulated and clinical datasets, including scans with metal artifacts, show that our method achieves good accuracy and robustness: a mean SSIM of 90.7% for restorations, with translational errors of 2.88 mm and rotational errors of 3.18°for reductions in real datasets. Our method takes 52.9 s to complete the surgical planning in the phantom study, representing a significant acceleration compared to standard clinical workflows. Our method may facilitate effective surgical planning for pelvic fractures tailored to individual patients in clinical settings.
Collapse
Affiliation(s)
- Bolun Zeng
- Institute of Biomedical Manufacturing and Life Quality Engineering, School of Mechanical Engineering, Shanghai Jiao Tong University, China
| | - Huixiang Wang
- Department of Orthopedics, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Xingguang Tao
- Department of Orthopedics, Qingpu Branch of Zhongshan Hospital Affiliated to Fudan University, Shanghai, China
| | - Haochen Shi
- Institute of Biomedical Manufacturing and Life Quality Engineering, School of Mechanical Engineering, Shanghai Jiao Tong University, China
| | - Leo Joskowicz
- School of Computer Science and Engineering and the Edmond and Lily Safra Center for Brain Sciences, Hebrew University of Jerusalem, Jerusalem, Israel
| | - Xiaojun Chen
- Institute of Biomedical Manufacturing and Life Quality Engineering, School of Mechanical Engineering, Shanghai Jiao Tong University, China; Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai 200240, China.
| |
Collapse
|
2
|
Wu Z, Dai Y, Zeng Y. Intelligent robot-assisted fracture reduction system for the treatment of unstable pelvic fractures. J Orthop Surg Res 2024; 19:271. [PMID: 38689343 PMCID: PMC11059586 DOI: 10.1186/s13018-024-04761-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/13/2024] [Accepted: 04/21/2024] [Indexed: 05/02/2024] Open
Abstract
BACKGROUND Precise and minimally invasive closed reduction is the premise of minimally invasive internal fixation. This paper aims to explore the safety and efficacy of a robot-assisted fracture reduction system (RAFR) in the treatment of pelvic fractures and to analyze its clinical advantages and existing problems. METHODS The RAFR system intelligently designed the optimal reduction path and target position based on a preoperative three-dimensional(3D) CT scan of the patient. The reduction robotic arm automatically reduced the affected hemipelvis according to the pre-planned reduction path. RESULTS The average residual displacement was the 6.65 ± 3.59 mm. According to Matta's criteria, there were 7 excellent, 10 good, and 3 fair, and the excellent and good rate was 85%. No postoperative complications occurred. CONCLUSION In our study, the RAFR system could complete accurate and minimally invasive closed reduction for most patients with unstable pelvic fractures, which could achieve good fracture reduction quality and short-term efficacy.
Collapse
Affiliation(s)
- Zhengjie Wu
- The Eighth Clinical Medical College of Guangzhou University of Chinese Medicine, Foshan, Guangdong, China.
| | - Yonghong Dai
- The Eighth Clinical Medical College of Guangzhou University of Chinese Medicine, Foshan, Guangdong, China
| | - Yanhui Zeng
- The Eighth Clinical Medical College of Guangzhou University of Chinese Medicine, Foshan, Guangdong, China
| |
Collapse
|
3
|
Lin Z, Lei C, Yang L. Modern Image-Guided Surgery: A Narrative Review of Medical Image Processing and Visualization. SENSORS (BASEL, SWITZERLAND) 2023; 23:9872. [PMID: 38139718 PMCID: PMC10748263 DOI: 10.3390/s23249872] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/01/2023] [Revised: 11/15/2023] [Accepted: 12/13/2023] [Indexed: 12/24/2023]
Abstract
Medical image analysis forms the basis of image-guided surgery (IGS) and many of its fundamental tasks. Driven by the growing number of medical imaging modalities, the research community of medical imaging has developed methods and achieved functionality breakthroughs. However, with the overwhelming pool of information in the literature, it has become increasingly challenging for researchers to extract context-relevant information for specific applications, especially when many widely used methods exist in a variety of versions optimized for their respective application domains. By being further equipped with sophisticated three-dimensional (3D) medical image visualization and digital reality technology, medical experts could enhance their performance capabilities in IGS by multiple folds. The goal of this narrative review is to organize the key components of IGS in the aspects of medical image processing and visualization with a new perspective and insights. The literature search was conducted using mainstream academic search engines with a combination of keywords relevant to the field up until mid-2022. This survey systemically summarizes the basic, mainstream, and state-of-the-art medical image processing methods as well as how visualization technology like augmented/mixed/virtual reality (AR/MR/VR) are enhancing performance in IGS. Further, we hope that this survey will shed some light on the future of IGS in the face of challenges and opportunities for the research directions of medical image processing and visualization.
Collapse
Affiliation(s)
- Zhefan Lin
- School of Mechanical Engineering, Zhejiang University, Hangzhou 310030, China;
- ZJU-UIUC Institute, International Campus, Zhejiang University, Haining 314400, China;
| | - Chen Lei
- ZJU-UIUC Institute, International Campus, Zhejiang University, Haining 314400, China;
| | - Liangjing Yang
- School of Mechanical Engineering, Zhejiang University, Hangzhou 310030, China;
- ZJU-UIUC Institute, International Campus, Zhejiang University, Haining 314400, China;
| |
Collapse
|
4
|
Killeen BD, Gao C, Oguine KJ, Darcy S, Armand M, Taylor RH, Osgood G, Unberath M. An autonomous X-ray image acquisition and interpretation system for assisting percutaneous pelvic fracture fixation. Int J Comput Assist Radiol Surg 2023; 18:1201-1208. [PMID: 37213057 PMCID: PMC11002911 DOI: 10.1007/s11548-023-02941-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2023] [Accepted: 04/25/2023] [Indexed: 05/23/2023]
Abstract
PURPOSE Percutaneous fracture fixation involves multiple X-ray acquisitions to determine adequate tool trajectories in bony anatomy. In order to reduce time spent adjusting the X-ray imager's gantry, avoid excess acquisitions, and anticipate inadequate trajectories before penetrating bone, we propose an autonomous system for intra-operative feedback that combines robotic X-ray imaging and machine learning for automated image acquisition and interpretation, respectively. METHODS Our approach reconstructs an appropriate trajectory in a two-image sequence, where the optimal second viewpoint is determined based on analysis of the first image. A deep neural network is responsible for detecting the tool and corridor, here a K-wire and the superior pubic ramus, respectively, in these radiographs. The reconstructed corridor and K-wire pose are compared to determine likelihood of cortical breach, and both are visualized for the clinician in a mixed reality environment that is spatially registered to the patient and delivered by an optical see-through head-mounted display. RESULTS We assess the upper bounds on system performance through in silico evaluation across 11 CTs with fractures present, in which the corridor and K-wire are adequately reconstructed. In post hoc analysis of radiographs across 3 cadaveric specimens, our system determines the appropriate trajectory to within 2.8 ± 1.3 mm and 2.7 ± 1.8[Formula: see text]. CONCLUSION An expert user study with an anthropomorphic phantom demonstrates how our autonomous, integrated system requires fewer images and lower movement to guide and confirm adequate placement compared to current clinical practice. Code and data are available.
Collapse
Affiliation(s)
| | - Cong Gao
- Johns Hopkins University, Baltimore, 21210, MD, USA
| | | | - Sean Darcy
- Johns Hopkins University, Baltimore, 21210, MD, USA
| | - Mehran Armand
- Johns Hopkins University, Baltimore, 21210, MD, USA
- Department of Orthopaedic Surgery, Johns Hopkins University, Baltimore, USA
| | | | - Greg Osgood
- Department of Orthopaedic Surgery, Johns Hopkins University, Baltimore, USA
| | | |
Collapse
|
5
|
Zhao C, Cao Q, Sun X, Wu X, Zhu G, Wang Y. Intelligent robot-assisted minimally invasive reduction system for reduction of unstable pelvic fractures. Injury 2023; 54:604-614. [PMID: 36371315 DOI: 10.1016/j.injury.2022.11.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/21/2022] [Revised: 07/15/2022] [Accepted: 11/02/2022] [Indexed: 11/06/2022]
Abstract
OBJECTIVE Currently, minimally invasive internal fixation is recommended for the surgical treatment of unstable pelvic fractures. The premise and difficulty of minimally invasive internal fixation are minimally invasive reduction of fractures. This review aimed to investigate the indications, surgical strategy and techniques, safety, and efficacy of intelligent robot-assisted fracture reduction (RAFR) system of pelvic ring injuries. METHODS This retrospective study reviewed a case series from March 2021 to November 2021. A total of 22 patients with unstable pelvic fracture injuries underwent minimally invasive internal fixations. All pelvic ring fractures were reduced with our intelligent RAFR system. The robot system intelligently designs the optimal position and reduction path based on the patient's preoperative 3D CT. During the operation, the three-dimensional visualization of the fracture is realized through image registration, and the Robot completes the automatic reduction of the fracture. The global 3D point cloud error between the preoperative planning results and the actual postoperative reduction results was calculated. The postoperative reduction results of residual displacement were graded by the Matta Criteria. RESULTS Minimally invasive closed reduction procedures were completed in all 22 cases with our RAFR system. The average global 3D point cloud reduction error between the preoperative planning results and the actual postoperative reduction results was 3.41mm±1.83mm. The mean residual displacement was 4.61mm±3.29mm. Given the Matta criteria, 16 cases were excellent, five were good, and one was fair, with an excellent and good rate of 95.5%. CONCLUSION Our new pelvic fracture reduction robot system can complete intelligent and minimally invasive fracture reduction for most patients with unstable pelvic fractures. The system has intelligent reduction position and path planning and realizes stable pelvis control through a unique holding arm and a robotic arm. The operation process will not cause additional damage to the patient, which fully meets the clinical requirements. Our study demonstrated the safety and effectiveness of our robotic reduction system and its applicability and usability in clinical practice, thus paving the way towards Robot minimally invasive pelvic fracture surgeries.
Collapse
Affiliation(s)
- Chunpeng Zhao
- Department of Orthopedics and Traumatology, Beijing Jishuitan Hospital, Beijing 100035, China
| | - Qiyong Cao
- Department of Orthopedics and Traumatology, Beijing Jishuitan Hospital, Beijing 100035, China
| | - Xu Sun
- Department of Orthopedics and Traumatology, Beijing Jishuitan Hospital, Beijing 100035, China
| | - Xinbao Wu
- Department of Orthopedics and Traumatology, Beijing Jishuitan Hospital, Beijing 100035, China.
| | - Gang Zhu
- Rossum Robot Co., Ltd., Beijing 100083, China
| | - Yu Wang
- School of Biological Science and Medical Engineering, Beihang University, Beijing 100083, China; Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing 100083, China
| |
Collapse
|
6
|
Fan X, Zhu Q, Tu P, Joskowicz L, Chen X. A review of advances in image-guided orthopedic surgery. Phys Med Biol 2023; 68. [PMID: 36595258 DOI: 10.1088/1361-6560/acaae9] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2022] [Accepted: 12/12/2022] [Indexed: 12/15/2022]
Abstract
Orthopedic surgery remains technically demanding due to the complex anatomical structures and cumbersome surgical procedures. The introduction of image-guided orthopedic surgery (IGOS) has significantly decreased the surgical risk and improved the operation results. This review focuses on the application of recent advances in artificial intelligence (AI), deep learning (DL), augmented reality (AR) and robotics in image-guided spine surgery, joint arthroplasty, fracture reduction and bone tumor resection. For the pre-operative stage, key technologies of AI and DL based medical image segmentation, 3D visualization and surgical planning procedures are systematically reviewed. For the intra-operative stage, the development of novel image registration, surgical tool calibration and real-time navigation are reviewed. Furthermore, the combination of the surgical navigation system with AR and robotic technology is also discussed. Finally, the current issues and prospects of the IGOS system are discussed, with the goal of establishing a reference and providing guidance for surgeons, engineers, and researchers involved in the research and development of this area.
Collapse
Affiliation(s)
- Xingqi Fan
- Institute of Biomedical Manufacturing and Life Quality Engineering, State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Qiyang Zhu
- Institute of Biomedical Manufacturing and Life Quality Engineering, State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Puxun Tu
- Institute of Biomedical Manufacturing and Life Quality Engineering, State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Leo Joskowicz
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Jerusalem, Israel
| | - Xiaojun Chen
- Institute of Biomedical Manufacturing and Life Quality Engineering, State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, People's Republic of China.,Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| |
Collapse
|
7
|
Baumann F, Becker C, Freigang V, Alt V. Imaging, post-processing and navigation: Surgical applications in pelvic fracture treatment. Injury 2022; 53 Suppl 3:S16-S22. [PMID: 36028373 DOI: 10.1016/j.injury.2022.08.049] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/09/2022] [Revised: 08/16/2022] [Accepted: 08/20/2022] [Indexed: 02/02/2023]
Abstract
Technical advancements of the past decade have led to massive improvements regarding imaging and visualization in trauma care. Digital imaging technology has fundamentally changed most processes in fracture management. However, the digital revolution in trauma surgery has just begun. Optical tracking navigation is currently the gold standard for positioning of implants for advanced applications in trauma surgery. Digital technology may enable the surgeon to achieve the same level of safety even in non-navigated placement of screws: We developed a new planning tool to transcript a preoperative into a semi-transparent "fluoroscopic like" image that can be identified intraoperatively and used as a map for the safe placement of sacro-iliac screws based on the "vestibule concept". In the future, development of artificial intelligence algorithms may provide features like automated segmentation of bone-fragments and other applications for a systematic fracture analysis to improve the standard of care in trauma surgery. Digital transformation has massive impact on diagnostics and surgical management of pelvic fractures. Improved visualization technology provides a better understanding of the surgical anatomy of the pelvis and may enable the surgeon to achieve greatest safety in percutaneous placement of screws even without using optical tracking navigation tools. The "para-axial fusion technique" is a useful tool to plan fluoroscopic views based on a 3D dataset prior to the surgery.
Collapse
Affiliation(s)
- Florian Baumann
- Department of Trauma Surgery, Regensburg University Medical Center, Regensburg, Germany.
| | - Claus Becker
- Institute of Diagnostic Radiology, Regensburg University Medical Center, Regensburg, Germany
| | - Viola Freigang
- Department of Trauma Surgery, Regensburg University Medical Center, Regensburg, Germany
| | - Volker Alt
- Department of Trauma Surgery, Regensburg University Medical Center, Regensburg, Germany
| |
Collapse
|
8
|
Moolenaar JZ, Tümer N, Checa S. Computer-assisted preoperative planning of bone fracture fixation surgery: A state-of-the-art review. Front Bioeng Biotechnol 2022; 10:1037048. [PMID: 36312550 PMCID: PMC9613932 DOI: 10.3389/fbioe.2022.1037048] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2022] [Accepted: 09/30/2022] [Indexed: 11/13/2022] Open
Abstract
Background: Bone fracture fixation surgery is one of the most commonly performed surgical procedures in the orthopedic field. However, fracture healing complications occur frequently, and the choice of the most optimal surgical approach often remains challenging. In the last years, computational tools have been developed with the aim to assist preoperative planning procedures of bone fracture fixation surgery. Objectives: The aims of this review are 1) to provide a comprehensive overview of the state-of-the-art in computer-assisted preoperative planning of bone fracture fixation surgery, 2) to assess the clinical feasibility of the existing virtual planning approaches, and 3) to assess their clinical efficacy in terms of clinical outcomes as compared to conventional planning methods. Methods: A literature search was performed in the MEDLINE-PubMed, Ovid-EMBASE, Ovid-EMCARE, Web of Science, and Cochrane libraries to identify articles reporting on the clinical use of computer-assisted preoperative planning of bone fracture fixation. Results: 79 articles were included to provide an overview of the state-of-the art in virtual planning. While patient-specific geometrical model construction, virtual bone fracture reduction, and virtual fixation planning are routinely applied in virtual planning, biomechanical analysis is rarely included in the planning framework. 21 of the included studies were used to assess the feasibility and efficacy of computer-assisted planning methods. The reported total mean planning duration ranged from 22 to 258 min in different studies. Computer-assisted planning resulted in reduced operation time (Standardized Mean Difference (SMD): -2.19; 95% Confidence Interval (CI): -2.87, -1.50), less blood loss (SMD: -1.99; 95% CI: -2.75, -1.24), decreased frequency of fluoroscopy (SMD: -2.18; 95% CI: -2.74, -1.61), shortened fracture healing times (SMD: -0.51; 95% CI: -0.97, -0.05) and less postoperative complications (Risk Ratio (RR): 0.64, 95% CI: 0.46, 0.90). No significant differences were found in hospitalization duration. Some studies reported improvements in reduction quality and functional outcomes but these results were not pooled for meta-analysis, since the reported outcome measures were too heterogeneous. Conclusion: Current computer-assisted planning approaches are feasible to be used in clinical practice and have been shown to improve clinical outcomes. Including biomechanical analysis into the framework has the potential to further improve clinical outcome.
Collapse
Affiliation(s)
- Jet Zoë Moolenaar
- Berlin Institute of Health at Charité, Universitätsmedizin Berlin, Julius Wolff Institute, Berlin, Germany
- Department of Biomechanical Engineering, Delft University of Technology (TU Delft), Delft, Netherlands
| | - Nazli Tümer
- Department of Biomechanical Engineering, Delft University of Technology (TU Delft), Delft, Netherlands
| | - Sara Checa
- Berlin Institute of Health at Charité, Universitätsmedizin Berlin, Julius Wolff Institute, Berlin, Germany
| |
Collapse
|
9
|
Ding AS, Lu A, Li Z, Galaiya D, Ishii M, Siewerdsen JH, Taylor RH, Creighton FX. Statistical Shape Model of the Temporal Bone Using Segmentation Propagation. Otol Neurotol 2022; 43:e679-e687. [PMID: 35761465 PMCID: PMC10072910 DOI: 10.1097/mao.0000000000003554] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
HYPOTHESIS Automated image registration techniques can successfully determine anatomical variation in human temporal bones with statistical shape modeling. BACKGROUND There is a lack of knowledge about inter-patient anatomical variation in the temporal bone. Statistical shape models (SSMs) provide a powerful method for quantifying variation of anatomical structures in medical images but are time-intensive to manually develop. This study presents SSMs of temporal bone anatomy using automated image-registration techniques. METHODS Fifty-three cone-beam temporal bone CTs were included for SSM generation. The malleus, incus, stapes, bony labyrinth, and facial nerve were automatically segmented using 3D Slicer and a template-based segmentation propagation technique. Segmentations were then used to construct SSMs using MATLAB. The first three principal components of each SSM were analyzed to describe shape variation. RESULTS Principal component analysis of middle and inner ear structures revealed novel modes of anatomical variation. The first three principal components for the malleus represented variability in manubrium length (mean: 4.47 mm; ±2-SDs: 4.03-5.03 mm) and rotation about its long axis (±2-SDs: -1.6° to 1.8° posteriorly). The facial nerve exhibits variability in first and second genu angles. The bony labyrinth varies in the angle between the posterior and superior canals (mean: 88.9°; ±2-SDs: 83.7°-95.7°) and cochlear orientation (±2-SDs: -4.0° to 3.0° anterolaterally). CONCLUSIONS SSMs of temporal bone anatomy can inform surgeons on clinically relevant inter-patient variability. Anatomical variation elucidated by these models can provide novel insight into function and pathophysiology. These models also allow further investigation of anatomical variation based on age, BMI, sex, and geographical location.
Collapse
Affiliation(s)
- Andy S. Ding
- Department of Otolaryngology – Head and Neck Surgery, Johns Hopkins University School of Medicine, Baltimore, Maryland
- Department of Biomedical Engineering, Johns Hopkins University Whiting School of Engineering, Baltimore, Maryland
| | - Alexander Lu
- Department of Otolaryngology – Head and Neck Surgery, Johns Hopkins University School of Medicine, Baltimore, Maryland
- Department of Biomedical Engineering, Johns Hopkins University Whiting School of Engineering, Baltimore, Maryland
| | - Zhaoshuo Li
- Department of Computer Science, Johns Hopkins University Whiting School of Engineering, Baltimore, Maryland
| | - Deepa Galaiya
- Department of Otolaryngology – Head and Neck Surgery, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | - Masaru Ishii
- Department of Otolaryngology – Head and Neck Surgery, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | - Jeffrey H. Siewerdsen
- Department of Biomedical Engineering, Johns Hopkins University Whiting School of Engineering, Baltimore, Maryland
- Department of Computer Science, Johns Hopkins University Whiting School of Engineering, Baltimore, Maryland
| | - Russell H. Taylor
- Department of Computer Science, Johns Hopkins University Whiting School of Engineering, Baltimore, Maryland
| | - Francis X. Creighton
- Department of Otolaryngology – Head and Neck Surgery, Johns Hopkins University School of Medicine, Baltimore, Maryland
| |
Collapse
|
10
|
Sheth N, Vagdargi P, Sisniega A, Uneri A, Osgood G, Siewerdsen JH. Preclinical evaluation of a prototype freehand drill video guidance system for orthopedic surgery. J Med Imaging (Bellingham) 2022; 9:045004. [PMID: 36046335 PMCID: PMC9411797 DOI: 10.1117/1.jmi.9.4.045004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2022] [Accepted: 08/09/2022] [Indexed: 08/28/2023] Open
Abstract
Purpose: Internal fixation of pelvic fractures is a challenging task requiring the placement of instrumentation within complex three-dimensional bone corridors, typically guided by fluoroscopy. We report a system for two- and three-dimensional guidance using a drill-mounted video camera and fiducial markers with evaluation in first preclinical studies. Approach: The system uses a camera affixed to a surgical drill and multimodality (optical and radio-opaque) markers for real-time trajectory visualization in fluoroscopy and/or CT. Improvements to a previously reported prototype include hardware components (mount, camera, and fiducials) and software (including a system for detecting marker perturbation) to address practical requirements necessary for translation to clinical studies. Phantom and cadaver experiments were performed to quantify the accuracy of video-fluoroscopy and video-CT registration, the ability to detect marker perturbation, and the conformance in placing guidewires along realistic pelvic trajectories. The performance was evaluated in terms of geometric accuracy and conformance within bone corridors. Results: The studies demonstrated successful guidewire delivery in a cadaver, with a median entry point error of 1.00 mm (1.56 mm IQR) and median angular error of 1.94 deg (1.23 deg IQR). Such accuracy was sufficient to guide K-wire placement through five of the six trajectories investigated with a strong level of conformance within bone corridors. The sixth case demonstrated a cortical breach due to extrema in the registration error. The system was able to detect marker perturbations and alert the user to potential registration issues. Feasible workflows were identified for orthopedic-trauma scenarios involving emergent cases (with no preoperative imaging) or cases with preoperative CT. Conclusions: A prototype system for guidewire placement was developed providing guidance that is potentially compatible with orthopedic-trauma workflow. First preclinical (cadaver) studies demonstrated accurate guidance of K-wire placement in pelvic bone corridors and the ability to automatically detect perturbations that degrade registration accuracy. The preclinical prototype demonstrated performance and utility supporting translation to clinical studies.
Collapse
Affiliation(s)
- Niral Sheth
- Johns Hopkins University, Department of Biomedical Engineering, Baltimore, Maryland, United States
| | - Prasad Vagdargi
- Johns Hopkins University, Department of Computer Science, Baltimore, Maryland, United States
| | - Alejandro Sisniega
- Johns Hopkins University, Department of Biomedical Engineering, Baltimore, Maryland, United States
| | - Ali Uneri
- Johns Hopkins University, Department of Biomedical Engineering, Baltimore, Maryland, United States
| | - Gregory Osgood
- Johns Hopkins Medicine, Department of Orthopedic Surgery, Baltimore, Maryland, United States
| | - Jeffrey H. Siewerdsen
- Johns Hopkins University, Department of Biomedical Engineering, Baltimore, Maryland, United States
- Johns Hopkins University, Department of Computer Science, Baltimore, Maryland, United States
| |
Collapse
|
11
|
Zhao C, Wang Y, Wu X, Zhu G, Shi S. Design and evaluation of an intelligent reduction robot system for the minimally invasive reduction in pelvic fractures. J Orthop Surg Res 2022; 17:205. [PMID: 35379278 PMCID: PMC8981738 DOI: 10.1186/s13018-022-03089-2] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/24/2022] [Accepted: 03/22/2022] [Indexed: 11/10/2022] Open
Abstract
Introduction Pelvic fracture is a severe high-energy injury with the highest disability and mortality of all fractures. Traditional open surgery is associated with extensive soft tissue damages and many complications. Minimally invasive surgery potentially mitigates the risks of open surgical procedures and is becoming a new standard for pelvic fracture treatment. The accurate reduction has been recognized as the cornerstone of minimally invasive surgery for pelvic fracture. At present, the closed reduction in pelvic fractures is limited by the current sub-optimal 2D intra-operative imaging (fluoroscopy) and by the high forces of soft tissue involved in the fragment manipulation, which might result in fracture malreduction. To overcome these shortcomings and facilitate pelvic fracture reduction, we developed an intelligent robot-assisted fracture reduction (RAFR) system for pelvic fracture. Methods The presented method is divided into three parts. The first part is the preparation of 20 pelvic fracture models. In the second part, we offer an automatic reduction algorithm of our robotic reduction system, including Intraoperative real-time 3D navigation, reduction path planning, control and fixation, and robotic-assisted fracture reduction. In the third part, image registration accuracy and fracture reduction accuracy were calculated and analyzed. Results All 20 pelvic fracture bone models were reduced by the RAFR system; the mean registration error E1 of the 20 models was 1.29 ± 0.57 mm. The mean reduction error E2 of the 20 models was 2.72 ± 0.82 mm. The global error analysis of registration and reduction results showed that higher errors are mainly located at the edge of the pelvis, such as the iliac wing. Conclusion The accuracy of image registration error and fracture reduction error in our study was excellent, which could reach the requirements of the clinical environment. Our study demonstrated the precision and effectiveness of our RAFR system and its applicability and usability in clinical practice, thus paving the way toward robot minimally invasive pelvic fracture surgeries.
Collapse
Affiliation(s)
- Chunpeng Zhao
- Department of Orthopedics and Traumatology, Beijing Jishuitan Hospital, Beijing, 100035, China
| | - Yu Wang
- School of Biological Science and Medical Engineering, Beihang University, Beijing, 100083, China.,Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing, 100083, China
| | - Xinbao Wu
- Department of Orthopedics and Traumatology, Beijing Jishuitan Hospital, Beijing, 100035, China.
| | - Gang Zhu
- Rossum Robot Co., Ltd., Beijing, 100083, China
| | | |
Collapse
|
12
|
Han J, Davids J, Ashrafian H, Darzi A, Elson DS, Sodergren M. A systematic review of robotic surgery: From supervised paradigms to fully autonomous robotic approaches. Int J Med Robot 2022; 18:e2358. [PMID: 34953033 DOI: 10.1002/rcs.2358] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2021] [Revised: 11/23/2021] [Accepted: 12/21/2021] [Indexed: 12/20/2022]
Abstract
BACKGROUND From traditional open surgery to laparoscopic surgery and robot-assisted surgery, advances in robotics, machine learning, and imaging are pushing the surgical approach to-wards better clinical outcomes. Pre-clinical and clinical evidence suggests that automation may standardise techniques, increase efficiency, and reduce clinical complications. METHODS A PRISMA-guided search was conducted across PubMed and OVID. RESULTS Of the 89 screened articles, 51 met the inclusion criteria, with 10 included in the final review. Automatic data segmentation, trajectory planning, intra-operative registration, trajectory drilling, and soft tissue robotic surgery were discussed. CONCLUSION Although automated surgical systems remain conceptual, several research groups have developed supervised autonomous robotic surgical systems with increasing consideration for ethico-legal issues for automation. Automation paves the way for precision surgery and improved safety and opens new possibilities for deploying more robust artificial intelligence models, better imaging modalities and robotics to improve clinical outcomes.
Collapse
Affiliation(s)
- Jinpei Han
- Hamlyn Centre for Robotic Surgery and Artificial Intelligence, Imperial College London, London, UK
| | - Joseph Davids
- Hamlyn Centre for Robotic Surgery and Artificial Intelligence, Imperial College London, London, UK
- National Hospital for Neurology and Neurosurgery, London, UK
| | - Hutan Ashrafian
- Hamlyn Centre for Robotic Surgery and Artificial Intelligence, Imperial College London, London, UK
| | - Ara Darzi
- Hamlyn Centre for Robotic Surgery and Artificial Intelligence, Imperial College London, London, UK
| | - Daniel S Elson
- Hamlyn Centre for Robotic Surgery and Artificial Intelligence, Imperial College London, London, UK
| | - Mikael Sodergren
- Hamlyn Centre for Robotic Surgery and Artificial Intelligence, Imperial College London, London, UK
| |
Collapse
|
13
|
Killeen BD, Chakraborty S, Osgood G, Unberath M. Toward Perception-based Anticipation of Cortical Breach During K-wire Fixation of the Pelvis. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2022; 12031:120311N. [PMID: 38617810 PMCID: PMC11016333 DOI: 10.1117/12.2612989] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/16/2024]
Abstract
Intraoperative imaging using C-arm X-ray systems enables percutaneous management of fractures by providing real-time visualization of tool to tissue relationships. However, estimating appropriate positioning of surgical instruments, such as K-wires, relative to safe bony corridors is challenging due to the projective nature of X-ray images: tool pose in the plane containing the principal ray is difficult to assess, necessitating the acquisition of numerous views onto the anatomy. This task is especially demanding in complex anatomy, such as the superior pubic ramus of the pelvis, and results in high cognitive load and repeat attempts even in experienced trauma surgeons. A perception-based algorithm that interprets interventional radiographs during internal fixation to infer the likelihood of cortical breach - especially early on, when the wire has not been advanced - might reduce both the amount of X-rays acquired for verification and the likelihood of repeat attempts. In this manuscript, we present first steps towards developing such an algorithm. We devise a strategy for in silico collection and annotation of X-ray images suitable for detecting cortical breach of a K-wire in the superior pubic ramus, including those with visible fractures. Beginning with minimal manual annotations of correct trajectories, we randomly perturb entry and exit points and project the 3D scene using a physics-based forward model to obtain a large number of 2D X-ray images with and without cortical breach. We report baseline results for anticipating cortical breach at various K-wire insertion depths, achieving an AUROC score of 0.68 for 50% insertion. Code and data are available at github.com/benjamindkilleen/cortical-breach-detection.
Collapse
Affiliation(s)
- Benjamin D Killeen
- Johns Hopkins University, 3400 N Charles St., Baltimore, MD, United States
| | - Shreya Chakraborty
- Johns Hopkins University, 3400 N Charles St., Baltimore, MD, United States
| | - Greg Osgood
- Johns Hopkins University, 3400 N Charles St., Baltimore, MD, United States
| | - Mathias Unberath
- Johns Hopkins University, 3400 N Charles St., Baltimore, MD, United States
| |
Collapse
|
14
|
Vijayan RC, Han R, Wu P, Sheth NM, Ketcha MD, Vagdargi P, Vogt S, Kleinszig G, Osgood GM, Siewerdsen JH, Uneri A. Development of a fluoroscopically guided robotic assistant for instrument placement in pelvic trauma surgery. J Med Imaging (Bellingham) 2021; 8:035001. [PMID: 34124283 PMCID: PMC8189698 DOI: 10.1117/1.jmi.8.3.035001] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2020] [Accepted: 05/21/2021] [Indexed: 11/14/2022] Open
Abstract
Purpose: A method for fluoroscopic guidance of a robotic assistant is presented for instrument placement in pelvic trauma surgery. The solution uses fluoroscopic images acquired in standard clinical workflow and helps avoid repeat fluoroscopy commonly performed during implant guidance. Approach: Images acquired from a mobile C-arm are used to perform 3D-2D registration of both the patient (via patient CT) and the robot (via CAD model of a surgical instrument attached to its end effector, e.g; a drill guide), guiding the robot to target trajectories defined in the patient CT. The proposed approach avoids C-arm gantry motion, instead manipulating the robot to acquire disparate views of the instrument. Phantom and cadaver studies were performed to determine operating parameters and assess the accuracy of the proposed approach in aligning a standard drill guide instrument. Results: The proposed approach achieved average drill guide tip placement accuracy of 1.57 ± 0.47 mm and angular alignment of 0.35 ± 0.32 deg in phantom studies. The errors remained within 2 mm and 1 deg in cadaver experiments, comparable to the margins of errors provided by surgical trackers (but operating without the need for external tracking). Conclusions: By operating at a fixed fluoroscopic perspective and eliminating the need for encoded C-arm gantry movement, the proposed approach simplifies and expedites the registration of image-guided robotic assistants and can be used with simple, non-calibrated, non-encoded, and non-isocentric C-arm systems to accurately guide a robotic device in a manner that is compatible with the surgical workflow.
Collapse
Affiliation(s)
- Rohan C. Vijayan
- Johns Hopkins University, Department of Biomedical Engineering, Baltimore, Maryland, United States
| | - Runze Han
- Johns Hopkins University, Department of Biomedical Engineering, Baltimore, Maryland, United States
| | - Pengwei Wu
- Johns Hopkins University, Department of Biomedical Engineering, Baltimore, Maryland, United States
| | - Niral M. Sheth
- Johns Hopkins University, Department of Biomedical Engineering, Baltimore, Maryland, United States
| | - Michael D. Ketcha
- Johns Hopkins University, Department of Biomedical Engineering, Baltimore, Maryland, United States
| | - Prasad Vagdargi
- Johns Hopkins University, Department of Computer Science, Baltimore, Maryland, United States
| | | | | | - Greg M. Osgood
- Johns Hopkins Medicine, Department of Orthopaedic Surgery, Baltimore, Maryland, United States
| | - Jeffrey H. Siewerdsen
- Johns Hopkins University, Department of Biomedical Engineering, Baltimore, Maryland, United States
- Johns Hopkins University, Department of Computer Science, Baltimore, Maryland, United States
| | - Ali Uneri
- Johns Hopkins University, Department of Biomedical Engineering, Baltimore, Maryland, United States
| |
Collapse
|
15
|
Vagdargi P, Sheth N, Sisniega A, Uneri A, De Silva T, Osgood GM, Siewerdsen JH. Drill-mounted video guidance for orthopaedic trauma surgery. J Med Imaging (Bellingham) 2021; 8:015002. [PMID: 33604409 DOI: 10.1117/1.jmi.8.1.015002] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2020] [Accepted: 01/19/2021] [Indexed: 11/14/2022] Open
Abstract
Purpose: Percutaneous fracture fixation is a challenging procedure that requires accurate interpretation of fluoroscopic images to insert guidewires through narrow bone corridors. We present a guidance system with a video camera mounted onboard the surgical drill to achieve real-time augmentation of the drill trajectory in fluoroscopy and/or CT. Approach: The camera was mounted on the drill and calibrated with respect to the drill axis. Markers identifiable in both video and fluoroscopy are placed about the surgical field and co-registered by feature correspondences. If available, a preoperative CT can also be co-registered by 3D-2D image registration. Real-time guidance is achieved by virtual overlay of the registered drill axis on fluoroscopy or in CT. Performance was evaluated in terms of target registration error (TRE), conformance within clinically relevant pelvic bone corridors, and runtime. Results: Registration of the drill axis to fluoroscopy demonstrated median TRE of 0.9 mm and 2.0 deg when solved with two views (e.g., anteroposterior and lateral) and five markers visible in both video and fluoroscopy-more than sufficient to provide Kirschner wire (K-wire) conformance within common pelvic bone corridors. Registration accuracy was reduced when solved with a single fluoroscopic view ( TRE = 3.4 mm and 2.7 deg) but was also sufficient for K-wire conformance within pelvic bone corridors. Registration was robust with as few as four markers visible within the field of view. Runtime of the initial implementation allowed fluoroscopy overlay and/or 3D CT navigation with freehand manipulation of the drill up to 10 frames / s . Conclusions: A drill-mounted video guidance system was developed to assist with K-wire placement. Overall workflow is compatible with fluoroscopically guided orthopaedic trauma surgery and does not require markers to be placed in preoperative CT. The initial prototype demonstrates accuracy and runtime that could improve the accuracy of K-wire placement, motivating future work for translation to clinical studies.
Collapse
Affiliation(s)
- Prasad Vagdargi
- Johns Hopkins University, Department of Computer Science, Baltimore, Maryland, United States
| | - Niral Sheth
- Johns Hopkins University, Department of Biomedical Engineering, Baltimore, Maryland, United States
| | - Alejandro Sisniega
- Johns Hopkins University, Department of Biomedical Engineering, Baltimore, Maryland, United States
| | - Ali Uneri
- Johns Hopkins University, Department of Biomedical Engineering, Baltimore, Maryland, United States
| | - Tharindu De Silva
- Johns Hopkins University, Department of Biomedical Engineering, Baltimore, Maryland, United States
| | - Greg M Osgood
- Johns Hopkins Medicine, Department of Orthopaedic Surgery, Baltimore, Maryland, United States
| | - Jeffrey H Siewerdsen
- Johns Hopkins University, Department of Computer Science, Baltimore, Maryland, United States.,Johns Hopkins University, Department of Biomedical Engineering, Baltimore, Maryland, United States
| |
Collapse
|
16
|
Han R, Uneri A, Vijayan RC, Wu P, Vagdargi P, Sheth N, Vogt S, Kleinszig G, Osgood GM, Siewerdsen JH. Fracture reduction planning and guidance in orthopaedic trauma surgery via multi-body image registration. Med Image Anal 2020; 68:101917. [PMID: 33341493 DOI: 10.1016/j.media.2020.101917] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2020] [Revised: 11/16/2020] [Accepted: 11/23/2020] [Indexed: 02/06/2023]
Abstract
PURPOSES Surgical reduction of pelvic fracture is a challenging procedure, and accurate restoration of natural morphology is essential to obtaining positive functional outcome. The procedure often requires extensive preoperative planning, long fluoroscopic exposure time, and trial-and-error to achieve accurate reduction. We report a multi-body registration framework for reduction planning using preoperative CT and intraoperative guidance using routine 2D fluoroscopy that could help address such challenges. METHOD The framework starts with semi-automatic segmentation of fractured bone fragments in preoperative CT using continuous max-flow. For reduction planning, a multi-to-one registration is performed to register bone fragments to an adaptive template that adjusts to patient-specific bone shapes and poses. The framework further registers bone fragments to intraoperative fluoroscopy to provide 2D fluoroscopy guidance and/or 3D navigation relative to the reduction plan. The framework was investigated in three studies: (1) a simulation study of 40 CT images simulating three fracture categories (unilateral two-body, unilateral three-body, and bilateral two-body); (2) a proof-of-concept cadaver study to mimic clinical scenario; and (3) a retrospective clinical study investigating feasibility in three cases of increasing severity and accuracy requirement. RESULTS Segmentation of simulated pelvic fracture demonstrated Dice coefficient of 0.92±0.06. Reduction planning using the adaptive template achieved 2-3 mm and 2-3° error for the three fracture categories, significantly better than planning based on mirroring of contralateral anatomy. 3D-2D registration yielded ~2 mm and 0.5° accuracy, providing accurate guidance with respect to the preoperative reduction plan. The cadaver study and retrospective clinical study demonstrated comparable accuracy: ~0.90 Dice coefficient in segmentation, ~3 mm accuracy in reduction planning, and ~2 mm accuracy in 3D-2D registration. CONCLUSION The registration framework demonstrated planning and guidance accuracy within clinical requirements in both simulation and clinical feasibility studies for a broad range of fracture-dislocation patterns. Using routinely acquired preoperative CT and intraoperative fluoroscopy, the framework could improve the accuracy of pelvic fracture reduction, reduce radiation dose, and could integrate well with common clinical workflow without the need for additional navigation systems.
Collapse
Affiliation(s)
- R Han
- Department of Biomedical Engineering, The Johns Hopkins University, BaltimoreMD, United States
| | - A Uneri
- Department of Biomedical Engineering, The Johns Hopkins University, BaltimoreMD, United States
| | - R C Vijayan
- Department of Biomedical Engineering, The Johns Hopkins University, BaltimoreMD, United States
| | - P Wu
- Department of Biomedical Engineering, The Johns Hopkins University, BaltimoreMD, United States
| | - P Vagdargi
- Department of Computer Science, The Johns Hopkins University, BaltimoreMD, United States
| | - N Sheth
- Department of Biomedical Engineering, The Johns Hopkins University, BaltimoreMD, United States
| | - S Vogt
- Siemens Healthineers, ErlangenGermany
| | | | - G M Osgood
- Department of Orthopaedic Surgery, The Johns Hopkins Hospital, BaltimoreMD, United States
| | - J H Siewerdsen
- Department of Biomedical Engineering, The Johns Hopkins University, BaltimoreMD, United States.
| |
Collapse
|
17
|
Scorza D, El Hadji S, Cortés C, Bertelsen Á, Cardinale F, Baselli G, Essert C, Momi ED. Surgical planning assistance in keyhole and percutaneous surgery: A systematic review. Med Image Anal 2020; 67:101820. [PMID: 33075642 DOI: 10.1016/j.media.2020.101820] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2019] [Revised: 08/07/2020] [Accepted: 09/07/2020] [Indexed: 11/29/2022]
Abstract
Surgical planning of percutaneous interventions has a crucial role to guarantee the success of minimally invasive surgeries. In the last decades, many methods have been proposed to reduce clinician work load related to the planning phase and to augment the information used in the definition of the optimal trajectory. In this survey, we include 113 articles related to computer assisted planning (CAP) methods and validations obtained from a systematic search on three databases. First, a general formulation of the problem is presented, independently from the surgical field involved, and the key steps involved in the development of a CAP solution are detailed. Secondly, we categorized the articles based on the main surgical applications, which have been object of study and we categorize them based on the type of assistance provided to the end-user.
Collapse
Affiliation(s)
- Davide Scorza
- Vicomtech Foundation, Basque Research and Technology Alliance (BRTA), Donostia-San Sebastián, Spain; Department of Electronics, Information and Bioengineering (DEIB), Politecnico di Milano, Milan, Italy; Biodonostia Health Research Institute, Donostia-San Sebastián, Spain.
| | - Sara El Hadji
- Department of Electronics, Information and Bioengineering (DEIB), Politecnico di Milano, Milan, Italy.
| | - Camilo Cortés
- Vicomtech Foundation, Basque Research and Technology Alliance (BRTA), Donostia-San Sebastián, Spain; Biodonostia Health Research Institute, Donostia-San Sebastián, Spain
| | - Álvaro Bertelsen
- Vicomtech Foundation, Basque Research and Technology Alliance (BRTA), Donostia-San Sebastián, Spain; Biodonostia Health Research Institute, Donostia-San Sebastián, Spain
| | - Francesco Cardinale
- Claudio Munari Centre for Epilepsy and Parkinson surgery, Azienda Socio-Sanitaria Territoriale Grande Ospedale Metropolitano Niguarda (ASST GOM Niguarda), Milan, Italy
| | - Giuseppe Baselli
- Department of Electronics, Information and Bioengineering (DEIB), Politecnico di Milano, Milan, Italy
| | - Caroline Essert
- ICube Laboratory, CNRS, UMR 7357, Université de Strasbourg, Strasbourg, France
| | - Elena De Momi
- Department of Electronics, Information and Bioengineering (DEIB), Politecnico di Milano, Milan, Italy
| |
Collapse
|
18
|
Vagdargi P, Uneri A, Sheth N, Sisniega A, De Silva T, Osgood GM, Siewerdsen JH. Calibration and Registration of a Freehand Video-Guided Surgical Drill for Orthopaedic Trauma. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2020; 11315. [PMID: 32476703 DOI: 10.1117/12.2550001] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
Pelvic trauma surgical procedures rely heavily on guidance with 2D fluoroscopy views for navigation in complex bone corridors. This "fluoro-hunting" paradigm results in extended radiation exposure and possible suboptimal guidewire placement from limited visualization of the fractures site with overlapped anatomy in 2D fluoroscopy. A novel computer vision-based navigation system for freehand guidewire insertion is proposed. The navigation framework is compatible with the rapid workflow in trauma surgery and bridges the gap between intraoperative fluoroscopy and preoperative CT images. The system uses a drill-mounted camera to detect and track poses of simple multimodality (optical/radiographic) markers for registration of the drill axis to fluoroscopy and, in turn, to CT. Surgical navigation is achieved with real-time display of the drill axis position on fluoroscopy views and, optionally, in 3D on the preoperative CT. The camera was corrected for lens distortion effects and calibrated for 3D pose estimation. Custom marker jigs were constructed to calibrate the drill axis and tooltip with respect to the camera frame. A testing platform for evaluation of the navigation system was developed, including a robotic arm for precise, repeatable, placement of the drill. Experiments were conducted for hand-eye calibration between the drill-mounted camera and the robot using the Park and Martin solver. Experiments using checkerboard calibration demonstrated subpixel accuracy [-0.01 ± 0.23 px] for camera distortion correction. The drill axis was calibrated using a cylindrical model and demonstrated sub-mm accuracy [0.14 ± 0.70 mm] and sub-degree angular deviation.
Collapse
Affiliation(s)
- P Vagdargi
- Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA 21218
| | - A Uneri
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA 21218
| | - N Sheth
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA 21218
| | - A Sisniega
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA 21218
| | - T De Silva
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA 21218
| | - G M Osgood
- Department of Orthopedic Surgery, Johns Hopkins Medicine, Baltimore, MD, USA 21218
| | - J H Siewerdsen
- Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA 21218.,Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA 21218
| |
Collapse
|
19
|
Vijayan RC, Han R, Wu P, Sheth NM, Ketcha MD, Vagdargi P, Vogt S, Kleinszig G, Osgood GM, Siewerdsen JH, Uneri A. Image-Guided Robotic K-Wire Placement for Orthopaedic Trauma Surgery. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2020; 11315:113151A. [PMID: 36082206 PMCID: PMC9450105 DOI: 10.1117/12.2549713] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
PURPOSE We report the initial development of an image-based solution for robotic assistance of pelvic fracture fixation. The approach uses intraoperative radiographs, preoperative CT, and an end effector of known design to align the robot with target trajectories in CT. The method extends previous work to solve the robot-to-patient registration from a single radiographic view (without C-arm rotation) and addresses the workflow challenges associated with integrating robotic assistance in orthopaedic trauma surgery in a form that could be broadly applicable to isocentric or non-isocentric C-arms. METHODS The proposed method uses 3D-2D known-component registration to localize a robot end effector with respect to the patient by: (1) exploiting the extended size and complex features of pelvic anatomy to register the patient; and (2) capturing multiple end effector poses using precise robotic manipulation. These transformations, along with an offline hand-eye calibration of the end effector, are used to calculate target robot poses that align the end effector with planned trajectories in the patient CT. Geometric accuracy of the registrations was independently evaluated for the patient and the robot in phantom studies. RESULTS The resulting translational difference between the ground truth and patient registrations of a pelvis phantom using a single (AP) view was 1.3 mm, compared to 0.4 mm using dual (AP+Lat) views. Registration of the robot in air (i.e., no background anatomy) with five unique end effector poses achieved mean translational difference ~1.4 mm for K-wire placement in the pelvis, comparable to tracker-based margins of error (commonly ~2 mm). CONCLUSIONS The proposed approach is feasible based on the accuracy of the patient and robot registrations and is a preliminary step in developing an image-guided robotic guidance system that more naturally fits the workflow of fluoroscopically guided orthopaedic trauma surgery. Future work will involve end-to-end development of the proposed guidance system and assessment of the system with delivery of K-wires in cadaver studies.
Collapse
Affiliation(s)
- R. C. Vijayan
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore MD
| | - R. Han
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore MD
| | - P. Wu
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore MD
| | - N. M. Sheth
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore MD
| | - M. D. Ketcha
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore MD
| | - P. Vagdargi
- Department of Computer Science, Johns Hopkins University, Baltimore MD
| | - S. Vogt
- Siemens Healthineers, Forchheim, Germany
| | | | - G. M. Osgood
- Department of Orthopaedic Surgery, Johns Hopkins Medicine, Baltimore MD
| | - J. H. Siewerdsen
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore MD
- Department of Computer Science, Johns Hopkins University, Baltimore MD
| | - A. Uneri
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore MD
| |
Collapse
|