Vision-based real-time position control of a semi-automated system for robot-assisted joint fracture surgery

[Development and clinical application of robot-assisted technology in traumatic orthopedics]

OBJECTIVE

To review and evaluate the basic principles and advantages of orthopedic robot-assisted technology, research progress, clinical applications, and limitations in the field of traumatic orthopedics, especially in fracture reduction robots.

METHODS

An extensive review of research literature on the principles of robot-assisted technology and fracture reduction robots was conducted to analyze the technical advantages and clinical efficacy and shortcomings, and to discuss the future development trends in this field.

RESULTS

Orthopedic surgical robots can assist orthopedists in intuitive preoperative planning, precise intraoperative control, and minimally invasive operations. It greatly expands the ability of doctors to evaluate and treat orthopedic trauma. Trauma orthopedic surgery robot has achieved a breakthrough from basic research to clinical application, and the preliminary results show that the technology can significantly improve surgical precision and reduce surgical trauma. However, there are still problems such as insufficient evaluation of effectiveness, limited means of technology realization, and narrow clinical indications that need to be solved.

CONCLUSION

Robot-assisted technology has a broad application prospect in traumatic orthopedics, but the current development is still in the initial stage. It is necessary to strengthen the cooperative medical-industrial research, the construction of doctors' communication platform, standardized training and data sharing in order to continuously promote the development of robot-assisted technology in traumatic orthopedics and better play its clinical application value.

Human-robot-robot cooperative control using positioning robot and 1-DOF traction device for robot-assisted fracture reduction system

While performing musculoskeletal long bone fracture reduction surgery, assistant surgeons can often suffer from physical fatigue as they provide resistance against the tension from surrounding muscles pulling on the patient’s broken bones. These days, robotic systems are being actively developed to mitigate this physical workload by realigning and holding these fractured bones for surgeons. This has led to one consortium proposing the development of a robot-assisted fracture reduction system consisting of a 6-DOF positioning robot along with a 1-DOF traction device. With the introduction of the 1-DOF traction device, the positioning robot does not have to fight these contraction forces so can be compact improving its maneuverability and overall convenience; however, considering surgeon-robot interactions, this approach adds the requirement of controlling two different types of robots simultaneously. As such, an advanced cooperative control methodology is required to control the proposed bone fracture reduction robot system. In this paper, a human-robot-robot cooperative control (HRRCC) scheme is proposed for collaboration between the surgeon, the positioning robot, and the traction device. First, the mathematical background of this HRRCC scheme is provided. Next, we describe a series of experiments that show how the proposed scheme facilitates a reduction in the load placed on the positioning robot from strong muscular contraction forces making it possible to conduct fracture reduction procedures more safely despite the muscular forces.

Design and evaluation of an intelligent reduction robot system for the minimally invasive reduction in pelvic fractures

Introduction

Pelvic fracture is a severe high-energy injury with the highest disability and mortality of all fractures. Traditional open surgery is associated with extensive soft tissue damages and many complications. Minimally invasive surgery potentially mitigates the risks of open surgical procedures and is becoming a new standard for pelvic fracture treatment. The accurate reduction has been recognized as the cornerstone of minimally invasive surgery for pelvic fracture. At present, the closed reduction in pelvic fractures is limited by the current sub-optimal 2D intra-operative imaging (fluoroscopy) and by the high forces of soft tissue involved in the fragment manipulation, which might result in fracture malreduction. To overcome these shortcomings and facilitate pelvic fracture reduction, we developed an intelligent robot-assisted fracture reduction (RAFR) system for pelvic fracture.

Methods

The presented method is divided into three parts. The first part is the preparation of 20 pelvic fracture models. In the second part, we offer an automatic reduction algorithm of our robotic reduction system, including Intraoperative real-time 3D navigation, reduction path planning, control and fixation, and robotic-assisted fracture reduction. In the third part, image registration accuracy and fracture reduction accuracy were calculated and analyzed.

Results

All 20 pelvic fracture bone models were reduced by the RAFR system; the mean registration error E1 of the 20 models was 1.29 ± 0.57 mm. The mean reduction error E2 of the 20 models was 2.72 ± 0.82 mm. The global error analysis of registration and reduction results showed that higher errors are mainly located at the edge of the pelvis, such as the iliac wing.

Conclusion

The accuracy of image registration error and fracture reduction error in our study was excellent, which could reach the requirements of the clinical environment. Our study demonstrated the precision and effectiveness of our RAFR system and its applicability and usability in clinical practice, thus paving the way toward robot minimally invasive pelvic fracture surgeries.

Constraint of musculoskeletal tissue and path planning of robot-assisted fracture reduction with collision avoidance

BACKGROUND

For the robot-assisted fracture reduction, due to the complex fracture musculoskeletal environment, it is necessary to consider the influence of soft tissue traction on preoperative reduction path planning.

METHOD

An improved 3D A* algorithm is adopted to plan the fracture reduction path. The distal fragment point clouds are updated to avoid the collision, and the end point coordinates of the muscles are updated to calculate muscular lengths during the path search.

RESULTS

3D reduction path of long-bone fracture is planned, effectively avoiding the fracture fragments collision and ensuring the length of the corresponding muscle is always less than the allowable maximum muscle length after elongation.

CONCLUSION

The proposed method can effectively avoid the collision between the distal fragment and the proximal fragment during the fracture reduction, can avoid secondary injury of the muscles around the femoral bone caused by over-distraction, and effectively improve the safety of robot reduction operation. This article is protected by copyright. All rights reserved.

Design and Evaluation of a Percutaneous Fragment Manipulation Device for Minimally Invasive Fracture Surgery

Reduction of fractures in the minimally invasive (MI) manner can avoid risks associated with open fracture surgery. The MI approach requires specialized tools called percutaneous fragment manipulation devices (PFMD) to enable surgeons to safely grasp and manipulate fragments. PFMDs developed for long-bone manipulation are not suitable for intra-articular fractures where small bone fragments are involved. With this study, we offer a solution to potentially move the current fracture management practice closer to the use of a MI approach. We investigate the design and testing of a new PFMD design for manual as well as robot-assisted manipulation of small bone fragments. This new PFMD design is simulated using FEA in three loading scenarios (force/torque: 0 N/2.6 Nm, 75.7 N/3.5 N, 147 N/6.8 Nm) assessing structural properties, breaking points, and maximum bending deformations. The PFMD is tested in a laboratory setting on Sawbones models (0 N/2.6 Nm), and on ex-vivo swine samples (F = 80 N ± 8 N, F = 150 ± 15 N). A commercial optical tracking system was used for measuring PFMD deformations under external loading and the results were verified with an electromagnetic tracking system. The average error difference between the tracking systems was 0.5 mm, being within their accuracy limits. Final results from reduction maneuvers performed both manually and with the robot assistance are obtained from 7 human cadavers with reduction forces in the range of (F = 80 N ± 8 N, F = 150 ± 15 N, respectively). The results show that structurally, the system performs as predicted by the simulation results. The PFMD did not break during ex-vivo and cadaveric trials. Simulation, laboratory, and cadaveric tests produced similar results regarding the PFMD bending. Specifically, for forces applied perpendicularly to the axis of the PFMD of 80 N ± 8 N deformations of 2.8, 2.97, and 3.06 mm are measured on the PFMD, while forces of 150 ± 15 N produced deformations of 5.8, 4.44, and 5.19 mm. This study has demonstrated that the proposed PFMD undergoes predictable deformations under typical bone manipulation loads. Testing of the device on human cadavers proved that these deformations do not affect the anatomic reduction quality. The PFMD is, therefore, suitable to reliably achieve and maintain fracture reductions, and to, consequently, allow external fracture fixation.

Medical Robotics in Bone Fracture Reduction Surgery: A Review

Since the advantages of precise operation and effective reduction of radiation, robots have become one of the best choices for solving the defects of traditional fracture reduction surgery. This paper focuses on the application of robots in fracture reduction surgery, design of the mechanism, navigation technology, robotic control, interaction technology, and the bone-robot connection technology. Through literature review, the problems in current fracture reduction robot and its future development are discussed.

Robot-Assisted Fracture Surgery: Surgical Requirements and System Design

The design of medical devices is a complex and crucial process to ensure patient safety. It has been shown that improperly designed devices lead to errors and associated accidents and costs. A key element for a successful design is incorporating the views of the primary and secondary stakeholders early in the development process. They provide insights into current practice and point out specific issues with the current processes and equipment in use. This work presents how information from a user-study conducted in the early stages of the RAFS (Robot Assisted Fracture Surgery) project informed the subsequent development and testing of the system. The user needs were captured using qualitative methods and converted to operational, functional, and non-functional requirements based on the methods derived from product design and development. This work presents how the requirements inform a new workflow for intra-articular joint fracture reduction using a robotic system. It is also shown how the various elements of the system are developed to explicitly address one or more of the requirements identified, and how intermediate verification tests are conducted to ensure conformity. Finally, a validation test in the form of a cadaveric trial confirms the ability of the designed system to satisfy the aims set by the original research question and the needs of the users.

Comparison of algorithms for automated femur fracture reduction

PURPOSE

We designed an experiment to determine the comparative effectiveness of computer algorithms for performing automated long bone fracture reduction.

METHODS

Automated reduction of 10 3D fracture models was performed using two computer algorithms, random sample matching (RANSAM) and Z-buffering (Z-Buffer), and one of five options of post-processing: none; iterative closest point algorithm (ICP); ICP-X1; ICP-X2; and ICP-X3. We measured the final alignment between the two fragments for each algorithm and post-processing option.

RESULTS

The RANSAM algorithm combined with postprocessing algorithm ICP-X1 or ICP-X3 resulted in the most accurate fracture reduction in the translational plane. No discernible difference was observed in the rotational plane. Automated reduction had more accurate translational displacement than telemanipulated manual reductions.

CONCLUSION

This study supports the use of the RANSAM algorithm for automated fracture reduction procedures. The use of ICP algorithms provides further optimization of the initial reduction.

Image-Guided Surgical Robotic System for Percutaneous Reduction of Joint Fractures

Complex joint fractures often require an open surgical procedure, which is associated with extensive soft tissue damages and longer hospitalization and rehabilitation time. Percutaneous techniques can potentially mitigate these risks but their application to joint fractures is limited by the current sub-optimal 2D intra-operative imaging (fluoroscopy) and by the high forces involved in the fragment manipulation (due to the presence of soft tissue, e.g., muscles) which might result in fracture malreduction. Integration of robotic assistance and 3D image guidance can potentially overcome these issues. The authors propose an image-guided surgical robotic system for the percutaneous treatment of knee joint fractures, i.e., the robot-assisted fracture surgery (RAFS) system. It allows simultaneous manipulation of two bone fragments, safer robot-bone fixation system, and a traction performing robotic manipulator. This system has led to a novel clinical workflow and has been tested both in laboratory and in clinically relevant cadaveric trials. The RAFS system was tested on 9 cadaver specimens and was able to reduce 7 out of 9 distal femur fractures (T- and Y-shape 33-C1) with acceptable accuracy (≈1 mm, ≈5°), demonstrating its applicability to fix knee joint fractures. This study paved the way to develop novel technologies for percutaneous treatment of complex fractures including hip, ankle, and shoulder, thus representing a step toward minimally-invasive fracture surgeries.

Intra-operative fiducial-based CT/fluoroscope image registration framework for image-guided robot-assisted joint fracture surgery

Purpose

Joint fractures must be accurately reduced minimising soft tissue damages to avoid negative surgical outcomes. To this regard, we have developed the RAFS surgical system, which allows the percutaneous reduction of intra-articular fractures and provides intra-operative real-time 3D image guidance to the surgeon. Earlier experiments showed the effectiveness of the RAFS system on phantoms, but also key issues which precluded its use in a clinical application. This work proposes a redesign of the RAFS’s navigation system overcoming the earlier version’s issues, aiming to move the RAFS system into a surgical environment.

Methods

The navigation system is improved through an image registration framework allowing the intra-operative registration between pre-operative CT images and intra-operative fluoroscopic images of a fractured bone using a custom-made fiducial marker. The objective of the registration is to estimate the relative pose between a bone fragment and an orthopaedic manipulation pin inserted into it intra-operatively. The actual pose of the bone fragment can be updated in real time using an optical tracker, enabling the image guidance.

Results

Experiments on phantom and cadavers demonstrated the accuracy and reliability of the registration framework, showing a reduction accuracy (sTRE) of about \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$0.88~\pm 0.2\,\hbox {mm}$$\end{document}0.88±0.2mm (phantom) and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.15\pm 0.8\,\hbox {mm}$$\end{document}1.15±0.8mm (cadavers). Four distal femur fractures were successfully reduced in cadaveric specimens using the improved navigation system and the RAFS system following the new clinical workflow (reduction error \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.2\pm 0.3\,\hbox {mm}$$\end{document}1.2±0.3mm, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2\pm 1{^{\circ }})$$\end{document}2±1∘).

Conclusion

Experiments showed the feasibility of the image registration framework. It was successfully integrated into the navigation system, allowing the use of the RAFS system in a realistic surgical application.

Intra-operative 3D imaging system for robot-assisted fracture manipulation

Reduction is a crucial step in the treatment of broken bones. Achieving precise anatomical alignment of bone fragments is essential for a good fast healing process. Percutaneous techniques are associated with faster recovery time and lower infection risk. However, deducing intra-operatively the desired reduction position is quite challenging due to the currently available technology. The 2D nature of this technology (i.e. the image intensifier) doesn't provide enough information to the surgeon regarding the fracture alignment and rotation, which is actually a three-dimensional problem. This paper describes the design and development of a 3D imaging system for the intra-operative virtual reduction of joint fractures. The proposed imaging system is able to receive and segment CT scan data of the fracture, to generate the 3D models of the bone fragments, and display them on a GUI. A commercial optical tracker was included into the system to track the actual pose of the bone fragments in the physical space, and generate the corresponding pose relations in the virtual environment of the imaging system. The surgeon virtually reduces the fracture in the 3D virtual environment, and a robotic manipulator connected to the fracture through an orthopedic pin executes the physical reductions accordingly. The system is here evaluated through fracture reduction experiments, demonstrating a reduction accuracy of 1.04 ± 0.69 mm (translational RMSE) and 0.89 ± 0.71 ° (rotational RMSE).

Design and real-time control of a robotic system for fracture manipulation

This paper presents the design, development and control of a new robotic system for fracture manipulation. The objective is to improve the precision, ergonomics and safety of the traditional surgical procedure to treat joint fractures. The achievements toward this direction are here reported and include the design, the real-time control architecture and the evaluation of a new robotic manipulator system. The robotic manipulator is a 6-DOF parallel robot with the struts developed as linear actuators. The control architecture is also described here. The high-level controller implements a host-target structure composed by a host computer (PC), a real-time controller, and an FPGA. A graphical user interface was designed allowing the surgeon to comfortably automate and monitor the robotic system. The real-time controller guarantees the determinism of the control algorithms adding an extra level of safety for the robotic automation. The system's positioning accuracy and repeatability have been demonstrated showing a maximum positioning RMSE of 1.18 ± 1.14mm (translations) and 1.85 ± 1.54° (rotations).

Navigation system for robot-assisted intra-articular lower-limb fracture surgery

Purpose

In the surgical treatment for lower-leg intra-articular fractures, the fragments have to be positioned and aligned to reconstruct the fractured bone as precisely as possible, to allow the joint to function correctly again. Standard procedures use 2D radiographs to estimate the desired reduction position of bone fragments. However, optimal correction in a 3D space requires 3D imaging. This paper introduces a new navigation system that uses pre-operative planning based on 3D CT data and intra-operative 3D guidance to virtually reduce lower-limb intra-articular fractures. Physical reduction in the fractures is then performed by our robotic system based on the virtual reduction.

Methods

3D models of bone fragments are segmented from CT scan. Fragments are pre-operatively visualized on the screen and virtually manipulated by the surgeon through a dedicated GUI to achieve the virtual reduction in the fracture. Intra-operatively, the actual position of the bone fragments is provided by an optical tracker enabling real-time 3D guidance. The motion commands for the robot connected to the bone fragment are generated, and the fracture physically reduced based on the surgeon’s virtual reduction. To test the system, four femur models were fractured to obtain four different distal femur fracture types. Each one of them was subsequently reduced 20 times by a surgeon using our system.

Results

The navigation system allowed an orthopaedic surgeon to virtually reduce the fracture with a maximum residual positioning error of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$0.95 \pm 0.3\,\hbox {mm}$$\end{document}0.95±0.3mm (translational) and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.4^{\circ } \pm 0.5^{\circ }$$\end{document}1.4∘±0.5∘ (rotational). Correspondent physical reductions resulted in an accuracy of 1.03 ± 0.2 mm and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.56^{\circ }\pm 0.1^{\circ }$$\end{document}1.56∘±0.1∘, when the robot reduced the fracture.

Conclusions

Experimental outcome demonstrates the accuracy and effectiveness of the proposed navigation system, presenting a fracture reduction accuracy of about 1 mm and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.5^{\circ }$$\end{document}1.5∘, and meeting the clinical requirements for distal femur fracture reduction procedures.

Electronic supplementary material

The online version of this article (doi:10.1007/s11548-016-1418-z) contains supplementary material, which is available to authorized users.