Three-dimensional augmented reality surgical navigation with hybrid optical and electromagnetic tracking for distal intramedullary nail interlocking

Distortion-Free Magnetic Tracking of Metal Instruments in Image-Guided Interventions

Electromagnetic tracking (EMT) can benefit image-guided interventions in cases where line of sight is unavailable. However, EMT can suffer from electromagnetic distortion in the presence of metal instruments. Metal instruments are widely used in laparoscopic surgery, ENT surgery, arthroscopy and many other clinical applications. In this work, we investigate the feasibility of tracking such metal instruments by placing the inductive sensor within the instrument shaft. We propose a magnetostatic model of the field within the instrument, and verify the results experimentally for frequencies from 6 kHz to 60 kHz. The impact of the instrument's dimensions, conductivity and transmitting field frequency is quantified for ranges representative of typical metal instruments used in image-guided interventions. We then performed tracking using the open-source Anser EMT system and quantify the error caused by the presence of the rod as a function of the frequency of the eight emitting coils for the system. The work clearly demonstrates why smaller tool diameters (less than 8 mm) are less susceptible to distortion, as well as identifying optimal frequencies (1 kHz to 2 kHz) for transmitter design to minimise for distortion in larger instruments.

Development of Augmented Reality Vision for Osteosynthesis Using a 3D Camera

BACKGROUND

We developed a 3D camera system to track motion in a surgical field. This system has the potential to introduce augmented reality (AR) systems non-invasively, eliminating the need for the invasive AR markers conventionally required. The present study was performed to verify the real-time tracking accuracy of this system, assess the feasibility of integrating this system into the surgical workflow, and establish its potential to enhance the accuracy and efficiency of orthopedic procedures.

METHODS

To evaluate the accuracy of AR technology using a 3D camera, a forearm bone model was created. The forearm model was depicted using a 3D camera, and its accuracy was verified in terms of the positional relationship with a 3D bone model created from previously imaged CT data. Images of the surgical field (capturing the actual forearm) were taken and saved in nine poses by rotating the forearm from pronation to supination. The alignment of the reference points was computed at the three points of CT versus the three points of the 3D camera, yielding a 3D rotation matrix representing the positional relationship. In the original system, a stereo vision-based 3D camera, with a depth image resolution of 1280×720 pixels, 30 frames per second, and a lens field of view of 64 specifications, with a baseline of 3 cm, capable of optimally acquiring real-time 3D data at a distance of 40-60 cm from the subject was used. In the modified system, the following modifications were made to improve tracking performance: (1) color filter processing was changed from HSV to RGB, (2) positional detection accuracy was modified with supporting marker sizes of 8 mm in diameter, and (3) the detection of marker positions was stabilized by calculating the marker position for each frame. Tracking accuracy was examined with the original system and modified system for the following parameters: differences in the rotation matrix, maximum and minimum inter-reference point errors between CT-based and camera-based 3D data, and the average error for the three reference points.

RESULTS

In the original system, the average difference in rotation matrices was 5.51±2.68 mm. Average minimum and maximum errors were 1.10±0.61 and 15.53±12.51 mm, respectively. The average error of reference points was 6.26±4.49 mm. In the modified system, the average difference in rotation matrices was 4.22±1.73 mm. Average minimum and maximum errors were 0.79±0.49 and 1.94±0.87 mm, respectively. The average error of reference points was 1.41±0.58 mm. In the original system, once tracking failed, it was difficult to recover tracking accuracy. This resulted in a large maximum error in supination positions. These issues were resolved by the modified system. Significant improvements were achieved in maximum errors and average errors using the modified system (P<0.05).

CONCLUSION

AR technology using a 3D camera was developed. This system allows direct comparisons of 3D data from preoperative CT scans with 3D data acquired from the surgical field using a 3D camera. This method has the advantage of introducing AR into the surgical field without invasive markers.

Determination of intramedullary nail based on centerline adaptive registration

Objective: Internal fixation with intramedullary nails is a gold standard for the treatment of femoral shaft fractures. However, both the mismatch between intramedullary nails and the medullary cavity and inaccurate positioning of entry points will lead to deformation of intramedullary nails after implantation. The study aimed to determine a suitable intramedullary nail with an optimal entry point for a specific patient based on centerline adaptive registration.

Method: A homotopic thinning algorithm is employed to extract centerlines of the femoral medullary cavity and the intramedullary nail. The two centerlines are registered to obtain a transformation. The medullary cavity and the intramedullary nail are registered based on the transformation. Next, a plane projection method is employed to calculate the surface points of the intramedullary nail laid outside the medullary cavity. According to the distribution of compenetration points, an iterative adaptive registration strategy is designed to decide an optimal position of the intramedullary nail in medullary cavity. The isthmus centerline is extended to the femur surface, where the entry point of the intramedullary nail is located. The suitability of an intramedullary nail for a specific patient was calculated by measuring the geometric quantities reflecting the interference between the femur and nail, and the suitability values of all nails are compared and the most suitable one is determined.

Results: The growth experiment indicated that the bone to nail alignment is indeed affected by the extension of the isthmus centerline, including the extension direction and velocity. The geometrical experiment showed that this method could find the best registration position of intramedullary nails and select the optimal intramedullary nail for a specific patient. In the model experiments, the determined intramedullary nail could be successfully placed into the medullary cavity through the optimal entry point. A pre-screening tool to determine nails which can be successfully used has been given. In addition, the distal hole was accurately located within 14.28 s.

Conclusion: These results suggest that the proposed method can select a suitable intramedullary nail with an optimal entry point. The position of the intramedullary nail can be determined in the medullary cavity, while deformation is avoided. The proposed method can determine the largest diameter intramedullary nail with as little damage to the intramedullary tissue as possible. The proposed method provides preparation aid for internal fixation with intramedullary nails guided by navigation systems or extracorporeal aimers.

Preoperative Mixed-Reality Visualization of Complex Tibial Plateau Fractures and Its Benefit Compared to CT and 3D Printing

BACKGROUND

Various studies have shown the benefit of three-dimensional (3D) computed tomography (CT) reconstruction and especially 3D printing in the treatment of tibial plateau fractures (TPFs). This study aimed to investigate whether mixed-reality visualization (MRV) using mixed-reality glasses can provide a benefit for CT and/or 3D printing in planning treatment strategies for complex TPFs.

METHODS

Three complex TPFs were selected for the study and processed for 3D imaging. Subsequently, the fractures were presented to specialists in trauma surgery using CT (including 3D CT reconstruction), MRV (hardware: Microsoft HoloLens 2; software: mediCAD MIXED REALITY) and 3D prints. A standardized questionnaire on fracture morphology and treatment strategy was completed after each imaging session.

RESULTS

23 surgeons from 7 hospitals were interviewed. A total of 69.6% (n = 16) of those had treated at least 50 TPFs. A change in fracture classification according to Schatzker was recorded in 7.1% of the cases and in 78.6% an adjustment of the ten-segment classification was observed after MRV. In addition, the intended patient positioning changed in 16.1% of the cases, the surgical approach in 33.9% and osteosynthesis in 39.3%. A total of 82.1% of the participants rated MRV as beneficial compared to CT regarding fracture morphology and treatment planning. An additional benefit of 3D printing was reported in 57.1% of the cases (five-point Likert scale).

CONCLUSIONS

Preoperative MRV of complex TPFs leads to improved fracture understanding, better treatment strategies and a higher detection rate of fractures in posterior segments, and it thus has the potential to improve patient care and outcomes.

Visualization, registration and tracking techniques for augmented reality guided surgery: a review

Augmented reality (AR) surgical navigation has developed rapidly in recent years. This paper reviews and analyzes the visualization, registration, and tracking techniques used in AR surgical navigation systems, as well as the application of these AR systems in different surgical fields. The types of AR visualization are divided into two categories ofin situvisualization and nonin situvisualization. The rendering contents of AR visualization are various. The registration methods include manual registration, point-based registration, surface registration, marker-based registration, and calibration-based registration. The tracking methods consist of self-localization, tracking with integrated cameras, external tracking, and hybrid tracking. Moreover, we describe the applications of AR in surgical fields. However, most AR applications were evaluated through model experiments and animal experiments, and there are relatively few clinical experiments, indicating that the current AR navigation methods are still in the early stage of development. Finally, we summarize the contributions and challenges of AR in the surgical fields, as well as the future development trend. Despite the fact that AR-guided surgery has not yet reached clinical maturity, we believe that if the current development trend continues, it will soon reveal its clinical utility.

Clinical applications of augmented reality in orthopaedic surgery: a comprehensive narrative review

PURPOSE

The development of augmented reality (AR) technology allows orthopaedic surgeons to incorporate and visualize surgical data, assisting the execution of both routine and complex surgical operations. Uniquely, AR technology allows a surgeon to view the surgical field and superimpose peri-operative imaging, anatomical landmarks, navigation guidance, and more, all in one view without the need for conjugate gaze between multiple screens. The aim of this literature review was to introduce the fundamental requirements for an augmented reality system and to assess the current applications, outcomes, and potential limitations to this technology.

METHODS

A literature search was performed using MEDLINE and Embase databases, by two independent reviewers, who then collaboratively synthesized and collated the results of the literature search into a narrative review focused on the applications of augmented reality in major orthopaedic sub-specialties.

RESULTS

Current technology requires that pre-operative patient data be acquired, and AR-compatible models constructed. Intra-operatively, to produce manipulatable virtual images into the user's view in real time, four major components are required including a camera, computer image processing technology, tracking tools, and an output screen. The user is provided with a heads-up display, which is a transparent display, enabling the user to look at both their natural view and the computer-generated images. Currently, high-quality evidence for clinical implementation of AR technology in the orthopaedic surgery operating room is lacking; however, growing in vitro literature highlights a multitude of potential applications, including increasing operative accuracy, improved biomechanical angular and alignment parameters, and potentially reduced operative time.

CONCLUSION

While the application of AR systems in surgery is currently in its infancy, we anticipate rapid and widespread implementation of this technology in various orthopaedic sub-specialties.

A review of advances in image-guided orthopedic surgery

Orthopedic surgery remains technically demanding due to the complex anatomical structures and cumbersome surgical procedures. The introduction of image-guided orthopedic surgery (IGOS) has significantly decreased the surgical risk and improved the operation results. This review focuses on the application of recent advances in artificial intelligence (AI), deep learning (DL), augmented reality (AR) and robotics in image-guided spine surgery, joint arthroplasty, fracture reduction and bone tumor resection. For the pre-operative stage, key technologies of AI and DL based medical image segmentation, 3D visualization and surgical planning procedures are systematically reviewed. For the intra-operative stage, the development of novel image registration, surgical tool calibration and real-time navigation are reviewed. Furthermore, the combination of the surgical navigation system with AR and robotic technology is also discussed. Finally, the current issues and prospects of the IGOS system are discussed, with the goal of establishing a reference and providing guidance for surgeons, engineers, and researchers involved in the research and development of this area.

From Diagnosis to Therapy: The Role of Virtual and Augmented Reality in Orthopaedic Trauma Surgery

By reducing procedure-related problems, advancements in computer-assisted surgery (CAS) and surgical training aim to boost operative precision and enhance patient safety. Orthopaedic training and practice have started to change as a result of the incorporation of reality technologies like virtual reality (VR), augmented reality (AR), and mixed reality (MR) into CAS. Today's trainees can engage in realistic and highly involved operational simulations without supervision. With the coronavirus disease 2019 (COVID-19) pandemic, there is a greater need for breakthrough technology adoption. VR is an interactive technology that enables personalised care and could support successful patient-centered rehabilitation. It is a valid and trustworthy evaluation method for determining joint range of motion, function, and balance in physical rehabilitation. It may make it possible to customise care, encourage patients, boost compliance, and track their advancement. AR supplementation in orthopaedic surgery has shown promising results in pre-clinical settings, with improvements in surgical accuracy and reproducibility, decreased operating times, and less radiation exposure. As little patient observation is needed, this may lessen the workload clinicians must bear. The ability to use it for home-based therapy is often available commercially as well.

The objectives of this review are to evaluate the technology available, comprehend the available evidence regarding the benefit, and take into account implementation problems in clinical practice. The use of this technology, its practical and moral ramifications, and how it will affect orthopaedic doctors and their patients are also covered. This review offers a current and thorough analysis of the reality technologies and their uses in orthopaedic surgery.

Surgery Training and Simulation Using Virtual and Augmented Reality for Knee Arthroplasty

A range of extended reality technology integration, including immersive virtual reality (IVR), augmented reality (AR), as well as mixed reality, has lately acquired favour in orthopaedics. The utilization of extended reality machinery in knee arthroplasty is examined in this review study. Virtual reality (VR) and AR are usually exercised together in orthopaedic surgical training as alluring training outside of the operation theatre is acknowledged as a good surgical training tool. The use of this technology, its consequences for orthopaedic surgeons and their patients, and its moral and practical issues are also covered. Head-mounted displays (HMDs) are a potential addition directed toward improving surgical precision along with instruction. Although the hardware is cutting-edge, substantial effort needs to be done to develop software that enables seamless, trustworthy integration into clinical practice and training. Remote virtual rehabilitation has drawn increasing attention in recent years, and its significance has increased in light of the recent outbreak of the COVID-19 epidemic. Numerous medical sectors have shown the benefits of telerehabilitation, gamification, VR, and AR. Given the rising demand for orthopaedic therapy and its rising costs, this is a requirement. A remote surgeon can impart knowledge without being present, by virtually placing his or her hands in the visual field of a local surgeon using AR technology. With the use of this innovation, orthopaedic surgery seems to have been able to participate in the telemedicine revolution. This technology may also have an impact on how surgeons collaborate and train for orthopaedic residencies in the future. Volatility in the HMD market will probably stall improvements in surgical education.

Validation of an RF Image System for Real-Time Tracking Neurosurgical Tools

A radio frequency (RF)-based system for surgical navigation is presented. Surgical navigation technologies are widely used nowadays for aiding the surgical team with many interventions. However, the currently available options still pose considerable limitations, such as line-of-sight occlusion prevention or restricted materials and equipment allowance. In this work, we suggest a different approach based on a microwave broadband antenna system. We combine techniques from microwave medical imaging, which can overcome the current limitations in surgical navigation technologies, and we propose methods to develop RF-based systems for real-time tracking neurosurgical tools. The design of the RF system to perform the measurements is shown and discussed, and two methods (Multiply and Sum and Delay Multiply and Sum) for building the medical images are analyzed. From these measurements, a surgical tool's position tracking system is developed and experimentally assessed in an emulated surgical scenario. The reported results are coherent with other approaches found in the literature, while overcoming their main practical limitations. The discussion of the results discloses some hints on the validity of the system, the optimal configurations depending on the requirements, and the possibilities for future enhancements.

Augmented Reality: Mapping Methods and Tools for Enhancing the Human Role in Healthcare HMI

Background: Augmented Reality (AR) represents an innovative technology to improve data visualization and strengthen the human perception. Among Human–Machine Interaction (HMI), medicine can benefit most from the adoption of these digital technologies. In this perspective, the literature on orthopedic surgery techniques based on AR was evaluated, focusing on identifying the limitations and challenges of AR-based healthcare applications, to support the research and the development of further studies. Methods: Studies published from January 2018 to December 2021 were analyzed after a comprehensive search on PubMed, Google Scholar, Scopus, IEEE Xplore, Science Direct, and Wiley Online Library databases. In order to improve the review reporting, the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines were used. Results: Authors selected sixty-two articles meeting the inclusion criteria, which were categorized according to the purpose of the study (intraoperative, training, rehabilitation) and according to the surgical procedure used. Conclusions: AR has the potential to improve orthopedic training and practice by providing an increasingly human-centered clinical approach. Further research can be addressed by this review to cover problems related to hardware limitations, lack of accurate registration and tracking systems, and absence of security protocols.

Smart Technology and Orthopaedic Surgery: Current Concepts Regarding the Impact of Smartphones and Wearable Technology on Our Patients and Practice

PURPOSE OF REVIEW

While limited to case reports or small case series, emerging evidence advocates the inclusion of smartphone-interfacing mobile platforms and wearable technologies, consisting of internet-powered mobile and wearable devices that interface with smartphones, in the orthopaedic surgery practice. The purpose of this review is to investigate the relevance and impact of this technology in orthopaedic surgery.

RECENT FINDINGS

Smartphone-interfacing mobile platforms and wearable technologies are capable of improving the patients' quality of life as well as the extent of their therapeutic engagement, while promoting the orthopaedic surgeons' abilities and level of care. Offered advantages include improvements in diagnosis and examination, preoperative templating and planning, and intraoperative assistance, as well as postoperative monitoring and rehabilitation. Supplemental surgical exposure, through haptic feedback and realism of audio and video, may add another perspective to these innovations by simulating the operative environment and potentially adding a virtual tactile feature to the operator's visual experience. Although encouraging in the field of orthopaedic surgery, surgeons should be cautious when using smartphone-interfacing mobile platforms and wearable technologies, given the lack of a current academic governing board certification and clinical practice validation processes.

Endoscopy-assisted distal locking of an intramedullary nail: A new experimental technique to reduce radiation exposure during distal locking of the intramedullary nails

Objectives

In this study, we aimed to introduce a new technique in which distal locking step was easier and safer for orthopedic trauma surgeons involving in the treatment of long bone fractures using an intranail endoscopic visualization and illumination method.

Materials and methods

A total of 20 fresh bovine hind limbs derived from healthy adults and killed for sale were dissected to obtain tibial bones. Two equal groups including 10 samples in each were prepared. We applied a standard nailing process in both groups: the study group (n=10) was locked by the new technique, intranail endoscopic illumination guidance and intranail visualization assistance locking technique and the control group (n=10) was locked by the classical free-hand fluoroscopic guidance technique. We measured the surgical period time and the radiation exposure time required for the distal locking in both groups.

Results

The radiation exposure time was statistically significantly lower in the study group compared to the control group. Also, the time period required for distal locking in the study group were statistically significantly lower than the control group. With the use of the intranail endoscopic illumination guidance and visualization assistance technique, the median period time required for the distal locking procedure reduced from 477.5 to 223.5 sec (p<0.001). The median time for radiation exposure dramatically reduced from 13.5 to 2 sec (p<0.001). The median attempt number reduced from 6.5 to 2 times (p<0.001).

Conclusion

This experimental study indicates that the endoscopic illumination and intranail visualization assistance technique can reduce the radiation exposure time and the period time required for distal locking compared to the free-hand fluoroscopic guidance.

Augmented Reality (AR) in Orthopedics: Current Applications and Future Directions

PURPOSE OF REVIEW

Imaging technologies (X-ray, CT, MRI, and ultrasound) have revolutionized orthopedic surgery, allowing for the more efficient diagnosis, monitoring, and treatment of musculoskeletal aliments. The current review investigates recent literature surrounding the impact of augmented reality (AR) imaging technologies on orthopedic surgery. In particular, it investigates the impact that AR technologies may have on provider cognitive burden, operative times, occupational radiation exposure, and surgical precision and outcomes.

RECENT FINDINGS

Many AR technologies have been shown to lower provider cognitive burden and reduce operative time and radiation exposure while improving surgical precision in pre-clinical cadaveric and sawbones models. So far, only a few platforms focusing on pedicle screw placement have been approved by the FDA. These technologies have been implemented clinically with mixed results when compared to traditional free-hand approaches. It remains to be seen if current AR technologies can deliver upon their multitude of promises, and the ability to do so seems contingent upon continued technological progress. Additionally, the impact of these platforms will likely be highly conditional on clinical indication and provider type. It remains unclear if AR will be broadly accepted and utilized or if it will be reserved for niche indications where it adds significant value. One thing is clear, orthopedics' high utilization of pre- and intra-operative imaging, combined with the relative ease of tracking rigid structures like bone as compared to soft tissues, has made it the clear beachhead market for AR technologies in medicine.

Intramedullary nail holes laser indicator, a non-invasive technique for interlocking of intramedullary nails

Interlocking of intramedullary nails is a challenging procedure in orthopedic trauma surgery. Numerous methods have been described to facilitate this process. But they are exposed patient and surgical team to X-rays or involves trial and error. An accurate and non-invasive method has been provided to easily interlocking intramedullary nails. By transferring a safe visible light inside the nail, a drilling position appears which use to drilling bone toward the nail hole. The wavelength of this light was obtained from ex-vivo spectroscopy on biological tissues which has optimal transmission, reflectance, and absorption properties. Moreover, animal and human experiments were performed to evaluate performance of the proposed system. Ex-vivo performance experiments were performed successfully on two groups of cow and sheep samples. Output parameters were procedure time and drilling quality which there were significant differences between the two groups in procedure time (P < 0.05). But no significant differences were observed in drilling quality (P > 0.05). Moreover, an In-vivo performance experiment was performed successfully on a middle-aged man. To compare the provided method, targeting-arm, and free-hand techniques, two human experiments were performed on a middle-aged and a young man. The results indicate the advantage of the proposed technique in the procedure time (P < 0.05), while the drilling quality is equal to the free-hand technique (P = 0.05). Intramedullary nail holes laser indicator is a safe and accurate method that reduced surgical time and simplifies the process. This new technology makes it easier to interlocking the intramedullary nail which can have good clinical applications.

The Clinical Application of Augmented Reality in Orthopaedics: Where Do We Stand?

PURPOSE OF REVIEW

The surgical community is constantly working to improve accuracy and reproducibility in patient care, with the goal to improve patient outcomes and efficiency. One area of growing interest with potential to meet these goals is in the use of augmented reality (AR) in surgery. There is still a paucity of published research on the clinical benefits of AR over traditional techniques, but this article aims to present an update on the current state of AR within orthopaedics over the past 5 years.

RECENT FINDINGS

AR systems are being developed and studied for use in all areas of orthopaedics. Most recently published research has focused on the areas of fracture care, adult reconstruction, orthopaedic oncology, spine, and resident education. These studies have shown some promising results, particularly in surgical accuracy, decreased surgical time, and less radiation exposure. However, the majority of recently published research is still in the pre-clinical setting, with very few studies using living patients. AR supplementation in orthopaedic surgery has shown promising results in pre-clinical settings, with improvements in surgical accuracy and reproducibility, decreased operating times, and less radiation exposure. Most AR systems, however, are still not approved for clinical use. Further research is needed to validate the benefits of AR use in orthopaedic surgery before it is widely adopted into practice.

A novel dynamic electromagnetic tracking navigation system for distal locking of intramedullary nails

BACKGROUND

The accurate distal locking of intramedullary (IM) nails is a clinical challenge for surgeons. Although many navigation systems have been developed, a real-time guide method with free radiation exposure, better user convenience, and high cost performance has not been proposed.

METHODS

This paper aims to develop an electromagnetic navigation system named TianXuan-MDTS that provides surgeons with a proven surgical solution. And the registration method with external landmarks for IM nails and calibration algorithm for guiders were proposed. A puncture experiment, model experiments measured by 3D Slicer and cadaver experiments (2 cadaveric leg specimens and 6 drilling operations) are conducted to evaluate its performance and stability.

RESULTS

The registration deviations (TRE) is 1.05± 0.13 mm. In the puncture experiment, a success rate of 96% can be achieved in 45.94 s. TianXuan-MDTS were evaluated on 3 tibia model. The results demonstrated that all 9 screw holes were successfully prepared at a rate of 100% in 91.67 s. And the entry point, end point, and angular deviations were 1.60±0.20 mm, 1.47±0.18 mm, and 3.10±0.84°, respectively. Postoperative fluoroscopy in cadaver experiments showed that all drills were in the distal locking holes, with a success rate of 100% and the average time 143.17± 18.27 s.

CONCLUSIONS

The experimental results indicate that our system with novel registration and calibration methods could serve as a feasible and promising tool to assist surgeons during distal locking.

Augmented reality based navigation for distal interlocking of intramedullary nails utilizing Microsoft HoloLens 2

BACKGROUND AND OBJECTIVE

The distal interlocking of intramedullary nail remains a technically demanding procedure. Existing augmented reality based solutions still suffer from hand-eye coordination problem, prolonged operation time, and inadequate resolution. In this study, an augmented reality based navigation system for distal interlocking of intramedullary nail is developed using Microsoft HoloLens 2, the state-of-the-art optical see-through head-mounted display.

METHODS

A customized registration cube is designed to assist surgeons with better depth perception when performing registration procedures. During drilling, surgeons can obtain accurate and in-situ visualization of intramedullary nail and drilling path, and dynamic navigation is enabled. An intraoperative warning system is proposed to provide intuitive feedback of real-time deviations and electromagnetic disturbances.

RESULTS

The preclinical phantom experiment showed that the reprojection errors along the X, Y, and Z axes were 1.55 ± 0.27 mm, 1.71 ± 0.40 mm, and 2.84 ± 0.78 mm, respectively. The end-to-end evaluation method indicated the distance error was 1.61 ± 0.44 mm, and the 3D angle error was 1.46 ± 0.46°. A cadaver experiment was also conducted to evaluate the feasibility of the system.

CONCLUSION

Our system has potential advantages over the 2D-screen based navigation system and the pointing device based navigation system in terms of accuracy and time consumption, and has tremendous application prospects.

Augmented Reality, Virtual Reality and Artificial Intelligence in Orthopedic Surgery: A Systematic Review

Background: The application of virtual and augmented reality technologies to orthopaedic surgery training and practice aims to increase the safety and accuracy of procedures and reducing complications and costs. The purpose of this systematic review is to summarise the present literature on this topic while providing a detailed analysis of current flaws and benefits. Methods: A comprehensive search on the PubMed, Cochrane, CINAHL, and Embase database was conducted from inception to February 2021. The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines were used to improve the reporting of the review. The Cochrane Risk of Bias Tool and the Methodological Index for Non-Randomized Studies (MINORS) was used to assess the quality and potential bias of the included randomized and non-randomized control trials, respectively. Results: Virtual reality has been proven revolutionary for both resident training and preoperative planning. Thanks to augmented reality, orthopaedic surgeons could carry out procedures faster and more accurately, improving overall safety. Artificial intelligence (AI) is a promising technology with limitless potential, but, nowadays, its use in orthopaedic surgery is limited to preoperative diagnosis. Conclusions: Extended reality technologies have the potential to reform orthopaedic training and practice, providing an opportunity for unidirectional growth towards a patient-centred approach.

Development and Pre-Clinical Analysis of Spatiotemporal-Aware Augmented Reality in Orthopedic Interventions

Suboptimal interaction with patient data and challenges in mastering 3D anatomy based on ill-posed 2D interventional images are essential concerns in image-guided therapies. Augmented reality (AR) has been introduced in the operating rooms in the last decade; however, in image-guided interventions, it has often only been considered as a visualization device improving traditional workflows. As a consequence, the technology is gaining minimum maturity that it requires to redefine new procedures, user interfaces, and interactions. The main contribution of this paper is to reveal how exemplary workflows are redefined by taking full advantage of head-mounted displays when entirely co-registered with the imaging system at all times. The awareness of the system from the geometric and physical characteristics of X-ray imaging allows the exploration of different human-machine interfaces. Our system achieved an error of 4.76 ± 2.91mm for placing K-wire in a fracture management procedure, and yielded errors of 1.57 ± 1.16° and 1.46 ± 1.00° in the abduction and anteversion angles, respectively, for total hip arthroplasty (THA). We compared the results with the outcomes from baseline standard operative and non-immersive AR procedures, which had yielded errors of [4.61mm, 4.76°, 4.77°] and [5.13mm, 1.78°, 1.43°], respectively, for wire placement, and abduction and anteversion during THA. We hope that our holistic approach towards improving the interface of surgery not only augments the surgeon's capabilities but also augments the surgical team's experience in carrying out an effective intervention with reduced complications and provide novel approaches of documenting procedures for training purposes.

Virtual Reality and Augmented Reality-Translating Surgical Training into Surgical Technique

PURPOSE OF REVIEW

As immersive learning outside of the operating room is increasingly recognized as a valuable method of surgical training, virtual reality (VR) and augmented reality (AR) are increasingly utilized in orthopedic surgical training. This article reviews the evolving nature of these training tools and provides examples of their use and efficacy. The practical and ethical implications of incorporating this technology and its impact on both orthopedic surgeons and their patients are also discussed.

RECENT FINDINGS

Head-mounted displays (HMDs) represent a possible adjunct to surgical accuracy and education. While the hardware is advanced, there is still much work to be done in developing software that allows for seamless, reliable, useful integration into clinical practice and training. Surgical training is changing: AR and VR will become mainstays of future training efforts. More evidence is needed to determine which training technology translates to improved clinical performance. Volatility within the HMD industry will likely delay advances in surgical training.

A Survey of Marker-Less Tracking and Registration Techniques for Health & Environmental Applications to Augmented Reality and Ubiquitous Geospatial Information Systems

Most existing augmented reality (AR) applications are suitable for cases in which only a small number of real world entities are involved, such as superimposing a character on a single surface. In this case, we only need to calculate pose of the camera relative to that surface. However, when an AR health or environmental application involves a one-to-one relationship between an entity in the real-world and the corresponding object in the computer model (geo-referenced object), we need to estimate the pose of the camera in reference to a common coordinate system for better geo-referenced object registration in the real-world. New innovations in developing cheap sensors, computer vision techniques, machine learning, and computing power have helped to develop applications with more precise matching between a real world and a virtual content. AR Tracking techniques can be divided into two subcategories: marker-based and marker-less approaches. This paper provides a comprehensive overview of marker-less registration and tracking techniques and reviews their most important categories in the context of ubiquitous Geospatial Information Systems (GIS) and AR focusing to health and environmental applications. Basic ideas, advantages, and disadvantages, as well as challenges, are discussed for each subcategory of tracking and registration techniques. We need precise enough virtual models of the environment for both calibrations of tracking and visualization. Ubiquitous GISs can play an important role in developing AR in terms of providing seamless and precise spatial data for outdoor (e.g., environmental applications) and indoor (e.g., health applications) environments.

Applicability of augmented reality in orthopedic surgery - A systematic review

BACKGROUND

Computer-assisted solutions are changing surgical practice continuously. One of the most disruptive technologies among the computer-integrated surgical techniques is Augmented Reality (AR). While Augmented Reality is increasingly used in several medical specialties, its potential benefit in orthopedic surgery is not yet clear. The purpose of this article is to provide a systematic review of the current state of knowledge and the applicability of AR in orthopedic surgery.

METHODS

A systematic review of the current literature was performed to find the state of knowledge and applicability of AR in Orthopedic surgery. A systematic search of the following three databases was performed: "PubMed", "Cochrane Library" and "Web of Science". The systematic review followed the Preferred Reporting Items on Systematic Reviews and Meta-analysis (PRISMA) guidelines and it has been published and registered in the international prospective register of systematic reviews (PROSPERO).

RESULTS

31 studies and reports are included and classified into the following categories: Instrument / Implant Placement, Osteotomies, Tumor Surgery, Trauma, and Surgical Training and Education. Quality assessment could be performed in 18 studies. Among the clinical studies, there were six case series with an average score of 90% and one case report, which scored 81% according to the Joanna Briggs Institute Critical Appraisal Checklist (JBI CAC). The 11 cadaveric studies scored 81% according to the QUACS scale (Quality Appraisal for Cadaveric Studies).

CONCLUSION

This manuscript provides 1) a summary of the current state of knowledge and research of Augmented Reality in orthopedic surgery presented in the literature, and 2) a discussion by the authors presenting the key remarks required for seamless integration of Augmented Reality in the future surgical practice.

TRIAL REGISTRATION

PROSPERO registration number: CRD42019128569.

Design and Analysis of Customized Fixation Plate for Femoral Shaft

BACKGROUND

In the recent years, several techniques have been used to treat femur diaphyseal fracture. Among all the traditional fixation techniques, unstable fixation remains the biggest challenge for orthopedists. Researchers have recommended new approaches to deal with diaphyseal femur fracture. However, solely few had been successful in getting some better results. In the present work, a methodology comprising of design and finite-element analysis of a counter fit customized fixation plate has been suggested to provide a stable fixation.

MATERIALS AND METHODS

In the present work, reverse engineering (RE) approach has been invoked to create a 3D model of a fresh fractured femur diaphysis bone using the computed tomography (CT) scan data available in digital imaging and communications in medicine (DICOM) format. To provide stable fixation, a counter fit customized fixation plate at medial side has been designed and simulated under static physiological loading conditions for three different biocompatible materials, viz., titanium alloy (Ti6Al4V), stainless steel (SS-316L), and cobalt-chromium-molybdenum alloy (Co-Cr-Mo).

RESULTS

Static stress distribution and deformation analysis of the clinical setup have been performed for the aforementioned materials. It has been observed that the stresses and deformation developed in all the materials are quite low. It implies that customized fixation plates will provide stable fixation resulting in improved fracture union.

CONCLUSION

The proposed work will assist the medical practitioners regarding the design and analysis of customized implants. This will reduce the post surgical failures and residual pain due to non-union fractured region.

Image Overlay Surgery Based on Augmented Reality: A Systematic Review

Augmented Reality (AR) applied to surgical guidance is gaining relevance in clinical practice. AR-based image overlay surgery (i.e. the accurate overlay of patient-specific virtual images onto the body surface) helps surgeons to transfer image data produced during the planning of the surgery (e.g. the correct resection margins of tissue flaps) to the operating room, thus increasing accuracy and reducing surgery times. We systematically reviewed 76 studies published between 2004 and August 2018 to explore which existing tracking and registration methods and technologies allow healthcare professionals and researchers to develop and implement these systems in-house. Most studies used non-invasive markers to automatically track a patient's position, as well as customised algorithms, tracking libraries or software development kits (SDKs) to compute the registration between patient-specific 3D models and the patient's body surface. Few studies combined the use of holographic headsets, SDKs and user-friendly game engines, and described portable and wearable systems that combine tracking, registration, hands-free navigation and direct visibility of the surgical site. Most accuracy tests included a low number of subjects and/or measurements and did not normally explore how these systems affect surgery times and success rates. We highlight the need for more procedure-specific experiments with a sufficient number of subjects and measurements and including data about surgical outcomes and patients' recovery. Validation of systems combining the use of holographic headsets, SDKs and game engines is especially interesting as this approach facilitates an easy development of mobile AR applications and thus the implementation of AR-based image overlay surgery in clinical practice.

Knee arthroscopic navigation using virtual-vision rendering and self-positioning technology

PURPOSE

Knee arthroscopy suffers from a lack of depth information and easy occlusion of the visual field. To solve these limitations, we propose an arthroscopic navigation system based on self-positioning technology, with the guidance of virtual-vision views. This system can work without any external tracking devices or added markers, thus increasing the working range and improving the robustness of the rotating operation.

METHODS

The fly-through view and global positioning view for surgical guidance are rendered through virtual-vision rendering in real time. The fly-through view provides surgeons with navigating the arthroscope in the internal anatomical structures using a virtual camera perspective. The global positioning view shows the posture of the arthroscope relative to the preoperative model in a transparent manner. The posture of the arthroscope is estimated from the fusion of visual and inertial data based on the visual-inertial stereo slam. A flexible calibration method that transforms the posture of the arthroscope in the physical world into the virtual-vision rendering framework is proposed for the arthroscopic navigation system with self-positioning information.

RESULTS

Quantitative experiments for evaluating self-positioning accuracy were performed. For translation, the acquired mean error was 0.41 ± 0.28 mm; for rotation, it was 0.11° ± 0.07°. The tracking range of the proposed system was approximately 1.4 times that of the traditional external optical tracking system for the rotating operation. Simulated surgical operations were performed on the phantom. The fly-through and global positing views were paired with original arthroscopic images for intuitive surgical guidance.

CONCLUSION

The proposed system provides surgeons with both fly-through and global positioning views without a dependence on the traditional external tracking systems for surgical guidance. The feasibility and robustness of the system are evaluated, and it shows promise for medical applications.

Augmented reality in orthopaedics

Aims

Computer-based applications are increasingly being used by orthopaedic surgeons in their clinical practice. With the integration of technology in surgery, augmented reality (AR) may become an important tool for surgeons in the future. By superimposing a digital image on a user’s view of the physical world, this technology shows great promise in orthopaedics. The aim of this review is to investigate the current and potential uses of AR in orthopaedics.

Materials and Methods

A systematic review of the PubMed, MEDLINE, and Embase databases up to January 2019 using the keywords ‘orthopaedic’ OR ‘orthopedic AND augmented reality’ was performed by two independent reviewers.

Results

A total of 41 publications were included after screening. Applications were divided by subspecialty: spine (n = 15), trauma (n = 16), arthroplasty (n = 3), oncology (n = 3), and sports (n = 4). Out of these, 12 were clinical in nature. AR-based technologies have a wide variety of applications, including direct visualization of radiological images by overlaying them on the patient and intraoperative guidance using preoperative plans projected onto real anatomy, enabling hands-free real-time access to operating room resources, and promoting telemedicine and education.

Conclusion

There is an increasing interest in AR among orthopaedic surgeons. Although studies show similar or better outcomes with AR compared with traditional techniques, many challenges need to be addressed before this technology is ready for widespread use. Cite this article: Bone Joint J 2019;101-B:1479–1488

Optical and Electromagnetic Tracking Systems for Biomedical Applications: A Critical Review on Potentialities and Limitations

Optical and electromagnetic tracking systems represent the two main technologies integrated into commercially-available surgical navigators for computer-assisted image-guided surgery so far. Optical Tracking Systems (OTSs) work within the optical spectrum to track the position and orientation, i.e., pose of target surgical instruments. OTSs are characterized by high accuracy and robustness to environmental conditions. The main limitation of OTSs is the need of a direct line-of-sight between the optical markers and the camera sensor, rigidly fixed into the operating theatre. Electromagnetic Tracking Systems (EMTSs) use electromagnetic field generator to detect the pose of electromagnetic sensors. EMTSs do not require such a direct line-of-sight, however the presence of metal or ferromagnetic sources in the operating workspace can significantly affect the measurement accuracy. The aim of the proposed review is to provide a complete and detailed overview of optical and electromagnetic tracking systems, including working principles, source of error and validation protocols. Moreover, commercial and research-oriented solutions, as well as clinical applications, are described for both technologies. Finally, a critical comparative analysis of the state of the art which highlights the potentialities and the limitations of each tracking system for a medical use is provided.

Augmented Reality in Orthopedics: Current State and Future Directions

Augmented reality (AR) comprises special hardware and software, which is used in order to offer computer-processed imaging data to the surgeon in real time, so that real-life objects are combined with computer-generated images. AR technology has recently gained increasing interest in the surgical practice. Preclinical research has provided substantial evidence that AR might be a useful tool for intra-operative guidance and decision-making. AR has been applied to a wide spectrum of orthopedic procedures, such as tumor resection, fracture fixation, arthroscopy, and component's alignment in total joint arthroplasty. The present study aimed to summarize the current state of the application of AR in orthopedics, in preclinical and clinical level, providing future directions and perspectives concerning potential further benefits from this technology.