Augmented reality-based navigation system applied to tibial bone resection in total knee arthroplasty

A novel portable augmented reality surgical navigation system for maxillofacial surgery: technique and accuracy study

Surgical navigation, despite its potential benefits, faces challenges in widespread adoption in clinical practice. Possible reasons include the high cost, increased surgery time, attention shifts during surgery, and the mental task of mapping from the monitor to the patient. To address these challenges, a portable, all-in-one surgical navigation system using augmented reality (AR) was developed, and its feasibility and accuracy were investigated. The system achieves AR visualization by capturing a live video stream of the actual surgical field using a visible light camera and merging it with preoperative virtual images. A skull model with reference spheres was used to evaluate the accuracy. After registration, virtual models were overlaid on the real skull model. The discrepancies between the centres of the real spheres and the virtual model were measured to assess the AR visualization accuracy. This AR surgical navigation system demonstrated precise AR visualization, with an overall overlap error of 0.53 ± 0.21 mm. By seamlessly integrating the preoperative virtual plan with the intraoperative field of view in a single view, this novel AR navigation system could provide a feasible solution for the use of AR visualization to guide the surgeon in performing the operation as planned.

A pilot study on preferences from surgeons to deal with an innovative customized and connected knee prosthesis - A discret choice experiment

Background

To address the increasing global demand for Total Knee Arthroplasty and reduce the need for revisions, several technologies combining 3D planning and artificial intelligence have emerged. These innovations aim to enhance customization, improve component positioning accuracy and precision. The integration of these advancements paves the way for the development of personalized and connected knee implant.

Questions/purposes

These groundbreaking advancements may necessitate changes in surgical practices. Hence, it is important to comprehend surgeons' intentions in integrating these technologies into their routine procedures. Our study aims to assess how surgeons' preferences will affect the acceptability of using this new implant and associated technologies within the entire care chain.

Methods

We employed a Discrete Choice Experiment, a predictive technique mirroring real-world healthcare decisions, to assess surgeons' trade-off evaluations and preferences.

Results

A total of 90 experienced surgeons, performing a significant number of procedures annually (mostly over 51) answered. Analysis indicates an affinity for technology but limited interest in integrating digital advancements like preoperative software and robotics. However, they are receptive to practice improvements and considering the adoption of future sensors.

Conclusions

In conclusion, surgeons prefer customized prostheses via augmented reality, accepting extra cost. Embedded sensor technology is deemed premature by them.

An Image-Based Augmented Reality System for Achieving Accurate Bone Resection in Total Knee Arthroplasty

Background and objective With the steady advancement of computer-assisted surgical techniques, the importance of assessing and researching technology related to total knee arthroplasty (TKA) procedures has increased. Augmented reality (AR), a recently proposed next-generation technology, is expected to enhance the precision of orthopedic surgery by providing a more efficient and cost-effective approach. However, the accuracy of image-based AR in TKA surgery has not been established. Therefore, this study aimed to determine whether accurate bone resection can be achieved in TKA surgery using image-based AR. Methods In this study, we replaced traditional CT imaging and reconstructions for creating a bone 3D model by direct 3D scanning of the femur and tibia. The preoperative planning involved identifying anatomical landmarks and determining the surgical details. During surgery, markers were employed to create a local coordinate system for an AR-assisted surgical system using a Polaris camera. This approach helped minimize discrepancies between the 3D model and actual positioning, ensuring accurate alignment. Results The AR-assisted surgery using the image method resulted in fewer errors [average error: 0.32 mm; standard deviation (SD): 0.143] between the bone resection depth of the preoperative surgical plan and the bone model test results. Conclusions Our findings demonstrated the accuracy of bone resectioning by using image-based AR-assisted navigation for TKA surgery. Image-based AR-assisted navigation in TKA surgery is a valuable tool not only for enhancing accuracy by using smart glasses and sensors but also for improving the efficiency of the procedure. Therefore, we anticipate that image-based AR-assisted navigation in TKA surgery will gain wide acceptance in practice.

Accuracy of augmented reality-assisted pedicle screw placement: a systematic review

OBJECTIVE

Conventional freehand methods of pedicle screw placement are associated with significant complications due to close proximity to neural and vascular structures. Recent advances in augmented reality surgical navigation (ARSN) have led to its adoption into spine surgery. However, little is known regarding its overall accuracy. The purpose of this study is to delineate the overall accuracy of ARSN pedicle screw placement across various models.

METHODS

A systematic review was conducted of Medline/PubMed, Cochrane and Embase Library databases according to the PRISMA guidelines. Relevant data extracted included reports of pedicle screw placement accuracy and breaches, as defined by the Gertzbein-Robbins classification, in addition to deviation from pre-planned trajectory and entry point. Accuracy was defined as the summation of grade 0 and grade 1 events per the Gertzbein-Robbins classification.

RESULTS

Twenty studies reported clinically accurate placed screws. The range of clinically accurate placed screws was 26.3-100%, with 2095 screws (93.1%) being deemed clinically accurate. Furthermore, 5.4% (112/2088) of screws were reported as grade two breaches, 1.6% (33/2088) grade 3 breaches, 3.1% (29/926) medial breaches and 2.3% (21/926) lateral breaches. Mean linear deviation ranged from 1.3 to 5.99 mm, while mean angular/trajectory deviation ranged 1.6°-5.88°.

CONCLUSION

The results of this study highlight the overall accuracy of ARSN pedicle screw placement. However, further robust prospective studies are needed to accurately compare to conventional methods of pedicle screw placement.

Is Total Knee Arthroplasty Surgical Performance Enhanced Using Augmented Reality? A Single-Center Study on 76 Consecutive Patients

BACKGROUND

Augmented reality (AR) is a powerful multipurpose tool. With a dedicated visor, AR allows the visualization of a series of information and/or images superimposed on the user's field of vision. For this reason, it was recently introduced as a surgical assistant tool. This single-center study aimed to evaluate the intraoperative outcomes of total knee arthroplasties performed with AR assistance in terms of time required and the difference between preplanned and achieved implant positioning (in terms of tibial cut varus and slope angles).

METHODS

A total of 76 consecutive patients was selected. Preplanning was performed according to the AR protocol, and the target varus and slope angles were defined to instruct the device, which subsequently guided the tibial cuts intraoperatively. Surgeries were performed starting from the tibial cut, and the time required to perform the calibration, registration, and fixation of the resection block was recorded. The varus and slope angles achieved were recorded to compare with the preplanned ones to determine the means and SDs of the differences.

RESULTS

The mean usage time of the AR tool was 5 ± 1 minutes. Results showed a mean difference of 0.59 ± 0.55° for varus angles and 0.70 ± 0.75° for the slope. For varus angles, the differences were <1° for 96% of the cases. Concerning the slope, 89% of the cases were <1°.

CONCLUSION

The results showed excellent accuracy of the surgical cuts and a limited increase in surgery duration. Therefore, these outcomes highlighted the potential of this new technology as a valid option for surgical assistance.

Mixed Reality and Artificial Intelligence: A Holistic Approach to Multimodal Visualization and Extended Interaction in Knee Osteotomy

OBJECTIVE

Recent advancements in augmented reality led to planning and navigation systems for orthopedic surgery. However little is known about mixed reality (MR) in orthopedics. Furthermore, artificial intelligence (AI) has the potential to boost the capabilities of MR by enabling automation and personalization. The purpose of this work is to assess Holoknee prototype, based on AI and MR for multimodal data visualization and surgical planning in knee osteotomy, developed to run on the HoloLens 2 headset.

METHODS

Two preclinical test sessions were performed with 11 participants (eight surgeons, two residents, and one medical student) executing three times six tasks, corresponding to a number of holographic data interactions and preoperative planning steps. At the end of each session, participants answered a questionnaire on user perception and usability.

RESULTS

During the second trial, the participants were faster in all tasks than in the first one, while in the third one, the time of execution decreased only for two tasks ("Patient selection" and "Scrolling through radiograph") with respect to the second attempt, but without statistically significant difference (respectively [Formula: see text] = 0.14 and [Formula: see text] = 0.13, [Formula: see text]). All subjects strongly agreed that MR can be used effectively for surgical training, whereas 10 (90.9%) strongly agreed that it can be used effectively for preoperative planning. Six (54.5%) agreed and two of them (18.2%) strongly agreed that it can be used effectively for intraoperative guidance.

DISCUSSION/CONCLUSION

In this work, we presented Holoknee, the first holistic application of AI and MR for surgical planning for knee osteotomy. It reported promising results on its potential translation to surgical training, preoperative planning, and surgical guidance. Clinical and Translational Impact Statement - Holoknee can be helpful to support surgeons in the preoperative planning of knee osteotomy. It has the potential to impact positively the training of the future generation of residents and aid surgeons in the intraoperative stage.

Exposure to Extended Reality and Artificial Intelligence-Based Manifestations: A Primer on the Future of Hip and Knee Arthroplasty

BACKGROUND

Software-infused services, from robot-assisted and wearable technologies to artificial intelligence (AI)-laden analytics, continue to augment clinical orthopaedics - namely hip and knee arthroplasty. Extended reality (XR) tools, which encompass augmented reality, virtual reality, and mixed reality technology, represent a new frontier for expanding surgical horizons to maximize technical education, expertise, and execution. The purpose of this review is to critically detail and evaluate the recent developments surrounding XR in the field of hip and knee arthroplasty and to address potential future applications as they relate to AI.

METHODS

In this narrative review surrounding XR, we discuss (1) definitions, (2) techniques, (3) studies, (4) current applications, and (5) future directions. We highlight XR subsets (augmented reality, virtual reality, and mixed reality) as they relate to AI in the increasingly digitized ecosystem within hip and knee arthroplasty.

RESULTS

A narrative review of the XR orthopaedic ecosystem with respect to XR developments is summarized with specific emphasis on hip and knee arthroplasty. The XR as a tool for education, preoperative planning, and surgical execution is discussed with future applications dependent upon AI to potentially obviate the need for robotic assistance and preoperative advanced imaging without sacrificing accuracy.

CONCLUSION

In a field where exposure is critical to clinical success, XR represents a novel stand-alone software-infused service that optimizes technical education, execution, and expertise but necessitates integration with AI and previously validated software solutions to offer opportunities that improve surgical precision with or without the use of robotics and computed tomography-based imaging.

The Accuracy and Absolute Reliability of a Knee Surgery Assistance System Based on ArUco-Type Sensors

Recent advances allow the use of Augmented Reality (AR) for many medical procedures. AR via optical navigators to aid various knee surgery techniques (e.g., femoral and tibial osteotomies, ligament reconstructions or menisci transplants) is becoming increasingly frequent. Accuracy in these procedures is essential, but evaluations of this technology still need to be made. Our study aimed to evaluate the system's accuracy using an in vitro protocol. We hypothesised that the system's accuracy was equal to or less than 1 mm and 1° for distance and angular measurements, respectively. Our research was an in vitro laboratory with a 316 L steel model. Absolute reliability was assessed according to the Hopkins criteria by seven independent evaluators. Each observer measured the thirty palpation points and the trademarks to acquire direct angular measurements on three occasions separated by at least two weeks. The system's accuracy in assessing distances had a mean error of 1.203 mm and an uncertainty of 2.062, and for the angular values, a mean error of 0.778° and an uncertainty of 1.438. The intraclass correlation coefficient was for all intra-observer and inter-observers, almost perfect or perfect. The mean error for the distance's determination was statistically larger than 1 mm (1.203 mm) but with a trivial effect size. The mean error assessing angular values was statistically less than 1°. Our results are similar to those published by other authors in accuracy analyses of AR systems.

Integration of Square Fiducial Markers in Patient-Specific Instrumentation and Their Applicability in Knee Surgery

Computer technologies play a crucial role in orthopaedic surgery and are essential in personalising different treatments. Recent advances allow the usage of augmented reality (AR) for many orthopaedic procedures, which include different types of knee surgery. AR assigns the interaction between virtual environments and the physical world, allowing both to intermingle (AR superimposes information on real objects in real-time) through an optical device and allows personalising different processes for each patient. This article aims to describe the integration of fiducial markers in planning knee surgeries and to perform a narrative description of the latest publications on AR applications in knee surgery. Augmented reality-assisted knee surgery is an emerging set of techniques that can increase accuracy, efficiency, and safety and decrease the radiation exposure (in some surgical procedures, such as osteotomies) of other conventional methods. Initial clinical experience with AR projection based on ArUco-type artificial marker sensors has shown promising results and received positive operator feedback. Once initial clinical safety and efficacy have been demonstrated, the continued experience should be studied to validate this technology and generate further innovation in this rapidly evolving field.

Augmented Reality Navigation Can Achieve Accurate Coronal Component Alignment During Total Knee Arthroplasty

Background Computer-navigated knee arthroplasty has been shown to improve accuracy over conventional instruments. The next generation of computer assistance is being developed using augmented reality. The accuracy of augmented reality navigation has not been established. Methods From April 2021 to October 2021, a prospective, consecutive series of 20 patients underwent total knee arthroplasty utilising an augmented reality-assisted navigation system (ARAN). The coronal and sagittal alignment of the femoral and tibial bone cuts was measured using the ARAN and the final position of the components was measured on postoperative CT scans. The absolute difference between the measurements was recorded to determine the accuracy of the ARAN. Results Two cases were excluded due to segmentation errors, leaving 18 cases for analysis. The ARAN produced a mean absolute error of 1.4°, 2.0°, 1.1° and 1.6° for the femoral coronal, femoral sagittal, tibial coronal and tibial sagittal alignments, respectively. No outliers (absolute error of >3°) were identified in femoral coronal or tibial coronal alignment measurements. Three outliers were identified in tibial sagittal alignment, with all cases demonstrating less tibial slope (by 3.1°, 3.3° and 4°). Five outliers were identified in femoral sagittal alignment and in all cases, the component was more extended (3.1°, 3.2°, 3.2°, 3.4° and 3.9°). The mean operative time significantly decreased from the first nine augmented reality cases to the final nine cases by 11 minutes (p<0.05). There was no difference in the accuracy between the early and late ARAN cases. Conclusion Augmented reality navigation can achieve accurate alignment of total knee arthroplasty with a low rate of component malposition in the coronal plane. Acceptable and consistent accuracy can be achieved from the initial adoption of this technique, however, some sagittal outliers were identified and there is a clear learning curve with respect to operating time. The level of evidence was IV.

Clinical applications of augmented reality in orthopaedic surgery: a comprehensive narrative review

PURPOSE

The development of augmented reality (AR) technology allows orthopaedic surgeons to incorporate and visualize surgical data, assisting the execution of both routine and complex surgical operations. Uniquely, AR technology allows a surgeon to view the surgical field and superimpose peri-operative imaging, anatomical landmarks, navigation guidance, and more, all in one view without the need for conjugate gaze between multiple screens. The aim of this literature review was to introduce the fundamental requirements for an augmented reality system and to assess the current applications, outcomes, and potential limitations to this technology.

METHODS

A literature search was performed using MEDLINE and Embase databases, by two independent reviewers, who then collaboratively synthesized and collated the results of the literature search into a narrative review focused on the applications of augmented reality in major orthopaedic sub-specialties.

RESULTS

Current technology requires that pre-operative patient data be acquired, and AR-compatible models constructed. Intra-operatively, to produce manipulatable virtual images into the user's view in real time, four major components are required including a camera, computer image processing technology, tracking tools, and an output screen. The user is provided with a heads-up display, which is a transparent display, enabling the user to look at both their natural view and the computer-generated images. Currently, high-quality evidence for clinical implementation of AR technology in the orthopaedic surgery operating room is lacking; however, growing in vitro literature highlights a multitude of potential applications, including increasing operative accuracy, improved biomechanical angular and alignment parameters, and potentially reduced operative time.

CONCLUSION

While the application of AR systems in surgery is currently in its infancy, we anticipate rapid and widespread implementation of this technology in various orthopaedic sub-specialties.

What are the optimal targeting visualizations for performing surgical navigation of iliosacral screws? A user study

INTRODUCTION

Complex orthopaedic procedures, such as iliosacral screw (ISS) fixations, can take advantage of surgical navigation technology to achieve accurate results. Although the impact of surgical navigation on outcomes has been studied, no studies to date have quantified how the design of the targeting display used for navigation affects ISS targeting performance. However, it is known in other contexts that how task information is displayed can have significant effects on both accuracy and time required to perform motor tasks, and that this can be different among users with different experience levels. This study aimed to investigate which visualization techniques helped experienced surgeons and inexperienced users most efficiently and accurately align a surgical tool to a target axis.

METHODS

We recruited 21 participants and conducted a user study to investigate five proposed 2D visualizations (bullseye, rotated bullseye, target-fixed, tool-fixed in translation, and tool-fixed in translation and rotation) with varying representations of the ISS targets and tool, and one 3D visualization. We measured the targeting accuracy achieved by each participant, as well as the time required to perform the task using each of the visualizations.

RESULTS

We found that all 2D visualizations had equivalent translational and rotational errors, with mean translational errors below 0.9 mm and rotational errors below 1.1[Formula: see text]. The 3D visualization had statistically greater mean translational and rotational errors (4.29 mm and 5.47[Formula: see text], p < 0.001) across all users. We also found that the 2D bullseye view allowed users to complete the simulated task most efficiently (mean 30.2 s; 95% CI 26.4-35.7 s), even when combined with other visualizations.

CONCLUSIONS

Our results show that 2D bullseye views helped both experienced orthopaedic trauma surgeons and inexperienced users target iliosacral screws accurately and efficiently. These findings could inform the design of visualizations for use in a surgical navigation system for screw insertions for both training and surgical practice.

Surgery Training and Simulation Using Virtual and Augmented Reality for Knee Arthroplasty

A range of extended reality technology integration, including immersive virtual reality (IVR), augmented reality (AR), as well as mixed reality, has lately acquired favour in orthopaedics. The utilization of extended reality machinery in knee arthroplasty is examined in this review study. Virtual reality (VR) and AR are usually exercised together in orthopaedic surgical training as alluring training outside of the operation theatre is acknowledged as a good surgical training tool. The use of this technology, its consequences for orthopaedic surgeons and their patients, and its moral and practical issues are also covered. Head-mounted displays (HMDs) are a potential addition directed toward improving surgical precision along with instruction. Although the hardware is cutting-edge, substantial effort needs to be done to develop software that enables seamless, trustworthy integration into clinical practice and training. Remote virtual rehabilitation has drawn increasing attention in recent years, and its significance has increased in light of the recent outbreak of the COVID-19 epidemic. Numerous medical sectors have shown the benefits of telerehabilitation, gamification, VR, and AR. Given the rising demand for orthopaedic therapy and its rising costs, this is a requirement. A remote surgeon can impart knowledge without being present, by virtually placing his or her hands in the visual field of a local surgeon using AR technology. With the use of this innovation, orthopaedic surgery seems to have been able to participate in the telemedicine revolution. This technology may also have an impact on how surgeons collaborate and train for orthopaedic residencies in the future. Volatility in the HMD market will probably stall improvements in surgical education.

Augmented reality-aided unicompartmental knee arthroplasty

Purpose

To illustrate a surgical technique for augmented reality (AR)-assisted unicompartmental knee arthroplasty (UKA) and report preliminary data.

Methods

We developed an AR-based navigation system that enables the surgeon to see the tibial mechanical axis superimposed on the patient’s leg in addition to the tibial cutting angle. We measured the tibial resection angle in 11 UKAs using postoperative radiographs and calculated the absolute difference between preoperative target angle and postoperative measured angle. The target angle was determined for each patient: mean values were 0.7° ± 1.0° varus in coronal alignment and 5.3° ± 1.4° posterior slope in sagittal alignment.

Results

The angles measured on postoperative radiographs were 2.6° ± 1.2° varus in the coronal plane and 4.8° ± 2.5° posterior slope in the sagittal plane. The absolute differences between the target and measured angles were 1.9° ± 1.5° in coronal alignment and 2.6° ± 1.2° in sagittal alignment. No patients experienced complications, including surgical site infection and periprosthetic fracture.

Conclusion

The AR-based portable navigation system may provide passable accuracy in terms of proximal tibial resection during UKA.

Level of Evidence

IV

Augmented Reality in Orthopedic Surgery and Its Application in Total Joint Arthroplasty: A Systematic Review

The development of augmented reality (AR) and its application in total joint arthroplasty aims at improving the accuracy and precision in implant components’ positioning, hopefully leading to increased outcomes and survivorship. However, this field is far from being thoroughly explored. We therefore performed a systematic review of the literature in order to examine the application, the results, and the different AR systems available in TJA. A systematic review of the literature according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines was performed. A comprehensive search of PubMed, MEDLINE, EMBASE, and the Cochrane Database of Systematic Reviews was conducted for English articles on the application of augmented reality in total joint arthroplasty using various combinations of keywords since the inception of the database to 31 March 2022. Accuracy was intended as the mean error from the targeted positioning angle and compared as mean values and standard deviations. In all, 14 articles met the inclusion criteria. Among them, four studies reported on the application of AR in total knee arthroplasty, six studies on total hip arthroplasty, three studies reported on reverse shoulder arthroplasty, and one study on total elbow arthroplasty. Nine of the included studies were preclinical (sawbones or cadaveric), while five of them reported results of AR’s clinical application. The main common feature was the high accuracy and precision when implant positioning was compared with preoperative targeted angles with errors ≤2 mm and/or ≤2°. Despite the promising results in terms of increased accuracy and precision, this technology is far from being widely adopted in daily clinical practice. However, the recent exponential growth in machine learning techniques and technologies may eventually lead to the resolution of the ongoing limitations including depth perception and their high complexity, favorably encouraging the widespread usage of AR systems.

Augmented Reality in Arthroplasty: An Overview of Clinical Applications, Benefits, and Limitations

Augmented reality (AR) is a natural extension of computer-assisted surgery whereby a computer-generated image is superimposed on the surgeon's field of vision to assist in the planning and execution of the procedure. This emerging technology shows great potential in the field of arthroplasty, improving efficiency, limb alignment, and implant position. AR has shown the capacity to build on computer navigation systems while providing more elaborate information in a streamlined workflow to the user. This review investigates the current uses of AR in the field of arthroplasty and discusses outcomes, limitations, and potential future directions.

Artificial intelligence in knee arthroplasty: current concept of the available clinical applications

Background

Artificial intelligence (AI) is defined as the study of algorithms that allow machines to reason and perform cognitive functions such as problem-solving, objects, images, word recognition, and decision-making. This study aimed to review the published articles and the comprehensive clinical relevance of AI-based tools used before, during, and after knee arthroplasty.

Methods

The search was conducted through PubMed, EMBASE, and MEDLINE databases from 2000 to 2021 using the 2009 Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocol (PRISMA).

Results

A total of 731 potential articles were reviewed, and 132 were included based on the inclusion criteria and exclusion criteria. Some steps of the knee arthroplasty procedure were assisted and improved by using AI-based tools. Before surgery, machine learning was used to aid surgeons in optimizing decision-making. During surgery, the robotic-assisted systems improved the accuracy of knee alignment, implant positioning, and ligamentous balance. After surgery, remote patient monitoring platforms helped to capture patients’ functional data.

Conclusion

In knee arthroplasty, the AI-based tools improve the decision-making process, surgical planning, accuracy, and repeatability of surgical procedures.

Augmented Reality: Mapping Methods and Tools for Enhancing the Human Role in Healthcare HMI

Background: Augmented Reality (AR) represents an innovative technology to improve data visualization and strengthen the human perception. Among Human–Machine Interaction (HMI), medicine can benefit most from the adoption of these digital technologies. In this perspective, the literature on orthopedic surgery techniques based on AR was evaluated, focusing on identifying the limitations and challenges of AR-based healthcare applications, to support the research and the development of further studies. Methods: Studies published from January 2018 to December 2021 were analyzed after a comprehensive search on PubMed, Google Scholar, Scopus, IEEE Xplore, Science Direct, and Wiley Online Library databases. In order to improve the review reporting, the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines were used. Results: Authors selected sixty-two articles meeting the inclusion criteria, which were categorized according to the purpose of the study (intraoperative, training, rehabilitation) and according to the surgical procedure used. Conclusions: AR has the potential to improve orthopedic training and practice by providing an increasingly human-centered clinical approach. Further research can be addressed by this review to cover problems related to hardware limitations, lack of accurate registration and tracking systems, and absence of security protocols.

Mixed Reality Technology in Total Knee Arthroplasty: An Updated Review With a Preliminary Case Report

Background

Augmented reality and mixed reality have been used to help surgeons perform complex surgeries. With the development of technology, mixed reality (MR) technology has been used to improve the success rate of complex hip arthroplasty due to its unique advantages. At present, there are few reports on the application of MR technology in total knee arthroplasty. We presented a case of total knee arthroplasty with the help of mixed reality technology.

Case Presentation

We presented a case of a 71-year-old woman who was diagnosed with bilateral knee osteoarthritis with varus deformity, especially on the right side. After admission, the right total knee arthroplasty was performed with the assistance of MR technology. Before the operation, the three-dimensional virtual model of the knee joint of the patient was reconstructed for condition analysis, operation plan formulation, and operation simulation. During the operation, the three-dimensional virtual images of the femur and tibia coincided with the real body of the patient, showing the osteotomy plane designed before the operation, which can accurately guide the completion of osteotomy and prosthesis implantation.

Conclusions

As far as we know, this is the first report on total knee arthroplasty under the guidance of mixed reality technology.

Virtual and augmented reality for surgical training and simulation in knee arthroplasty

BACKGROUND

Immersive virtual reality (IVR), augmented reality and mixed reality form a spectrum of extended reality technology integration that has gained popularity in orthopaedics recently. This review article examines the role of extended reality technologies in knee arthroplasty.

METHODS

Existing literature on the applications of extended reality technologies in preoperative planning and intraoperative navigation were reviewed. A sample workflow of a novel IVR simulator for improving surgical training was also provided to demonstrate its utility in educating trainees on knee arthroplasty techniques.

RESULTS

Extended reality technologies enable the surgeon to visualise patient-specific anatomy in real-time, enhancing preoperative planning and providing intraoperative guidance. IVR technology has the potential to revolutionise modern surgical training and optimise surgical performance in a cost-efficient manner, with current evidence demonstrating favourable immediate skill acquisition and transfer.

CONCLUSIONS

Extended reality technologies have a myriad of potential applications in orthopaedic surgery. Further research is needed to evaluate the cost-effectiveness of its incorporation into training programmes.

AUGMENTED REALITY NAVIGATION FRAMEWORK FOR TOTAL HIP ARTHROPLASTY SURGERY

In a total hip arthroplasty surgery, correctly implanting the artificial acetabulum and the femoral head is essential for a successful treatment. An augmented reality (AR) navigation framework is proposed in this paper to provide accurate surgical guidance in a total hip arthroplasty procedure. The AR framework consists of three parts: (1) preoperative surgical planning to generate virtual information for AR; (2) computer vision-based tracking for the real-time localization of both acetabular cup positioner and bony landmarks during surgery; (3) registration of a virtual object onto a real-world operative field to properly overlay the preoperative surgical planning data onto a three-dimensional (3D)-printed pelvis model. The cost-effective framework is designed with our clinical partner based on real surgical conditions. The bony landmarks are automatically detected and used for the registration between virtual and real objects. The AR performance is evaluated with a pelvis model, and it presents mean errors of 2.2[Formula: see text]mm and 0.8∘ in position and orientation, respectively, between real and virtual spaces. These small errors are within the tolerance of positive surgical outcomes.

The Clinical Application of Augmented Reality in Orthopaedics: Where Do We Stand?

PURPOSE OF REVIEW

The surgical community is constantly working to improve accuracy and reproducibility in patient care, with the goal to improve patient outcomes and efficiency. One area of growing interest with potential to meet these goals is in the use of augmented reality (AR) in surgery. There is still a paucity of published research on the clinical benefits of AR over traditional techniques, but this article aims to present an update on the current state of AR within orthopaedics over the past 5 years.

RECENT FINDINGS

AR systems are being developed and studied for use in all areas of orthopaedics. Most recently published research has focused on the areas of fracture care, adult reconstruction, orthopaedic oncology, spine, and resident education. These studies have shown some promising results, particularly in surgical accuracy, decreased surgical time, and less radiation exposure. However, the majority of recently published research is still in the pre-clinical setting, with very few studies using living patients. AR supplementation in orthopaedic surgery has shown promising results in pre-clinical settings, with improvements in surgical accuracy and reproducibility, decreased operating times, and less radiation exposure. Most AR systems, however, are still not approved for clinical use. Further research is needed to validate the benefits of AR use in orthopaedic surgery before it is widely adopted into practice.

Augmented Reality-Assisted Femoral Bone Resection in Total Knee Arthroplasty

An augmented reality (AR)-based navigation system allows visualization of the center of the femoral head and femoral mechanical axis superimposed on the surgical field during total knee arthroplasty (TKA) and may help surgeons to improve the accuracy of distal femoral resection.

The use of augmented reality for limb and component alignment in total knee arthroplasty: systematic review of the literature and clinical pilot study

PURPOSE

A systematic review of the literature has been carried out to assess the actual evidence of the use of augmented reality in total knee arthroplasty (TKA). We then conducted a pilot clinical study to examine the accuracy of the Knee + augmented reality navigation in performing TKA. The present augmented reality (AR) system allows the surgeon to view the tibial and femur axis superimposed on the surgical field through the smart glasses. It provides real-time information during surgery and intraoperative feedback.

METHODS

A systematic review of the PubMed, MEDLINE, and Embase databases up to May 2021 using the keywords "augmented reality", "knee arthroplasty", "computer assisted surgery", "navigation knee arthroplasty" was performed by two independent reviewers. We performed five TKAs using the Knee + system. Patients were 4 females, with mean age of 76.4 years old (range 73-79) and mean Body Max Index (BMI) of 31.9 kg/m2 (range 27-35). The axial alignment of the limb and the orientation of the components were evaluated on standardized pre and postoperative full leg length weight-bearing radiographs, anteroposterior radiographs, and lateral radiographs of the knee. The time of tourniquet was recorded. The perception of motion sickness was assessed by Virtual Reality Sickness Questionnaire (VRSQ) subjected to surgeon immediately after surgery.

RESULTS

After duplicate removal, a total of 31 abstracts were found. However, only two studies concerned knee arthroplasty. Unfortunately, both were preclinical studies. Knee + system is able to perform a cutting error of less than 1° of difference about coronal alignment of femur and tibia and less than 2° about flexion/extension of femur and posterior tibial slope. The absolute differences between the values obtained during surgery and the measurement of varus femur, varus tibia, posterior slope, and femur flexion angle on post-operative radiographs were 0.6° ± 1.34°, 0.8° ± 0.84°, 0.8° ± 1.79°, and 0.4 mm ± 0.55 mm, respectively.

CONCLUSIONS

On light of our preliminary results, the Knee + system is accurate and effective to perform TKA. The translation from pilot study to high-level prospective studies is warranted to assess accuracy and cost-effective analysis compared to conventional techniques.

LEVEL OF EVIDENCE

IV.

Head-Mounted Augmented Reality Platform for Markerless Orthopaedic Navigation

Visual augmented reality (AR) has the potential to improve the accuracy, efficiency and reproducibility of computer-assisted orthopaedic surgery (CAOS). AR Head-mounted displays (HMDs) further allow non-eye-shift target observation and egocentric view. Recently, a markerless tracking and registration (MTR) algorithm was proposed to avoid the artificial markers that are conventionally pinned into the target anatomy for tracking, as their use prolongs surgical workflow, introduces human-induced errors, and necessitates additional surgical invasion in patients. However, such an MTR-based method has neither been explored for surgical applications nor integrated into current AR HMDs, making the ergonomic HMD-based markerless AR CAOS navigation hard to achieve. To these aims, we present a versatile, device-agnostic and accurate HMD-based AR platform. Our software platform, supporting both video see-through (VST) and optical see-through (OST) modes, integrates two proposed fast calibration procedures using a specially designed calibration tool. According to the camera-based evaluation, our AR platform achieves a display error of 6.31 2.55 arcmin for VST and 7.72 3.73 arcmin for OST. A proof-of-concept markerless surgical navigation system to assist in femoral bone drilling was then developed based on the platform and Microsoft HoloLens 1. According to the user study, both VST and OST markerless navigation systems are reliable, with the OST system providing the best usability. The measured navigation error is 4.90 1.04 mm, 5.96 2.22 for VST system and 4.36 0.80 mm, 5.65 1.42 for OST system.

Towards markerless computer assisted surgery: Application to total knee arthroplasty

PURPOSE

A new approach is proposed to localise surgical instruments for Computer Assisted Orthopaedic Surgery (CAOS) that aims at overpassing the limitations of conventional CAOS solutions. This approach relies on both a depth sensor and a 6D pose estimation algorithm.

METHODS

The Point-Pair Features (PPF) algorithm was used to estimate the pose of a Patient-Specific Instrument (PSI) for Total Knee Arthroplasty (TKA). Four depth sensors have been compared. Three scores have been computed to assess the performances: The Depth Fitting Error (DFE), the Pose Errors, and the Success Rate.

RESULTS

The obtained results demonstrate higher performances for the Microsoft Kinect Azure in terms of DFE. The Occipital Structure core shows better behavior in terms of Pose Errors and Success Rate.

CONCLUSION

This comparative study presents the first depth-sensor based solution allowing the intraoperative markerless localization of surgical instruments in orthopedics.

Augmented Reality, Virtual Reality and Artificial Intelligence in Orthopedic Surgery: A Systematic Review

Background: The application of virtual and augmented reality technologies to orthopaedic surgery training and practice aims to increase the safety and accuracy of procedures and reducing complications and costs. The purpose of this systematic review is to summarise the present literature on this topic while providing a detailed analysis of current flaws and benefits. Methods: A comprehensive search on the PubMed, Cochrane, CINAHL, and Embase database was conducted from inception to February 2021. The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines were used to improve the reporting of the review. The Cochrane Risk of Bias Tool and the Methodological Index for Non-Randomized Studies (MINORS) was used to assess the quality and potential bias of the included randomized and non-randomized control trials, respectively. Results: Virtual reality has been proven revolutionary for both resident training and preoperative planning. Thanks to augmented reality, orthopaedic surgeons could carry out procedures faster and more accurately, improving overall safety. Artificial intelligence (AI) is a promising technology with limitless potential, but, nowadays, its use in orthopaedic surgery is limited to preoperative diagnosis. Conclusions: Extended reality technologies have the potential to reform orthopaedic training and practice, providing an opportunity for unidirectional growth towards a patient-centred approach.

Augmented Reality in Orthopedic Surgery Is Emerging from Proof of Concept Towards Clinical Studies: a Literature Review Explaining the Technology and Current State of the Art

PURPOSE OF REVIEW

Augmented reality (AR) is becoming increasingly popular in modern-day medicine. Computer-driven tools are progressively integrated into clinical and surgical procedures. The purpose of this review was to provide a comprehensive overview of the current technology and its challenges based on recent literature mainly focusing on clinical, cadaver, and innovative sawbone studies in the field of orthopedic surgery. The most relevant literature was selected according to clinical and innovational relevance and is summarized.

RECENT FINDINGS

Augmented reality applications in orthopedic surgery are increasingly reported. In this review, we summarize basic principles of AR including data preparation, visualization, and registration/tracking and present recently published clinical applications in the area of spine, osteotomies, arthroplasty, trauma, and orthopedic oncology. Higher accuracy in surgical execution, reduction of radiation exposure, and decreased surgery time are major findings presented in the literature. In light of the tremendous progress of technological developments in modern-day medicine and emerging numbers of research groups working on the implementation of AR in routine clinical procedures, we expect the AR technology soon to be implemented as standard devices in orthopedic surgery.