Augmented reality in orthopaedics

Total Knee Arthroplasty With Robotic and Augmented Reality Guidance: A Hierarchical Task Analysis

Background

Total knee arthroplasty (TKA) is a commonly performed procedure that has traditionally utilized reproducible steps using a set of mechanical instruments. The number of TKAs performed using robotic assistance is increasing, and augmented reality (AR) navigation systems are being developed. Hierarchical task analysis (HTA) aims to describe the steps of a specific task in detail to reduce errors and ensure reproducibility. The objective of this study was to develop and validate HTAs for conventional, robotic-assisted, and AR-navigated TKA.

Methods

The development of HTAs for conventional TKA involved an iterative review process that incorporated the input of 4 experienced arthroplasty surgeons. The HTAs were then adapted for robotic-assisted and AR-navigated TKA by incorporating specific steps associated with the use of these systems. The accuracy and completeness of the HTAs were validated by observing 10 conventional and 10 robotic-assisted TKA procedures.

Results

HTAs for conventional, robotic-assisted, and AR-navigated TKA were developed and validated. The resulting HTAs provide a comprehensive and standardized plan for each procedure and can aid in the identification of potential areas of inefficiency and risk. Robotic-assisted and AR-navigated approaches require additional steps, and there are an increased number of instances where complications may occur.

Conclusions

The HTAs developed in this study can provide valuable insights into the potential pitfalls of robotic-assisted and AR-navigated TKA procedures. As AR-navigation systems are developed, they should be optimized by critical analysis using the developed HTAs to ensure maximum efficiency, reliability, accessibility, reduction of human error, and costs.

Comparison of virtual reality and computed tomography in the preoperative planning of complex tibial plateau fractures

INTRODUCTION

Preoperative planning is a critical step in the success of any complex surgery. The pur-pose of this study is to evaluate the advantage of VR glasses in surgical planning of complex tibial plateau fractures compared to CT planning.

MATERIALS AND METHODS

Five orthopedic surgeons performed preoperative planning for 30 fractures using either conventional CT slices or VR visualization with a VR headset. Planning was performed in a randomized order with a 3-month interval between planning sessions. A standardized questionnaire assessed planned operative time, planning time, fracture classification and understanding, and surgeons' subjective confidence in surgical planning.

RESULTS

The mean planned operative time of 156 (SD 47) minutes was significantly lower (p < 0.001) in the VR group than in the CT group (172 min; SD 44). The mean planning time in the VR group was 3.48 min (SD 2.4), 17% longer than in the CT group (2.98 min, SD 1.9; p = 0.027). Relevant parameters influencing planning time were surgeon experience (-0.61 min) and estimated complexity of fracture treatment (+ 0.65 min).

CONCLUSION

The use of virtual reality for surgical planning of complex tibial plateau fractures resulted in significantly shorter planned operative time, while planning time was longer compared to CT planning. After VR planning, more surgeons felt (very) well prepared for surgery.

Feasibility and Usability of Augmented Reality Technology in the Orthopaedic Operating Room

PURPOSE OF REVIEW

Augmented reality (AR) has gained popularity in various sectors, including gaming, entertainment, and healthcare. The desire for improved surgical navigation within orthopaedic surgery has led to the evaluation of the feasibility and usability of AR in the operating room (OR). However, the safe and effective use of AR technology in the OR necessitates a proper understanding of its capabilities and limitations. This review aims to describe the fundamental elements of AR, highlight limitations for use within the field of orthopaedic surgery, and discuss potential areas for development.

RECENT FINDINGS

To date, studies have demonstrated evidence that AR technology can be used to enhance navigation and performance in orthopaedic procedures. General hardware and software limitations of the technology include the registration process, ergonomics, and battery life. Other limitations are related to the human response factors such as inattentional blindness, which may lead to the inability to see complications within the surgical field. Furthermore, the prolonged use of AR can cause eye strain and headache due to phenomena such as the vergence-convergence conflict. AR technology may prove to be a better alternative to current orthopaedic surgery navigation systems. However, the current limitations should be mitigated to further improve the feasibility and usability of AR in the OR setting. It is important for both non-clinicians and clinicians to work in conjunction to guide the development of future iterations of AR technology and its implementation into the OR workflow.

Navigate biopsy with ultrasound under augmented reality device: Towards higher system performance

PURPOSE

Biopsies play a crucial role in determining the classification and staging of tumors. Ultrasound is frequently used in this procedure to provide real-time anatomical information. Using augmented reality (AR), surgeons can visualize ultrasound data and spatial navigation information seamlessly integrated with real tissues. This innovation facilitates faster and more precise biopsy operations.

METHODS

We have developed an augmented reality biopsy navigation system characterized by low display latency and high accuracy. Ultrasound data is initially read by an image capture card and streamed to Unity via net communication. In Unity, navigation information is rendered and transmitted to the HoloLens 2 device using holographic remoting. Concurrently, a retro-reflective tool tracking method is implemented on the HoloLens 2, enabling the simultaneous tracking of the ultrasound probe and biopsy needle. Distinct navigation information is provided during in-plane and out-of-plane punctuation. To evaluate the effectiveness of our system, we conducted a study involving ten participants, assessing puncture accuracy and biopsy time in comparison to traditional methods.

RESULTS

Ultrasound image was streamed from the ultrasound device to augmented reality headset with 122.49±11.61ms latency, while only 16.22±11.25ms was taken after data acquisition from image capture card. Navigation accuracy reached 1.23±0.68mm in the image plane and 0.95±0.70mm outside the image plane, within a depth range of 200 millimeters. Remarkably, the utilization of our system led to 98% and 95% success rate in out-of-plane and in-plane biopsy, among ten participants with little ultrasound experience.

CONCLUSION

To sum up, this paper introduces an AR-based ultrasound biopsy navigation system characterized by high navigation accuracy and minimal latency. The system provides distinct visualization contents during in-plane and out-of-plane operations according to their different characteristics. Use case study in this paper proved that our system can help young surgeons perform biopsy faster and more accurately.

Comparison between accuracy of augmented reality computed tomography-based and portable augmented reality-based navigation systems for cup insertion in total hip arthroplasty

Augmented reality (AR) has been used for navigation during total hip arthroplasty (THA). AR computed tomography (CT)-based navigation systems and AR-based portable navigation systems that use smartphones can also be used. This study compared the accuracy of cup insertion during THA using AR-CT-based and portable AR-based navigation systems. Patients with symptomatic hip disease who underwent primary THA in the supine position using both AR CT-based and portable AR-based navigation systems simultaneously between October 2021 and July 2023 were included. The primary outcome of this study was the absolute difference between cup angles in the intraoperative navigation record and those measured on postoperative CT. The secondary outcome was to determine the factors affecting the absolute value of the navigation error in radiographic inclination (RI) and radiographic anteversion (RA) of the cup, including sex, age, body mass index, left or right side, approach, and preoperative pelvic tilt. This study included 94 consecutive patients. There were 11 men and 83 women, with a mean age of 68 years. The mean absolute errors of RI were 2.7° ± 2.0° in the AR CT-based and 3.3° ± 2.4° in the portable AR-based navigation system. The mean absolute errors of RA were 2.5° ± 2.1° in the AR CT-based navigation system and 2.3° ± 2.2° in the portable AR-based navigation system. No significant differences were observed in RI or RA of the cup between the two navigation systems (RI: p = 0.706; RA: p = 0.329). No significant factors affected the absolute value of the navigation errors in RI and RA. In conclusion, there were no differences in the accuracy of cup insertion between the AR CT-based and portable AR-based navigation systems.

Does Extended Reality Simulation Improve Surgical/Procedural Learning and Patient Outcomes When Compared With Standard Training Methods?: A Systematic Review

INTRODUCTION

The use of extended reality (XR) technologies, including virtual, augmented, and mixed reality, has increased within surgical and procedural training programs. Few studies have assessed experiential learning- and patient-based outcomes using XR compared with standard training methods.

METHODS

As a working group for the Society for Simulation in Healthcare, we used Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines and a PICO strategy to perform a systematic review of 4238 articles to assess the effectiveness of XR technologies compared with standard training methods. Outcomes were grouped into knowledge, time-to-completion, technical proficiency, reactions, and patient outcomes. Because of study heterogeneity, a meta-analysis was not feasible.

RESULTS

Thirty-two studies met eligibility criteria: 18 randomized controlled trials, 7 comparative studies, and 7 systematic reviews. Outcomes of most studies included Kirkpatrick levels of evidence I-III (reactions, knowledge, and behavior), while few reported level IV outcomes (patient). The overall risk of bias was low. With few exceptions, included studies showed XR technology to be more effective than standard training methods in improving objective skills and performance, shortening procedure time, and receiving more positive learner ratings. However, XR use did not show significant differences in gained knowledge.

CONCLUSIONS

Surgical or procedural XR training may improve technical skill development among trainees and is generally favored over standard training methods. However, there should be an additional focus on how skill development translates to clinically relevant outcomes. We recommend longitudinal studies to examine retention and transfer of training to clinical settings, methods to improve timely, adaptive feedback for deliberate practice, and cost analyses.

Femoral prosthesis alignment of augmented reality-assisted versus accelerometer-based navigation in total knee arthroplasty: A noninferiority analysis

INTRODUCTION

The purpose of this study was to examine the comparative precision of the augmented reality (AR)-assisted navigation system and the accelerometer-based navigation system in total knee arthroplasty (TKA).

MATERIALS AND METHODS

We performed noninferiority analysis in a retrospective cohort. The coronal alignment of femoral prosthesis was compared between 109 TKAs performed using the AR-assisted navigation system and 118 TKAs performed using the accelerometer-based navigation system. All femoral prostheses were planned to be positioned perpendicular to the mechanical axis of the femur. The primary outcome was the success rate of coronal alignment of the femoral prosthesis defined as alignment error relative to neutral alignment <3°. We calculated the noninferiority margin as 7%-points using the 95%-95 % method and also confirmed the validity of the noninferiority margin using the fixed margin method. Noninferiority would be shown if the lower boundary of the 95 % confidence interval (CI) for the between-group difference in percentage of the success rate was not less than 0.93 (i.e., 1.00 - 0.07).

RESULTS

Treatment success was achieved in 104 of 109 patients (95.4 %) in the AR-assisted navigation group and 110 of 118 (93.2 %) in the accelerometer-based navigation group. The risk ratio of success between the AR-assisted navigation group versus accelerometer-based navigation group was 1.02 (95 % CI, 0.96 to 1.09): the CIs did not include the noninferiority margin of 0.93.

CONCLUSION

The AR-assisted navigation system was noninferior to the accelerometer-based navigation system in terms of coronal alignment of the femoral prosthesis in TKA.

The Accuracy and Absolute Reliability of a Knee Surgery Assistance System Based on ArUco-Type Sensors

Recent advances allow the use of Augmented Reality (AR) for many medical procedures. AR via optical navigators to aid various knee surgery techniques (e.g., femoral and tibial osteotomies, ligament reconstructions or menisci transplants) is becoming increasingly frequent. Accuracy in these procedures is essential, but evaluations of this technology still need to be made. Our study aimed to evaluate the system's accuracy using an in vitro protocol. We hypothesised that the system's accuracy was equal to or less than 1 mm and 1° for distance and angular measurements, respectively. Our research was an in vitro laboratory with a 316 L steel model. Absolute reliability was assessed according to the Hopkins criteria by seven independent evaluators. Each observer measured the thirty palpation points and the trademarks to acquire direct angular measurements on three occasions separated by at least two weeks. The system's accuracy in assessing distances had a mean error of 1.203 mm and an uncertainty of 2.062, and for the angular values, a mean error of 0.778° and an uncertainty of 1.438. The intraclass correlation coefficient was for all intra-observer and inter-observers, almost perfect or perfect. The mean error for the distance's determination was statistically larger than 1 mm (1.203 mm) but with a trivial effect size. The mean error assessing angular values was statistically less than 1°. Our results are similar to those published by other authors in accuracy analyses of AR systems.

Accuracy of augmented reality with computed tomography-based navigation in total hip arthroplasty

BACKGROUND

Augmented reality (AR) provides the surgeon with direct visualization of radiological images by overlaying them on the patient. This study aimed to evaluate the accuracy of cup placement using a computed tomography (CT)-based AR navigation system.

METHODS

Sixty-five prospectively enrolled patients underwent primary cementless total hip arthroplasty (THA) in a supine position using this novel AR navigation system, and changes in pelvic flexion angle (PFA) were evaluated. Absolute navigation errors were defined as the absolute differences between angles in the intraoperative navigation record and those measured on postoperative CT. Factors affecting the absolute navigation error in cup alignment were determined.

RESULTS

Mean absolute change in PFA between preoperative CT and reduction was 2.1° ± 1.6°. Mean absolute navigation errors were 2.5° ± 1.7° in radiographic inclination (RI) and 2.5° ± 2.2° in radiographic anteversion (RA). While no factors significantly affecting absolute navigation error were found for RI, absolute change in PFA between preoperative CT and reduction correlated significantly with the absolute navigation error for RA.

CONCLUSION

This CT-based navigation system with AR enabled surgeons to place the cup more accurately than was possible by freehand placement during THA in a supine position.

Integration of Square Fiducial Markers in Patient-Specific Instrumentation and Their Applicability in Knee Surgery

Computer technologies play a crucial role in orthopaedic surgery and are essential in personalising different treatments. Recent advances allow the usage of augmented reality (AR) for many orthopaedic procedures, which include different types of knee surgery. AR assigns the interaction between virtual environments and the physical world, allowing both to intermingle (AR superimposes information on real objects in real-time) through an optical device and allows personalising different processes for each patient. This article aims to describe the integration of fiducial markers in planning knee surgeries and to perform a narrative description of the latest publications on AR applications in knee surgery. Augmented reality-assisted knee surgery is an emerging set of techniques that can increase accuracy, efficiency, and safety and decrease the radiation exposure (in some surgical procedures, such as osteotomies) of other conventional methods. Initial clinical experience with AR projection based on ArUco-type artificial marker sensors has shown promising results and received positive operator feedback. Once initial clinical safety and efficacy have been demonstrated, the continued experience should be studied to validate this technology and generate further innovation in this rapidly evolving field.

Preoperative Mixed-Reality Visualization of Complex Tibial Plateau Fractures and Its Benefit Compared to CT and 3D Printing

BACKGROUND

Various studies have shown the benefit of three-dimensional (3D) computed tomography (CT) reconstruction and especially 3D printing in the treatment of tibial plateau fractures (TPFs). This study aimed to investigate whether mixed-reality visualization (MRV) using mixed-reality glasses can provide a benefit for CT and/or 3D printing in planning treatment strategies for complex TPFs.

METHODS

Three complex TPFs were selected for the study and processed for 3D imaging. Subsequently, the fractures were presented to specialists in trauma surgery using CT (including 3D CT reconstruction), MRV (hardware: Microsoft HoloLens 2; software: mediCAD MIXED REALITY) and 3D prints. A standardized questionnaire on fracture morphology and treatment strategy was completed after each imaging session.

RESULTS

23 surgeons from 7 hospitals were interviewed. A total of 69.6% (n = 16) of those had treated at least 50 TPFs. A change in fracture classification according to Schatzker was recorded in 7.1% of the cases and in 78.6% an adjustment of the ten-segment classification was observed after MRV. In addition, the intended patient positioning changed in 16.1% of the cases, the surgical approach in 33.9% and osteosynthesis in 39.3%. A total of 82.1% of the participants rated MRV as beneficial compared to CT regarding fracture morphology and treatment planning. An additional benefit of 3D printing was reported in 57.1% of the cases (five-point Likert scale).

CONCLUSIONS

Preoperative MRV of complex TPFs leads to improved fracture understanding, better treatment strategies and a higher detection rate of fractures in posterior segments, and it thus has the potential to improve patient care and outcomes.

Next step trauma and orthopaedic surgery: integration of augmented reality for reduction and nail implantation of tibial fractures

INTRODUCTION

There is a tremendous scope of hardware and software development going on in augmented reality (AR), also in trauma and orthopaedic surgery. However, there are only a few systems available for intra-operative 3D imaging and guidance, most of them rely on peri- and intra-operative X-ray imaging. Especially in complex situations such as pelvic surgery or multifragmentary multilevel fractures, intra-operative 3D imaging and implant tracking systems have proven to be of great advantage for the outcome of the surgery and can help reduce X-ray exposure, at least for the surgical team (Ochs et al. in Injury 41:1297 1305, 2010). Yet, the current systems do not provide the ability to have a dynamic live view from the perspective of the surgeon. Our study describes a prototype AR-based system for live tracking which does not rely on X-rays.

MATERIALS AND METHODS

A protype live-view intra-operative guidance system using an AR head-mounted device (HMD) was developed and tested on the implantation of a medullary nail in a tibia fracture model. Software algorithms that allow live view and tracking of the implant, fracture fragments and soft tissue without the intra-operative use of X-rays were derived.

RESULTS

The implantation of a medullar tibia nail is possible while only relying on AR-guidance and live view without the intra-operative use of X-rays.

CONCLUSIONS

The current paper describes a feasibility study with a prototype of an intra-operative dynamic live tracking and imaging system that does not require intra-operative use of X-rays and dynamically adjust to the perspective of the surgeons due to an AR HMD. To our knowledge, the current literature does not describe any similar systems. This could be the next step in surgical imaging and education and a promising way to improve patient care.

What are the optimal targeting visualizations for performing surgical navigation of iliosacral screws? A user study

INTRODUCTION

Complex orthopaedic procedures, such as iliosacral screw (ISS) fixations, can take advantage of surgical navigation technology to achieve accurate results. Although the impact of surgical navigation on outcomes has been studied, no studies to date have quantified how the design of the targeting display used for navigation affects ISS targeting performance. However, it is known in other contexts that how task information is displayed can have significant effects on both accuracy and time required to perform motor tasks, and that this can be different among users with different experience levels. This study aimed to investigate which visualization techniques helped experienced surgeons and inexperienced users most efficiently and accurately align a surgical tool to a target axis.

METHODS

We recruited 21 participants and conducted a user study to investigate five proposed 2D visualizations (bullseye, rotated bullseye, target-fixed, tool-fixed in translation, and tool-fixed in translation and rotation) with varying representations of the ISS targets and tool, and one 3D visualization. We measured the targeting accuracy achieved by each participant, as well as the time required to perform the task using each of the visualizations.

RESULTS

We found that all 2D visualizations had equivalent translational and rotational errors, with mean translational errors below 0.9 mm and rotational errors below 1.1[Formula: see text]. The 3D visualization had statistically greater mean translational and rotational errors (4.29 mm and 5.47[Formula: see text], p < 0.001) across all users. We also found that the 2D bullseye view allowed users to complete the simulated task most efficiently (mean 30.2 s; 95% CI 26.4-35.7 s), even when combined with other visualizations.

CONCLUSIONS

Our results show that 2D bullseye views helped both experienced orthopaedic trauma surgeons and inexperienced users target iliosacral screws accurately and efficiently. These findings could inform the design of visualizations for use in a surgical navigation system for screw insertions for both training and surgical practice.

Validating the accuracy of a novel virtual reality platform for determining implant orientation in simulated primary total hip replacement

Introduction

Accurate acetabular cup and femoral stem component orientation are critical for optimising patient outcomes, reducing complications and increasing component longevity following total hip replacement (THR). This study aimed to determine the accuracy of a novel virtual reality (VR) platform in assessing component orientation in a simulated THR model.

Methods

The VR platform (HTC Vive Pro® system hardware) was compared against the validated Vicon® optical motion capture (MoCap) system. An acetabular cup and femoral stem were manually implanted across a range of orientations into pelvic and femur sawbones, respectively. Simultaneous readings of the acetabular cup operative anteversion (OA) and inclination (OI) and femoral stem alignment (FSA) and neck anteversion (FNA) were obtained from the VR and MoCap systems. Statistical analysis was performed using Pearson product-moment correlation coefficient (PPMCC) (Pearson’s r) and linear regression (R²).

Results

A total of 55 readings were obtained for the acetabular cup and 68 for the femoral stem model. The mean average differences in OA, OI, FSA and FNA between the systems were 3.44°, −0.01°, 0.01° and −0.04°, respectively. Strong positive correlations were demonstrated between both systems in OA, OI, FSA and FNA, with Pearson’s r = 0.92, 0.94, 0.99 and 0.99, and adjusted R² = 0.82, 0.9, 0.98 and 0.98, respectively.

Conclusion

The novel VR platform is highly accurate and reliable in determining both acetabular cup and femoral stem component orientations in simulated THR models. This adaptable and cost-effective digital tracking platform may be modified for use in a range of simulated surgical training and educational purposes, particularly in orthopaedic surgery.

Augmented reality-assisted systematic mapping of anterolateral thigh perforators

PURPOSE

In soft tissue reconstructive surgery, perforator localization and flap harvesting have always been critical challenges, but augmented reality (AR) has become a dominant technology to help map perforators.

METHODS

The lateral circumflex femoral artery (LCFA) and its perforators were reconstructed by CTA in consecutive patients (N = 14). Then, the anterolateral thigh perforators and the points from which the perforators emerged from the deep fascia were marked and projected onto the skin surface. As the virtual images were projected onto patients according to bony markers, the courses of the LCFA and its perforators were depicted on the skin surface for intraoperative guidance. Finally, the locations of the emergence points were verified by intraoperative findings and compared to those determined by handheld Doppler ultrasound.

RESULTS

The sources, locations, and numbers of perforators were determined by CTA. The perforators and their emergence points were accurately mapped on the skin surface by a portable projector to harvest the anterolateral thigh perforator flap. During the operation, the accuracy of the CTA & AR method was 90.2% (37/41), and the sensitivity reached 97.4% (37/38), which were much higher than the corresponding values of Doppler ultrasound. Additionally, the differences between the AR-marked points and the intraoperative findings were much smaller than those seen with Doppler ultrasound (P < 0.001). Consequently, all of the flaps were well designed and survived, and only one complication occurred.

CONCLUSION

Augmented reality, namely, CTA combined with projection in this study, plays a vital and reliable role in locating the perforator emergence points and guiding the procedure to harvest flaps and has fewer potential risks.

Sensor technology usage in orthopedic trauma

Medicine in general is quickly transitioning to a digital presence. Orthopaedic surgery is also being impacted by the tenets of digital health but there are also direct efforts with trauma surgery. Sensors are the pen and paper of the next wave of data acquisition. Orthopaedic trauma can and will be part of this new wave of medicine. Early sensor products that are now coming to market, or are in early development, will directly change the way we think about surgical diagnosis and outcomes. Sensor development for biometrics is already here. Wellness devices, pressure, temperature, and other parameters are already being measured. Data acquisition and analysis is going to be a fruitful addition to our research armamentarium with the volume of information now available. A combination of broadband internet, micro electrical machine systems (MEMS), and new wireless communication standards is driving this new wave of medicine. The Internet of Things (IoT) [1] now has a subset which is the Internet of Medical Devices [2-5] permitting a much more in-depth dive into patient procedures and outcomes. IoT devices are now being used to enable remote health monitoring, in hospital treatment, and guide therapies. This article reviews current sensor technology that looks to impact trauma care.

Augmented reality-aided unicompartmental knee arthroplasty

Purpose

To illustrate a surgical technique for augmented reality (AR)-assisted unicompartmental knee arthroplasty (UKA) and report preliminary data.

Methods

We developed an AR-based navigation system that enables the surgeon to see the tibial mechanical axis superimposed on the patient’s leg in addition to the tibial cutting angle. We measured the tibial resection angle in 11 UKAs using postoperative radiographs and calculated the absolute difference between preoperative target angle and postoperative measured angle. The target angle was determined for each patient: mean values were 0.7° ± 1.0° varus in coronal alignment and 5.3° ± 1.4° posterior slope in sagittal alignment.

Results

The angles measured on postoperative radiographs were 2.6° ± 1.2° varus in the coronal plane and 4.8° ± 2.5° posterior slope in the sagittal plane. The absolute differences between the target and measured angles were 1.9° ± 1.5° in coronal alignment and 2.6° ± 1.2° in sagittal alignment. No patients experienced complications, including surgical site infection and periprosthetic fracture.

Conclusion

The AR-based portable navigation system may provide passable accuracy in terms of proximal tibial resection during UKA.

Level of Evidence

IV

Augmented Reality in Arthroplasty: An Overview of Clinical Applications, Benefits, and Limitations

Augmented reality (AR) is a natural extension of computer-assisted surgery whereby a computer-generated image is superimposed on the surgeon's field of vision to assist in the planning and execution of the procedure. This emerging technology shows great potential in the field of arthroplasty, improving efficiency, limb alignment, and implant position. AR has shown the capacity to build on computer navigation systems while providing more elaborate information in a streamlined workflow to the user. This review investigates the current uses of AR in the field of arthroplasty and discusses outcomes, limitations, and potential future directions.

Utility of optical see-through head mounted displays in augmented reality-assisted surgery: A systematic review

This article presents a systematic review of optical see-through head mounted display (OST-HMD) usage in augmented reality (AR) surgery applications from 2013 to 2020. Articles were categorised by: OST-HMD device, surgical speciality, surgical application context, visualisation content, experimental design and evaluation, accuracy and human factors of human-computer interaction. 91 articles fulfilled all inclusion criteria. Some clear trends emerge. The Microsoft HoloLens increasingly dominates the field, with orthopaedic surgery being the most popular application (28.6%). By far the most common surgical context is surgical guidance (n=58) and segmented preoperative models dominate visualisation (n=40). Experiments mainly involve phantoms (n=43) or system setup (n=21), with patient case studies ranking third (n=19), reflecting the comparative infancy of the field. Experiments cover issues from registration to perception with very different accuracy results. Human factors emerge as significant to OST-HMD utility. Some factors are addressed by the systems proposed, such as attention shift away from the surgical site and mental mapping of 2D images to 3D patient anatomy. Other persistent human factors remain or are caused by OST-HMD solutions, including ease of use, comfort and spatial perception issues. The significant upward trend in published articles is clear, but such devices are not yet established in the operating room and clinical studies showing benefit are lacking. A focused effort addressing technical registration and perceptual factors in the lab coupled with design that incorporates human factors considerations to solve clear clinical problems should ensure that the significant current research efforts will succeed.

Sensors and digital medicine in orthopaedic surgery

Digital health principles are starting to be evident in medicine. Orthopaedic trauma surgery is also being impacted —indirectly by all other improvements in the health ecosystem but also in particular efforts aimed at trauma surgery. Data acquisition is changing how evidence is gathered and utilized. Sensors are the pen and paper of the next wave of data acquisition. Sensors are gathering wide arrays of information to facilitate digital health relevance and adoption. Early adaption of sensor technology by the nonlegacy health environment is what has made sensor driven data acquisition so palatable to the normal health care system. As it applies to orthopaedic trauma, current sensor driven diagnostics and surveillance are nowhere near as developed as in the larger medical community. Digital health is being explored for health care records, data acquisition in diagnostics and rehabilitation, wellness to health care translation, intraoperative monitoring, surgical technique improvement, as well as some early-stage projects in long-term monitoring with implantable devices. The internet of things is the next digital wave that will undoubtedly affect medicine and orthopaedics. Internet of things (loT) devices are now being used to enable remote health monitoring and emergency notification systems. This article reviews current and future concepts in digital health that will impact trauma care.

The current and possible future role of 3D modelling within oesophagogastric surgery: a scoping review

BACKGROUND

3D reconstruction technology could revolutionise medicine. Within surgery, 3D reconstruction has a growing role in operative planning and procedures, surgical education and training as well as patient engagement. Whilst virtual and 3D printed models are already used in many surgical specialities, oesophagogastric surgery has been slow in their adoption. Therefore, the authors undertook a scoping review to clarify the current and future roles of 3D modelling in oesophagogastric surgery, highlighting gaps in the literature and implications for future research.

METHODS

A scoping review protocol was developed using a comprehensive search strategy based on internationally accepted guidelines and tailored for key databases (MEDLINE, Embase, Elsevier Scopus and ISI Web of Science). This is available through the Open Science Framework (osf.io/ta789) and was published in a peer-reviewed journal. Included studies underwent screening and full text review before inclusion. A thematic analysis was performed using pre-determined overarching themes: (i) surgical training and education, (ii) patient education and engagement, and (iii) operative planning and surgical practice. Where applicable, subthemes were generated.

RESULTS

A total of 56 papers were included. Most research was low-grade with 88% (n = 49) of publications at or below level III evidence. No randomised control trials or systematic reviews were found. Most literature (86%, n = 48) explored 3D reconstruction within operative planning. These were divided into subthemes of pre-operative (77%, n = 43) and intra-operative guidance (9%, n = 5). Few papers reported on surgical training and education (14%, n = 8), and were evenly subcategorised into virtual reality simulation (7%, n = 4) and anatomical teaching (7%, n = 4). No studies utilising 3D modelling for patient engagement and education were found.

CONCLUSION

The use of 3D reconstruction is in its infancy in oesophagogastric surgery. The quality of evidence is low and key themes, such as patient engagement and education, remain unexplored. Without high quality research evaluating the application and benefits of 3D modelling, oesophagogastric surgery may be left behind.

Support for using a three-dimensional anatomy application over anatomical atlases in a randomized comparison

To investigate to what extent the use of a three-dimensional (3D) anatomy computer application can improve the acquisition of anatomical knowledge compared with anatomical atlases, junior and advanced medical students participated in an experiment. Participants were asked to answer anatomical questions with the use of a 3D anatomy application (developed at the University Medical Center in Utrecht, the Netherlands) or anatomy atlases. Every student had to complete two assignments, either with an atlas or with the 3D anatomy application. One assignment consisted of 20 questions about the anatomy of the hand, the other one had 20 questions about the anatomy of the foot. The scores on the assignments and time to complete the assignments were registered and investigated. A total of 76 students participated. Students scored significantly higher and were significantly faster when they used the 3D anatomy application. Junior medical students were significantly faster than advanced medical students and particularly, advanced students who worked with an atlas needed most time. These results suggest that the 3D anatomy application is more effective as a studying tool, when compared to the use of paper atlases, for both junior and advanced medical students. The difference in time could indicate an influence of the increased number of mental steps it takes to convert two-dimensional atlas images to a 3D mental representation compared to using the 3D anatomy application, although practical issues explaining this cannot be ruled out. Future studies should establish whether the application leads to better learning/retention and to more time-efficient studying.

Examining the benefits of extended reality in neurosurgery: A systematic review

While well-established in other surgical subspecialties, the benefits of extended reality, consisting of virtual reality (VR), augmented reality (AR), and mixed reality (MR) technologies, remains underexplored in neurosurgery despite its increasing utilization. To address this gap, we conducted a systematic review of the effects of extended reality (XR) in neurosurgery with an emphasis on the perioperative period, to provide a guide for future clinical optimization. Seven primary electronic databases were screened following guidelines outlined by PRISMA and the Institute of Medicine. Reported data related to outcomes in the perioperative period and resident training were all examined, and a focused analysis of studies reporting controlled, clinical outcomes was completed. After removal of duplicates, 2548 studies were screened with 116 studies reporting measurable effects of XR in neurosurgery. The majority (82%) included cranial based applications related to tumor surgery with 34% showing improved resection rates and functional outcomes. A rise in high-quality studies was identified from 2017 to 2020 compared to all previous years (p = 0.004). Primary users of the technology were: 56% neurosurgeon (n = 65), 28% residents (n = 33) and 5% patients (n = 6). A final synthesis was conducted on 10 controlled studies reporting patient outcomes. XR technologies have demonstrated benefits in preoperative planning and multimodal neuronavigation especially for tumor surgery. However, few studies have reported patient outcomes in a controlled design demonstrating a need for higher quality data. XR platforms offer several advantages to improve patient outcomes and specifically, the patient experience for neurosurgery.

Immersive technologies for total knee arthroplasty surgical education

The need to adapt surgical curricula to meet the demands of an increasingly restrictive training environment is rising. Modern constraints of surgical trainees including work-hour restrictions and concerns surrounding patient safety have created an opportunity to supplement traditional teaching methods with developing immersive technologies including virtual and augmented reality. Virtual reality (VR) and augmented reality (AR) have been preliminarily investigated as it relates to total joint arthroplasty. The purpose of this article is to discuss VR and AR as it applies to modern total knee replacement (TKR) surgical education.

Augmented Reality-Assisted Femoral Bone Resection in Total Knee Arthroplasty

An augmented reality (AR)-based navigation system allows visualization of the center of the femoral head and femoral mechanical axis superimposed on the surgical field during total knee arthroplasty (TKA) and may help surgeons to improve the accuracy of distal femoral resection.

A Review of Orthopaedic Resident Case Logs to Identify Fluctuations in Exposure to Adult Orthopaedic Procedures

INTRODUCTION

Orthopaedic surgery resident case exposure is an important component of surgical training and is monitored by the Accreditation Council for Graduate Medical Education (ACGME) to ensure resident readiness for graduation. The purpose of this study was to investigate trends in exposure to adult orthopaedic surgical procedures and analyze the impact of the 2013 update in ACGME case logging expectations.

METHODS

A retrospective review of ACGME case log data was conducted for adult orthopaedic procedures performed by graduating orthopaedic surgery residents from 2012 to 2020. Trends in the number of cases logged and the case share by anatomical location were investigated. Linear regression analysis was performed to analyze changes in case number over the 9-year period.

RESULTS

For all surgical categories, there was stability in the average case number per resident from 2012 to 2013, followed by a precipitous decrease from 2013 to 2014. From 2014 to 2020, there has been a gradual increase in case number for all categories except "other musculoskeletal (MSK)," resulting in a total 46% recovery since the 2014 decline. Concomitant with the decline, there was a relative increase in pelvis/hip and femur/knee procedures and decrease in shoulder, other MSK, and spine procedures. From 2014 to 2020, shoulder, humerus/elbow, pelvis/hip, leg/ankle, foot/toes, and spine cases have gradually accounted for a larger proportion of total cases while femur/knee and "other MSK" cases have accounted for less.

CONCLUSIONS

The 2013 update in ACGME case logging expectations was associated with a significant decrease in case number. This is likely a reflection of residents correctly entering 1 primary Current Procedural Terminology code for each surgical case. Programs should be aware of a general increase in case number since 2014 and acknowledge the fact that some procedure types may be given priority from a logging standpoint when multiple Current Procedural Terminology codes apply.

Opportunities and challenges of using augmented reality and heads-up display in orthopaedic surgery: A narrative review

BACKGROUND & AIM

Utilization of augmented reality (AR) and heads-up displays (HUD) to aid orthopaedic surgery has the potential to benefit surgeons and patients alike through improved accuracy, safety, and educational benefits. With the COVID-19 pandemic, the opportunity for adoption of novel technology is more relevant. The aims are to assess the technology available, to understand the current evidence regarding the benefit and to consider challenges to implementation in clinical practice.

METHODS & RESULTS

PRISMA guidelines were used to filter the literature. Of 1004 articles returned the following exclusion criteria were applied: 1) reviews/commentaries 2) unrelated to orthopaedic surgery 3) use of other AR wearables beyond visual aids leaving 42 papers for review.This review illustrates benefits including enhanced accuracy and reduced time of surgery, reduced radiation exposure and educational benefits.

CONCLUSION

Whilst there are obstacles to overcome, there are already reports of technology being used. As with all novel technologies, a greater understanding of the learning curve is crucial, in addition to shielding our patients from this learning curve. Improvements in usability and implementing surgeons' specific needs should increase uptake.

Advances and innovations in total hip arthroplasty

Total hip arthroplasty (THA) has been quoted as one of the most successful and cost-effective procedures in Orthopaedics. The last decade has seen an exponential rise in the number of THAs performed globally and a sharp increase in the percentage of young patients hoping to improve their quality of life and return to physically demanding activities. Hence, it is imperative to review the various applications of technology in total hip arthroplasty for improving outcomes. The development of state-of-the-art robotic technology has enabled more reproducible and accurate acetabular positioning, while long-term data are needed to assess its cost-effectiveness. This opinion piece aims to outline and present the advances and innovations in total hip arthroplasty, from virtual reality and three-dimensional printing to patient-specific instrumentation and dual mobility bearings. This illustrates and reflects the debate that will be at the centre of hip surgery for the next decade.

Experimental pilot study for augmented reality-enhanced elbow arthroscopy

The purpose of this study was to develop and evaluate a novel elbow arthroscopy system with superimposed bone and nerve visualization using preoperative computed tomography (CT) and magnetic resonance imaging (MRI) data. We obtained bone and nerve segmentation data by CT and MRI, respectively, of the elbow of a healthy human volunteer and cadaveric Japanese monkey. A life size 3-dimensional (3D) model of human organs and frame was constructed using a stereo-lithographic 3D printer. Elbow arthroscopy was performed using the elbow of a cadaveric Japanese monkey. The augmented reality (AR) range of error during rotation of arthroscopy was examined at 20 mm scope-object distances. We successfully performed AR arthroscopy using the life-size 3D elbow model and the elbow of the cadaveric Japanese monkey by making anteromedial and posterior portals. The target registration error was 1.63 ± 0.49 mm (range 1-2.7 mm) with respect to the rotation angle of the lens cylinder from 40° to - 40°. We attained reasonable accuracy and demonstrated the operation of the designed system. Given the multiple applications of AR-enhanced arthroscopic visualization, it has the potential to be a next-generation technology for arthroscopy. This technique will contribute to the reduction of serious complications associated with elbow arthroscopy.

Development and Pre-Clinical Analysis of Spatiotemporal-Aware Augmented Reality in Orthopedic Interventions

Suboptimal interaction with patient data and challenges in mastering 3D anatomy based on ill-posed 2D interventional images are essential concerns in image-guided therapies. Augmented reality (AR) has been introduced in the operating rooms in the last decade; however, in image-guided interventions, it has often only been considered as a visualization device improving traditional workflows. As a consequence, the technology is gaining minimum maturity that it requires to redefine new procedures, user interfaces, and interactions. The main contribution of this paper is to reveal how exemplary workflows are redefined by taking full advantage of head-mounted displays when entirely co-registered with the imaging system at all times. The awareness of the system from the geometric and physical characteristics of X-ray imaging allows the exploration of different human-machine interfaces. Our system achieved an error of 4.76 ± 2.91mm for placing K-wire in a fracture management procedure, and yielded errors of 1.57 ± 1.16° and 1.46 ± 1.00° in the abduction and anteversion angles, respectively, for total hip arthroplasty (THA). We compared the results with the outcomes from baseline standard operative and non-immersive AR procedures, which had yielded errors of [4.61mm, 4.76°, 4.77°] and [5.13mm, 1.78°, 1.43°], respectively, for wire placement, and abduction and anteversion during THA. We hope that our holistic approach towards improving the interface of surgery not only augments the surgeon's capabilities but also augments the surgical team's experience in carrying out an effective intervention with reduced complications and provide novel approaches of documenting procedures for training purposes.

The impact of COVID-19 on the future of orthopaedic training in the UK

The COVID-19 pandemic has had a major impact on global healthcare systems, has drastically affected patient care, and has had widespread effects upon medical education. As plans are being devised to reinstate elective surgical services, it is important to consider the impact that the pandemic has had and will continue to have on surgical training. We describe the effect COVID-19 has had at all levels of training in the UK within trauma and orthopaedics and evaluate how training might change in the future. We found that the COVID-19 pandemic has significantly impacted trainees within trauma and orthopaedics at all levels of training. It had led to reduced operative exposure, cancellations of examinations and courses, and modifications to speciality recruitment and annual appraisals. This cohort of trainees is witnessing novel methods of delivering orthopaedic services, which will continue to develop and become part of routine practice even once the pandemic has resolved. It will be important to observe the extent to which the rapid changes currently being introduced will impact the personal health, safety, and career progression of current trainees.

[Potential of digitalization in undergraduate and postgraduate medical education and training in orthopedics and trauma surgery]

INTRODUCTION

In its digital agenda the German Federal Government pursues the ambitious objective to facilitate digital competence and perform research into digital learning and teaching processes. Considerable investments are to be concentrated into the future viability of education, academic research and digitalization. As far as academic teaching and further education are concerned, not only in the field of orthopedics and trauma surgery, three aspects can be identified: digital organization, digital competence and digital tools.

DIGITAL APPLICATIONS

New formats, such as the elective subject digital health of the Charité in Berlin, enable digital competences to be mediated in a multimodal and interdisciplinary way. With the help of a newly developed app the University of Essen provides teachers and students with mobile and flexible access to information on lectures in terms of content and organization. Especially because of transparency, high legal compliance and predictability, the digital logbook for the resident training program promises a real innovation for trainees in the further training reformation. Augmented and virtual reality play a crucial role in the imparting of practical skills and interconnect high-tech with classical craftsmanship. Digital training course formats have significantly gained in importance and are meanwhile well-established tools for efficient advanced medical training.

OUTLOOK

If orthopedic and trauma surgeons take an active role in the process of digitalization of teaching, they can take part in decisions, adequately prepare the colleagues of tomorrow, optimize patient care, encourage innovations and altogether improve the discipline even more.

[Digital OR]

BACKGROUND

Numerous processes are involved in the orthopedic and trauma surgery operating room (OR). Technical progress, particularly in the area of digitalization, is increasingly changing routine surgical procedures.

OBJECTIVE

This article highlights the possibilities and also limitations regarding this matter.

MATERIAL AND METHODS

Based on the current literature this article provides insights into innovations in the areas of digitalization of surgical devices, hybrid OR, machine-2-machine networking, management systems for perioperative efficiency improvement, 3D printing technology and robotics.

RESULTS

The technical possibilities for the use of digital applications in the surgical environment are rapidly increasing. Close cooperation with industrial partners is important in this context. Technologies from the automotive, gaming and mobile phone industries are being adopted.

CONCLUSION

Digital technology in the OR can improve treatment quality, patient and staff safety and cost efficiency; however, the networking of devices, implementation of innovations in existing structures and the sometimes high acquisition costs are still limiting factors.

[Extended reality technologies for support of surgical workflows]

Current developments in the field of extended reality (XR) could prove useful in the optimization of surgical workflows, time effectiveness and postoperative outcome. Although still primarily a subject of research, the state of XR technologies is rapidly improving and approaching feasibility for a broad clinical application. Surgical fields of application of XR technologies are currently primarily training, preoperative planning and intraoperative assistance. For all three areas, products already exist (some clinically approved) and technical feasibility studies have been conducted. In teaching, the use of XR can already be assessed as fundamentally practical and meaningful but still needs to be evaluated in large multicenter studies. In preoperative planning XR can also offer advantages, although technical limitations often impede routine use; however, for cases of intraoperative use informative evaluation studies are mostly lacking, so that an assessment is not yet possible in a meaningful way. Furthermore, there is a lack of assessments regarding cost-effectiveness in all three areas. The XR technologies enable proven advantages in surgical workflows despite the lack of high-quality evaluation with respect to the practical and clinical use of XR. New concepts for effective interaction with XR media also need to be developed. In the future, further research progress and technical developments in the field can be expected.

Ten techniques for improving navigated spinal surgery

With the identification of literature shortfalls on the techniques employed in intraoperative navigated (ION) spinal surgery, we outline a number of measures which have been synthesised into a coherent operative technique. These include positioning, dissection, management of the reference frame, the grip, the angle of attack, the drill, the template, the pedicle screw, the wire, and navigated intrathecal analgesia. Optimizing techniques to improve accuracy allow an overall reduction of the repetition of the surgical steps with its associated productivity benefits including time, cost, radiation, and safety.

Cite this article: Bone Joint J 2020;102-B(3):371–375.