Augmented reality for base plate component placement in reverse total shoulder arthroplasty: a feasibility study

Intraoperative Navigation in Reverse Shoulder Arthroplasty: Advantages and Future Prospects

Intraoperative navigation is a novel technology that can provide real-time feedback to the surgeon during implantation and enhance the accuracy and precision of glenoid component positioning. Applications of intraoperative navigation systems have demonstrated increased precision in baseplate version and inclination, as well as improved baseplate screw placement, with fewer screws used and greater purchase length achieved when compared to standard instrumentation. Early clinical studies have shown favorable results, with significantly improved patient-reported and clinical outcomes and decreased complications. The implementation of intraoperative navigation is associated with a short learning curve and a minimal increase in operative time. Nevertheless, further research is necessary to substantiate the clinical benefit of navigation and evaluate its economic cost-effectiveness and impact on implant survival. Augmented reality and robotic-assisted surgery are additional emerging technologies that, while novel, hold the potential to further advance the field of shoulder arthroplasty.

Mixed-reality improves execution of templated glenoid component positioning in shoulder arthroplasty: a CT imaging analysis

INTRODUCTION

Glenoid placement is critical for successful outcomes in total shoulder arthroplasty (TSA). Preoperative templating with three-dimensional imaging has improved implant positioning, but deviations from the planned inclination and version still occur. Mixed-Reality (MR) is a novel technology that allows surgeons intra-operative access to three-dimensional imaging and templates, capable of overlaying the surgical field to help guide component positioning. The purpose of this study was to compare the execution of preoperative templates using MR vs.standard instruments (SIs).

METHODS

Retrospective review of 97 total shoulder arthroplasties (18 anatomic, 79 reverse) from a single high-volume shoulder surgeon between January 2021 and February 2023, including only primary diagnoses of osteoarthritis, rotator cuff arthropathy, or a massive irreparable rotator cuff tear. To be included, patients needed a templated preoperative plan and then a postoperative computed tomography scan. Allocation to MR vs. SI was based on availability of the MR headset, industry technical personnel, and the templated preoperative plan loaded into the software, but preoperative or intraoperative patient factors did not contribute to the allocation decision. Postoperative inclination and version were measured by two independent, blinded physicians and compared to the preoperative template. From these measurements, we calculated the mean difference, standard deviation (SD), and variance to compare MR and SI.

RESULTS

Comparing 25 MR to 72 SI cases, MR significantly improved both inclination (P < .001) and version (P < .001). Specifically, MR improved the mean difference from preoperative templates (by 1.9° inclination, 2.4° version), narrowed the SD (by 1.7° inclination, 1.8° version), and decreased the variance (11.7-3.0 inclination, 14.9-4.3 version). A scatterplot of the data demonstrates a concentration of MR cases within 5° of plan relative to SI cases typically within 10° of plan. There was no difference in operative time.

CONCLUSION

MR improved the accuracy and precision of glenoid positioning. Although it is unlikely that 2° makes a detectable clinical difference, our results demonstrate the potential ability for technology like MR to narrow the bell curve and decrease the outliers in glenoid placement. This will be particularly relevant as MR and other similar technologies continue to evolve into more effective methods in guiding surgical execution.

Augmented reality-assisted intraoperative navigation increases precision of glenoid inclination in reverse shoulder arthroplasty

BACKGROUND

Reverse total shoulder arthroplasty (RTSA) is the standard of care for patients with glenohumeral osteoarthritis and rotator cuff deficiency. Preoperative RTSA planning based on medical images and patient-specific instruments has been established over the last decade. This study aims to determine the effects of using augmented reality-assisted intraoperative navigation (ARIN) for baseplate positioning in RTSA compared to preoperative planning. It is hypothesized that ARIN will decrease deviation between preoperative planning and postoperative baseplate positioning. Moreover, ARIN will decrease deviation between the (senior) more (>50 RTSAs/yr) and less experienced (junior) surgeon (5-10 RTSAs/yr).

METHODS

Preoperative CT scans of 16 fresh-frozen cadaveric shoulders were obtained. Baseplate placements were planned using a validated software. The data were then converted and uploaded to the augmented reality system (NextAR; Medacta International). Each of the 8 RTSAs were implanted by a senior and a junior surgeon, with 4 RTSAs using ARIN and 4 without. A postoperative CT scan was performed in all cases. The scanned scapulae were segmented, and the preoperative scan was laid over the postoperative scapula by the nearest iterative point cloud analysis. The deviation from the planned entry point and trajectory was calculated regarding the inclination, retroversion, medialization (reaming depth) and lateralization, anteroposterior position, and superoinferior position of the baseplate. Data are reported as mean ± standard deviation (SD) or mean and 95% confidence interval (CI). P values < .05 were considered statistically significant.

RESULTS

The use of ARIN yielded a reduction in the absolute difference between planned and obtained inclination from 9° (SD: 4°) to 3° (SD: 2°) (P = .011). Mean difference in planned-obtained inclination between surgeons was 3° in free-hand surgeries (95% CI: -4 to 10; P = .578), whereas this difference reduced to 1° (95% CI: -6 to 7, P = .996) using ARIN. Retroversion, medialization (reaming depth) and lateralization, anteroposterior position, and superoinferior position of the baseplate were not affected by using ARIN. Surgical duration was increased using ARIN for both the senior (10 minutes) and junior (18 minutes) surgeon.

CONCLUSIONS

The implementation of ARIN leads to greater accuracy of glenoid component placement, specifically with respect to inclination. Further studies have to verify if this increased accuracy is clinically important. Furthermore, ARIN allows less experienced surgeons to achieve a similar level of accuracy in component placement comparable to more experienced surgeons. However, the potential advantages of ARIN in RTSA are counterbalanced by an increase in operative time.

Advanced technology in shoulder arthroplasty

Background

Glenoid component positioning is an important and challenging aspect of total shoulder arthroplasty. The use of freehand technique with standard instrumentation or preoperative planning based on 2-dimensional computed tomography (CT) scans provides an opportunity for improvement in terms of component accuracy, precision, and deformity correction. These techniques have produced varying outcomes.

Methods

Preoperative planning software (PPS), patient specific instrumentation (PSI), and intraoperative navigation (NAV) have been developed to improve the accuracy of implant placement and deformity correction with the ultimate goals of improved patient outcomes and implant longevity. Literature search was conducted on published and available studies comparing the accuracy of glenoid component placement and improvements in surgical and patient outcomes amongst the aforementioned techniques.

Results

PPS, PSI, and NAV have demonstrated improved accuracy over freehand techniques with standard instrumentation. However, data demonstrating the clinical benefit and cost effectiveness of these new technologies are lacking.

Discussion

In this paper, we reviewed the evidence available to answer the question of whether or not advanced shoulder arthroplasty technologies have been beneficial and reviewed future technologies in development such as virtual/mixed-reality and robotic assisted shoulder surgery.

Level of Evidence

4.

Three-dimensional planning, navigation, patient-specific instrumentation and mixed reality in shoulder arthroplasty: a digital orthopedic renaissance

Accurate component placement in shoulder arthroplasty is crucial for avoiding complications, achieving superior biomechanical performance and optimizing functional outcomes. Shoulder and elbow surgeons have explored various methods to improve surgical understanding and precise execution including preoperative planning with 3D computed tomography (CT), patient-specific instrumentation (PSI), intraoperative navigation, and mixed reality (MR). 3D preoperative planning facilitated by CT scans and advanced software, enhances surgical precision, influences decision-making for implant types and approaches, reduces errors in guide pin placement, and contributes to cost-effectiveness. Navigation demonstrates benefits in reducing malpositioning, optimizing baseplate stability, improving humeral cut, and potentially conserving bone stock, although challenges such as varied operating times and costs warrant further investigation. The personalized patient care and enhanced operational efficiency associated with PSI are not only attractive for achieving desired component positions but also hold promise for improved outcomes in complex cases involving glenoid bone loss. Augmented reality (AR) and virtual reality (VR) technologies play a pivotal role in reshaping shoulder arthroplasty. They offer benefits in preoperative planning, intraoperative guidance, and interactive surgery. Studies demonstrate their effectiveness in AR-guided guidewire placement, providing real-time surgical advice during reverse total shoulder arthroplasty (RTSA). Additionally, these technologies show promise in orthopedic training, delivering superior realism and accelerating learning compared to conventional methods.

Metaverse, virtual reality and augmented reality in total shoulder arthroplasty: a systematic review

PURPOSE

This systematic review aims to provide an overview of the current knowledge on the role of the metaverse, augmented reality, and virtual reality in reverse shoulder arthroplasty.

METHODS

A systematic review was performed using the PRISMA guidelines. A comprehensive review of the applications of the metaverse, augmented reality, and virtual reality in in-vivo intraoperative navigation, in the training of orthopedic residents, and in the latest innovations proposed in ex-vivo studies was conducted.

RESULTS

A total of 22 articles were included in the review. Data on navigated shoulder arthroplasty was extracted from 14 articles: seven hundred ninety-three patients treated with intraoperative navigated rTSA or aTSA were included. Also, three randomized control trials (RCTs) reported outcomes on a total of fifty-three orthopedics surgical residents and doctors receiving VR-based training for rTSA, which were also included in the review. Three studies reporting the latest VR and AR-based rTSA applications and two proof of concept studies were also included in the review.

CONCLUSIONS

The metaverse, augmented reality, and virtual reality present immense potential for the future of orthopedic surgery. As these technologies advance, it is crucial to conduct additional research, foster development, and seamlessly integrate them into surgical education to fully harness their capabilities and transform the field. This evolution promises enhanced accuracy, expanded training opportunities, and improved surgical planning capabilities.

Feasibility and Usability of Augmented Reality Technology in the Orthopaedic Operating Room

PURPOSE OF REVIEW

Augmented reality (AR) has gained popularity in various sectors, including gaming, entertainment, and healthcare. The desire for improved surgical navigation within orthopaedic surgery has led to the evaluation of the feasibility and usability of AR in the operating room (OR). However, the safe and effective use of AR technology in the OR necessitates a proper understanding of its capabilities and limitations. This review aims to describe the fundamental elements of AR, highlight limitations for use within the field of orthopaedic surgery, and discuss potential areas for development.

RECENT FINDINGS

To date, studies have demonstrated evidence that AR technology can be used to enhance navigation and performance in orthopaedic procedures. General hardware and software limitations of the technology include the registration process, ergonomics, and battery life. Other limitations are related to the human response factors such as inattentional blindness, which may lead to the inability to see complications within the surgical field. Furthermore, the prolonged use of AR can cause eye strain and headache due to phenomena such as the vergence-convergence conflict. AR technology may prove to be a better alternative to current orthopaedic surgery navigation systems. However, the current limitations should be mitigated to further improve the feasibility and usability of AR in the OR setting. It is important for both non-clinicians and clinicians to work in conjunction to guide the development of future iterations of AR technology and its implementation into the OR workflow.

Validation of mixed-reality surgical navigation for glenoid axis pin placement in shoulder arthroplasty using a cadaveric model

BACKGROUND

Mixed reality may offer an alternative for computer-assisted navigation in shoulder arthroplasty. The purpose of this study was to determine the accuracy and precision of mixed-reality guidance for the placement of the glenoid axis pin in cadaver specimens. This step is essential for accurate glenoid placement in total shoulder arthroplasty.

METHODS

Fourteen cadaveric shoulders underwent simulated shoulder replacement surgery by 7 experienced shoulder surgeons. The surgeons exposed the cadavers through a deltopectoral approach and then used mixed-reality surgical navigation to insert a guide pin in a preplanned position and trajectory in the glenoid. The mixed-reality system used the Microsoft Hololens 2 headset, navigation software, dedicated instruments with fiducial marker cubes, and a securing pin. Computed tomography scans obtained before and after the procedure were used to plan the surgeries and determine the difference between the planned and executed values for the entry point, version, and inclination. One specimen had to be discarded from the analysis because the guide pin was removed accidentally before obtaining the postprocedure computed tomography scan.

RESULTS

Regarding the navigated entry point on the glenoid, the mean difference between planned and executed values was 1.7 ± 0.8 mm; this difference was 1.2 ± 0.6 mm in the superior-inferior direction and 0.9 ± 0.8 mm in the anterior-posterior direction. The maximum deviation from the entry point for all 13 specimens analyzed was 3.1 mm. Regarding version, the mean difference between planned and executed version values was 1.6° ± 1.2°, with a maximum deviation in version for all 13 specimens of 4.1°. Regarding inclination, the mean angular difference was 1.7° ± 1.5°, with a maximum deviation in inclination of 5°.

CONCLUSIONS

The mixed-reality navigation system used in this study allowed surgeons to insert the glenoid guide pin on average within 2 mm from the planned entry point and within 2° of version and inclination. The navigated values did not exceed 3 mm or 5°, respectively, for any of the specimens analyzed. This approach may help surgeons more accurately place the definitive glenoid component.

[Reverse Shoulder Arthroplasty - Current Concepts]

Due to first promising long term outcome data, reverse shoulder arthroplasty experienced an immense increase of usage during the past decade. Moreover, the initial Grammont concept has constantly been refined and adapted to current scientific findings. Therefore, clinical and radiological problems like scapular notching and postoperative instability were constantly addressed but do still remain an area of concern.This article summarises current concepts in reverse shoulder arthroplasty and gives an overview of actual indications like cuff tear arthropathy, severe osteoarthritis, proximal humerus fractures, tumours, fracture sequelae as well as revision surgery and their corresponding clinical and radiological results.

The Value of Computed Tomography-Based Planning in Shoulder Arthroplasty Compared to Intra-/Interobserver Reliability of X-ray Planning

Background: Reversed total shoulder arthroplasty (RTSA) is an established surgery for many pathologies of the shoulder and the demand continues to rise with an aging population. Preoperative planning is mandatory to support the surgeon's understanding of the patient's individual anatomy and, therefore, is crucial for the patient's outcome. Methods: In this observational study, we identified 30 patients who underwent RTSA with two- and three-dimensional preoperative planning. Each patient underwent new two-dimensional planning from a medical student and an orthopedic resident as well as through a mid-volume and high-volume shoulder surgeon, which was repeated after a minimum of 4 weeks. The intra- and interobserver reliability was then analyzed and compared to the 3D planning and the implanted prosthesis. The evaluated parameters were the size of the pegged glenoid baseplate, glenosphere, and humeral short stem. Results: The inter-rater reliability showed higher deviations in all four raters compared to the 3D planning of the base plate, glenosphere, and shaft. The intra-rater reliability showed a better correlation in more experienced raters, especially in the planning of the shaft. Conclusions: Our study shows that 3D planning is more accurate than traditional planning on plain X-rays, despite experienced shoulder surgeons showing better results in 2D planning than inexperienced ones.

The application of extended reality technology-assisted intraoperative navigation in orthopedic surgery

Extended reality (XR) technology refers to any situation where real-world objects are enhanced with computer technology, including virtual reality, augmented reality, and mixed reality. Augmented reality and mixed reality technologies have been widely applied in orthopedic clinical practice, including in teaching, preoperative planning, intraoperative navigation, and surgical outcome evaluation. The primary goal of this narrative review is to summarize the effectiveness and superiority of XR-technology-assisted intraoperative navigation in the fields of trauma, joint, spine, and bone tumor surgery, as well as to discuss the current shortcomings in intraoperative navigation applications. We reviewed titles of more than 200 studies obtained from PubMed with the following search terms: extended reality, mixed reality, augmented reality, virtual reality, intraoperative navigation, and orthopedic surgery; of those 200 studies, 69 related papers were selected for abstract review. Finally, the full text of 55 studies was analyzed and reviewed. They were classified into four groups-trauma, joint, spine, and bone tumor surgery-according to their content. Most of studies that we reviewed showed that XR-technology-assisted intraoperative navigation can effectively improve the accuracy of implant placement, such as that of screws and prostheses, reduce postoperative complications caused by inaccurate implantation, facilitate the achievement of tumor-free surgical margins, shorten the surgical duration, reduce radiation exposure for patients and surgeons, minimize further damage caused by the need for visual exposure during surgery, and provide richer and more efficient intraoperative communication, thereby facilitating academic exchange, medical assistance, and the implementation of remote healthcare.

Artificial intelligence- and computer-assisted navigation for shoulder surgery

Background: Over the last few decades, shoulder surgery has undergone rapid advancements, with ongoing exploration and the development of innovative technological approaches. In the coming years, technologies such as robot-assisted surgeries, virtual reality, artificial intelligence, patient-specific instrumentation, and different innovative perioperative and preoperative planning tools will continue to fuel a revolution in the medical field, thereby pushing it toward new frontiers and unprecedented advancements. In relation to this, shoulder surgery will experience significant breakthroughs. Main body: Recent advancements and technological innovations in the field were comprehensively analyzed. We aimed to provide a detailed overview of the current landscape, emphasizing the roles of technologies. Computer-assisted surgery utilizing robotic- or image-guided technologies is widely adopted in various orthopedic specialties. The most advanced components of computer-assisted surgery are navigation and robotic systems, with functions and applications that are continuously expanding. Surgical navigation requires a visual system that presents real-time positional data on surgical instruments or implants in relation to the target bone, displayed on a computer monitor. There are three primary categories of surgical planning that utilize navigation systems. The initial category involves volumetric images, such as ultrasound echogram, computed tomography, and magnetic resonance images. The second type is based on intraoperative fluoroscopic images, and the third type incorporates kinetic information about joints or morphometric data about the target bones acquired intraoperatively. Conclusion: The rapid integration of artificial intelligence and deep learning into the medical domain has a significant and transformative influence. Numerous studies utilizing deep learning-based diagnostics in orthopedics have remarkable achievements and performance.

Glenoid component placement in reverse shoulder arthroplasty assisted with augmented reality through a head-mounted display leads to low deviation between planned and postoperative parameters

BACKGROUND

Navigated augmented reality (AR) through a head-mounted display (HMD) may lead to accurate glenoid component placement in reverse shoulder arthroplasty (RSA). The purpose of this study is to evaluate the deviation between planned, intra- and postoperative inclination, retroversion, entry point, depth, and rotation of the glenoid component placement assisted by a navigated AR through HMD during RSA.

METHODS

Both shoulders of 6 fresh frozen human cadavers, free from fractures or other bony pathologies, were used. Preoperative computed tomography (CT) scans were used for the 3-dimensional (3D) planning. The glenoid component placement was assisted using a navigated AR system through an HMD in all specimens. Intraoperative inclination, retroversion, depth, and rotation were measured by the system. A postoperative CT scan was performed. The pre- and postoperative 3D CT scan reconstructions were superimposed to calculate the deviation between planned and postoperative inclination, retroversion, entry point, depth, and rotation of the glenoid component placement. Additionally, a comparison between intra- and postoperative values was calculated. Outliers were defined as >10° inclination, >10° retroversion, >3 mm entry point.

RESULTS

The registration algorithm of the scapulae prior to the procedure was correctly completed for all cases. The deviations between planned and postoperative values were 1.0° ± 0.7° for inclination, 1.8° ± 1.3° for retroversion, 1.1 ± 0.4 mm for entry point, 0.7 ± 0.6 mm for depth, and 1.7° ± 1.6° for rotation. The deviation between intra- and postoperative values were 0.9° ± 0.8° for inclination, 1.2° ± 1.1° for retroversion, 0.6 ± 0.5 mm for depth, and 0.3° ± 0.2° for rotation. There were no outliers between planned and postoperative parameters.

CONCLUSION

In this study, the use of a navigated AR system through an HMD for RSA led to low deviation between planned and postoperative values and between intra- and postoperative parameters.

Preoperative planning in reverse shoulder arthroplasty: plain radiographs vs. computed tomography scan vs. navigation vs. augmented reality

Reverse shoulder arthroplasty (RSA) has become a highly successful treatment option for various shoulder conditions, leading to a significant increase in its utilization since its approval in 2003. However, postoperative complications, including scapular notching, prosthetic instability, and component loosening, remain a concern. These complications can often be attributed to technical errors during component implantation, emphasizing the importance of proper preoperative planning and accurate positioning of prosthetic components. Improper baseplate and glenosphere positioning in RSA have been linked to impingement, reduced range of motion, and increased scapular notching. Additionally, the relationship between component positioning and intrinsic stability of RSA has been established, with glenoid component retroversion exceeding 10° posing a risk to implant stability. Adequate initial glenoid baseplate fixation, achieved through optimal seating and the use of appropriate screws, is crucial for long-term success and prevention of early failure. Factors such as lateralization and distalization also influence outcomes and complications in RSA, yet standardized guidelines for preoperative planning in these parameters are still lacking. Despite the impact of component position on outcomes, glenoid component implantation remains challenging, with position errors being common even among experienced surgeons. Challenges arise due to factors such as deformity, bone defects, limited exposure, and the absence of reliable bony landmarks intraoperatively. With the evolving understanding of RSA biomechanics and the significance of implant configuration and positioning, advancements in preoperative planning and surgical aids have emerged. This review article explores the current evidence on preoperative planning techniques in RSA, including plain radiographs, three-dimensional imaging, computer planning software, intraoperative navigation, and augmented reality (AR), highlighting their potential benefits and advancements in improving implant position accuracy.

Augmented reality and shoulder replacement: a state-of-the-art review article

Since its implementation, the rates of failure of total shoulder arthroplasty which may be due to malpositioning pushed to improve this surgery by creating new techniques and tools to help perioperatively. Augmented reality, a newly used tool in orthopedic surgery can help bypass this problem and reduce the rates of failure faced in shoulder replacement surgeries. Although this technology has revolutionized orthopedic surgery and helped improve the accuracy in shoulder prosthesis components positioning, it still has some limitations such as inaccurate over-imposition that should be addressed before it becomes of standard usage.

An accurate scapula registration process in shoulder arthroplasty using mixed reality

PURPOSE

Arthroplasty surgery can be challenging because of limited exposure of the bones. However, in shoulder arthroplasty for example, a good positioning of the glenoid component is essential to mitigate risks of revision surgeries. To improve the procedure's outcomes, mixed reality can be used as a guidance system relying on a fundamental registration step between the patient's bone and its corresponding 3D model.

METHODS

We present a complete registration workflow for shoulder arthroplasty using Hololens 2 Head Mounted Display. We rely on acquisitions made thanks to our marker-based tracking system, an improved Iterative Closest Point algorithm and verification steps. Our accuracy targets are 1.5 ± 1.5 mm for the glenoid guidewire entry point on both antero-posterior and supero-inferior axes, and 1.5 ± 1[Formula: see text] for inclination and version. The overall process must last less than 5 min.

RESULTS

We have evaluated our process on a cohort of 13 3D printed glenoid bones of all types, showing an average accuracy of 0.84 ± 0.58 mm on the antero-posterior axis and 0.49 ± 0.41 mm on the supero-inferior one for the entry point. As for inclination and version, we have 0.89 ± 0.6[Formula: see text] and 0.97 ± 0.8[Formula: see text], respectively. The mean process time is about 1 min 24 s.

CONCLUSIONS

We have developed a complete, embedded registration workflow and a verification protocol to evaluate our accuracy. Our results are promising for the improvement of the glenoid guidewire placement. Moreover, everything is performed in the field of view of the surgeon, which allows them to fully concentrate on the surgical site.

Emerging Technologies in Shoulder Arthroplasty: Navigation, Mixed Reality, and Preoperative Planning

Shoulder arthroplasty is a rapidly improving and utilized management for end-stage arthritis that is associated with improved functional outcomes, pain relief, and long-term implant survival. Accurate placement of the glenoid and humeral components is critical for improved outcomes. Traditionally, preoperative planning was limited to radiographs and 2-dimensional computed tomography (CT); however, 3-dimensional CT is becoming more commonly utilized and necessary to understand complex glenoid and humeral deformities. To further increase accurate component placement, intraoperative assistive devices-patient-specific instrumentation, navigation, and mixed reality-minimize malpositioning, increase surgeon accuracy, and maximize fixation. These intraoperative technologies likely represent the future of shoulder arthroplasty.

The Evolution of Reverse Total Shoulder Arthroplasty-Where Do We Stand and What Comes Next?

Over 35 years ago, the pioneer Paul Grammont from Lyon published his ideas of a reversed semi-constraint prosthesis improving the moment arm of the deltoid by medializing the center of rotation and lengthening of the arm and thus increasing deltoid muscle tension [...].

Remote Interactive Surgery Platform (RISP): Proof of Concept for an Augmented-Reality-Based Platform for Surgical Telementoring

The “Remote Interactive Surgery Platform” (RISP) is an augmented reality (AR)-based platform for surgical telementoring. It builds upon recent advances of mixed reality head-mounted displays (MR-HMD) and associated immersive visualization technologies to assist the surgeon during an operation. It enables an interactive, real-time collaboration with a remote consultant by sharing the operating surgeon’s field of view through the Microsoft (MS) HoloLens2 (HL2). Development of the RISP started during the Medical Augmented Reality Summer School 2021 and is currently still ongoing. It currently includes features such as three-dimensional annotations, bidirectional voice communication and interactive windows to display radiographs within the sterile field. This manuscript provides an overview of the RISP and preliminary results regarding its annotation accuracy and user experience measured with ten participants.

Visualization, registration and tracking techniques for augmented reality guided surgery: a review

Augmented reality (AR) surgical navigation has developed rapidly in recent years. This paper reviews and analyzes the visualization, registration, and tracking techniques used in AR surgical navigation systems, as well as the application of these AR systems in different surgical fields. The types of AR visualization are divided into two categories ofin situvisualization and nonin situvisualization. The rendering contents of AR visualization are various. The registration methods include manual registration, point-based registration, surface registration, marker-based registration, and calibration-based registration. The tracking methods consist of self-localization, tracking with integrated cameras, external tracking, and hybrid tracking. Moreover, we describe the applications of AR in surgical fields. However, most AR applications were evaluated through model experiments and animal experiments, and there are relatively few clinical experiments, indicating that the current AR navigation methods are still in the early stage of development. Finally, we summarize the contributions and challenges of AR in the surgical fields, as well as the future development trend. Despite the fact that AR-guided surgery has not yet reached clinical maturity, we believe that if the current development trend continues, it will soon reveal its clinical utility.

Clinical applications of augmented reality in orthopaedic surgery: a comprehensive narrative review

PURPOSE

The development of augmented reality (AR) technology allows orthopaedic surgeons to incorporate and visualize surgical data, assisting the execution of both routine and complex surgical operations. Uniquely, AR technology allows a surgeon to view the surgical field and superimpose peri-operative imaging, anatomical landmarks, navigation guidance, and more, all in one view without the need for conjugate gaze between multiple screens. The aim of this literature review was to introduce the fundamental requirements for an augmented reality system and to assess the current applications, outcomes, and potential limitations to this technology.

METHODS

A literature search was performed using MEDLINE and Embase databases, by two independent reviewers, who then collaboratively synthesized and collated the results of the literature search into a narrative review focused on the applications of augmented reality in major orthopaedic sub-specialties.

RESULTS

Current technology requires that pre-operative patient data be acquired, and AR-compatible models constructed. Intra-operatively, to produce manipulatable virtual images into the user's view in real time, four major components are required including a camera, computer image processing technology, tracking tools, and an output screen. The user is provided with a heads-up display, which is a transparent display, enabling the user to look at both their natural view and the computer-generated images. Currently, high-quality evidence for clinical implementation of AR technology in the orthopaedic surgery operating room is lacking; however, growing in vitro literature highlights a multitude of potential applications, including increasing operative accuracy, improved biomechanical angular and alignment parameters, and potentially reduced operative time.

CONCLUSION

While the application of AR systems in surgery is currently in its infancy, we anticipate rapid and widespread implementation of this technology in various orthopaedic sub-specialties.

A review of advances in image-guided orthopedic surgery

Orthopedic surgery remains technically demanding due to the complex anatomical structures and cumbersome surgical procedures. The introduction of image-guided orthopedic surgery (IGOS) has significantly decreased the surgical risk and improved the operation results. This review focuses on the application of recent advances in artificial intelligence (AI), deep learning (DL), augmented reality (AR) and robotics in image-guided spine surgery, joint arthroplasty, fracture reduction and bone tumor resection. For the pre-operative stage, key technologies of AI and DL based medical image segmentation, 3D visualization and surgical planning procedures are systematically reviewed. For the intra-operative stage, the development of novel image registration, surgical tool calibration and real-time navigation are reviewed. Furthermore, the combination of the surgical navigation system with AR and robotic technology is also discussed. Finally, the current issues and prospects of the IGOS system are discussed, with the goal of establishing a reference and providing guidance for surgeons, engineers, and researchers involved in the research and development of this area.

Augmented reality through head-mounted display for navigation of baseplate component placement in reverse total shoulder arthroplasty: a cadaveric study

BACKGROUND

To achieve an optimal clinical outcome in reverse total shoulder arthroplasty (RSA), accurate placement of the components is essential. The recently introduced navigation technology of augmented reality (AR) through head-mounted displays (HMD) offers a promising new approach to visualize the anatomy and navigate component positioning in various orthopedic surgeries. We hypothesized that AR through HMD is feasible, reliable, and accurate for guidewire placement in RSA baseplate positioning.

METHODS

Twelve human cadaver shoulders were scanned with computed tomography (CT) and RSA baseplate positioning was 3-D planned using dedicated software. The shoulders were prepared through a deltopectoral approach and an augmented reality hologram was superimposed using the HMD Microsoft HoloLense. The central guidewire was then navigated through the HMD to achieve the planned entry point and trajectory. Postoperatively, the shoulders were CT-scanned a second time and the deviation from the planning was calculated.

RESULTS

The mean deviation of the entry point was 3.5 mm  ± 1.7 mm (95% CI 2.4 mm; 4.6 mm). The mean deviation of the planned trajectory was 3.8°  ± 1.7° (95% CI 2.6°; 4.9°).

CONCLUSION

Augmented reality seems feasible and reliable for baseplate guidewire positioning in reverse total shoulder arthroplasty. The achieved values were accurate.

Augmenting Performance: A Systematic Review of Optical See-Through Head-Mounted Displays in Surgery

We conducted a systematic review of recent literature to understand the current challenges in the use of optical see-through head-mounted displays (OST-HMDs) for augmented reality (AR) assisted surgery. Using Google Scholar, 57 relevant articles from 1 January 2021 through 18 March 2022 were identified. Selected articles were then categorized based on a taxonomy that described the required components of an effective AR-based navigation system: data, processing, overlay, view, and validation. Our findings indicated a focus on orthopedic (n=20) and maxillofacial surgeries (n=8). For preoperative input data, computed tomography (CT) (n=34), and surface rendered models (n=39) were most commonly used to represent image information. Virtual content was commonly directly superimposed with the target site (n=47); this was achieved by surface tracking of fiducials (n=30), external tracking (n=16), or manual placement (n=11). Microsoft HoloLens devices (n=24 in 2021, n=7 in 2022) were the most frequently used OST-HMDs; gestures and/or voice (n=32) served as the preferred interaction paradigm. Though promising system accuracy in the order of 2–5 mm has been demonstrated in phantom models, several human factors and technical challenges—perception, ease of use, context, interaction, and occlusion—remain to be addressed prior to widespread adoption of OST-HMD led surgical navigation.

UltrARsound: in situ visualization of live ultrasound images using HoloLens 2

Purpose

Augmented Reality (AR) has the potential to simplify ultrasound (US) examinations which usually require a skilled and experienced sonographer to mentally align narrow 2D cross-sectional US images in the 3D anatomy of the patient. This work describes and evaluates a novel approach to track retroreflective spheres attached to the US probe using an inside-out technique with the AR glasses HoloLens 2. Finally, live US images are displayed in situ on the imaged anatomy.

Methods

The Unity application UltrARsound performs spatial tracking of the US probe and attached retroreflective markers using the depth camera integrated into the AR glasses—thus eliminating the need for an external tracking system. Additionally, a Kalman filter is implemented to improve the noisy measurements of the camera. US images are streamed wirelessly via the PLUS toolkit to HoloLens 2. The technical evaluation comprises static and dynamic tracking accuracy, frequency and latency of displayed images.

Results

Tracking is performed with a median accuracy of 1.98 mm/1.81\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document}∘ for the static setting when using the Kalman filter. In a dynamic scenario, the median error was 2.81 mm/1.70\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document}∘. The tracking frequency is currently limited to 20 Hz. 83% of the displayed US images had a latency lower than 16 ms.

Conclusions

In this work, we showed that spatial tracking of retroreflective spheres with the depth camera of HoloLens 2 is feasible, achieving a promising accuracy for in situ visualization of live US images. For tracking, no additional hardware nor modifications to HoloLens 2 are required making it a cheap and easy-to-use approach. Moreover, a minimal latency of displayed images enables a real-time perception for the sonographer.

Augmented Reality in Orthopedic Surgery and Its Application in Total Joint Arthroplasty: A Systematic Review

The development of augmented reality (AR) and its application in total joint arthroplasty aims at improving the accuracy and precision in implant components’ positioning, hopefully leading to increased outcomes and survivorship. However, this field is far from being thoroughly explored. We therefore performed a systematic review of the literature in order to examine the application, the results, and the different AR systems available in TJA. A systematic review of the literature according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines was performed. A comprehensive search of PubMed, MEDLINE, EMBASE, and the Cochrane Database of Systematic Reviews was conducted for English articles on the application of augmented reality in total joint arthroplasty using various combinations of keywords since the inception of the database to 31 March 2022. Accuracy was intended as the mean error from the targeted positioning angle and compared as mean values and standard deviations. In all, 14 articles met the inclusion criteria. Among them, four studies reported on the application of AR in total knee arthroplasty, six studies on total hip arthroplasty, three studies reported on reverse shoulder arthroplasty, and one study on total elbow arthroplasty. Nine of the included studies were preclinical (sawbones or cadaveric), while five of them reported results of AR’s clinical application. The main common feature was the high accuracy and precision when implant positioning was compared with preoperative targeted angles with errors ≤2 mm and/or ≤2°. Despite the promising results in terms of increased accuracy and precision, this technology is far from being widely adopted in daily clinical practice. However, the recent exponential growth in machine learning techniques and technologies may eventually lead to the resolution of the ongoing limitations including depth perception and their high complexity, favorably encouraging the widespread usage of AR systems.

Augmented Reality in Arthroplasty: An Overview of Clinical Applications, Benefits, and Limitations

Augmented reality (AR) is a natural extension of computer-assisted surgery whereby a computer-generated image is superimposed on the surgeon's field of vision to assist in the planning and execution of the procedure. This emerging technology shows great potential in the field of arthroplasty, improving efficiency, limb alignment, and implant position. AR has shown the capacity to build on computer navigation systems while providing more elaborate information in a streamlined workflow to the user. This review investigates the current uses of AR in the field of arthroplasty and discusses outcomes, limitations, and potential future directions.

Review and Future/Potential Application of Mixed Reality Technology in Orthopaedic Oncology

In orthopaedic oncology, surgical planning and intraoperative execution errors may result in positive tumor resection margins that increase the risk of local recurrence and adversely affect patients’ survival. Computer navigation and 3D-printed resection guides have been reported to address surgical inaccuracy by replicating the surgical plans in complex cases. However, limitations include surgeons’ attention shift from the operative field to view the navigation monitor and expensive navigation facilities in computer navigation surgery. Practical concerns are lacking real-time visual feedback of preoperative images and the lead-time in manufacturing 3D-printed objects. Mixed Reality (MR) is a technology of merging real and virtual worlds to produce new environments with enhanced visualizations, where physical and digital objects coexist and allow users to interact with both in real-time. The unique MR features of enhanced medical images visualization and interaction with holograms allow surgeons real-time and on-demand medical information and remote assistance in their immediate working environment. Early application of MR technology has been reported in surgical procedures. Its role is unclear in orthopaedic oncology. This review aims to provide orthopaedic tumor surgeons with up-to-date knowledge of the emerging MR technology. The paper presents its essential features and clinical workflow, reviews the current literature and potential clinical applications, and discusses the limitations and future development in orthopaedic oncology. The emerging MR technology adds a new dimension to digital assistive tools with a more accessible and less costly alternative in orthopaedic oncology. The MR head-mounted display and hand-free control may achieve clinical point-of-care inside or outside the operating room and improve service efficiency and patient safety. However, lacking an accurate hologram-to-patient matching, an MR platform dedicated to orthopaedic oncology, and clinical results may hinder its wide adoption. Industry-academic partnerships are essential to advance the technology with its clinical role determined through future clinical studies.

Point your SmartPhone at the code above. If you have a QR code reader the video abstract will appear. Or use:

https://youtu.be/t4hl_Anh_kM

Augmented Reality: Mapping Methods and Tools for Enhancing the Human Role in Healthcare HMI

Background: Augmented Reality (AR) represents an innovative technology to improve data visualization and strengthen the human perception. Among Human–Machine Interaction (HMI), medicine can benefit most from the adoption of these digital technologies. In this perspective, the literature on orthopedic surgery techniques based on AR was evaluated, focusing on identifying the limitations and challenges of AR-based healthcare applications, to support the research and the development of further studies. Methods: Studies published from January 2018 to December 2021 were analyzed after a comprehensive search on PubMed, Google Scholar, Scopus, IEEE Xplore, Science Direct, and Wiley Online Library databases. In order to improve the review reporting, the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines were used. Results: Authors selected sixty-two articles meeting the inclusion criteria, which were categorized according to the purpose of the study (intraoperative, training, rehabilitation) and according to the surgical procedure used. Conclusions: AR has the potential to improve orthopedic training and practice by providing an increasingly human-centered clinical approach. Further research can be addressed by this review to cover problems related to hardware limitations, lack of accurate registration and tracking systems, and absence of security protocols.

Glenoid Component Placement Assisted by Augmented Reality Through a Head-Mounted Display During Reverse Shoulder Arthroplasty

Component positioning is a key factor for avoiding complications and improving functional outcomes in reverse shoulder arthroplasty. Preoperative planning can improve component positioning. However, translating the preoperative plan into the surgical procedure can be challenging. This is particularly the case for the glenoid component positioning in severe deformity or limited visualization of the scapula. Different computational-assisted techniques have been developed to aid implementation of the preoperative plan into the surgical procedure. Navigated augmented reality (AR) refers to the real world augmented with virtual real-time information about the position and orientation of instruments and components. This information can be presented through a head-mounted display (HMD), which enables the user to visualize the virtual information directly overlaid onto the real world. Navigated AR systems through HMD have been validated for shoulder arthroplasty using phantoms and cadavers. This article details a step-by-step guide use of a navigated AR system through HMD, in the placement of the glenoid bony-augmented component.

Optimal insertion site of glenoid baseplate in reverse total shoulder arthroplasty: anatomical simulation using three dimensional image processing software

PURPOSE

Conventionally, the central structure of the baseplate is inserted through the point where the vertical and horizontal axes of the glenoid intersect (conventional insertion site (CIS)). However, there is scanty theoretical evidence that CIS has the optimal bone stock. We evaluated the optimal insertion site for the glenoid baseplate through the three-dimensional volumetric measurement of the glenoid bone stock.

METHODS

Pre-operative computed tomography (CT) images of 30 consecutive reverse total shoulder arthroplasty procedures were analyzed. Three-dimensional image processing software was used to reconstruct CT and volumetrically measure the glenoid bone stock according to the simulated central peg. A simulated central peg was inserted to the medial pole of the scapula from 49 points determined along with the intersect point of the vertical and horizontal axes of the glenoid CIS at 2-mm intervals. The overlapped volume between the simulated central peg and glenoid vault, representing the amount of glenoid bone stock along the passage of the central peg, was then automatically calculated.

RESULTS

The depth of the glenoid vault was 25.5 ± 3.0 mm (range, 19.3-31.5), and the mean overlapped volume between the simulated central peg and the glenoid vault was 623.0 ± 185.8 ml. The optimal insertion site for the bony purchase of the central peg was 2 mm inferior and posterior from the CIS (765.3 ± 157.5).

CONCLUSION

The optimal insertion site of the baseplate is located slightly inferiorly and posteriorly to the CIS. This anatomical information may be used as a reference to determine the optimal insertion site of the baseplate according to an implant of a surgeon's choice.

Augmented-Reality-Assisted K-Wire Placement for Glenoid Component Positioning in Reversed Shoulder Arthroplasty: A Proof-of-Concept Study

The accuracy of the implant's post-operative position and orientation in reverse shoulder arthroplasty is known to play a significant role in both clinical and functional outcomes. Whilst technologies such as navigation and robotics have demonstrated superior radiological outcomes in many fields of surgery, the impact of augmented reality (AR) assistance in the operating room is still unknown. Malposition of the glenoid component in shoulder arthroplasty is known to result in implant failure and early revision surgery. The use of AR has many promising advantages, including allowing the detailed study of patient-specific anatomy without the need for invasive procedures such as arthroscopy to interrogate the joint's articular surface. In addition, this technology has the potential to assist surgeons intraoperatively in aiding the guidance of surgical tools. It offers the prospect of increased component placement accuracy, reduced surgical procedure time, and improved radiological and functional outcomes, without recourse to the use of large navigation or robotic instruments, with their associated high overhead costs. This feasibility study describes the surgical workflow from a standardised CT protocol, via 3D reconstruction, 3D planning, and use of a commercial AR headset, to AR-assisted k-wire placement. Post-operative outcome was measured using a high-resolution laser scanner on the patient-specific 3D printed bone. In this proof-of-concept study, the discrepancy between the planned and the achieved glenoid entry point and guide-wire orientation was approximately 3 mm with a mean angulation error of 5°.

Augmented Reality, Virtual Reality and Artificial Intelligence in Orthopedic Surgery: A Systematic Review

Background: The application of virtual and augmented reality technologies to orthopaedic surgery training and practice aims to increase the safety and accuracy of procedures and reducing complications and costs. The purpose of this systematic review is to summarise the present literature on this topic while providing a detailed analysis of current flaws and benefits. Methods: A comprehensive search on the PubMed, Cochrane, CINAHL, and Embase database was conducted from inception to February 2021. The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines were used to improve the reporting of the review. The Cochrane Risk of Bias Tool and the Methodological Index for Non-Randomized Studies (MINORS) was used to assess the quality and potential bias of the included randomized and non-randomized control trials, respectively. Results: Virtual reality has been proven revolutionary for both resident training and preoperative planning. Thanks to augmented reality, orthopaedic surgeons could carry out procedures faster and more accurately, improving overall safety. Artificial intelligence (AI) is a promising technology with limitless potential, but, nowadays, its use in orthopaedic surgery is limited to preoperative diagnosis. Conclusions: Extended reality technologies have the potential to reform orthopaedic training and practice, providing an opportunity for unidirectional growth towards a patient-centred approach.