Clinical Accuracy, Technical Precision, and Workflow of the First in Human Use of an Augmented-Reality Head-Mounted Display Stereotactic Navigation System for Spine Surgery

A decade of progress: bringing mixed reality image-guided surgery systems in the operating room

Advancements in mixed reality (MR) have led to innovative approaches in image-guided surgery (IGS). In this paper, we provide a comprehensive analysis of the current state of MR in image-guided procedures across various surgical domains. Using the Data Visualization View (DVV) Taxonomy, we analyze the progress made since a 2013 literature review paper on MR IGS systems. In addition to examining the current surgical domains using MR systems, we explore trends in types of MR hardware used, type of data visualized, visualizations of virtual elements, and interaction methods in use. Our analysis also covers the metrics used to evaluate these systems in the operating room (OR), both qualitative and quantitative assessments, and clinical studies that have demonstrated the potential of MR technologies to enhance surgical workflows and outcomes. We also address current challenges and future directions that would further establish the use of MR in IGS.

Integrating Augmented Reality in Spine Surgery: Redefining Precision with New Technologies

INTRODUCTION

The integration of augmented reality (AR) in spine surgery marks a significant advancement, enhancing surgical precision and patient outcomes. AR provides immersive, three-dimensional visualizations of anatomical structures, facilitating meticulous planning and execution of spine surgeries. This technology not only improves spatial understanding and real-time navigation during procedures but also aims to reduce surgical invasiveness and operative times. Despite its potential, challenges such as model accuracy, user interface design, and the learning curve for new technology must be addressed. AR's application extends beyond the operating room, offering valuable tools for medical education and improving patient communication and satisfaction.

MATERIAL AND METHODS

A literature review was conducted by searching PubMed and Scopus databases using keywords related to augmented reality in spine surgery, covering publications from January 2020 to January 2024.

RESULTS

In total, 319 articles were identified through the initial search of the databases. After screening titles and abstracts, 11 articles in total were included in the qualitative synthesis.

CONCLUSION

Augmented reality (AR) is becoming a transformative force in spine surgery, enhancing precision, education, and outcomes despite hurdles like technical limitations and integration challenges. AR's immersive visualizations and educational innovations, coupled with its potential synergy with AI and machine learning, indicate a bright future for surgical care. Despite the existing obstacles, AR's impact on improving surgical accuracy and safety marks a significant leap forward in patient treatment and care.

Feasibility of a novel augmented reality overlay for cervical screw placement in phantom spine models

STUDY DESIGN

Feasibility study.

PURPOSE

A phantom model was used to evaluate the accuracy of a novel augmented reality (AR) system for cervical screw placement.

OVERVIEW OF LITERATURE

The use of navigation systems is becoming increasingly common in spine procedures. However, numerous factors limit the feasibility of regular and widespread use of navigation tools during spine surgery. AR is a new technology that has already demonstrated utility as a navigation tool during spine surgery. However, advancements in AR technology are needed to increase its adoption by the medical community.

METHODS

AR technology that uses a fiducial-less registration system was tested in a preclinical cervical spine phantom model study for accuracy during spinal screw placement. A three-dimensional reconstruction of the spine along with trajectory lines was superimposed onto the phantom model using an AR headset. Participants used the AR system to guide screw placement, and post-instrumentation scans were compared for accuracy assessment.

RESULTS

Twelve cervical screws were placed under AR guidance. All screws were placed in an acceptable anatomic position. The average distance error for the insertion point was 2.73±0.55 mm, whereas that for the endpoint was 2.71±0.69 mm. The average trajectory angle error for all insertions was 2.69°±0.59°.

CONCLUSIONS

This feasibility study describes a novel registration approach that superimposes spinal anatomy and trajectories onto the surgeon's real-world view of the spine. These results demonstrate reasonable accuracy in the preclinical model. The results of this study demonstrate that this technology can assist with accurate screw placement. Further investigation using cadaveric and clinical models is warranted.

Beyond Traditional Training: Exploring the Benefits of Virtual Reality Simulator in Lumbar Pedicle Screw Insertion - A Randomized Controlled Trial

INTRODUCTION

This study compares the effectiveness of virtual reality simulators (VRS) and a saw bone model for learning lumbar pedicle screw insertion (LPSI) in neurosurgery.

METHODS

A single-center, cross-sectional, randomized controlled laboratory investigation was conducted involving residents and fellows from a tertiary care referral hospital. Participants were divided into two groups (A and B). Group A performed 3 LPSI tasks: the first on a saw bone model, the second on VRS, and the third on another saw bone model. Group B completed 2 LPSI tasks: the first on a saw bone model and the second on another saw bone model. The accuracy of LPSI was evaluated through noncontrast computed tomography scans for the saw bone models, while the in-built application of VRS was utilized to check for accuracy of screw placement using the simulator.

RESULTS

The study included 38 participants (19 in each group). Group A participants showed reduced mean entry point error (0.11 mm, P 0.024), increased mean purchase length (4.66 cm, P 0.007), and no cortical breaches (P 0.031) when placing the second saw bone model screw. Similar improvements were observed among group A participants in PGY 1-3 while placing the second saw bone model screws.

CONCLUSIONS

Virtual reality simulators (VRS) prove to be an invaluable tool for teaching complex neurosurgical skills, such as LPSI, to trainees. This technology investment can enhance the learning curve while maintaining patient safety.

Development of Augmented Reality Vision for Osteosynthesis Using a 3D Camera

BACKGROUND

We developed a 3D camera system to track motion in a surgical field. This system has the potential to introduce augmented reality (AR) systems non-invasively, eliminating the need for the invasive AR markers conventionally required. The present study was performed to verify the real-time tracking accuracy of this system, assess the feasibility of integrating this system into the surgical workflow, and establish its potential to enhance the accuracy and efficiency of orthopedic procedures.

METHODS

To evaluate the accuracy of AR technology using a 3D camera, a forearm bone model was created. The forearm model was depicted using a 3D camera, and its accuracy was verified in terms of the positional relationship with a 3D bone model created from previously imaged CT data. Images of the surgical field (capturing the actual forearm) were taken and saved in nine poses by rotating the forearm from pronation to supination. The alignment of the reference points was computed at the three points of CT versus the three points of the 3D camera, yielding a 3D rotation matrix representing the positional relationship. In the original system, a stereo vision-based 3D camera, with a depth image resolution of 1280×720 pixels, 30 frames per second, and a lens field of view of 64 specifications, with a baseline of 3 cm, capable of optimally acquiring real-time 3D data at a distance of 40-60 cm from the subject was used. In the modified system, the following modifications were made to improve tracking performance: (1) color filter processing was changed from HSV to RGB, (2) positional detection accuracy was modified with supporting marker sizes of 8 mm in diameter, and (3) the detection of marker positions was stabilized by calculating the marker position for each frame. Tracking accuracy was examined with the original system and modified system for the following parameters: differences in the rotation matrix, maximum and minimum inter-reference point errors between CT-based and camera-based 3D data, and the average error for the three reference points.

RESULTS

In the original system, the average difference in rotation matrices was 5.51±2.68 mm. Average minimum and maximum errors were 1.10±0.61 and 15.53±12.51 mm, respectively. The average error of reference points was 6.26±4.49 mm. In the modified system, the average difference in rotation matrices was 4.22±1.73 mm. Average minimum and maximum errors were 0.79±0.49 and 1.94±0.87 mm, respectively. The average error of reference points was 1.41±0.58 mm. In the original system, once tracking failed, it was difficult to recover tracking accuracy. This resulted in a large maximum error in supination positions. These issues were resolved by the modified system. Significant improvements were achieved in maximum errors and average errors using the modified system (P<0.05).

CONCLUSION

AR technology using a 3D camera was developed. This system allows direct comparisons of 3D data from preoperative CT scans with 3D data acquired from the surgical field using a 3D camera. This method has the advantage of introducing AR into the surgical field without invasive markers.

Multidisciplinary Surgical Approach Using Augmented Reality Preplanning for Resection of Giant Thoracic Schwannoma With Robotic-Assisted Thoracoscopic Mobilization

BACKGROUND AND IMPORTANCE

In adults, primary spinal cord tumors account for 5% of all primary tumors of the central nervous system, with schwannomas making up about 74% of all nerve sheath tumors. Thoracic schwannomas can pose a threat to neurovasculature, presenting a significant challenge to safe and complete surgical resection. For patients presenting with complex pathologies including tumors, a dual surgeon approach may be used to optimize patient care and improve outcomes.

CLINICAL PRESENTATION

A 73-year-old female previously diagnosed with a nerve sheath tumor of the fourth thoracic vertebra presented with significant thoracic pain and a history of falls. Imaging showed that the tumor had doubled in size ranging from T3 to T5. Augmented reality volumetric rendering was used to clarify anatomic relationships of the mass for perioperative evaluation and decision-making. A dual surgeon approach was used for complete resection. First, a ventrolateral left video-assisted thoracoscopic surgery was performed with robotic assistance followed by a posterior tumor resection and thoracic restabilization. The patient did well postoperatively.

CONCLUSION

Although surgical treatment of large thoracic dumbbell tumors presents a myriad of risks, perioperative evaluation with augmented reality, new robotic surgical techniques, and a dual surgeon approach can be implemented to mitigate these risks.

Virtual, Augmented, and Mixed Reality Applications for Surgical Rehearsal, Operative Execution, and Patient Education in Spine Surgery: A Scoping Review

Background and Objectives: Advances in virtual reality (VR), augmented reality (AR), and mixed reality (MR) technologies have resulted in their increased application across many medical specialties. VR's main application has been for teaching and preparatory roles, while AR has been mostly used as a surgical adjunct. The objective of this study is to discuss the various applications and prospects for VR, AR, and MR specifically as they relate to spine surgery. Materials and Methods: A systematic review was conducted to examine the current applications of VR, AR, and MR with a focus on spine surgery. A literature search of two electronic databases (PubMed and Scopus) was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). The study quality was assessed using the MERSQI score for educational research studies, QUACS for cadaveric studies, and the JBI critical appraisal tools for clinical studies. Results: A total of 228 articles were identified in the primary literature review. Following title/abstract screening and full-text review, 46 articles were included in the review. These articles comprised nine studies performed in artificial models, nine cadaveric studies, four clinical case studies, nineteen clinical case series, one clinical case-control study, and four clinical parallel control studies. Teaching applications utilizing holographic overlays are the most intensively studied aspect of AR/VR; the most simulated surgical procedure is pedicle screw placement. Conclusions: VR provides a reproducible and robust medium for surgical training through surgical simulations and for patient education through various platforms. Existing AR/MR platforms enhance the accuracy and precision of spine surgeries and show promise as a surgical adjunct.

Utilization of Augmented Reality Head-Mounted Display for the Surgical Management of Thoracolumbar Spinal Trauma

Background and Objectives: Augmented reality head-mounted display (AR-HMD) is a novel technology that provides surgeons with a real-time CT-guided 3-dimensional recapitulation of a patient's spinal anatomy. In this case series, we explore the use of AR-HMD alongside more traditional robotic assistance in surgical spine trauma cases to determine their effect on operative costs and perioperative outcomes. Materials and Methods: We retrospectively reviewed trauma patients who underwent pedicle screw placement surgery guided by AR-HMD or robotic-assisted platforms at an academic tertiary care center between 1 January 2021 and 31 December 2022. Outcome distributions were compared using the Mann-Whitney U test. Results: The AR cohort (n = 9) had a mean age of 66 years, BMI of 29.4 kg/m2, Charlson Comorbidity Index (CCI) of 4.1, and Surgical Invasiveness Index (SII) of 8.8. In total, 77 pedicle screws were placed in this cohort. Intra-operatively, there was a mean blood loss of 378 mL, 0.78 units transfused, 398 min spent in the operating room, and a 20-day LOS. The robotic cohort (n = 13) had a mean age of 56 years, BMI of 27.1 kg/m2, CCI of 3.8, and SII of 14.2. In total, 128 pedicle screws were placed in this cohort. Intra-operatively, there was a mean blood loss of 432 mL, 0.46 units transfused units used, 331 min spent in the operating room, and a 10.4-day LOS. No significant difference was found between the two cohorts in any outcome metrics. Conclusions: Although the need to address urgent spinal conditions poses a significant challenge to the implementation of innovative technologies in spine surgery, this study represents an initial effort to show that AR-HMD can yield comparable outcomes to traditional robotic surgical techniques. Moreover, it highlights the potential for AR-HMD to be readily integrated into Level 1 trauma centers without requiring extensive modifications or adjustments.

The application of extended reality technology-assisted intraoperative navigation in orthopedic surgery

Extended reality (XR) technology refers to any situation where real-world objects are enhanced with computer technology, including virtual reality, augmented reality, and mixed reality. Augmented reality and mixed reality technologies have been widely applied in orthopedic clinical practice, including in teaching, preoperative planning, intraoperative navigation, and surgical outcome evaluation. The primary goal of this narrative review is to summarize the effectiveness and superiority of XR-technology-assisted intraoperative navigation in the fields of trauma, joint, spine, and bone tumor surgery, as well as to discuss the current shortcomings in intraoperative navigation applications. We reviewed titles of more than 200 studies obtained from PubMed with the following search terms: extended reality, mixed reality, augmented reality, virtual reality, intraoperative navigation, and orthopedic surgery; of those 200 studies, 69 related papers were selected for abstract review. Finally, the full text of 55 studies was analyzed and reviewed. They were classified into four groups-trauma, joint, spine, and bone tumor surgery-according to their content. Most of studies that we reviewed showed that XR-technology-assisted intraoperative navigation can effectively improve the accuracy of implant placement, such as that of screws and prostheses, reduce postoperative complications caused by inaccurate implantation, facilitate the achievement of tumor-free surgical margins, shorten the surgical duration, reduce radiation exposure for patients and surgeons, minimize further damage caused by the need for visual exposure during surgery, and provide richer and more efficient intraoperative communication, thereby facilitating academic exchange, medical assistance, and the implementation of remote healthcare.

Augmented Reality Registration System for Visualization of Skull Landmarks

BACKGROUND

Augmented reality (AR) is an emerging technology in neurosurgery with the potential to become a strategic tool in the delivery of care and education for trainees. Advances in technology have demonstrated promising use for improving visualization and spatial awareness of critical neuroanatomic structures. In this report, we employ a novel AR registration system for the visualization and targeting of skull landmarks.

METHODS

A markerless AR system was used to register 3-dimensional reconstructions of suture lines onto the head via a head-mounted display. Participants were required to identify craniometric points with and without AR assistance. Targeting error was measured as the Euclidian distance between the user-defined location and the true craniometric point on the subjects' heads.

RESULTS

All participants successfully registered 3-dimensional reconstructions onto the subjects' heads. Targeting accuracy was significantly improved with AR (3.59 ± 1.29 mm). Across all target points, AR increased accuracy by an average of 19.96 ± 3.80 mm. Posttest surveys revealed that participants felt the technology increased their confidence in identifying landmarks (4.6/5) and that the technology will be useful for clinical care (4.2/5).

CONCLUSIONS

While several areas of improvement and innovation can further enhance the use of AR in neurosurgery, this report demonstrates the feasibility of a markerless headset-based AR system for visualizing craniometric points on the skull. As the technology continues to advance, AR is expected to play an increasingly significant role in neurosurgery, transforming how surgeries are performed and improving patient care.

Augmented reality in spine surgery - past, present, and future

BACKGROUND CONTEXT

Augmented reality (AR) is increasingly recognized as a valuable tool in spine surgery. Here we provides an overview of the key developments and technological milestones that have laid the foundation for AR applications in this field. We also assess the quality of existing studies on AR systems in spine surgery and explore potential future applications.

PURPOSE

The purpose of this narrative review is to examine the role of AR in spine surgery. It aims to highlight the evolution of AR technology in this context, evaluate the existing body of research, and outline potential future directions for integrating AR into spine surgery.

STUDY DESIGN

Narrative review.

METHODS

We conducted a thorough literature search to identify studies and developments related to AR in spine surgery. Relevant articles, reports, and technological advancements were analyzed to establish the historical context and current state of AR in this field.

RESULTS

The review identifies significant milestones in the development of AR technology for spine surgery. It discusses the growing body of research and highlights the strengths and weaknesses of existing investigations. Additionally, it presents insights into the potential for AR to enhance spine surgical education and speculates on future applications.

CONCLUSIONS

Augmented reality has emerged as a promising adjunct in spine surgery, with notable advancements and research efforts. The integration of AR into the spine surgery operating room holds promise, as does its potential to revolutionize surgical education. Future applications of AR in spine surgery may include real-time navigation, enhanced visualization, and improved patient outcomes. Continued development and evaluation of AR technology are essential for its successful implementation in this specialized surgical field.

Factors affecting augmented reality head-mounted device performance in real OR

PURPOSE

Over the last years, interest and efforts to implement augmented reality (AR) in orthopedic surgery through head-mounted devices (HMD) have increased. However, the majority of experiments were preclinical and within a controlled laboratory environment. The operating room (OR) is a more challenging environment with various confounding factors potentially affecting the performance of an AR-HMD. The aim of this study was to assess the performance of an AR-HMD in a real-life OR setting.

METHODS

An established AR application using the HoloLens 2 HMD was tested in an OR and in a laboratory by two users. The accuracy of the hologram overlay, the time to complete the trial, the number of rejected registration attempts, the delay in live overlay of the hologram, and the number of completely failed runs were recorded. Further, different OR setting parameters (light condition, setting up partitions, movement of personnel, and anchor placement) were modified and compared.

RESULTS

Time for full registration was higher with 48 s (IQR 24 s) in the OR versus 33 s (IQR 10 s) in the laboratory setting (p < 0.001). The other investigated parameters didn't differ significantly if an optimal OR setting was used. Within the OR, the strongest influence on performance of the AR-HMD was different light conditions with direct light illumination on the situs being the least favorable.

CONCLUSION

AR-HMDs are affected by different OR setups. Standardization measures for better AR-HMD performance include avoiding direct light illumination on the situs, setting up partitions, and minimizing the movement of personnel.

Applications of Augmented Reality in Orthopaedic Spine Surgery

The application of augmented reality (AR) in surgical settings has primarily been as a navigation tool in the operating room because of its ease of use and minimal effect on surgical procedures. The surgeon can directly face the surgical field while viewing 3D anatomy virtually, thus reducing the need to look at an external display, such as a navigation system. Applications of AR are being explored in spine surgery. The basic principles of AR include data preparation, registration, tracking, and visualization. Current literature provides sufficient preclinical and clinical data evidence for the use of AR technology in spine surgery. AR systems are efficient assistive devices, providing greater accuracy for insertion points, more comfort for surgeons, and reduced operating time. AR technology also has beneficial applications in surgical training, education, and telementorship for spine surgery. However, costs associated with specially designed imaging equipment and physicians' comfort in using this technology continue to remain barriers to its adoption. As this technology evolves to a more widespread use, future applications will be directed by the cost-effectiveness of AR-assisted surgeries.

Augmented Reality in Minimally Invasive Spinal Surgery: A Narrative Review of Available Technology

INTRODUCTION

Spine surgery has undergone significant changes in approach and technique. With the adoption of intraoperative navigation, minimally invasive spinal surgery (MISS) has arguably become the gold standard. Augmented reality (AR) has now emerged as a front-runner in anatomical visualization and narrower operative corridors. In effect, AR is poised to revolutionize surgical training and operative outcomes. Our study examines the current literature on AR-assisted MISS, synthesizes findings, and creates a narrative highlighting the history and future of AR in spine surgery.

MATERIAL AND METHODS

Relevant literature was gathered using the PubMed (Medline) database from 1975 to 2023. Pedicle screw placement models were the primary intervention in AR. These were compared to the outcomes of traditional MISS RESULTS: We found that AR devices on the market show promising clinical outcomes in preoperative training and intraoperative use. Three prominent systems were as follows: XVision, HoloLens, and ImmersiveTouch. In the studies, surgeons, residents, and medical students had opportunities to operate AR systems, showcasing their educational potential across each phase of learning. Specifically, one facet described training with cadaver models to gauge accuracy in pedicle screw placement. AR-MISS exceeded free-hand methods without unique complications or contraindications.

CONCLUSIONS

While still in its infancy, AR has already proven beneficial for educational training and intraoperative MISS applications. We believe that with continued research and advancement of this technology, AR is poised to become a dominant player within the fundamentals of surgical education and MISS operative technique.

A Pilot Human Cadaveric Study on Accuracy of the Augmented Reality Surgical Navigation System for Thoracolumbar Pedicle Screw Insertion Using a New Intraoperative Rapid Registration Method

To evaluate the feasibility and accuracy of AR-assisted pedicle screw placement using a new intraoperative rapid registration method of combining preoperative CT scanning and intraoperative C-arm 2D fluoroscopy in cadavers. Five cadavers with intact thoracolumbar spines were employed in this study. Intraoperative registration was performed using anteroposterior and lateral views of preoperative CT scanning and intraoperative 2D fluoroscopic images. Patient-specific targeting guides were used for pedicle screw placement from Th1-L5, totaling 166 screws. Instrumentation for each side was randomized (augmented reality surgical navigation (ARSN) vs. C-arm) with an equal distribution of 83 screws in each group. CT was performed to evaluate the accuracy of both techniques by assessing the screw positions and the deviations between the inserted screws and planned trajectories. Postoperative CT showed that 98.80% (82/83) screws in ARSN group and 72.29% (60/83) screws in C-arm group were within the 2-mm safe zone (p < 0.001). The mean time for instrumentation per level in ARSN group was significantly shorter than that in C-arm group (56.17 ± 3.33 s vs. 99.22 ± 9.03 s, p < 0.001). The overall intraoperative registration time was 17.2 ± 3.5 s per segment. AR-based navigation technology can provide surgeons with accurate guidance of pedicle screw insertion and save the operation time by using the intraoperative rapid registration method of combining preoperative CT scanning and intraoperative C-arm 2D fluoroscopy.

Virtual and Augmented Reality in Spine Surgery: A Systematic Review

BACKGROUND

Augmented reality (AR) and virtual reality (VR) implementation in spinal surgery has expanded rapidly over the past decade. This systematic review summarizes the use of AR/VR technology in surgical education, preoperative planning, and intraoperative guidance.

METHODS

A search query for AR/VR technology in spine surgery was conducted through PubMed, Embase, and Scopus. After exclusions, 48 studies were included. Included studies were then grouped into relevant subsections. Categorization into subsections yielded 12 surgical training studies, 5 preoperative planning, 24 intraoperative usage, and 10 radiation exposure.

RESULTS

VR-assisted training significantly reduced penetration rates or increased accuracy rates compared to lecture-based groups in 5 studies. Preoperative VR planning significantly influenced surgical recommendations and reduced radiation exposure, operating time, and estimated blood loss. For 3 patient studies, AR-assisted pedicle screw placement accuracy ranged from 95.77% to 100% using the Gertzbein grading scale. Head-mounted display was the most common interface used intraoperatively followed by AR microscope and projector. AR/VR also had applications in tumor resection, vertebroplasty, bone biopsy, and rod bending. Four studies reported significantly reduced radiation exposure in AR group compared to fluoroscopy group.

CONCLUSIONS

AR/VR technologies have the potential to usher in a paradigm shift in spine surgery. However, the current evidence indicates there is still a need for 1) defined quality and technical requirements for AR/VR devices, 2) more intraoperative studies that explore usage outside of pedicle screw placement, and 3) technological advancements to overcome registration errors via the development of an automatic registration method.

How to Avoid Errors When Using Navigation to Place Implants - A Narrative Review

PURPOSE

Surgeons placing implants use navigation for implant placement accuracy. The importance of this review is to document the sources of error that are involved with navigation so surgeons can recognize factors to decrease error. The objective is to provide surgeons with a reference to optimize navigation.

METHODS

Pubmed.gov was the information source. Years reviewed included 2010 to 2022. The inclusion criteria included only articles in peer-reviewed journals. In vitro results were included only if they involved testing of variables microgap, cone beam computerized tomography (CBCT) accuracy evaluation, or accuracy of printed models. Variables were searched and evaluated. Data collected included the objectives and outcomes of the study including statistical significance. The conclusions made by the authors were confirmed by evaluating the data analysis, and then these conclusions were listed in each error-related topic.

RESULTS

The search used terms which included guided implant surgery complications (n = 4,029), accuracy of CBCT scanners (n = 319), accuracy of implant navigation (n = 983), and the error between drills and static guides (n = 3). From this search, 70 articles were collated that satisfied the inclusion criteria. There are multiple sources of error that are less than 1 mm, including but not limited to errors associated with the scanner and method for scanning, errors associated with merging scanned files with the CBCT scan, errors using different guide stent fabrication methods, errors associated with intraoperative techniques, the learning curve, and planning error. If small errors are not taken into consideration, implant placement errors can exceed 1-2 mm of platform location and angulation errors in excess of 8°.

CONCLUSION

The surgeon needs to take into consideration controllable factors that will result in the avoidance of implant malposition and thus be able to effectively utilize navigation for accurate implant placement.

Clinical applications of augmented reality in orthopaedic surgery: a comprehensive narrative review

PURPOSE

The development of augmented reality (AR) technology allows orthopaedic surgeons to incorporate and visualize surgical data, assisting the execution of both routine and complex surgical operations. Uniquely, AR technology allows a surgeon to view the surgical field and superimpose peri-operative imaging, anatomical landmarks, navigation guidance, and more, all in one view without the need for conjugate gaze between multiple screens. The aim of this literature review was to introduce the fundamental requirements for an augmented reality system and to assess the current applications, outcomes, and potential limitations to this technology.

METHODS

A literature search was performed using MEDLINE and Embase databases, by two independent reviewers, who then collaboratively synthesized and collated the results of the literature search into a narrative review focused on the applications of augmented reality in major orthopaedic sub-specialties.

RESULTS

Current technology requires that pre-operative patient data be acquired, and AR-compatible models constructed. Intra-operatively, to produce manipulatable virtual images into the user's view in real time, four major components are required including a camera, computer image processing technology, tracking tools, and an output screen. The user is provided with a heads-up display, which is a transparent display, enabling the user to look at both their natural view and the computer-generated images. Currently, high-quality evidence for clinical implementation of AR technology in the orthopaedic surgery operating room is lacking; however, growing in vitro literature highlights a multitude of potential applications, including increasing operative accuracy, improved biomechanical angular and alignment parameters, and potentially reduced operative time.

CONCLUSION

While the application of AR systems in surgery is currently in its infancy, we anticipate rapid and widespread implementation of this technology in various orthopaedic sub-specialties.

Augmented reality for minimally invasive spinal surgery

Background

Augmented reality (AR) is an emerging technology that can overlay computer graphics onto the real world and enhance visual feedback from information systems. Within the past several decades, innovations related to AR have been integrated into our daily lives; however, its application in medicine, specifically in minimally invasive spine surgery (MISS), may be most important to understand. AR navigation provides auditory and haptic feedback, which can further enhance surgeons' capabilities and improve safety.

Purpose

The purpose of this article is to address previous and current applications of AR, AR in MISS, limitations of today's technology, and future areas of innovation.

Methods

A literature review related to applications of AR technology in previous and current generations was conducted.

Results

AR systems have been implemented for treatments related to spinal surgeries in recent years, and AR may be an alternative to current approaches such as traditional navigation, robotically assisted navigation, fluoroscopic guidance, and free hand. As AR is capable of projecting patient anatomy directly on the surgical field, it can eliminate concern for surgeon attention shift from the surgical field to navigated remote screens, line-of-sight interruption, and cumulative radiation exposure as the demand for MISS increases.

Conclusion

AR is a novel technology that can improve spinal surgery, and limitations will likely have a great impact on future technology.

Mixed Reality in Modern Surgical and Interventional Practice: Narrative Review of the Literature

BACKGROUND

Mixed reality (MR) and its potential applications have gained increasing interest within the medical community over the recent years. The ability to integrate virtual objects into a real-world environment within a single video-see-through display is a topic that sparks imagination. Given these characteristics, MR could facilitate preoperative and preinterventional planning, provide intraoperative and intrainterventional guidance, and aid in education and training, thereby improving the skills and merits of surgeons and residents alike.

OBJECTIVE

In this narrative review, we provide a broad overview of the different applications of MR within the entire spectrum of surgical and interventional practice and elucidate on potential future directions.

METHODS

A targeted literature search within the PubMed, Embase, and Cochrane databases was performed regarding the application of MR within surgical and interventional practice. Studies were included if they met the criteria for technological readiness level 5, and as such, had to be validated in a relevant environment.

RESULTS

A total of 57 studies were included and divided into studies regarding preoperative and interventional planning, intraoperative and interventional guidance, as well as training and education.

CONCLUSIONS

The overall experience with MR is positive. The main benefits of MR seem to be related to improved efficiency. Limitations primarily seem to be related to constraints associated with head-mounted display. Future directions should be aimed at improving head-mounted display technology as well as incorporation of MR within surgical microscopes, robots, and design of trials to prove superiority.

In-Human Report of S2 Alar-Iliac Screw Placement Using Augmented Reality Assistance

BACKGROUND

S2 alar-iliac (S2AI) screws provide spinopelvic fixation with the advantages of minimized dissection, easier rod contouring, and decreased symptomatic screw-head prominence. However, placement of S2AI screws may be challenging because of the anatomy of the lumbosacral junction. Augmented reality is a nascent technology that may enhance placement of S2AI screws.

OBJECTIVE

To report the first in-human placement of augmented reality (AR)-assisted S2 alar-iliac screws and evaluate the accuracy of screw placement.

METHODS

A retrospective review was performed of patients who underwent AR-assisted S2AI screw placement. All surgeries were performed by 2 neurosurgeons using an AR head-mounted display (Xvision, Augmedics). Screw accuracy was analyzed in a blinded fashion by an independent neuroradiologist using the cortical breach grading scale.

RESULTS

Twelve patients underwent AR-assisted S2AI screw placement for a total of 23 screws. Indications for surgery included deformity, degenerative disease, and tumor. Twenty-two screws (95.6%) were accurate-defined as grade 0 or grade 1. Twenty-one screws (91.3%) were classified as grade 0, 1 screw (4.3%) was grade 1, and 1 screw (4.3%) was grade 3. All breaches were asymptomatic.

CONCLUSION

AR-assisted S2AI screw placement had an overall accuracy rate of 95.6% (grade 0 and grade 1 screws) in a cohort of 12 patients and 23 screws. This compares favorably with freehand and robotic placement. 1,2 AR enables spine surgeons to both better visualize anatomy and accurately place spinal instrumentation. Future studies are warranted to research the learning curve and cost analysis of AR-assisted spine surgery.

Augmented and virtual reality usage in awake craniotomy: a systematic review

Augmented and virtual reality (AR, VR) are becoming promising tools in neurosurgery. AR and VR can reduce challenges associated with conventional approaches via the simulation and mimicry of specific environments of choice for surgeons. Awake craniotomy (AC) enables the resection of lesions from eloquent brain areas while monitoring higher cortical and subcortical functions. Evidence suggests that both surgeons and patients benefit from the various applications of AR and VR in AC. This paper investigates the application of AR and VR in AC and assesses its prospective utility in neurosurgery. A systematic review of the literature was performed using PubMed, Scopus, and Web of Science databases in accordance with the PRISMA guidelines. Our search results yielded 220 articles. A total of six articles consisting of 118 patients have been included in this review. VR was used in four papers, and the other two used AR. Tumour was the most common pathology in 108 patients, followed by vascular lesions in eight patients. VR was used for intraoperative mapping of language, vision, and social cognition, while AR was incorporated in preoperative training of white matter dissection and intraoperative visualisation and navigation. Overall, patients and surgeons were satisfied with the applications of AR and VR in their cases. AR and VR can be safely incorporated during AC to supplement, augment, or even replace conventional approaches in neurosurgery. Future investigations are required to assess the feasibility of AR and VR in various phases of AC.

Extended Reality-Assisted Surgery as a Surgical Training Tool: Pilot Study Presenting First HoloLens-Assisted Complex Penile Revision Surgery

BACKGROUND

Extended reality-assisted urologic surgery (XRAS) is a novel technology that superimposes a computer-generated image on the physician's field to integrate common elements of the surgical process in more advanced detail. An extended reality (XR) interface is generated using optical head-mounted display (OHMD) devices.

AIM

To present the first case of HoloLens-assisted complex penile revision surgery.

METHODS

We describe our pilot study of HoloLens-assisted penile revision surgery and present a thorough review of the literature regarding XRAS technology and innovative OHMD devices.

OUTCOMES

The ability of XRAS technology to superimpose a computer-generated image of the patient and integrate common elements of the surgical planning process with long-distance experts.

RESULTS

XRAS is a feasible technology for application in complex penile surgical planning processes.

CLINICAL TRANSLATION

XRAS and OHMD devices are novel technologies applicable to urological surgical training and planning.

STRENGTHS AND LIMITATIONS

Evidence suggests that the potential use of OHMD devices is safe and beneficial for surgeons. We intend to pioneer HoloLens technology in the surgical planning process of a malfunctioning penile implant due to herniation of the cylinder. This novel technology has not been used in prosthetic surgery, and current data about XRAS are limited.

CONCLUSION

OHMD devices are effective in the operative setting. Herein, we successfully demonstrated the integration of Microsoft HoloLens 2 into a penile surgical planning process for the first time. Further development and studies for this technology are necessary to better characterize the XRAS as a training and surgical planning tool. Quesada-Olarte J, Carrion RE, Fernandez-Crespo R, et al. Extended Reality-Assisted Surgery as a Surgical Training Tool: Pilot Study Presenting First HoloLens-Assisted Complex Penile Revision Surgery. J Sex Med 2022;19:1580-1586.

Augmented Reality in Surgery: A Scoping Review

Augmented reality (AR) is an innovative system that enhances the real world by superimposing virtual objects on reality. The aim of this study was to analyze the application of AR in medicine and which of its technical solutions are the most used. We carried out a scoping review of the articles published between 2019 and February 2022. The initial search yielded a total of 2649 articles. After applying filters, removing duplicates and screening, we included 34 articles in our analysis. The analysis of the articles highlighted that AR has been traditionally and mainly used in orthopedics in addition to maxillofacial surgery and oncology. Regarding the display application in AR, the Microsoft HoloLens Optical Viewer is the most used method. Moreover, for the tracking and registration phases, the marker-based method with a rigid registration remains the most used system. Overall, the results of this study suggested that AR is an innovative technology with numerous advantages, finding applications in several new surgery domains. Considering the available data, it is not possible to clearly identify all the fields of application and the best technologies regarding AR.

The Future of Minimally Invasive Spinal Surgery

Strong forces are pushing minimally invasive spinal surgery (MISS) to the forefront of spine care. Less-invasive surgical techniques have been enabled by a variety of technical advances. Despite the promise of MISS, however, several factors, including few training opportunities, perception of a steep learning curve, and high upfront costs, have limited the adoption of these techniques. The "6 T's" is a framework highlighting key factors that must be accounted for to ensure safe and effective MISS as techniques continually evolve. Further, technological advancement in endoscopy, robotics, and augmented/virtual reality is enhancing minimally invasive surgeries to make them even less invasive and safer for patients. The evolution of these new techniques and technologies is driving the future of MISS.

The XVS System During Open Spinal Fixation Procedures in Patients Requiring Pedicle Screw Placement in the Lumbosacral Spine

OBJECTIVE

This pilot study was undertaken to evaluate the safety, performance, and usability of the Xvision-Spine (XVS) System (Augmedics, Arlington Heights, IL) during open spinal fixation procedures in patients requiring pedicle screw placement in the lumbosacral spine.

METHODS

The XVS System is an augmented reality head-mounted display (HMD) based on a computer navigation system designed to assist surgeons in accurately placing pedicle screws. It uses an HMD-mounted tracking camera to provide optical tracking technology, and provides the surgeon a translucent direct near-eye display of the navigated surgical instrument's location relative to the computed tomographic image. We report the preliminary results of a prospective series of all consecutive patients who underwent augmented reality-assisted pedicle screw placement in the lumbosacral vertebrae at 3 institutions. Clinical accuracy for each pedicle screw was graded with Gertzbein-Robbins scores by 2 independent and blinded neuroradiologists.

RESULTS

The 19 study participants included 8 men and 11 women with a mean age of 59.13 ± 12.09 and 59.91 ± 12.89 years, respectively. Seventeen procedures were successfully completed via the XVS System. Two procedures were not completed due to technical issues with the system's intraoperative scanner. A total of 86 screws were inserted. The accuracy of the XVS System was 97.7%.

CONCLUSIONS

The XVS System's performance in accurate placement of pedicle screws in the lumbosacral vertebrae had an overall accuracy of 97.7%. These preliminary results were comparable to the accuracy of other manual computer-assisted navigation systems reported in the literature.

Review and Future/Potential Application of Mixed Reality Technology in Orthopaedic Oncology

In orthopaedic oncology, surgical planning and intraoperative execution errors may result in positive tumor resection margins that increase the risk of local recurrence and adversely affect patients’ survival. Computer navigation and 3D-printed resection guides have been reported to address surgical inaccuracy by replicating the surgical plans in complex cases. However, limitations include surgeons’ attention shift from the operative field to view the navigation monitor and expensive navigation facilities in computer navigation surgery. Practical concerns are lacking real-time visual feedback of preoperative images and the lead-time in manufacturing 3D-printed objects. Mixed Reality (MR) is a technology of merging real and virtual worlds to produce new environments with enhanced visualizations, where physical and digital objects coexist and allow users to interact with both in real-time. The unique MR features of enhanced medical images visualization and interaction with holograms allow surgeons real-time and on-demand medical information and remote assistance in their immediate working environment. Early application of MR technology has been reported in surgical procedures. Its role is unclear in orthopaedic oncology. This review aims to provide orthopaedic tumor surgeons with up-to-date knowledge of the emerging MR technology. The paper presents its essential features and clinical workflow, reviews the current literature and potential clinical applications, and discusses the limitations and future development in orthopaedic oncology. The emerging MR technology adds a new dimension to digital assistive tools with a more accessible and less costly alternative in orthopaedic oncology. The MR head-mounted display and hand-free control may achieve clinical point-of-care inside or outside the operating room and improve service efficiency and patient safety. However, lacking an accurate hologram-to-patient matching, an MR platform dedicated to orthopaedic oncology, and clinical results may hinder its wide adoption. Industry-academic partnerships are essential to advance the technology with its clinical role determined through future clinical studies.

Point your SmartPhone at the code above. If you have a QR code reader the video abstract will appear. Or use:

https://youtu.be/t4hl_Anh_kM

Spine Surgery Assisted by Augmented Reality: Where Have We Been?

This present systematic review examines spine surgery literature supporting augmented reality (AR) technology and summarizes its current status in spinal surgery technology. Database search strategies were retrieved from PubMed, Web of Science, Cochrane Library, Embase, from the earliest records to April 1, 2021. Our review briefly examines the history of AR, and enumerates different device application workflows in a variety of spinal surgeries. We also sort out the pros and cons of current mainstream AR devices and the latest updates. A total of 45 articles are included in our review. The most prevalent surgical applications included are the augmented reality surgical navigation system and head-mounted display. The most popular application of AR is pedicle screw instrumentation in spine surgery, and the primary responsible surgical levels are thoracic and lumbar. AR guidance systems show high potential value in practical clinical applications for the spine. The overall number of cases in AR-related studies is still rare compared to traditional surgical-assisted techniques. These lack long-term clinical efficacy and robust surgical-related statistical data. Changing healthcare laws as well as the increasing prevalence of spinal surgery are generating critical data that determines the value of AR technology.

Key Ergonomics Requirements and Possible Mechanical Solutions for Augmented Reality Head-Mounted Displays in Surgery

In the context of a European project, we identified over 150 requirements for the development of an augmented reality (AR) head-mounted display (HMD) specifically tailored to support highly challenging manual surgical procedures. The requirements were established by surgeons from different specialties and by industrial players working in the surgical field who had strong commitments to the exploitation of this technology. Some of these requirements were specific to the project, while others can be seen as key requirements for the implementation of an efficient and reliable AR headset to be used to support manual activities in the peripersonal space. The aim of this work is to describe these ergonomic requirements that impact the mechanical design of the HMDs, the possible innovative solutions to these requirements, and how these solutions have been used to implement the AR headset in surgical navigation. We also report the results of a preliminary qualitative evaluation of the AR headset by three surgeons.

Augmented Reality to Assist Skin Paddle Harvesting in Osteomyocutaneous Fibular Flap Reconstructive Surgery: A Pilot Evaluation on a 3D-Printed Leg Phantom

Background

Augmented Reality (AR) represents an evolution of navigation-assisted surgery, providing surgeons with a virtual aid contextually merged with the real surgical field. We recently reported a case series of AR-assisted fibular flap harvesting for mandibular reconstruction. However, the registration accuracy between the real and the virtual content needs to be systematically evaluated before widely promoting this tool in clinical practice. In this paper, after description of the AR based protocol implemented for both tablet and HoloLens 2 smart glasses, we evaluated in a first test session the achievable registration accuracy with the two display solutions, and in a second test session the success rate in executing the AR-guided skin paddle incision task on a 3D printed leg phantom.

Methods

From a real computed tomography dataset, 3D virtual models of a human leg, including fibula, arteries and skin with planned paddle profile for harvesting, were obtained. All virtual models were imported into Unity software to develop a marker-less AR application suitable to be used both via tablet and via HoloLens 2 headset. The registration accuracy for both solutions was verified on a 3D printed leg phantom obtained from the virtual models, by repeatedly applying the tracking function and computing pose deviations between the AR-projected virtual skin paddle profile and the real one transferred to the phantom via a CAD/CAM cutting guide. The success rate in completing the AR-guided task of skin paddle harvesting was evaluated using CAD/CAM templates positioned on the phantom model surface.

Results

On average, the marker-less AR protocol showed comparable registration errors (ranging within 1-5 mm) for tablet-based and HoloLens-based solution. Registration accuracy seems to be quite sensitive to ambient light conditions. We found a good success rate in completing the AR-guided task within an error margin of 4 mm (97% and 100% for tablet and HoloLens, respectively). All subjects reported greater usability and ergonomics for HoloLens 2 solution.

Conclusions

Results revealed that the proposed marker-less AR based protocol may guarantee a registration error within 1-5 mm for assisting skin paddle harvesting in the clinical setting. Optimal lightening conditions and further improvement of marker-less tracking technologies have the potential to increase the efficiency and precision of this AR-assisted reconstructive surgery.

Virtual, Augmented, and Alternate Reality in Medical Education: Socially Distanced but Fully Immersed

BACKGROUND

Advancements in technology continue to transform the landscape of medical education. The need for technology-enhanced distance learning has been further accelerated by the coronavirus disease (COVID-19) pandemic. The relatively recent emergence of virtual reality (VR), augmented reality (AR), and alternate reality has expanded the possible applications of simulation-based education (SBE) outside of the traditional simulation laboratory, making SBE accessible asynchronously and in geographically diverse locations.

OBJECTIVE

In this review, we will explore the evidence base for use of emerging technologies in SBE as well as the strengths and limitations of each modality in a variety of settings.

METHODS

PubMed was searched for peer-reviewed articles published between 1995 and 2021 that focused on VR in medical education. The search terms included medical education, VR, simulation, AR, and alternate reality. We also searched reference lists from selected articles to identify additional relevant studies.

RESULTS

VR simulations have been used successfully in resuscitation, communication, and bronchoscopy training. In contrast, AR has demonstrated utility in teaching anatomical correlates with the use of diagnostic imaging, such as point-of-care ultrasound. Alternate reality has been used as a tool for developing clinical reasoning skills, longitudinal patient panel management, and crisis resource management via multiplayer platforms.

CONCLUSION

Although each of these modalities has a variety of educational applications in health profession education, there are benefits and limitations to each that are important to recognize prior to the design and implementation of educational content, including differences in equipment requirements, cost, and scalability.

Augmented Reality Assisted Endoscopic Transforaminal Lumbar Interbody Fusion: 2-Dimensional Operative Video

Augmented reality (AR) is a novel technology for spine navigation. This tracking camera-integrated head-mounted display (HMD) represents a novel stereotactic computer navigation modality that has demonstrated excellent precision and accuracy with spinal instrumentation.1 Standard computer-assisted spine navigation systems have two major shortcomings: attention shift and line-of-sight limitations. The HMD allows visualization of the surgical field and navigation data concurrently in the same field of view.2,3 However, the use of AR in spine surgery has been limited to use for instrumentation, not for endoscopy.  Fully endoscopic transforaminal interbody fusion under conscious sedation is an effective treatment option for degenerative spondylolisthesis and spinal stenosis. Although this technique has a steep learning curve, the advantages are vast, including preservation of normal tissue, smaller incisional requirement, and reduced postoperative pain, all enabling rapid recovery after surgery. As with other endoscopic spine surgeries, this procedure has a steep learning curve and requires a robust understanding of foraminal anatomy in order to safely access the disc space.4,5 However, with the introduction of AR, the safety and precision of this procedure could be greatly improved upon.  In this video, we present a case of a 60-yr-old female who presented with a grade 1 spondylolisthesis and severe spinal stenosis and was treated with an L4-L5 interbody fusion. All instrumentation steps and localization for the endoscopic portion of the case were performed with assistance from the AR-HMD system. Informed written consent was obtained from the patient. The participant and any identifiable individuals consented to the publication of his/her image.

Clinical accuracy and initial experience with augmented reality-assisted pedicle screw placement: the first 205 screws

OBJECTIVE

Augmented reality (AR) is a novel technology which, when applied to spine surgery, offers the potential for efficient, safe, and accurate placement of spinal instrumentation. The authors report the accuracy of the first 205 pedicle screws consecutively placed at their institution by using AR assistance with a unique head-mounted display (HMD) navigation system.

METHODS

A retrospective review was performed of the first 28 consecutive patients who underwent AR-assisted pedicle screw placement in the thoracic, lumbar, and/or sacral spine at the authors' institution. Clinical accuracy for each pedicle screw was graded using the Gertzbein-Robbins scale by an independent neuroradiologist working in a blinded fashion.

RESULTS

Twenty-eight consecutive patients underwent thoracic, lumbar, or sacral pedicle screw placement with AR assistance. The median age at the time of surgery was 62.5 (IQR 13.8) years and the median body mass index was 31 (IQR 8.6) kg/m2. Indications for surgery included degenerative disease (n = 12, 43%); deformity correction (n = 12, 43%); tumor (n = 3, 11%); and trauma (n = 1, 4%). The majority of patients (n = 26, 93%) presented with low-back pain, 19 (68%) patients presented with radicular leg pain, and 10 (36%) patients had documented lower extremity weakness. A total of 205 screws were consecutively placed, with 112 (55%) placed in the lumbar spine, 67 (33%) in the thoracic spine, and 26 (13%) at S1. Screw placement accuracy was 98.5% for thoracic screws, 97.8% for lumbar/S1 screws, and 98.0% overall.

CONCLUSIONS

AR depicted through a unique HMD is a novel and clinically accurate technology for the navigated insertion of pedicle screws. The authors describe the first 205 AR-assisted thoracic, lumbar, and sacral pedicle screws consecutively placed at their institution with an accuracy of 98.0% as determined by a Gertzbein-Robbins grade of A or B.

Augmented Reality in Spine Surgery: A Narrative Review

Augmented reality (AR) navigation refers to novel technologies that superimpose images, such as radiographs and navigation pathways, onto a view of the operative field. The development of AR navigation has focused on improving the safety and efficacy of neurosurgical and orthopedic procedures. In this review, the authors focus on 3 types of AR technology used in spine surgery: AR surgical navigation, microscope-mediated heads-up display, and AR head-mounted displays. Microscope AR and head-mounted displays offer the advantage of reducing attention shift and line-of-sight interruptions inherent in traditional navigation systems. With the U.S. Food and Drug Administration's recent clearance of the XVision AR system (Augmedics, Arlington Heights, IL), the adoption and refinement of AR technology by spine surgeons will only accelerate.

Augmented-Reality-Assisted K-Wire Placement for Glenoid Component Positioning in Reversed Shoulder Arthroplasty: A Proof-of-Concept Study

The accuracy of the implant's post-operative position and orientation in reverse shoulder arthroplasty is known to play a significant role in both clinical and functional outcomes. Whilst technologies such as navigation and robotics have demonstrated superior radiological outcomes in many fields of surgery, the impact of augmented reality (AR) assistance in the operating room is still unknown. Malposition of the glenoid component in shoulder arthroplasty is known to result in implant failure and early revision surgery. The use of AR has many promising advantages, including allowing the detailed study of patient-specific anatomy without the need for invasive procedures such as arthroscopy to interrogate the joint's articular surface. In addition, this technology has the potential to assist surgeons intraoperatively in aiding the guidance of surgical tools. It offers the prospect of increased component placement accuracy, reduced surgical procedure time, and improved radiological and functional outcomes, without recourse to the use of large navigation or robotic instruments, with their associated high overhead costs. This feasibility study describes the surgical workflow from a standardised CT protocol, via 3D reconstruction, 3D planning, and use of a commercial AR headset, to AR-assisted k-wire placement. Post-operative outcome was measured using a high-resolution laser scanner on the patient-specific 3D printed bone. In this proof-of-concept study, the discrepancy between the planned and the achieved glenoid entry point and guide-wire orientation was approximately 3 mm with a mean angulation error of 5°.

Real-time navigation guidance with intraoperative CT imaging for pedicle screw placement using an augmented reality head-mounted display: a proof-of-concept study

OBJECTIVE

Augmented reality (AR) has the potential to improve the accuracy and efficiency of instrumentation placement in spinal fusion surgery, increasing patient safety and outcomes, optimizing ergonomics in the surgical suite, and ultimately lowering procedural costs. The authors sought to describe the use of a commercial prototype Spine AR platform (SpineAR) that provides a commercial AR head-mounted display (ARHMD) user interface for navigation-guided spine surgery incorporating real-time navigation images from intraoperative imaging with a 3D-reconstructed model in the surgeon's field of view, and to assess screw placement accuracy via this method.

METHODS

Pedicle screw placement accuracy was assessed and compared with literature-reported data of the freehand (FH) technique. Accuracy with SpineAR was also compared between participants of varying spine surgical experience. Eleven operators without prior experience with AR-assisted pedicle screw placement took part in the study: 5 attending neurosurgeons and 6 trainees (1 neurosurgical fellow, 1 senior orthopedic resident, 3 neurosurgical residents, and 1 medical student). Commercially available 3D-printed lumbar spine models were utilized as surrogates of human anatomy. Among the operators, a total of 192 screws were instrumented bilaterally from L2-5 using SpineAR in 24 lumbar spine models. All but one trainee also inserted 8 screws using the FH method. In addition to accuracy scoring using the Gertzbein-Robbins grading scale, axial trajectory was assessed, and user feedback on experience with SpineAR was collected.

RESULTS

Based on the Gertzbein-Robbins grading scale, the overall screw placement accuracy using SpineAR among all users was 98.4% (192 screws). Accuracy for attendings and trainees was 99.1% (112 screws) and 97.5% (80 screws), respectively. Accuracy rates were higher compared with literature-reported lumbar screw placement accuracy using FH for attendings (99.1% vs 94.32%; p = 0.0212) and all users (98.4% vs 94.32%; p = 0.0099). The percentage of total inserted screws with a minimum of 5° medial angulation was 100%. No differences were observed between attendings and trainees or between the two methods. User feedback on SpineAR was generally positive.

CONCLUSIONS

Screw placement was feasible and accurate using SpineAR, an ARHMD platform with real-time navigation guidance that provided a favorable surgeon-user experience.

First in-human report of the clinical accuracy of thoracolumbar percutaneous pedicle screw placement using augmented reality guidance

OBJECTIVE

Augmented reality (AR) is an emerging technology that has great potential for guiding the safe and accurate placement of spinal hardware, including percutaneous pedicle screws. The goal of this study was to assess the accuracy of 63 percutaneous pedicle screws placed at a single institution using an AR head-mounted display (ARHMD) system.

METHODS

Retrospective analyses were performed for 9 patients who underwent thoracic and/or lumbar percutaneous pedicle screw placement guided by ARHMD technology. Clinical accuracy was assessed via the Gertzbein-Robbins scale by the authors and by an independent musculoskeletal radiologist. Thoracic pedicle subanalysis was also performed to assess screw accuracy based on pedicle morphology.

RESULTS

Nine patients received thoracic or lumbar AR-guided percutaneous pedicle screws. The mean age at the time of surgery was 71.9 ± 11.5 years and the mean number of screws per patient was 7. Indications for surgery were spinal tumors (n = 4, 44.4%), degenerative disease (n = 3, 33.3%), spinal deformity (n = 1, 11.1%), and a combination of deformity and infection (n = 1, 11.1%). Presenting symptoms were most commonly low-back pain (n = 7, 77.8%) and lower-extremity weakness (n = 5, 55.6%), followed by radicular lower-extremity pain, loss of lower-extremity sensation, or incontinence/urinary retention (n = 3 each, 33.3%). In all, 63 screws were placed (32 thoracic, 31 lumbar). The accuracy for these screws was 100% overall; all screws were Gertzbein-Robbins grade A or B (96.8% grade A, 3.2% grade B). This accuracy was achieved in the thoracic spine regardless of pedicle cancellous bone morphology.

CONCLUSIONS

AR-guided surgery demonstrated a 100% accuracy rate for the insertion of 63 percutaneous pedicle screws in 9 patients (100% rate of Gertzbein-Robbins grade A or B screw placement). Using an ARHMS system for the placement of percutaneous pedicle screws showed promise, but further validation using a larger cohort of patients across multiple surgeons and institutions will help to determine the true accuracy enabled by this technology.

A systematic review of virtual reality for the assessment of technical skills in neurosurgery

OBJECTIVE

Virtual reality (VR) and augmented reality (AR) systems are increasingly available to neurosurgeons. These systems may provide opportunities for technical rehearsal and assessments of surgeon performance. The assessment of neurosurgeon skill in VR and AR environments and the validity of VR and AR feedback has not been systematically reviewed.

METHODS

A systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines was conducted through MEDLINE and PubMed. Studies published in English between January 1990 and February 2021 describing the use of VR or AR to quantify surgical technical performance of neurosurgeons without the use of human raters were included. The types and categories of automated performance metrics (APMs) from each of these studies were recorded.

RESULTS

Thirty-three VR studies were included in the review; no AR studies met inclusion criteria. VR APMs were categorized as either distance to target, force, kinematics, time, blood loss, or volume of resection. Distance and time were the most well-studied APM domains, although all domains were effective at differentiating surgeon experience levels. Distance was successfully used to track improvements with practice. Examining volume of resection demonstrated that attending surgeons removed less simulated tumor but preserved more normal tissue than trainees. More recently, APMs have been used in machine learning algorithms to predict level of training with a high degree of accuracy. Key limitations to enhanced-reality systems include limited AR usage for automated surgical assessment and lack of external and longitudinal validation of VR systems.

CONCLUSIONS

VR has been used to assess surgeon performance across a wide spectrum of domains. The VR environment can be used to quantify surgeon performance, assess surgeon proficiency, and track training progression. AR systems have not yet been used to provide metrics for surgeon performance assessment despite potential for intraoperative integration. VR-based APMs may be especially useful for metrics that are difficult to assess intraoperatively, including blood loss and extent of resection.

"Disruptive Technology" in Spine Surgery and Education: Virtual and Augmented Reality

BACKGROUND

Technological advancements are the drivers of modern-day spine care. With the growing pressure to deliver faster and better care, surgical-assist technology is needed to harness computing power and enable the surgeon to improve outcomes. Virtual reality (VR) and augmented reality (AR) represent the pinnacle of emerging technology, not only to deliver higher quality education through simulated care, but also to provide valuable intraoperative information to assist in more efficient and more precise surgeries.

OBJECTIVE

To describe how the disruptive technologies of VR and AR interface in spine surgery and education.

METHODS

We review the relevance of VR and AR technologies in spine care, and describe the feasibility and limitations of the technologies.

RESULTS

We discuss potential future applications, and provide a case study demonstrating the feasibility of a VR program for neurosurgical spine education.

CONCLUSION

Initial experiences with VR and AR technologies demonstrate their applicability and ease of implementation. However, further prospective studies through multi-institutional and industry-academic partnerships are necessary to solidify the future of VR and AR in spine surgery education and clinical practice.

Head-Mounted Augmented Reality Platform for Markerless Orthopaedic Navigation

Visual augmented reality (AR) has the potential to improve the accuracy, efficiency and reproducibility of computer-assisted orthopaedic surgery (CAOS). AR Head-mounted displays (HMDs) further allow non-eye-shift target observation and egocentric view. Recently, a markerless tracking and registration (MTR) algorithm was proposed to avoid the artificial markers that are conventionally pinned into the target anatomy for tracking, as their use prolongs surgical workflow, introduces human-induced errors, and necessitates additional surgical invasion in patients. However, such an MTR-based method has neither been explored for surgical applications nor integrated into current AR HMDs, making the ergonomic HMD-based markerless AR CAOS navigation hard to achieve. To these aims, we present a versatile, device-agnostic and accurate HMD-based AR platform. Our software platform, supporting both video see-through (VST) and optical see-through (OST) modes, integrates two proposed fast calibration procedures using a specially designed calibration tool. According to the camera-based evaluation, our AR platform achieves a display error of 6.31 2.55 arcmin for VST and 7.72 3.73 arcmin for OST. A proof-of-concept markerless surgical navigation system to assist in femoral bone drilling was then developed based on the platform and Microsoft HoloLens 1. According to the user study, both VST and OST markerless navigation systems are reliable, with the OST system providing the best usability. The measured navigation error is 4.90 1.04 mm, 5.96 2.22 for VST system and 4.36 0.80 mm, 5.65 1.42 for OST system.