Machine vision augmented reality for pedicle screw insertion during spine surgery

Innovations in Spine Surgery: A Narrative Review of Current Integrative Technologies

Neurosurgical technologies have become increasingly more adaptive, featuring real-time and patient-specific guidance in preoperative, intraoperative, and postoperative settings. This review offers insight into how these integrative innovations compare with conventional approaches in spine surgery, focusing on machine learning (ML), artificial intelligence, augmented reality and virtual reality, and spinal navigation systems. Data on technology applications, diagnostic and procedural accuracy, intraoperative times, radiation exposures, postoperative outcomes, and costs were extracted and compared with conventional methods to assess their advantages and limitations. Preoperatively, augmented reality and virtual reality have applications in surgical training and planning that are more immersive, case specific, and risk-free and have been shown to enhance accuracy and reduce complications. ML algorithms have demonstrated high accuracy in predicting surgical candidacy (up to 92.1%) and tailoring personalized treatments based on patient-specific variables. Intraoperatively, advantages include more accurate pedicle screw insertion (96%-99% with ML), enhanced visualization, reduced radiation exposure (49 μSv with O-arm navigation vs. 556 μSv with fluoroscopy), increased efficiency, and potential for fewer intraoperative complications compared with conventional approaches. Postoperatively, certain ML and artificial intelligence models have outperformed conventional methods in predicting all postoperative complications of >6000 patients as well as predicting variables contributing to in-hospital and 90-day mortality. However, applying these technologies comes with limitations, such as longer operative times (up to 35.6% longer) with navigation, dependency on datasets, costs, accessibility, steep learning curve, and inherent software malfunctions. As these technologies advance, continuing to assess their efficacy and limitations will be crucial to their successful integration within spine surgery.

Simulated outcomes for durotomy repair in minimally invasive spine surgery

Minimally invasive spine surgery (MISS) is increasingly performed using endoscopic and microscopic visualization, and the captured video can be used for surgical education and development of predictive artificial intelligence (AI) models. Video datasets depicting adverse event management are also valuable, as predictive models not exposed to adverse events may exhibit poor performance when these occur. Given that no dedicated spine surgery video datasets for AI model development are publicly available, we introduce Simulated Outcomes for Durotomy Repair in Minimally Invasive Spine Surgery (SOSpine). A validated MISS cadaveric dural repair simulator was used to educate neurosurgery residents, and surgical microscope video recordings were paired with outcome data. Objects including durotomy, needle, grasper, needle driver, and nerve hook were then annotated. Altogether, SOSpine contains 15,698 frames with 53,238 annotations and associated durotomy repair outcomes. For validation, an AI model was fine-tuned on SOSpine video and detected surgical instruments with a mean average precision of 0.77. In summary, SOSpine depicts spine surgeons managing a common complication, providing opportunities to develop surgical AI models.

Machine Learning in Spine Surgery: A Narrative Review

Artificial intelligence and machine learning (ML) can offer revolutionary advances in their application to the field of spine surgery. Within the past 5 years, novel applications of ML have assisted in surgical decision-making, intraoperative imaging and navigation, and optimization of clinical outcomes. ML has the capacity to address many different clinical needs and improve diagnostic and surgical techniques. This review will discuss current applications of ML in the context of spine surgery by breaking down its implementation preoperatively, intraoperatively, and postoperatively. Ethical considerations to ML and challenges in ML implementation must be addressed to maximally benefit patients, spine surgeons, and the healthcare system. Areas for future research in augmented reality and mixed reality, along with limitations in generalizability and bias, will also be highlighted.

Effects of an augmented reality aided system on the placement precision of orthodontic miniscrews: A pilot study

Background/purpose

Augmented reality (AR) is gaining popularity in medical applications, which may aid clinicians in achieving improved clinical outcomes. The purpose of this study was to determine the positional and angle errors of orthodontic miniscrew placement by using a self-developed AR aided system.

Materials and methods

Cone beam computed tomography (CBCT) and patient printed models were used in in vitro experiments. The participants were divided into a control group and an AR group, in which traditional orthodontic methods and the AR-aided system were used respectively. After the information obtained from the CBCT images and navigation system was combined on the display device, the AR-aided system indicated the planned miniscrew position to guide the clinicians during the placement of miniscrews. Both methods were compared by a senior and a junior dentist, and the position and angle of miniscrew placement were statistically analyzed using Wilcoxon's signed-rank and Mann-Whitney U tests.

Results

When the AR-aided system was used, the accuracy of miniscrew placement in the mesiodistal position considerably increased (83%) when the procedure was performed by a senior clinician. In addition, the accuracy of miniscrew placement in the mesiodistal position and the angle of miniscrew placement considerably increased by approximately 67% and 72%, respectively, when the procedure was performed by a junior clinician. The position error of miniscrew placement was smaller for the junior clinician when the AR-aided system was used than for the senior clinician.

Conclusion

The AR-aided system improved the accuracy of miniscrew placement regardless of the clinician's level of experience.

Augmented reality in spine surgery - past, present, and future

BACKGROUND CONTEXT

Augmented reality (AR) is increasingly recognized as a valuable tool in spine surgery. Here we provides an overview of the key developments and technological milestones that have laid the foundation for AR applications in this field. We also assess the quality of existing studies on AR systems in spine surgery and explore potential future applications.

PURPOSE

The purpose of this narrative review is to examine the role of AR in spine surgery. It aims to highlight the evolution of AR technology in this context, evaluate the existing body of research, and outline potential future directions for integrating AR into spine surgery.

STUDY DESIGN

Narrative review.

METHODS

We conducted a thorough literature search to identify studies and developments related to AR in spine surgery. Relevant articles, reports, and technological advancements were analyzed to establish the historical context and current state of AR in this field.

RESULTS

The review identifies significant milestones in the development of AR technology for spine surgery. It discusses the growing body of research and highlights the strengths and weaknesses of existing investigations. Additionally, it presents insights into the potential for AR to enhance spine surgical education and speculates on future applications.

CONCLUSIONS

Augmented reality has emerged as a promising adjunct in spine surgery, with notable advancements and research efforts. The integration of AR into the spine surgery operating room holds promise, as does its potential to revolutionize surgical education. Future applications of AR in spine surgery may include real-time navigation, enhanced visualization, and improved patient outcomes. Continued development and evaluation of AR technology are essential for its successful implementation in this specialized surgical field.

A review of advances in image-guided orthopedic surgery

Orthopedic surgery remains technically demanding due to the complex anatomical structures and cumbersome surgical procedures. The introduction of image-guided orthopedic surgery (IGOS) has significantly decreased the surgical risk and improved the operation results. This review focuses on the application of recent advances in artificial intelligence (AI), deep learning (DL), augmented reality (AR) and robotics in image-guided spine surgery, joint arthroplasty, fracture reduction and bone tumor resection. For the pre-operative stage, key technologies of AI and DL based medical image segmentation, 3D visualization and surgical planning procedures are systematically reviewed. For the intra-operative stage, the development of novel image registration, surgical tool calibration and real-time navigation are reviewed. Furthermore, the combination of the surgical navigation system with AR and robotic technology is also discussed. Finally, the current issues and prospects of the IGOS system are discussed, with the goal of establishing a reference and providing guidance for surgeons, engineers, and researchers involved in the research and development of this area.

A Novel Point Set Registration-Based Hand-Eye Calibration Method for Robot-Assisted Surgery

Pedicle screw insertion with robot assistance dramatically improves surgical accuracy and safety when compared with manual implantation. In developing such a system, hand-eye calibration is an essential component that aims to determine the transformation between a position tracking and robot-arm systems. In this paper, we propose an effective hand-eye calibration method, namely registration-based hand-eye calibration (RHC), which estimates the calibration transformation via point set registration without the need to solve the AX=XB equation. Our hand-eye calibration method consists of tool-tip pivot calibrations in two-coordinate systems, in addition to paired-point matching, where the point pairs are generated via the steady movement of the robot arm in space. After calibration, our system allows for robot-assisted, image-guided pedicle screw insertion. Comprehensive experiments are conducted to verify the efficacy of the proposed hand-eye calibration method. A mean distance deviation of 0.70 mm and a mean angular deviation of 0.68° are achieved by our system when the proposed hand-eye calibration method is used. Further experiments on drilling trajectories are conducted on plastic vertebrae as well as pig vertebrae. A mean distance deviation of 1.01 mm and a mean angular deviation of 1.11° are observed when the drilled trajectories are compared with the planned trajectories on the pig vertebrae.

Spine Surgery Assisted by Augmented Reality: Where Have We Been?

This present systematic review examines spine surgery literature supporting augmented reality (AR) technology and summarizes its current status in spinal surgery technology. Database search strategies were retrieved from PubMed, Web of Science, Cochrane Library, Embase, from the earliest records to April 1, 2021. Our review briefly examines the history of AR, and enumerates different device application workflows in a variety of spinal surgeries. We also sort out the pros and cons of current mainstream AR devices and the latest updates. A total of 45 articles are included in our review. The most prevalent surgical applications included are the augmented reality surgical navigation system and head-mounted display. The most popular application of AR is pedicle screw instrumentation in spine surgery, and the primary responsible surgical levels are thoracic and lumbar. AR guidance systems show high potential value in practical clinical applications for the spine. The overall number of cases in AR-related studies is still rare compared to traditional surgical-assisted techniques. These lack long-term clinical efficacy and robust surgical-related statistical data. Changing healthcare laws as well as the increasing prevalence of spinal surgery are generating critical data that determines the value of AR technology.

Augmented Reality (AR) in Orthopedics: Current Applications and Future Directions

PURPOSE OF REVIEW

Imaging technologies (X-ray, CT, MRI, and ultrasound) have revolutionized orthopedic surgery, allowing for the more efficient diagnosis, monitoring, and treatment of musculoskeletal aliments. The current review investigates recent literature surrounding the impact of augmented reality (AR) imaging technologies on orthopedic surgery. In particular, it investigates the impact that AR technologies may have on provider cognitive burden, operative times, occupational radiation exposure, and surgical precision and outcomes.

RECENT FINDINGS

Many AR technologies have been shown to lower provider cognitive burden and reduce operative time and radiation exposure while improving surgical precision in pre-clinical cadaveric and sawbones models. So far, only a few platforms focusing on pedicle screw placement have been approved by the FDA. These technologies have been implemented clinically with mixed results when compared to traditional free-hand approaches. It remains to be seen if current AR technologies can deliver upon their multitude of promises, and the ability to do so seems contingent upon continued technological progress. Additionally, the impact of these platforms will likely be highly conditional on clinical indication and provider type. It remains unclear if AR will be broadly accepted and utilized or if it will be reserved for niche indications where it adds significant value. One thing is clear, orthopedics' high utilization of pre- and intra-operative imaging, combined with the relative ease of tracking rigid structures like bone as compared to soft tissues, has made it the clear beachhead market for AR technologies in medicine.

First reported use of real-time intraoperative computed tomography angiography image registration using the Machine-vision Image Guided Surgery system: illustrative case

BACKGROUND

Vertebral artery injury is a devastating potential complication of C1–2 posterior fusion. Intraoperative navigation can reduce the risk of neurovascular complications and improve screw placement accuracy. However, the use of intraoperative computed tomography (CT) increases radiation exposure and operative time, and it is unable to image vascular structures. The Machine-vision Image Guided Surgery (MvIGS) system uses optical topographic imaging and machine vision software to rapidly register using preoperative imaging. The authors presented the first report of intraoperative navigation with MvIGS registered using a preoperative CT angiogram (CTA) during C1–2 posterior fusion.

OBSERVATIONS

MvIGS can register in seconds, minimizing operative time with no additional radiation exposure. Furthermore, surgeons can better adjust for abnormal vertebral artery anatomy and increase procedure safety.

LESSONS

CTA-guided navigation generated a three-dimensional reconstruction of cervical spine anatomy that assisted surgeons during the procedure. Although further study is needed, the use of intraoperative MvIGS may reduce the risk of vertebral artery injury during C1–2 posterior fusion.

Machine Vision Navigation in Spine Surgery

The advancements in computing and digital localizer technologies has led to the evolving clinical application of image-guided technology for the surgical management of spinal disorders. Image-guided spinal navigation addresses the limitations of fluoroscopy and improves the accurate placement of fixation screws. Several navigation platforms are currently available, each having its own unique advantages and disadvantages. The most recent spinal navigation system developed utilizes machine vision structured light imaging which creates a precise and detailed three-dimensional image of the exposed surface anatomy and co-registers it to a pre-operatively or intra-operatively acquired image. This system improves upon the intraoperative workflow and efficiency of the navigation process. With the continued advancements in machine vision, there is a potential for clinical applications that extend beyond surgical navigation. These applications include reducing the potential for wrong level spine surgery and providing for real-time tracking of spinal deformity correction. As the adoption and clinical experience with navigation continues to expand and evolve, the technology that enables navigation also continues to evolve.

The Role of Machine Learning in Spine Surgery: The Future Is Now

The recent influx of machine learning centered investigations in the spine surgery literature has led to increased enthusiasm as to the prospect of using artificial intelligence to create clinical decision support tools, optimize postoperative outcomes, and improve technologies used in the operating room. However, the methodology underlying machine learning in spine research is often overlooked as the subject matter is quite novel and may be foreign to practicing spine surgeons. Improper application of machine learning is a significant bioethics challenge, given the potential consequences of over- or underestimating the results of such studies for clinical decision-making processes. Proper peer review of these publications requires a baseline familiarity of the language associated with machine learning, and how it differs from classical statistical analyses. This narrative review first introduces the overall field of machine learning and its role in artificial intelligence, and defines basic terminology. In addition, common modalities for applying machine learning, including classification and regression decision trees, support vector machines, and artificial neural networks are examined in the context of examples gathered from the spine literature. Lastly, the ethical challenges associated with adapting machine learning for research related to patient care, as well as future perspectives on the potential use of machine learning in spine surgery, are discussed specifically.