Early Experience With Virtual and Synchronized Augmented Reality Platform for Preoperative Planning and Intraoperative Navigation: A Case Series

Surgical planning in virtual reality: a systematic review

Purpose

Virtual reality (VR) technology has emerged as a promising tool for physicians, offering the ability to assess anatomical data in 3D with visuospatial interaction qualities. The last decade has witnessed a remarkable increase in the number of studies focusing on the application of VR to assess patient-specific image data. This systematic review aims to provide an up-to-date overview of the latest research on VR in the field of surgical planning.

Approach

A comprehensive literature search was conducted based on the preferred reporting items for systematic reviews and meta-analyses covering the period from April 1, 2021 to May 10, 2023. It includes research articles reporting on preoperative surgical planning using patient-specific medical images in virtual reality using head-mounted displays. The review summarizes the current state of research in this field, identifying key findings, technologies, study designs, methods, and potential directions for future research.

Results

The selected studies show a positive impact on surgical decision-making and anatomy understanding compared to other visualization modalities. A substantial number of studies are reporting anecdotal evidence and case-specific outcomes. Notably, surgical planning using VR led to more frequent changes in surgical plans compared to planning with other visualization methods when surgeons reassessed their initial plans. VR demonstrated benefits in reducing planning time and improving spatial localization of pathologies.

Conclusions

Results show that the application of VR for surgical planning is still in an experimental stage but is gradually advancing toward clinical use. The diverse study designs, methodologies, and varying reporting hinder a comprehensive analysis. Some findings lack statistical evidence and rely on subjective assumptions. To strengthen evaluation, future research should focus on refining study designs, improving technical reporting, defining visual and technical proficiency requirements, and enhancing VR software usability and design. Addressing these areas could pave the way for an effective implementation of VR in clinical settings.

Safety and Optimizing Ergonomics for Cardiothoracic Surgeons

Cardiothoracic surgery, demanding in nature, often results in surgeons suffering from musculoskeletal injuries, causing chronic pain and leading to premature retirement. A significant majority report experiencing pain, exacerbated by minimally invasive techniques such as video-assisted thoracoscopic surgery. Despite this, many surgeons delay seeking medical assistance. To mitigate these risks, preventative strategies such as strength exercises, stretching during operations, and taking brief breaks are crucial. However, the surgical community faces a shortage of institutional support and comprehensive ergonomic education. Advancements in technology, including artificial intelligence and virtual reality, could offer future solutions.

Evolution of the meta-neurosurgeon: A systematic review of the current technical capabilities, limitations, and applications of augmented reality in neurosurgery

Background

Augmented reality (AR) applications in neurosurgery have expanded over the past decade with the introduction of headset-based platforms. Many studies have focused on either preoperative planning to tailor the approach to the patient's anatomy and pathology or intraoperative surgical navigation, primarily realized as AR navigation through microscope oculars. Additional efforts have been made to validate AR in trainee and patient education and to investigate novel surgical approaches. Our objective was to provide a systematic overview of AR in neurosurgery, provide current limitations of this technology, as well as highlight several applications of AR in neurosurgery.

Methods

We performed a literature search in PubMed/Medline to identify papers that addressed the use of AR in neurosurgery. The authors screened three hundred and seventy-five papers, and 57 papers were selected, analyzed, and included in this systematic review.

Results

AR has made significant inroads in neurosurgery, particularly in neuronavigation. In spinal neurosurgery, this primarily has been used for pedicle screw placement. AR-based neuronavigation also has significant applications in cranial neurosurgery, including neurovascular, neurosurgical oncology, and skull base neurosurgery. Other potential applications include operating room streamlining, trainee and patient education, and telecommunications.

Conclusion

AR has already made a significant impact in neurosurgery in the above domains and has the potential to be a paradigm-altering technology. Future development in AR should focus on both validating these applications and extending the role of AR.

Extended reality in cranial and spinal neurosurgery - a bibliometric analysis

PURPOSE

This bibliometric analysis of the top 100 cited articles on extended reality (XR) in neurosurgery aimed to reveal trends in this research field. Gender differences in authorship and global distribution of the most-cited articles were also addressed.

METHODS

A Web of Science electronic database search was conducted. The top 100 most-cited articles related to the scope of this review were retrieved and analyzed for trends in publications, journal characteristics, authorship, global distribution, study design, and focus areas. After a brief description of the top 100 publications, a comparative analysis between spinal and cranial publications was performed.

RESULTS

From 2005, there was a significant increase in spinal neurosurgery publications with a focus on pedicle screw placement. Most articles were original research studies, with an emphasis on augmented reality (AR). In cranial neurosurgery, there was no notable increase in publications. There was an increase in studies assessing both AR and virtual reality (VR) research, with a notable emphasis on VR compared to AR. Education, surgical skills assessment, and surgical planning were more common themes in cranial studies compared to spinal studies. Female authorship was notably low in both groups, with no significant increase over time. The USA and Canada contributed most of the publications in the research field.

CONCLUSIONS

Research regarding the use of XR in neurosurgery increased significantly from 2005. Cranial research focused on VR and resident education while spinal research focused on AR and neuronavigation. Female authorship was underrepresented. North America provides most of the high-impact research in this area.

Intraoperative Augmented Reality for Complex Glioma Resection: A Case Report

Augmented reality (AR) is an emerging technology that can display three-dimensional patient anatomy in the surgeons' field of view. The use of this technology has grown considerably for both presurgical and intraoperative guidance. A patient diagnosed with breast cancer started to experience numbness in the left hand, which progressed to weakness in the left hand and arm. An MRI was performed demonstrating a 2.9 cm X 1.8 cm lesion with extensive surrounding edema in the posterior fronto-parietal lobes. Surgery was recommended for presumed metastatic disease. Preoperatively, an AR system and Brainlab navigation were registered to the patient. AR, traditional navigation, and ultrasound were all used to localize the lesion and determine the craniotomy site and size. The tumor was removed along the direction of the lesion. Intraoperatively, we used AR to reexamine the tumor details and could appreciate that we had to redirect our surgical trajectory anteriorly and laterally in order to follow along the main axis of the tumor. In doing this, we were able to more confidently remain with the tumor, which by this time was poorly defined by 2D navigation and by direct vision. Postoperative MRI confirmed gross total removal of the tumor. The patient had an uneventful postoperative course with resolution of preoperative symptoms and the final surgical pathology was grade 4 glioblastoma. Here, we describe the valuable use of AR for the resection of a glioma. The system has a seamless registration process and provides the surgeon with a unique view of 3D anatomy overlaid onto the patient's head. This exciting technology can add tremendous value to complex cranial surgeries.

Case Report: Use of novel AR registration system for presurgical planning during vestibular schwannoma resection surgery

Background and importance

Vestibular schwannomas are benign tumors and are the most common tumor found in the cerebellopontine angle. Surgical management of these lesions involves consideration of various operative approaches, which can have profound effects on procedural course and patient outcomes. Therefore, a comprehensive understanding of the location of the tumor and surrounding anatomical structures is vital for a positive outcome. We present a case of a 47-year-old female patient with vestibular schwannoma. A novel mixed reality (MR) system was used to register patient-specific 3D models onto the patient's head for operative planning and anatomical visualization.

Case description

A 47-year-old female presented with a history of left-sided hearing loss, tinnitus, and episodic left facial tingling. Magnetic Resonance Imaging (MRI) demonstrated a 3.3 cm enhancing lesion in the left cerebellopontine angle at the with mass effect on the brachium pontis and medulla. Surgical resection was performed via retrosigmoid craniotomy.

Conclusions

In this study, we report the use of Augmented Reality (AR) visualization for planning of vestibular schwannoma resection. This technology allows for efficient and accurate registration of a patient's 3D anatomical model onto their head while positioned in the operating room. This system is a powerful tool for operative planning as it allows the surgeon to visualize critical anatomical structures where they lie on the patient's head. The present case demonstrates the value and use of AR for operative planning of complex cranial lesions.

Optimizing Surgical Efficiency in Complex Spine Surgery Using Virtual Reality as a Communication Technology to Promote a Shared Mental Model: A Case Series and Review

BACKGROUND AND OBJECTIVES

Virtual reality (VR) is an emerging technology that can be used to promote a shared mental model among a surgical team. We present a case series demonstrating the use of 3-dimensional (3D) VR models to visually communicate procedural steps to a surgical team to promote a common operating objective. We also review the literature on existing uses of VR for preoperative communication and planning in spine surgery.

METHODS

Narrations of 3 to 4-minute walkthroughs were created in a VR visualization platform, converted, and distributed to team members through text and email the night before surgical intervention. A VR huddle was held immediately before the intervention to refine surgical goals. After the intervention, the participating team members' perceptions on the value of the tool were assessed using a survey that used a 5-point Likert scale. MEDLINE, Google Scholar, and Dimensions AI databases were queried from July 2010 to October 2022 to examine existing literature on preoperative VR use to plan spine surgery.

RESULTS

Three illustrative cases are presented with accompanying video. Postoperative survey results demonstrate a positive experience among surgical team members after reviewing preoperative plans created with patient-specific 3D VR models. Respondents felt that preoperative VR video review was "moderately useful" or more useful in improving their understanding of the operational sequence (71%, 5/7), in enhancing their ability to understand their role (86%, 6/7), and in improving the safety or efficiency of the case (86%, 6/7).

CONCLUSION

We present a proof of concept of a novel preoperative communication tool used to create a shared mental model of a common operating objective for surgical team members using narrated 3D VR models. Initial survey results demonstrate positive feedback among respondents. There is a paucity of literature investigating VR technology as a means for preoperative surgical communication in spine surgery.

ETHICS

Institutional review board approval (IRB-300009785) was obtained before this study.

Creating Augmented Reality Holograms for Polytrauma Patients Using 3D Slicer and Holomedicine Medical Image Platform

In traumatology physicians heavily rely on computed tomography (CT) 2D axial scans to identify and assess the patient's injuries after an accident. However, in some cases it can be difficult to rigorously evaluate the real extent of the damage considering only the bidimensional slices produced by the CT, and some life-threatening lesions can be missed. With the development of 3D holographic rendering and extended reality (XR) technology, CT images can be projected in a 3D format through head-mounted holographic displays, allowing multi-view from different angles and interactive slice intersections, thus increasing anatomical intelligibility. In this article, we explain how to import CT scans into holographic displays for 3D visualization and further compare the methodolgy with traditional bidimensional reading.

The Inaugural "Century" of Mixed Reality in Cranial Surgery: Virtual Reality Rehearsal/Augmented Reality Guidance and Its Learning Curve in the First 100-Case, Single-Surgeon Series

BACKGROUND AND OBJECTIVE

Virtual reality (VR) refers to a computer-generated three-dimensional space in which a surgeon can interact with patient-specific anatomic models for surgical planning. Augmented reality (AR) is the technology that places computer-generated objects, including those made in VR, into the surgeon's visual space. Together, VR and AR are called mixed reality (MxR), and it is gaining importance in neurosurgery. MxR is helpful for selecting and creating templates for an optimal surgical approach and identifying key anatomic landmarks intraoperatively. By reporting our experience with the first 100 consecutive cases planned with VR and executed with AR, our objective is to detail the learning curve and encountered obstacles while adopting the new technology.

METHODS

This series includes the first 100 consecutive complex cranial cases of a single surgeon for which MxR was intended for use. Effectiveness of the VR rehearsal and AR guidance was analyzed for four specific contributions: (1) opening size, (2) precise craniotomy placement, (3) guidance toward anatomic landmarks or target, and (4) antitarget avoidance. Seventeen cases in the study cohort were matched with historical non-MxR cases for comparison of outcome parameters. The cases in which MxR failed were plotted over time to determine the nature of the "learning curve."

RESULTS

AR guidance was abandoned in eight operations because of technical problems, but problem-free application of MxR increased between the 44th and 63rd cases. This provides some evidence of proficiency acquisition in between. Comparing the 17 pairs of matched MxR and non-MxR cases, no statistically significant differences exist in the groups regarding blood loss, length of stay nor duration of surgery. Cases where MxR had above-expectation performances are highlighted.

CONCLUSION

MxR is a powerful tool that can help tailor operations to patient-specific anatomy and provide efficient intraoperative guidance without additional time for surgery or hospitalization.

Validation of real-time inside-out tracking and depth realization technologies for augmented reality-based neuronavigation

PURPOSE

Concomitant with the significant advances in computing technology, the utilization of augmented reality-based navigation in clinical applications is being actively researched. In this light, we developed novel object tracking and depth realization technologies to apply augmented reality-based neuronavigation to brain surgery.

METHODS

We developed real-time inside-out tracking based on visual inertial odometry and a visual inertial simultaneous localization and mapping algorithm. The cube quick response marker and depth data obtained from light detection and ranging sensors are used for continuous tracking. For depth realization, order-independent transparency, clipping, and annotation and measurement functions were developed. In this study, the augmented reality model of a brain tumor patient was applied to its life-size three-dimensional (3D) printed model.

RESULTS

Using real-time inside-out tracking, we confirmed that the augmented reality model remained consistent with the 3D printed patient model without flutter, regardless of the movement of the visualization device. The coordination accuracy during real-time inside-out tracking was also validated. The average movement error of the X and Y axes was 0.34 ± 0.21 and 0.04 ± 0.08 mm, respectively. Further, the application of order-independent transparency with multilayer alpha blending and filtered alpha compositing improved the perception of overlapping internal brain structures. Clipping, and annotation and measurement functions were also developed to aid depth perception and worked perfectly during real-time coordination. We named this system METAMEDIP navigation.

CONCLUSIONS

The results validate the efficacy of the real-time inside-out tracking and depth realization technology. With these novel technologies developed for continuous tracking and depth perception in augmented reality environments, we are able to overcome the critical obstacles in the development of clinically applicable augmented reality neuronavigation.

Evaluation Metrics for Augmented Reality in Neurosurgical Preoperative Planning, Surgical Navigation, and Surgical Treatment Guidance: A Systematic Review

BACKGROUND AND OBJECTIVE

Recent years have shown an advancement in the development of augmented reality (AR) technologies for preoperative visualization, surgical navigation, and intraoperative guidance for neurosurgery. However, proving added value for AR in clinical practice is challenging, partly because of a lack of standardized evaluation metrics. We performed a systematic review to provide an overview of the reported evaluation metrics for AR technologies in neurosurgical practice and to establish a foundation for assessment and comparison of such technologies.

METHODS

PubMed, Embase, and Cochrane were searched systematically for publications on assessment of AR for cranial neurosurgery on September 22, 2022. The findings were reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.

RESULTS

The systematic search yielded 830 publications; 114 were screened full text, and 80 were included for analysis. Among the included studies, 5% dealt with preoperative visualization using AR, with user perception as the most frequently reported metric. The majority (75%) researched AR technology for surgical navigation, with registration accuracy, clinical outcome, and time measurements as the most frequently reported metrics. In addition, 20% studied the use of AR for intraoperative guidance, with registration accuracy, task outcome, and user perception as the most frequently reported metrics.

CONCLUSION

For quality benchmarking of AR technologies in neurosurgery, evaluation metrics should be specific to the risk profile and clinical objectives of the technology. A key focus should be on using validated questionnaires to assess user perception; ensuring clear and unambiguous reporting of registration accuracy, precision, robustness, and system stability; and accurately measuring task performance in clinical studies. We provided an overview suggesting which evaluation metrics to use per AR application and innovation phase, aiming to improve the assessment of added value of AR for neurosurgical practice and to facilitate the integration in the clinical workflow.

Optimizing Surgical Performance Using Preoperative Virtual Reality Planning: A Systematic Review

BACKGROUND

Surgery is often a complex process that requires detailed 3-dimensional anatomical knowledge and rigorous interplay between team members to attain ideal operational efficiency or "flow." Virtual Reality (VR) represents a technology by which to rehearse complex plans and communicate precise steps to a surgical team prior to entering the operating room. The objective of this study was to evaluate the use of VR for preoperative surgical team planning and interdisciplinary communication across all surgical specialties.

METHODS

A systematic review of the literature was performed examining existing research on VR use for preoperative surgical team planning and interdisciplinary communication across all surgical fields in order to optimize surgical efficiency. MEDLINE, SCOPUS, CINAHL databases were searched from inception to July 31, 2022 using standardized search clauses. A qualitative data synthesis was performed with particular attention to preoperative planning, surgical efficiency optimization, and interdisciplinary collaboration/communication techniques determined a priori. Preferred Reporting Items for Systematic Review and Meta-Analysis guidelines were followed. All included studies were appraised for their quality using the Medical Education Research Study Quality Instrument (MERSQI) tool.

RESULTS

One thousand and ninety-three non-duplicated articles with abstract and full text availability were identified. Thirteen articles that examined preoperative VR-based planning techniques for optimization of surgical efficiency and/or interdisciplinary communication fulfilled inclusion and exclusion criteria. These studies had a low-to-medium methodological quality with a MERSQI mean score of 10.04 out of 18 (standard deviation 3.61).

CONCLUSIONS

This review demonstrates that time spent rehearsing and visualizing patient-specific anatomical relationships in VR may improve operative efficiency and communication across multiple surgical specialties.

Intraoperative augmented reality fiber tractography complements cortical and subcortical mapping

Augmented reality (AR) has been found to be advantageous in enhancing visualization of complex neuroanatomy intraoperatively and in neurosurgical education. Another key tool that allows neurosurgeons to have enhanced visualization, namely of white matter tracts, is diffusion tensor imaging (DTI) that is processed with high-definition fiber tractography (HDFT). There remains an enduring challenge in the structural-functional correlation of white matter tracts that centers on the difficulty in clearly assigning function to any given fiber tract when evaluating them through separated as opposed to integrated modalities. Combining the technologies of AR with fiber tractography shows promise in helping to fill in this gap between structural-functional correlation of white matter tracts. This novel study demonstrates through a series of three cases of awake craniotomies for glioma resections a technique that allows the first and most direct evidence of fiber tract stimulation and assignment of function or deficit in vivo through the intraoperative, real-time fusion of electrical cortical stimulation, AR, and HDFT. This novel technique has qualitatively shown to be helpful in guiding intraoperative decision making on extent of resection of gliomas. Future studies could focus on larger, prospective cohorts of glioma patients who undergo this methodology and further correlate the post-operative imaging results to patient functional outcomes.

Image-guidance in endoscopic pituitary surgery: an in-silico study of errors involved in tracker-based techniques

Background

Endoscopic endonasal surgery is an established minimally invasive technique for resecting pituitary adenomas. However, understanding orientation and identifying critical neurovascular structures in this anatomically dense region can be challenging. In clinical practice, commercial navigation systems use a tracked pointer for guidance. Augmented Reality (AR) is an emerging technology used for surgical guidance. It can be tracker based or vision based, but neither is widely used in pituitary surgery.

Methods

This pre-clinical study aims to assess the accuracy of tracker-based navigation systems, including those that allow for AR. Two setups were used to conduct simulations: (1) the standard pointer setup, tracked by an infrared camera; and (2) the endoscope setup that allows for AR, using reflective markers on the end of the endoscope, tracked by infrared cameras. The error sources were estimated by calculating the Euclidean distance between a point's true location and the point's location after passing it through the noisy system. A phantom study was then conducted to verify the in-silico simulation results and show a working example of image-based navigation errors in current methodologies.

Results

The errors of the tracked pointer and tracked endoscope simulations were 1.7 and 2.5 mm respectively. The phantom study showed errors of 2.14 and 3.21 mm for the tracked pointer and tracked endoscope setups respectively.

Discussion

In pituitary surgery, precise neighboring structure identification is crucial for success. However, our simulations reveal that the errors of tracked approaches were too large to meet the fine error margins required for pituitary surgery. In order to achieve the required accuracy, we would need much more accurate tracking, better calibration and improved registration techniques.

Early Experience of Surgical Planning for STA-MCA Bypass Using Virtual Reality

BACKGROUND

The superficial temporal artery (STA)-to-middle cerebral artery (MCA) bypass requires precise preoperative planning, and 3-dimensional virtual reality (VR) models have recently been used to optimize planning of STA-MCA bypass. In the present report, we have described our experience with VR-based preoperative planning of STA-MCA bypass.

METHODS

Patients from August 2020 to February 2022 were analyzed. For the VR group, using 3-dimensional models from the patients' preoperative computed tomography angiograms, VR was used to locate the donor vessels, potential recipient, and anastomosis sites and plan the craniotomy, which were referenced throughout surgery. Computed tomography angiograms or digital subtraction angiograms were used to plan the craniotomy for the control group. The procedure time, bypass patency, craniotomy size, and postoperative complication rates were assessed.

RESULTS

The VR group included 17 patients (13 women; age, 49 ± 14 years) with Moyamoya disease (76.5%) and/or ischemic stroke (29.4%). The control group included 13 patients (8 women; age, 49 ± 12 years) with Moyamoya disease (92.3%) and/or ischemic stroke (7.3%). For all 30 patients, the preoperatively planned donor and recipient branches were effectively translated intraoperatively. No significant difference were found in the procedure time or craniotomy size between the 2 groups. Bypass patency was 94.1% for the VR group (16 of 17) and 84.6% for the control group (11 of 13). No permanent neurological deficits occurred in either group.

CONCLUSIONS

Our early experience has shown that VR can serve as a useful, interactive preoperative planning tool by enhancing visualization of the spatial relationship between the STA and MCA without compromising the surgical results.

Three-Dimensional Modeling and Extended Reality Simulations of the Cross-Sectional Anatomy of the Cerebrum, Cerebellum, and Brainstem

BACKGROUND

Understanding the anatomy of the human cerebrum, cerebellum, and brainstem and their 3-dimensional (3D) relationships is critical for neurosurgery. Although 3D photogrammetric models of cadaver brains and 2-dimensional images of postmortem brain slices are available, neurosurgeons lack free access to 3D models of cross-sectional anatomy of the cerebrum, cerebellum, and brainstem that can be simulated in both augmented reality (AR) and virtual reality (VR).

OBJECTIVE

To create 3D models and AR/VR simulations from 2-dimensional images of cross-sectionally dissected cadaveric specimens of the cerebrum, cerebellum, and brainstem.

METHODS

The Klingler method was used to prepare 3 cadaveric specimens for dissection in the axial, sagittal, and coronal planes. A series of 3D models and AR/VR simulations were then created using 360° photogrammetry.

RESULTS

High-resolution 3D models of cross-sectional anatomy of the cerebrum, cerebellum, and brainstem were obtained and used in creating AR/VR simulations. Eleven axial, 9 sagittal, and 7 coronal 3D models were created. The sections were planned to show important deep anatomic structures. These models can be freely rotated, projected onto any surface, viewed from all angles, and examined at various magnifications.

CONCLUSION

To our knowledge, this detailed study is the first to combine up-to-date technologies (photogrammetry, AR, and VR) for high-resolution 3D visualization of the cross-sectional anatomy of the entire human cerebrum, cerebellum, and brainstem. The resulting 3D images are freely available for use by medical professionals and students for better comprehension of the 3D relationship of the deep and superficial brain anatomy.

Efficacy of a Novel Augmented Reality Navigation System Using 3D Computer Graphic Modeling in Endoscopic Transsphenoidal Surgery for Sellar and Parasellar Tumors

In endoscopic transsphenoidal skull base surgery, knowledge of tumor location on imaging and the anatomic structures is required simultaneously. However, it is often difficult to accurately reconstruct the endoscopic vision of the surgical field from the pre-surgical radiographic images because the lesion remarkably displaces the geography of normal anatomic structures. We created a precise three-dimensional computer graphic model from preoperative radiographic data that was then superimposed on a visual image of the actual surgical field and displayed on a video monitor during endoscopic transsphenoidal surgery. We evaluated the efficacy of this augmented reality (AR) navigation system in 15 consecutive patients with sellar and parasellar tumors. The average score overall was 4.7 [95% confidence interval: 4.58-4.82], which indicates that the AR navigation system was as useful as or more useful than conventional navigation in certain patients. In two patients, AR navigation was assessed as less useful than conventional navigation because perception of the depth of the lesion was more difficult. The developed system was more useful than conventional navigation for facilitating an immediate three-dimensional understanding of the lesion and surrounding structures.

Three-dimensional technologies in presurgical planning of bone surgeries: current evidence and future perspectives

BACKGROUND

The recent development of three-dimensional (3D) technologies introduces a novel set of opportunities to the medical field in general, and specifically to surgery. The preoperative phase has proven to be a critical factor in surgical success. Utilization of 3D technologies has the potential to improve preoperative planning and overall surgical outcomes. In this narrative review article, the authors describe existing clinical data pertaining to the current use of 3D printing, virtual reality, and augmented reality in the preoperative phase of bone surgery.

METHODS

The methodology included keyword-based literature search in PubMed and Google Scholar for original articles published between 2014 and 2022. After excluding studies performed in nonbone surgery disciplines, data from 61 studies of five different surgical disciplines were processed to be included in this narrative review.

RESULTS

Among the mentioned technologies, 3D printing is currently the most advanced in terms of clinical use, predominantly creating anatomical models and patient-specific instruments that provide high-quality operative preparation. Virtual reality allows to set a surgical plan and to further simulate the procedure via a 2D screen or head mounted display. Augmented reality is found to be useful for surgical simulation upon 3D printed anatomical models or virtual phantoms.

CONCLUSIONS

Overall, 3D technologies are gradually becoming an integral part of a surgeon's preoperative toolbox, allowing for increased surgical accuracy and reduction of operation time, mainly in complex and unique surgical cases. This may eventually lead to improved surgical outcomes, thereby optimizing the personalized surgical approach.

Augmented Reality Neuronavigation for En Bloc Resection of Spinal Column Lesions

BACKGROUND

Primary tumors involving the spine are relatively rare but represent surgically challenging procedures with high patient morbidity. En bloc resection of these tumors necessitates large exposures, wide tumor margins, and poses risks to functionally relevant anatomical structures. Augmented reality neuronavigation (ARNV) represents a paradigm shift in neuronavigation, allowing on-demand visualization of 3D navigation data in real-time directly in line with the operative field.

METHODS

Here, we describe the first application of ARNV to perform distal sacrococcygectomies for the en bloc removal of sacral and retrorectal lesions involving the coccyx in 2 patients, as well as a thoracic 9-11 laminectomy with costotransversectomy for en bloc removal of a schwannoma in a third patient.

RESULTS

In our experience, ARNV allowed our teams to minimize the length of the incision, reduce the extent of bony resection, and enhanced visualization of critical adjacent anatomy. All tumors were resected en bloc, and the patients recovered well postoperatively, with no known complications. Pathologic analysis confirmed the en bloc removal of these lesions with negative margins.

CONCLUSIONS

We conclude that ARNV is an effective strategy for the precise, en bloc removal of spinal lesions including both sacrococcygeal tumors involving the retrorectal space and thoracic schwannomas.

Investigating uncertainty in augmented reality enhanced renal navigation using in vitro patient-specific tissue-mimicking phantoms

To improve the outcome of minimally invasive renal interventions, traditional video-guided needle navigation can be enhanced by tracking the needle, guiding the needle using video imaging, and augmenting the surgical scene with pre-procedural images or models of the anatomy. In our previous work we studied, both through simulations and in vitro experiments, the uncertainty associated with the model-to-phantom registration, as well as the camera-tracker calibration and video-guided navigation. In this work, we characterize the overall navigation uncertainty using tissue emulating patient-specific kidney phantoms featuring both virtual and physical internal targets. Pre-procedural models of the kidney phantoms and internal targets are generated from cone-beam CT images, and are registered to their intra-operative physical counter-parts. The user then guides the needle insertion to reach the internal targets using video-based imaging augmented with a virtual representation of the needle tracked in real time. Following navigation, we acquire post-procedural cone-beam CT images of the phantoms and inserted needles. These images are used to determine the ground truth needle navigation accuracy (i.e., needle to target distance) against which the intra-operative navigation accuracy (i.e., intra-op needle tip to target distance) is assessed. We also explore a method to update the pre-procedural model to physical phantom registration intra-operatively using tracked video imaging, with the overall goal to improve overall navigation accuracy in the event of sub-optimal initial image-to-phantom registration. Our results showed a navigation error of less than 3.5 mm in gelatin phantoms and less than 6.5 mm in PVA phantoms. Following registration correction intra-operatively, we showed an overall improvement in navigation from roughly 6 mm RMS to approximately 2 mm RMS error, which is acceptable given the inherent tracking, 3D printing and phantom manufacturing limitations.

A Smarter Health through the Internet of Surgical Things

(1) Background: In the last few years, technological developments in the surgical field have been rapid and are continuously evolving. One of the most revolutionizing breakthroughs was the introduction of the IoT concept within surgical practice. Our systematic review aims to summarize the most important studies evaluating the IoT concept within surgical practice, focusing on Telesurgery and surgical Telementoring. (2) Methods: We conducted a systematic review of the current literature, focusing on the Internet of Surgical Things in Telesurgery and Telementoring. Forty-eight (48) studies were included in this review. As secondary research questions, we also included brief overviews of the use of IoT in image-guided surgery, and patient Telemonitoring, by systematically analyzing fourteen (14) and nineteen (19) studies, respectively. (3) Results: Data from 219 patients and 757 healthcare professionals were quantitively analyzed. Study designs were primarily observational or based on model development. Palpable advantages from the IoT incorporation mainly include less surgical hours, accessibility to high quality treatment, and safer and more effective surgical education. Despite the described technological advances, and proposed benefits of the systems presented, there are still identifiable gaps in the literature that need to be further explored in a systematic manner. (4) Conclusions: The use of the IoT concept within the surgery domain is a widely incorporated but less investigated concept. Advantages have become palpable over the past decade, yet further research is warranted.

Single-stage cranioplasty with customized polyetheretherketone implant after tumor resection using virtual reality and augmented reality for precise implant customization and placement: illustrative case

BACKGROUND

Cranioplasties are routinely performed to restore cosmesis and to protect intracranial contents after trauma, resection of tumors, or other pathologies. Traditionally done as a second-stage procedure, new single-stage cranioplasty protocols have been developed to minimize recovery periods, decrease complications, and improve patient satisfaction. These protocols, however, still require the use of larger than planned implants or use larger than ideal incisions to accommodate three-dimensional (3D) templates, which may not be optimal in regions with complex bony anatomy.

OBSERVATIONS

A 50-year-old woman with a painful and progressively enlarging hemangioma of the left frontal bone underwent a single-stage resection followed by custom cranioplasty using a new extended reality (XR)-based workflow. Excellent cosmetic results, decreased operative time, and a feasible workflow were achieved.

LESSONS

The use of an XR-based visualization platform allows the surgeon to treat lesions and perform custom cranioplasties in one session while avoiding common pitfalls of current single-stage workflows, such as increased operative times for tailoring implants, as well as minimizing the use of 3D overlay models, which may not appropriately conform to complex regional bony anatomy intraoperatively.

Virtual and Augmented Reality in Neurosurgery: The Evolution of its Application and Study Designs

BACKGROUND

As the art of neurosurgery evolves in the 21st century, more emphasis is placed on minimally invasive techniques, which require technical precision. Simultaneously, the reduction on training hours continues, and teachers of neurosurgery faces "double jeopardy"-with harder skills to teach and less time to teach them. Mixed reality appears as the neurosurgical educators' natural ally: Virtual reality facilitates the learning of spatial relationships and permits rehearsal of skills, while augmented reality can make procedures safer and more efficient. Little wonder then, that the body of literature on mixed reality in neurosurgery has grown exponentially.

METHODS

Publications involving virtual and augmented reality in neurosurgery were examined. A total of 414 papers were included, and they were categorized according to study design and analyzed.

RESULTS

Half of the papers were published within the last 3 years alone. Whereas in the earlier half, most of the publications involved experiments in virtual reality simulation and the efficacy of skills acquisition, many of the more recent publication are proof-of-concept studies. This attests to the evolution of mixed reality in neurosurgery. As the technology advances, neurosurgeons are finding more applications, both in training and clinical practice.

CONCLUSIONS

With parallel advancement in Internet speed and artificial intelligence, the utilization of mixed reality will permeate neurosurgery. From solving staff problems in global neurosurgery, to mitigating the deleterious effect of duty-hour reductions, to improving individual operations, mixed reality will have a positive effect in many aspects of neurosurgery.