Augmented reality guidance in cerebrovascular surgery using microscopic video enhancement

Evolution of the meta-neurosurgeon: A systematic review of the current technical capabilities, limitations, and applications of augmented reality in neurosurgery

Background

Augmented reality (AR) applications in neurosurgery have expanded over the past decade with the introduction of headset-based platforms. Many studies have focused on either preoperative planning to tailor the approach to the patient's anatomy and pathology or intraoperative surgical navigation, primarily realized as AR navigation through microscope oculars. Additional efforts have been made to validate AR in trainee and patient education and to investigate novel surgical approaches. Our objective was to provide a systematic overview of AR in neurosurgery, provide current limitations of this technology, as well as highlight several applications of AR in neurosurgery.

Methods

We performed a literature search in PubMed/Medline to identify papers that addressed the use of AR in neurosurgery. The authors screened three hundred and seventy-five papers, and 57 papers were selected, analyzed, and included in this systematic review.

Results

AR has made significant inroads in neurosurgery, particularly in neuronavigation. In spinal neurosurgery, this primarily has been used for pedicle screw placement. AR-based neuronavigation also has significant applications in cranial neurosurgery, including neurovascular, neurosurgical oncology, and skull base neurosurgery. Other potential applications include operating room streamlining, trainee and patient education, and telecommunications.

Conclusion

AR has already made a significant impact in neurosurgery in the above domains and has the potential to be a paradigm-altering technology. Future development in AR should focus on both validating these applications and extending the role of AR.

Intraoperative Augmented Reality for Complex Glioma Resection: A Case Report

Augmented reality (AR) is an emerging technology that can display three-dimensional patient anatomy in the surgeons' field of view. The use of this technology has grown considerably for both presurgical and intraoperative guidance. A patient diagnosed with breast cancer started to experience numbness in the left hand, which progressed to weakness in the left hand and arm. An MRI was performed demonstrating a 2.9 cm X 1.8 cm lesion with extensive surrounding edema in the posterior fronto-parietal lobes. Surgery was recommended for presumed metastatic disease. Preoperatively, an AR system and Brainlab navigation were registered to the patient. AR, traditional navigation, and ultrasound were all used to localize the lesion and determine the craniotomy site and size. The tumor was removed along the direction of the lesion. Intraoperatively, we used AR to reexamine the tumor details and could appreciate that we had to redirect our surgical trajectory anteriorly and laterally in order to follow along the main axis of the tumor. In doing this, we were able to more confidently remain with the tumor, which by this time was poorly defined by 2D navigation and by direct vision. Postoperative MRI confirmed gross total removal of the tumor. The patient had an uneventful postoperative course with resolution of preoperative symptoms and the final surgical pathology was grade 4 glioblastoma. Here, we describe the valuable use of AR for the resection of a glioma. The system has a seamless registration process and provides the surgeon with a unique view of 3D anatomy overlaid onto the patient's head. This exciting technology can add tremendous value to complex cranial surgeries.

Augmented Reality Registration System for Visualization of Skull Landmarks

BACKGROUND

Augmented reality (AR) is an emerging technology in neurosurgery with the potential to become a strategic tool in the delivery of care and education for trainees. Advances in technology have demonstrated promising use for improving visualization and spatial awareness of critical neuroanatomic structures. In this report, we employ a novel AR registration system for the visualization and targeting of skull landmarks.

METHODS

A markerless AR system was used to register 3-dimensional reconstructions of suture lines onto the head via a head-mounted display. Participants were required to identify craniometric points with and without AR assistance. Targeting error was measured as the Euclidian distance between the user-defined location and the true craniometric point on the subjects' heads.

RESULTS

All participants successfully registered 3-dimensional reconstructions onto the subjects' heads. Targeting accuracy was significantly improved with AR (3.59 ± 1.29 mm). Across all target points, AR increased accuracy by an average of 19.96 ± 3.80 mm. Posttest surveys revealed that participants felt the technology increased their confidence in identifying landmarks (4.6/5) and that the technology will be useful for clinical care (4.2/5).

CONCLUSIONS

While several areas of improvement and innovation can further enhance the use of AR in neurosurgery, this report demonstrates the feasibility of a markerless headset-based AR system for visualizing craniometric points on the skull. As the technology continues to advance, AR is expected to play an increasingly significant role in neurosurgery, transforming how surgeries are performed and improving patient care.

Augmented Reality-Assisted Placement of Ommaya Reservoir for Cyst Aspiration: A Case Report

Image guidance technologies can significantly improve the accuracy and safety of intracranial catheter insertions. Augmented reality (AR) allows surgeons to visualize 3D information overlaid onto a patient's head. As such, AR has emerged as a novel image guidance technology that offers unique advantages when navigating intracranial targets. A 71-year-old woman with a history of brain metastasis from breast cancer and prior resection surgery and chemotherapy presented with altered mental status and generalized weakness worse on her left side. Magnetic resonance imaging (MRI) demonstrated right frontotemporoparietal edema with a contrast-enhancing mass. MR perfusion confirmed an active tumor with an enlarging right temporal pole cyst. A cyst aspiration was performed via Ommaya reservoir placement. Neuro-navigation (BrainLab, Munich, Germany) and AR navigation were used to plan the trajectory from the temporal gyrus to the cyst. Post-operative computed tomography (CT) demonstrated good placement of the reservoir, reconstitution of the temporal horn of the lateral ventricle with decreased external mass effect, and no areas of hemorrhage. AR has tremendous potential in the field of neurosurgery for improving the accuracy and safety of procedures. This case demonstrates an encouraging application of AR and can serve as an example to drive expanded clinical use of this technology.

Comparison Between the Stereoscopic Virtual Reality Display System and Conventional Computed Tomography Workstation in the Diagnosis and Characterization of Cerebral Arteriovenous Malformations

It is difficult to accurately understand the angioarchitecture of cerebral arteriovenous malformations (CAVMs) before surgery using existing imaging methods. This study aimed to evaluate the ability of the stereoscopic virtual reality display system (SVRDS) to display the angioarchitecture of CAVMs by comparing its accuracy with that of the conventional computed tomography workstation (CCTW). Nineteen patients with CAVM confirmed on digital subtraction angiography (DSA) or during surgery were studied. Computed tomography angiography images in the SVRDS and CCTW were retrospectively analyzed by two experienced neuroradiologists using a double-blind method. Angioarchitectural parameters, such as the location and size of the nidus, type and number of the arterial feeders and draining veins, and draining pattern of the vessels, were recorded and compared. The diameter of the nidus ranged from 1.1 to 9 cm. Both CCTW and SVRDS correctly diagnosed the location of the nidus in 19 patients with CAVM. Among the 19 patients, 35 arterial feeders and 25 draining veins were confirmed on DSA and during surgery. With the DSA and intraoperative results as the gold standard bases, the CCTW misjudged one arterial feeder and one draining vein and missed three arterial feeders and two draining veins; meanwhile, the SVRDS missed only two arterial feeders. SVRDS had some advantages in displaying nidus, arterial branches, and draining veins of the CAVM compared with CCTW, as well as SVRDS could more intuitively display the overall angio-architectural spatial picture of CAVM.

Augmented reality–assisted microsurgical resection of brain arteriovenous malformations: illustrative case

BACKGROUND

Arteriovenous malformations (AVMs) of the brain are vessel conglomerates of feeding arteries and draining veins that carry a risk of spontaneous and intraoperative rupture. Augmented reality (AR)-assisted neuronavigation permits continuous, real-time, updated visualization of navigation information through a heads-up display, thereby potentially improving the safety of surgical resection of AVMs.

OBSERVATIONS

The authors report a case of a 37-year-old female presenting with a 2-year history of recurrent falls due to intermittent right-sided weakness and increasing clumsiness in the right upper extremity. Magnetic resonance imaging, magnetic resonance angiography, and cerebral angiography of the brain revealed a left parietal Spetzler-Martin grade III AVM. After endovascular embolization of the AVM, microsurgical resection using an AR-assisted neuronavigation system was performed. Postoperative angiography confirmed complete obliteration of arteriovenous shunting. The postsurgical course was unremarkable, and the patient remains in excellent health.

LESSONS

Our case describes the operative setup and intraoperative employment of AR-assisted neuronavigation for AVM resection. Application of this technology may improve workflow and enhance patient safety.

Presurgical selection of the ideal aneurysm clip by the use of a three-dimensional planning system

Aneurysm occlusion rate after clipping is higher than after endovascular treatment. However, a certain percentage of incompletely clipped aneurysms remains. Presurgical selection of the proper aneurysm clips could potentially reduce the rate of incomplete clippings caused by inadequate clip geometry. The aim of the present study was to assess whether preoperative 3D image-based simulation allows for preoperative selection of a proper aneurysm clip for complete occlusion in individual cases. Patients harboring ruptured or unruptured cerebral aneurysms prior to surgical clipping were analyzed. CT angiography images were transferred to a 3D surgical-planning station (Dextroscope®) with imported models of 58 aneurysm clips. Intracranial vessels and aneurysms were segmented and the virtual aneurysm clips were placed at the aneurysm neck. Operating surgeons had information about the selected aneurysm clip, and patients underwent clipping. Intraoperative clip selection was documented and aneurysm occlusion rate was assessed by postoperative digital subtraction angiography. Nineteen patients were available for final analysis. In all patients, the most proximal clip at the aneurysm neck was the preselected clip. All aneurysms except one were fully occluded, as assessed by catheter angiography. One aneurysm had a small neck remnant that did not require secondary surgery and was occluded 15 months after surgery. 3D image-based preselection of a proper aneurysm clip can be translated to the operating room and avoids intraoperative clip selection. The associated occlusion rate of aneurysms is high.

Augmented reality surgical navigation system based on the spatial drift compensation method for glioma resection surgery

BACKGROUND

The number of patients who suffer from glioma has been increasing, and this malignancy is a serious threat to human health. The mainstream treatment for glioma is surgical resection; therefore, accurate resection can improve postoperative patient recovery.

PURPOSE

Many studies have investigated surgical navigation guided by mixed reality, with good outcomes. However, the limitations of mixed reality, such as spatial drift caused by environmental changes, limit its clinical application. Therefore, we present a mixed reality surgical navigation system for glioma resection. Preoperative information can be fused precisely with the real patient with the spatial compensation method to achieve clinically suitable accuracy.

METHODS

A head-mounted device was used to display virtual information, and a markerless spatial registration method was applied to precisely align the virtual anatomy with the real patient preoperatively. High-accuracy preoperative and intraoperative movement and spatial drift compensation methods were used to increase the positional accuracy of the mixed reality-guided glioma resection system when the patient's head is fixed to the bed frame. Several experiments were designed to validate the accuracy and efficacy of this system.

RESULTS

Phantom experiments were performed to test the efficacy and accuracy of this system under ideal conditions, and clinical tests were conducted to assess the performance of this system in clinical application. The accuracy of spatial registration was 1.18 mm in the phantom experiments and 1.86 mm in the clinical application.

CONCLUSIONS

Herein, we present a mixed reality-based multimodality fused surgical navigation system for assisting surgeons in intuitively identifying the glioma boundary intraoperatively. The experimental results indicate that this system has suitable accuracy and efficacy for clinical usage. This article is protected by copyright. All rights reserved.

Examining the benefits of extended reality in neurosurgery: A systematic review

While well-established in other surgical subspecialties, the benefits of extended reality, consisting of virtual reality (VR), augmented reality (AR), and mixed reality (MR) technologies, remains underexplored in neurosurgery despite its increasing utilization. To address this gap, we conducted a systematic review of the effects of extended reality (XR) in neurosurgery with an emphasis on the perioperative period, to provide a guide for future clinical optimization. Seven primary electronic databases were screened following guidelines outlined by PRISMA and the Institute of Medicine. Reported data related to outcomes in the perioperative period and resident training were all examined, and a focused analysis of studies reporting controlled, clinical outcomes was completed. After removal of duplicates, 2548 studies were screened with 116 studies reporting measurable effects of XR in neurosurgery. The majority (82%) included cranial based applications related to tumor surgery with 34% showing improved resection rates and functional outcomes. A rise in high-quality studies was identified from 2017 to 2020 compared to all previous years (p = 0.004). Primary users of the technology were: 56% neurosurgeon (n = 65), 28% residents (n = 33) and 5% patients (n = 6). A final synthesis was conducted on 10 controlled studies reporting patient outcomes. XR technologies have demonstrated benefits in preoperative planning and multimodal neuronavigation especially for tumor surgery. However, few studies have reported patient outcomes in a controlled design demonstrating a need for higher quality data. XR platforms offer several advantages to improve patient outcomes and specifically, the patient experience for neurosurgery.

Augmented reality head-mounted display-based incision planning in cranial neurosurgery: a prospective pilot study

OBJECTIVE

Monitor and wand-based neuronavigation stations (MWBNSs) for frameless intraoperative neuronavigation are routinely used in cranial neurosurgery. However, they are temporally and spatially cumbersome; the OR must be arranged around the MWBNS, at least one hand must be used to manipulate the MWBNS wand (interrupting a bimanual surgical technique), and the surgical workflow is interrupted as the surgeon stops to "check the navigation" on a remote monitor. Thus, there is need for continuous, real-time, hands-free, neuronavigation solutions. Augmented reality (AR) is poised to streamline these issues. The authors present the first reported prospective pilot study investigating the feasibility of using the OpenSight application with an AR head-mounted display to map out the borders of tumors in patients undergoing elective craniotomy for tumor resection, and to compare the degree of correspondence with MWBNS tracing.

METHODS

Eleven consecutive patients undergoing elective craniotomy for brain tumor resection were prospectively identified and underwent circumferential tumor border tracing at the time of incision planning by a surgeon wearing HoloLens AR glasses running the commercially available OpenSight application registered to the patient and preoperative MRI. Then, the same patient underwent circumferential tumor border tracing using the StealthStation S8 MWBNS. Postoperatively, both tumor border tracings were compared by two blinded board-certified neurosurgeons and rated as having an excellent, adequate, or poor correspondence degree based on a subjective sense of the overlap. Objective overlap area measurements were also determined.

RESULTS

Eleven patients undergoing craniotomy were included in the study. Five patient procedures were rated as having an excellent correspondence degree, 5 had an adequate correspondence degree, and 1 had poor correspondence. Both raters agreed on the rating in all cases. AR tracing was possible in all cases.

CONCLUSIONS

In this small pilot study, the authors found that AR was implementable in the workflow of a neurosurgery OR, and was a feasible method of preoperative tumor border identification for incision planning. Future studies are needed to identify strategies to improve and optimize AR accuracy.

Applications of augmented reality in the neurosurgical operating room: A systematic review of the literature

Advancements in imaging techniques are key forces of progress in neurosurgery. The importance of accurate visualization of intraoperative anatomy cannot be overemphasized and is commonly delivered through traditional neuronavigation. Augmented Reality (AR) technology has been tested and applied widely in various neurosurgical subspecialties in intraoperative, clinical use and shows promise for the future. This systematic review of the literature explores the ways in which AR technology has been successfully brought into the operating room (OR) and incorporated into clinical practice. A comprehensive literature search was performed in the following databases from inception-April 2020: Ovid MEDLINE, Ovid EMBASE, and The Cochrane Library. Studies retrieved were then screened for eligibility against predefined inclusion/exclusion criteria. A total of 54 articles were included in this systematic review. The studies were sub- grouped into brain and spine subspecialties and analyzed for their incorporation of AR in the neurosurgical clinical setting. AR technology has the potential to greatly enhance intraoperative visualization and guidance in neurosurgery beyond the traditional neuronavigation systems. However, there are several key challenges to scaling the use of this technology and bringing it into standard operative practice including accurate and efficient brain segmentation of magnetic resonance imaging (MRI) scans, accounting for brain shift, reducing coregistration errors, and improving the AR device hardware. There is also an exciting potential for future work combining AR with multimodal imaging techniques and artificial intelligence to further enhance its impact in neurosurgery.

Microscope integrated optical coherence tomography system combined with augmented reality

One of the disadvantages in microscope-integrated optical coherence tomography (MI-OCT) systems is that medical images acquired via different modalities are usually displayed independently. Hence, surgeons have to match two-dimensional and three-dimensional images of the same operative region subjectively. In this paper, we propose a simple registration method to overcome this problem by using guided laser points. This method combines augmented reality with an existing MI-OCT system. The basis of our idea is to introduce a guiding laser into the system, which allows us to identify fiducials in microscopic images. At first, the applied voltages of the scanning galvanometer mirror are used to calculate the fiducials' coordinates in an OCT model. After gathering data at the corresponding points' coordinates, the homography matrix and camera parameters are used to superimpose a reconstructed model on microscopic images. After performing experiments with artificial and animal eyes, we successfully obtain two-dimensional microscopic images of scanning regions with depth information. Moreover, the registration error is 0.04 mm, which is within the limits of medical and surgical errors. Our proposed method could have many potential applications in ophthalmic procedures.

Evaluation of the precision of operative augmented reality compared to standard neuronavigation using a 3D-printed skull

OBJECTIVE

Augmented reality (AR) in cranial surgery allows direct projection of preregistered overlaid images in real time on the microscope surgical field. In this study, the authors aimed to compare the precision of AR-assisted navigation and standard pointer-based neuronavigation (NV) by using a 3D-printed skull in surgical conditions.

METHODS

A commercial standardized 3D-printed skull was scanned, fused, and referenced with an MR image and a CT scan of a patient with a 2 × 2-mm right frontal sinus defect. The defect was identified, registered, and integrated into NV. The target was physically marked on the 3D-printed skull replicating the right frontal sinus defect. Twenty-six subjects participated, 25 of whom had no prior NV or AR experience and 1 with little AR experience. The subjects were briefly trained in how to use NV, AR, and AR recalibration tools. Participants were asked to do the following: 1) "target the center of the defect in the 3D-printed skull with a navigation pointer, assisted only by NV orientation," and 2) "use the surgical microscope and AR to focus on the center of the projected object" under conventional surgical conditions. For the AR task, the number of recalibrations was recorded. Confidence regarding NV and AR precision were assessed prior to and after the experiment by using a 9-level Likert scale.

RESULTS

The median distance to target was statistically lower for AR than for NV (1 mm [Q1: 1 mm, Q3: 2 mm] vs 3 mm [Q1: 2 mm, Q3: 4 mm] [p < 0.001]). In the AR task, the median number of recalibrations was 4 (Q1: 4, Q3: 4.75). The number of recalibrations was significantly correlated with the precision (Spearman rho: -0.71, p < 0.05). The trust assessment after performing the experiment scored a median of 8 for AR and 5.5 for NV (p < 0.01).

CONCLUSIONS

This study shows for the first time the superiority of AR over NV in terms of precision. AR is easy to use. The number of recalibrations performed using reference structures increases the precision of the navigation. The confidence regarding precision increases with experience.

Augmented reality in neurosurgical navigation: a survey

BACKGROUND

Neurosurgery has exceptionally high requirements for minimally invasive and safety. This survey attempts to analyze the practical application of AR in neurosurgical navigation. Also, this survey describes future trends in augmented reality neurosurgical navigation systems.

METHODS

In this survey, we searched related keywords "augmented reality", "virtual reality", "neurosurgery", "surgical simulation", "brain tumor surgery", "neurovascular surgery", "temporal bone surgery", and "spinal surgery" through Google Scholar, World Neurosurgery, PubMed and Science Direct. We collected 85 articles published over the past five years in areas related to this survey.

RESULTS

Detailed study has been conducted on the application of AR in neurosurgery and found that AR is constantly improving the overall efficiency of doctor training and treatment, which can help neurosurgeons learn and practice surgical procedures with zero risks.

CONCLUSIONS

Neurosurgical navigation is essential in neurosurgery. Despite certain technical limitations, it is still a necessary tool for the pursuit of maximum security and minimal intrusiveness. This article is protected by copyright. All rights reserved.

Applicability of Augmented Reality in an Organ Transplantation

Augmented reality (AR) delivers virtual information or some of its elements to the real world. This technology, which has been used primarily for entertainment and military applications, has vigorously entered medicine, especially in radiology and surgery, yet has never been used in organ transplantation. AR could be useful in training transplant surgeons, promoting organ donations, graft retrieval and allocation, and microscopic diagnosis of rejection, treatment of complications, and post-transplantation neoplasms. The availability of AR display tools such as Smartphone screens and head-mounted goggles, accessibility of software for automated image segmentation and 3-dimensional reconstruction, and algorithms allowing registration, make augmented reality an attractive tool for surgery including transplantation. The shortage of hospital IT specialists and insufficient investments from medical equipment manufacturers into the development of AR technology remain the most significant obstacles in its broader application.

Determining blood flow direction from short neurovascular surgical microscope videos

Neurovascular surgery aims to repair diseased or damaged blood vessels in the brain or spine. There are numerous procedures that fall under this category, and in all of them, the direction of blood flow through these vessels is crucial information. Current methods to determine this information intraoperatively include static pre-operative images combined with augmented reality, Doppler ultrasound, and injectable fluorescent dyes. Each of these systems has inherent limitations. This study includes the proposal and preliminary validation of a technique to identify the direction of blood flow through vessels using only video segments of a few seconds acquired from routinely used surgical microscopes. The video is enhanced to reveal subtle colour fluctuations related to blood pulsation, and these rhythmic signals are further analysed in Fourier space to reveal the direction of blood flow. The proposed method was validated using a novel physical phantom and retrospective analysis of surgical videos and demonstrated high accuracy in identifying the direction of blood flow.

Workflow Caveats in Augmented Reality-Assisted Pedicle Instrumentation: Cadaver Lab

BACKGROUND

Augmented reality (AR) is gaining popularity in gaming, industrial, military, and medical fields. Neurosurgical applications are currently limited and underdeveloped.

METHODS

The cadaver lab session was prepared with the currently available AR equipment and software. Pedicle instrumentation was performed from thoracic 1 to the pelvis with either fluoroscopy or AR.

RESULTS

A total of 38 screws were placed. There were no major breaches on the fluoroscopy-assisted side. Among the AR screws, 3 had a major medial breach and 4 had a major inferior breach. The cause of a 3 breaches appeared to be related to an error in the starting position, as their overall orientation remained correctly parallel to the original trajectory.

CONCLUSIONS

The article discusses the potential and limitations of AR in its current state and identifies strategies for successful AR application in future surgery.