Indocyanine Green Angiography Visualized by Augmented Reality in Aneurysm Surgery

A decade of progress: bringing mixed reality image-guided surgery systems in the operating room

Advancements in mixed reality (MR) have led to innovative approaches in image-guided surgery (IGS). In this paper, we provide a comprehensive analysis of the current state of MR in image-guided procedures across various surgical domains. Using the Data Visualization View (DVV) Taxonomy, we analyze the progress made since a 2013 literature review paper on MR IGS systems. In addition to examining the current surgical domains using MR systems, we explore trends in types of MR hardware used, type of data visualized, visualizations of virtual elements, and interaction methods in use. Our analysis also covers the metrics used to evaluate these systems in the operating room (OR), both qualitative and quantitative assessments, and clinical studies that have demonstrated the potential of MR technologies to enhance surgical workflows and outcomes. We also address current challenges and future directions that would further establish the use of MR in IGS.

Augmented reality technology shortens aneurysm surgery learning curve for residents

OBJECTIVES

We aimed to prospectively investigate the benefit of using augmented reality (AR) for surgery residents learning aneurysm surgery.

MATERIALS AND METHODS

Eight residents were included, and divided into an AR group and a control group (4 in each group). Both groups were asked to locate an aneurysm with a blue circle on the same screenshot after their viewing of surgery videos from both AR and non-AR tests. Only the AR group was allowed to inspect and manipulate an AR holographic representation of the aneurysm in AR tests. The actual location of the aneurysm was defined by a yellow circle by an attending physician after each test. Localization deviation was determined by the distance between the blue and yellow circle.

RESULTS

Localization deviation was lower in the AR group than in the control group in the last 2 tests (AR Test 2: 2.7 ± 1.0 mm vs. 5.8 ± 4.1 mm, p = 0.01, non-AR Test 2: 2.1 ± 0.8 mm vs. 5.9 ± 5.8 mm, p < 0.001). The mean deviation was lower in non-AR Test 2 as compared to non-AR Test 1 in both groups (AR: p < 0.001, control: p = 0.391). The localization deviation of the AR group decreased from 8.1 ± 3.8 mm in Test 2 to 2.7 ± 1.0 mm in AR Test 2 (p < 0.001).

CONCLUSION

AR technology provides an effective and interactive way for neurosurgery training, and shortens the learning curve for residents in aneurysm surgery.

Enabling Navigation and Augmented Reality in the Sitting Position in Posterior Fossa Surgery Using Intraoperative Ultrasound

Despite its broad use in cranial and spinal surgery, navigation support and microscope-based augmented reality (AR) have not yet found their way into posterior fossa surgery in the sitting position. While this position offers surgical benefits, navigation accuracy and thereof the use of navigation itself seems limited. Intraoperative ultrasound (iUS) can be applied at any time during surgery, delivering real-time images that can be used for accuracy verification and navigation updates. Within this study, its applicability in the sitting position was assessed. Data from 15 patients with lesions within the posterior fossa who underwent magnetic resonance imaging (MRI)-based navigation-supported surgery in the sitting position were retrospectively analyzed using the standard reference array and new rigid image-based MRI-iUS co-registration. The navigation accuracy was evaluated based on the spatial overlap of the outlined lesions and the distance between the corresponding landmarks in both data sets, respectively. Image-based co-registration significantly improved (p < 0.001) the spatial overlap of the outlined lesion (0.42 ± 0.30 vs. 0.65 ± 0.23) and significantly reduced (p < 0.001) the distance between the corresponding landmarks (8.69 ± 6.23 mm vs. 3.19 ± 2.73 mm), allowing for the sufficient use of navigation and AR support. Navigated iUS can therefore serve as an easy-to-use tool to enable navigation support for posterior fossa surgery in the sitting position.

ICGA combined with EP monitoring in microclipping of cerebral aneurysms

Cerebral aneurysm is one of the common cerebrovascular diseases in neurosurgery, and rupture of cerebral aneurysm is the most important cause of spontaneous subarachnoid hemorrhage. How to precisely clip the aneurysm has been a topic worth discussing, so the authors explore the value of ICGA combined with electrophysiological monitoring in the microclipping of cerebral aneurysms. Using the method of retrospective analysis of cases, 661 patients with cerebral aneurysms admitted to the Department of Neurosurgery, Zhongnan Hospital of Wuhan University, from 2021.8 to 2022.10 were studied, 390 patients with aneurysm clipping were included, and patients with Hunt-Hess classification ≥ 4 were excluded, and whether to use ICGA combined with EP in microclipping of the ruptured and unruptured aneurysm in pterional approach was investigated at the time of discharge, respectively. The MRS and total hospital days were compared to investigate the value of ICGA combined with EP in the microclipping of cerebral aneurysms. All 390 patients enrolled in the group had successful aneurysm clipping, 178 patients were screened for ruptured aneurysm pterional approach and 120 patients for unruptured aneurysm pterional approach access; the MRS at discharge was significantly lower in the ICGA combined with EP group than in the no-EP group for ruptured aneurysm pterional approach microclipping (p < 0.001), and the mean number of days in hospital was significantly lower (p < 0.01). Patients in the ICGA combined with EP group in microclipping of unruptured aneurysms with pterional approach also had significantly lower MRS at discharge compared with patients in the ICGA alone group (p < 0.001), with no statistically significant difference in the mean number of days in hospital (p = 0.09). In open cerebral aneurysm microclipping, ICGA combined with EP monitoring for both ruptured and unruptured aneurysms can effectively reduce the false-negative rate of ICGA, significantly reduce the incidence of postoperative neurological deficits, and shorten the total hospital stay to some extent. ICGA combined with EP monitoring may be an effective means to reduce the rate of false clipping of the penetrating vessels and to avoid stenosis or occlusion of the aneurysm-carrying artery and is worth promoting in microclipping of cerebral aneurysms except for Hunt-Hess ≥ 4.

Head-Mounted Augmented Reality in the Planning of Cerebrovascular Neurosurgical Procedures: A Single-Center Initial Experience

BACKGROUND

Augmented reality (AR) technology has played an increasing role in cerebrovascular neurosurgery over the last 2 decades. Hence, we aim to evaluate the technical and educational value of head-mounted AR in cerebrovascular procedures.

METHODS

This is a single-center retrospective study of patients who underwent open surgery for cranial and spinal cerebrovascular lesions between April and August 2022. In all cases, the Medivis Surgical AR platform and HoloLens 2 were used for preoperative and intraoperative (preincision) planning. Surgical plan adjustment due to the use of head-mounted AR and subjective educational value of the tool were recorded.

RESULTS

A total of 33 patients and 35 cerebrovascular neurosurgical procedures were analyzed. Procedures included 12 intracranial aneurysm clippings, 6 brain and 1 spinal arteriovenous malformation resections, 2 cranial dural arteriovenous fistula obliterations, 3 carotid endarterectomies, two extracranial-intracranial direct bypasses, two encephaloduroangiosynostosis for Moyamoya disease, 1 biopsy of the superficial temporal artery, 2 microvascular decompressions, 2 cavernoma resections, 1 combined intracranial aneurysm clipping and encephaloduroangiosynostosis for Moyamoya disease, and 1 percutaneous feeder catheterization for arteriovenous malformation embolization. Minor changes in the surgical plan were recorded in 16 of 35 procedures (45.7%). Subjective educational value was scored as "very helpful" for cranial, spinal arteriovenous malformations, and carotid endarterectomies; "helpful" for intracranial aneurysm, dural arteriovenous fistulas, direct bypass, encephaloduroangiosynostosis, and superficial temporal artery-biopsy; and "not helpful" for cavernoma resection and microvascular decompression.

CONCLUSIONS

Head-mounted AR can be used in cerebrovascular neurosurgery as an adjunctive tool that might influence surgical strategy, enable 3-dimensional understanding of complex anatomy, and provide great educational value in selected cases.

Augmented and virtual reality usage in awake craniotomy: a systematic review

Augmented and virtual reality (AR, VR) are becoming promising tools in neurosurgery. AR and VR can reduce challenges associated with conventional approaches via the simulation and mimicry of specific environments of choice for surgeons. Awake craniotomy (AC) enables the resection of lesions from eloquent brain areas while monitoring higher cortical and subcortical functions. Evidence suggests that both surgeons and patients benefit from the various applications of AR and VR in AC. This paper investigates the application of AR and VR in AC and assesses its prospective utility in neurosurgery. A systematic review of the literature was performed using PubMed, Scopus, and Web of Science databases in accordance with the PRISMA guidelines. Our search results yielded 220 articles. A total of six articles consisting of 118 patients have been included in this review. VR was used in four papers, and the other two used AR. Tumour was the most common pathology in 108 patients, followed by vascular lesions in eight patients. VR was used for intraoperative mapping of language, vision, and social cognition, while AR was incorporated in preoperative training of white matter dissection and intraoperative visualisation and navigation. Overall, patients and surgeons were satisfied with the applications of AR and VR in their cases. AR and VR can be safely incorporated during AC to supplement, augment, or even replace conventional approaches in neurosurgery. Future investigations are required to assess the feasibility of AR and VR in various phases of AC.

Augmented Reality to Compensate for Navigation Inaccuracies

This study aims to report on the capability of microscope-based augmented reality (AR) to evaluate registration and navigation accuracy with extracranial and intracranial landmarks and to elaborate on its opportunities and obstacles in compensation for navigation inaccuracies. In a consecutive single surgeon series of 293 patients, automatic intraoperative computed tomography-based registration was performed delivering a high initial registration accuracy with a mean target registration error of 0.84 ± 0.36 mm. Navigation accuracy is evaluated by overlaying a maximum intensity projection or pre-segmented object outlines within the recent focal plane onto the in situ patient anatomy and compensated for by translational and/or rotational in-plane transformations. Using bony landmarks (85 cases), there was two cases where a mismatch was seen. Cortical vascular structures (242 cases) showed a mismatch in 43 cases and cortex representations (40 cases) revealed two inaccurate cases. In all cases, with detected misalignment, a successful spatial compensation was performed (mean correction: bone (6.27 ± 7.31 mm), vascular (3.00 ± 1.93 mm, 0.38° ± 1.06°), and cortex (5.31 ± 1.57 mm, 1.75° ± 2.47°)) increasing navigation accuracy. AR support allows for intermediate and straightforward monitoring of accuracy, enables compensation of spatial misalignments, and thereby provides additional safety by increasing overall accuracy.

A systematic comparison of the accuracy of monocular RGB tracking and LiDAR for neuronavigation

With the advent of augmented reality (AR), the use of AR-guided systems in the field of medicine has gained traction. However, the wide-scale adaptation of these systems requires highly accurate and reliable tracking. In this work, the tracking accuracy of two technology platforms, LiDAR and Vuforia, are developed and rigorously tested for a catheter placement neurological procedure. Several experiments (900) are performed for each technology across various combinations of catheter lengths and insertion trajectories. This analysis shows that the LiDAR platform outperformed Vuforia; which is the state-of-the-art in monocular RGB tracking solutions. LiDAR had 75% less radial distance error and 26% less angle deviation error. Results provide key insights into the value and utility of LiDAR-based tracking in AR guidance systems.

Augmented Reality in Surgery: A Scoping Review

Augmented reality (AR) is an innovative system that enhances the real world by superimposing virtual objects on reality. The aim of this study was to analyze the application of AR in medicine and which of its technical solutions are the most used. We carried out a scoping review of the articles published between 2019 and February 2022. The initial search yielded a total of 2649 articles. After applying filters, removing duplicates and screening, we included 34 articles in our analysis. The analysis of the articles highlighted that AR has been traditionally and mainly used in orthopedics in addition to maxillofacial surgery and oncology. Regarding the display application in AR, the Microsoft HoloLens Optical Viewer is the most used method. Moreover, for the tracking and registration phases, the marker-based method with a rigid registration remains the most used system. Overall, the results of this study suggested that AR is an innovative technology with numerous advantages, finding applications in several new surgery domains. Considering the available data, it is not possible to clearly identify all the fields of application and the best technologies regarding AR.

Current status of augmented reality in cerebrovascular surgery: a systematic review

Augmented reality (AR) is an adjuvant tool in neuronavigation to improve spatial and anatomic understanding. The present review aims to describe the current status of intraoperative AR for the treatment of cerebrovascular pathology. A systematic review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The following databases were searched: PubMed, Science Direct, Web of Science, and EMBASE up to December, 2020. The search strategy consisted of "augmented reality," "AR," "cerebrovascular," "navigation," "neurovascular," "neurosurgery," and "endovascular" in both AND and OR combinations. Studies included were original research articles with intraoperative application. The manuscripts were thoroughly examined for study design, outcomes, and results. Sixteen studies were identified describing the use of intraoperative AR in the treatment of cerebrovascular pathology. A total of 172 patients were treated for 190 cerebrovascular lesions using intraoperative AR. The most common treated pathology was intracranial aneurysms. Most studies were cases and there was only a case-control study. A head-up display system in the microscope was the most common AR display. AR was found to be useful for tailoring the craniotomy, dura opening, and proper identification of donor and recipient vessels in vascular bypass. Most AR systems were unable to account for tissue deformation. This systematic review suggests that intraoperative AR is becoming a promising and feasible adjunct in the treatment of cerebrovascular pathology. It has been found to be a useful tool in the preoperative planning and intraoperative guidance. However, its clinical benefits remain to be seen.

Augmented reality visualization in brain lesions: a prospective randomized controlled evaluation of its potential and current limitations in navigated microneurosurgery

Background

Augmented reality (AR) has the potential to support complex neurosurgical interventions by including visual information seamlessly. This study examines intraoperative visualization parameters and clinical impact of AR in brain tumor surgery.

Methods

Fifty-five intracranial lesions, operated either with AR-navigated microscope (n = 39) or conventional neuronavigation (n = 16) after randomization, have been included prospectively. Surgical resection time, duration/type/mode of AR, displayed objects (n, type), pointer-based navigation checks (n), usability of control, quality indicators, and overall surgical usefulness of AR have been assessed.

Results

AR display has been used in 44.4% of resection time. Predominant AR type was navigation view (75.7%), followed by target volumes (20.1%). Predominant AR mode was picture-in-picture (PiP) (72.5%), followed by 23.3% overlay display. In 43.6% of cases, vision of important anatomical structures has been partially or entirely blocked by AR information. A total of 7.7% of cases used MRI navigation only, 30.8% used one, 23.1% used two, and 38.5% used three or more object segmentations in AR navigation. A total of 66.7% of surgeons found AR visualization helpful in the individual surgical case. AR depth information and accuracy have been rated acceptable (median 3.0 vs. median 5.0 in conventional neuronavigation). The mean utilization of the navigation pointer was 2.6 × /resection hour (AR) vs. 9.7 × /resection hour (neuronavigation); navigation effort was significantly reduced in AR (P < 0.001).

Conclusions

The main benefit of HUD-based AR visualization in brain tumor surgery is the integrated continuous display allowing for pointer-less navigation. Navigation view (PiP) provides the highest usability while blocking the operative field less frequently. Visualization quality will benefit from improvements in registration accuracy and depth impression.

German clinical trials registration number.

DRKS00016955.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00701-021-05045-1.