Augmented reality neurosurgical planning and navigation for surgical excision of parasagittal, falcine and convexity meningiomas

Evaluation Metrics for Augmented Reality in Neurosurgical Preoperative Planning, Surgical Navigation, and Surgical Treatment Guidance: A Systematic Review

BACKGROUND AND OBJECTIVE

Recent years have shown an advancement in the development of augmented reality (AR) technologies for preoperative visualization, surgical navigation, and intraoperative guidance for neurosurgery. However, proving added value for AR in clinical practice is challenging, partly because of a lack of standardized evaluation metrics. We performed a systematic review to provide an overview of the reported evaluation metrics for AR technologies in neurosurgical practice and to establish a foundation for assessment and comparison of such technologies.

METHODS

PubMed, Embase, and Cochrane were searched systematically for publications on assessment of AR for cranial neurosurgery on September 22, 2022. The findings were reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.

RESULTS

The systematic search yielded 830 publications; 114 were screened full text, and 80 were included for analysis. Among the included studies, 5% dealt with preoperative visualization using AR, with user perception as the most frequently reported metric. The majority (75%) researched AR technology for surgical navigation, with registration accuracy, clinical outcome, and time measurements as the most frequently reported metrics. In addition, 20% studied the use of AR for intraoperative guidance, with registration accuracy, task outcome, and user perception as the most frequently reported metrics.

CONCLUSION

For quality benchmarking of AR technologies in neurosurgery, evaluation metrics should be specific to the risk profile and clinical objectives of the technology. A key focus should be on using validated questionnaires to assess user perception; ensuring clear and unambiguous reporting of registration accuracy, precision, robustness, and system stability; and accurately measuring task performance in clinical studies. We provided an overview suggesting which evaluation metrics to use per AR application and innovation phase, aiming to improve the assessment of added value of AR for neurosurgical practice and to facilitate the integration in the clinical workflow.

Augmented reality for intracranial meningioma resection: a mini-review

Augmented reality (AR) integrates computer-generated content and real-world scenarios. Artificial intelligence's continuous development has allowed AR to be integrated into medicine. Neurosurgery has progressively introduced image-guided technologies. Integration of AR into the operating room has permitted a new perception of neurosurgical diseases, not only for neurosurgical planning, patient positioning, and incision design but also for intraoperative maneuvering and identification of critical neurovascular structures and tumor boundaries. Implementing AR, virtual reality, and mixed reality has introduced neurosurgeons into a new era of artificial interfaces. Meningiomas are the most frequent primary benign tumors commonly related to paramount neurovascular structures and bone landmarks. Integration of preoperative 3D reconstructions used for surgical planning into AR can now be inserted into the microsurgical field, injecting information into head-up displays and microscopes with integrated head-up displays, aiming to guide neurosurgeons intraoperatively to prevent potential injuries. This manuscript aims to provide a mini-review of the usage of AR for intracranial meningioma resection.

Iatrogenic inner ear dehiscence associated with lateral skull base surgery: a systematic analysis of drilling injuries and their causal factors

PURPOSE

Drilling injuries of the inner ear are an underreported complication of lateral skull base (LSB) surgery. Inner ear breaches can cause hearing loss, vestibular dysfunction, and third window phenomenon. This study aims to elucidate primary factors causing iatrogenic inner ear dehiscences (IED) in 9 patients who presented to a tertiary care center with postoperative symptoms of IED following LSB surgery for vestibular schwannoma, endolymphatic sac tumor, Meniere's disease, paraganglioma jugulare, and vagal schwannoma.

METHODS

Utilizing 3D Slicer image processing software, geometric and volumetric analysis was applied to both preoperative and postoperative imaging to identify causal factors iatrogenic inner ear breaches. Segmentation analyses, craniotomy analyses, and drilling trajectory analyses were performed. Cases of retrosigmoid approaches for vestibular schwannoma resection were compared to matched controls.

RESULTS

Excessive lateral drilling and breach of a single inner ear structure occurred in 3 cases undergoing transjugular (n=2) and transmastoid (n=1) approaches. Inadequate drilling trajectory breaching ≥1 inner ear structure occurred in 6 cases undergoing retrosigmoid (n=4), transmastoid (n=1), and middle cranial fossa approaches (n=1). In retrosigmoid approaches the 2-cm visualization window and craniotomy limits did not provide drilling angles to the entire tumor without causing IED in comparison to matched controls.

CONCLUSIONS

Inappropriate drill depth, errant lateral drilling, inadequate drill trajectory, or a combination of these led to iatrogenic IED. Image-based segmentation, individualized 3D anatomical model generation, and geometric and volumetric analyses can optimize operative plans and possibly reduce inner ear breaches from lateral skull base surgery.

The use of advanced technology for preoperative planning in cranial surgery - A survey by the EANS Young Neurosurgeons Committee

Introduction

Technological advancements provided several preoperative tools allowing for precise preoperative planning in cranial neurosurgery, aiming to increase the efficacy and safety of surgery. However, little data are available regarding if and how young neurosurgeons are trained in using such technologies, how often they use them in clinical practice, and how valuable they consider these technologies.

Research question

How frequently these technologies are used during training and clinical practice as well as to how their perceived value can be qualitatively assessed.

Materials and methods

The Young Neurosurgeons' Committee (YNC) of the European Association of Neurosurgical Societies (EANS) distributed a 14-items survey among young neurosurgeons between June 1st and August 31st, 2022.

Results

A total of 441 responses were collected. Most responders (42.34%) received "formal" training during their residency. Planning techniques were used mainly in neuro-oncology (90.86%), and 3D visualization of patients' DICOM dataset using open-source software was the most frequently used (>20 times/month, 20.34% of responders). Software for 3D visualization of patients' DICOM dataset was the most valuable technology, especially for planning surgical approach (42.03%). Conversely, simulation based on augmented/mixed/virtual reality was considered the less valuable tool, being rated below sufficiency by 39.7% of responders.

Discussion and conclusion

Training for using preoperative planning technologies in cranial neurosurgery is provided by neurosurgical residency programs. Software for 3D visualization of DICOM datasets is the most valuable and used tool, especially in neuro-oncology. Interestingly, simulation tools based on augmented/virtual/mixed reality are considered less valuable and, therefore, less used than other technologies.

Utility of MRI in surgical planning for parasagittal meningiomas

BACKGROUND

Surgical resection is the standard treatment for parasagittal meningioma (PSM), but complete resection may be challenging due to superior sagittal sinus (SSS) involvement. The SSS may be partially or completely obstructed, and collateral veins are commonly present. Thus, knowing the status of the SSS in PSM cases prior to treatment is essential to a successful outcome. MRI is utilized prior to surgery in order to determine SSS status and to check for presence of collateral veins. The objective of this study is to evaluate the reliability of MRI in predicting both SSS involvement and presence of collateral veins in subsequent comparison to actual intra-operative findings, and to report on complications and outcomes.

METHODS

27 patients were retrospectively analyzed for this study. A blinded radiologist reviewed all pre-operative images, noting SSS status and collateral vein presence. Intraoperative findings were obtained from hospital records to similarly categorize SSS status and collateral vein presence.

RESULTS

Sensitivity of the MRI to SSS status was found to be 100% and specificity was 93%. However, sensitivity and specificity of MRI to collateral vein presence was only 40% and 78.6%, respectively. Complications were experienced by 22% of patients, the majority neurologic in nature.

CONCLUSION

MRI accurately predicted SSS occlusion status, but was less consistent in identification of collateral veins. These findings suggest MRI should be used with caution prior to PSM resection surgery particularly with regards to the presence of collateral veins which may complicate resection.

Use of Mixed Reality in Neuro-Oncology: A Single Centre Experience

(1) Background: Intra-operative neuronavigation is currently an essential component to most neurosurgical operations. Recent progress in mixed reality (MR) technology has attempted to overcome the disadvantages of the neuronavigation systems. We present our experience using the HoloLens 2 in neuro-oncology for both intra- and extra-axial tumours. (2) Results: We describe our experience with three patients who underwent tumour resection. We evaluated surgeon experience, accuracy of superimposed 3D image in tumour localisation with standard neuronavigation both pre- and intra-operatively. Surgeon training and usage for HoloLens 2 was short and easy. The process of image overlay was relatively straightforward for the three cases. Registration in prone position with a conventional neuronavigation system is often difficult, which was easily overcome during use of HoloLens 2. (3) Conclusion: Although certain limitations were identified, the authors feel that this system is a feasible alternative device for intra-operative visualization of neurosurgical pathology. Further studies are being planned to assess its accuracy and suitability across various surgical disciplines.

Development of a 3D Printed Brain Model with Vasculature for Neurosurgical Procedure Visualisation and Training

BACKGROUND

Simulation-based techniques using three-dimensional models are gaining popularity in neurosurgical training. Most pre-existing models are expensive, so we felt a need to develop a real-life model using 3D printing technology to train in endoscopic third ventriculostomy.

METHODS

The brain model was made using a 3D-printed resin mold from patient-specific MRI data. The mold was filled with silicone Ecoflex™ 00-10 and mixed with Silc Pig^® pigment additives to replicate the color and consistency of brain tissue. The dura mater was made from quick-drying silicone paste admixed with gray dye. The blood vessels were made from a silicone 3D-printed mold based on magnetic resonance imaging. Liquid containing paprika oleoresin dye was used to simulate blood and was pumped through the vessels to simulate pulsatile motion.

RESULTS

Seven residents and eight senior neurosurgeons were recruited to test our model. The participants reported that the size and anatomy of the elements were very similar to real structures. The model was helpful for training neuroendoscopic 3D perception and navigation.

CONCLUSIONS

We developed an endoscopic third ventriculostomy training model using 3D printing technology that provides anatomical precision and a realistic simulation. We hope our model can provide an indispensable tool for young neurosurgeons to gain operative experience without exposing patients to risk.

A Tool-free Neuronavigation Method based on Single-view Hand Tracking

This work presents a novel tool-free neuronavigation method that can be used with a single RGB commodity camera. Compared with freehand craniotomy placement methods, the proposed system is more intuitive and less error prone. The proposed method also has several advantages over standard neuronavigation platforms. First, it has a much lower cost, since it doesn't require the use of an optical tracking camera or electromagnetic field generator, which are typically the most expensive parts of a neuronavigation system, making it much more accessible. Second, it requires minimal setup, meaning that it can be performed at the bedside and in circumstances where using a standard neuronavigation system is impractical. Our system relies on machine-learning-based hand pose estimation that acts as a proxy for optical tool tracking, enabling a 3D-3D pre-operative to intra-operative registration. Qualitative assessment from clinical users showed that the concept is clinically relevant. Quantitative assessment showed that on average a target registration error (TRE) of 1.3cm can be achieved. Furthermore, the system is framework-agnostic, meaning that future improvements to hand-tracking frameworks would directly translate to a higher accuracy.

A data-centric artificial intelligent and extended reality technology in smart healthcare systems

Extended reality (XR) solutions are quietly maturing, and their novel use cases are already being investigated, particularly in the healthcare industry. By 2022, the extended reality market is anticipated to be worth $209 billion. Certain diseases, such as Alzheimer's, Schizophrenia, Stroke rehabilitation stimulating specific areas of the patient's brain, healing brain injuries, surgeon training, realistic 3D visualization, touch-free interfaces, and teaching social skills to children with autism, have shown promising results with XR-assisted treatments. Similar effects have been used in video game therapies like Akili Interactive's EndeavorRx, which has previously been approved by the Food and Drug Administration (FDA) as a treatment regimen for children with attention deficit hyperactivity disorder (ADHD). However, while these improvements have received positive feedback, the field of XR-assisted patient treatment is in its infancy. The growth of XR in the healthcare sphere has the potential to transform the delivery of medical services. Imagine an elderly patient in a remote setting having a consultation with a world-renowned expert without ever having to leave their house. Rather than operating on cadavers in a medical facility, a surgical resident does surgery in a virtual setting at home. On the first try, a nurse uses a vein finder to implant an IV. Through cognitive treatment in a virtual world, a war veteran recovers from post-traumatic stress disorder (PTSD). The paper discusses the potential impact of XR in transforming the healthcare industry, as well as its use cases, challenges, XR tools and techniques for intelligent health care, recent developments of XR in intelligent healthcare services, and the potential benefits and future aspects of XR techniques in the medical domain.

Effects of a Mixed-Reality Headset on Procedural Outcomes in the Cardiac Catheterization Laboratory

BACKGROUND

Mixed reality head-mounted displays (MR-HMD) are a novel and emerging tool in healthcare. There is a paucity of data on the safety and efficacy of the use of MR-HMD in the cardiac catheterization laboratory (CCL). We sought to analyze and compare fluoroscopy time, procedure time, and complication rates with right heart catheterizations (RHCs) and coronary angiographies (CAs) performed with MR-HMD versus standard LCD medical displays.

METHODS

This is a non-randomized trial that included patients who underwent RHC and CA with MR-HMD between August 2019 and January 2020. Their outcomes were compared to a control group during the same time period. The primary endpoints were procedure time, fluoroscopy time, and dose area product (DAP). The secondary endpoints were contrast volume and intra and postprocedural complications rate.

RESULTS

50 patients were enrolled in the trial, 33 had a RHC done, and 29 had a diagnostic CA performed. They were compared to 232 patients in the control group. The use of MR-HMD was associated with a significantly lower procedure time (20 min (IQR 14-30) vs. 25 min (IQR 18-36), p = 0.038). There were no significant differences in median fluoroscopy time (1.5 min (IQR 0.7-4.9) in the study group vs. 1.3 min (IQR 0.8-3.1), p = 0.84) or median DAP (165.4 mGy·cm² (IQR 13-15,583) in the study group vs. 913 mGy·cm² (IQR 24-6291), p = 0.17). There was no significant increase in intra- or post-procedure complications.

CONCLUSION

MR-HMD use is safe and feasible and may decrease procedure time in the CCL.

Brain Tumor and Augmented Reality: New Technologies for the Future

In recent years, huge progress has been made in the management of brain tumors, due to the availability of imaging devices, which provide fundamental anatomical and pathological information not only for diagnostic purposes [...].

Digital Transformation Will Change Medical Education and Rehabilitation in Spine Surgery

The concept of minimally invasive spine therapy (MIST) has been proposed as a treatment strategy to reduce the need for overall patient care, including not only minimally invasive spine surgery (MISS) but also conservative treatment and rehabilitation. To maximize the effectiveness of patient care in spine surgery, the educational needs of medical students, residents, and patient rehabilitation can be enhanced by digital transformation (DX), including virtual reality (VR), augmented reality (AR), mixed reality (MR), and extended reality (XR), three-dimensional (3D) medical images and holograms; wearable sensors, high-performance video cameras, fifth-generation wireless system (5G) and wireless fidelity (Wi-Fi), artificial intelligence, and head-mounted displays (HMDs). Furthermore, to comply with the guidelines for social distancing due to the unexpected COVID-19 pandemic, the use of DX to maintain healthcare and education is becoming more innovative than ever before. In medical education, with the evolution of science and technology, it has become mandatory to provide a highly interactive educational environment and experience using DX technology for residents and medical students, known as digital natives. This study describes an approach to pre- and intraoperative medical education and postoperative rehabilitation using DX in the field of spine surgery that was implemented during the COVID-19 pandemic and will be utilized thereafter.

Evaluation of augmented-reality based navigation for brain tumor surgery

To date, several researchers have introduced augmented reality navigation (ARN) into neurological surgery. While its application in brain tumor surgery seems promising, reports on its utility have been limited, thus warranting further evaluation. To clarify the stages and approaches in which ARN is useful and assess the effect of presurgical discussion with surgeons, we assessed usefulness using a hand-held ARN system we had developed, which displays three-dimensional (3D) virtual structures overlaid on a real-time image of the surgical field via a tablet PC monitor. The system was tested in 20 patients undergoing various procedures, with the first 10 consecutive cases being unselected and the following 10 cases being selected, for whom 3D models were prepared per the surgeons' request. Thereafter, the surgeons ranked its usefulness during each stage of surgery. Consequently, case selection and presurgical discussions with surgeons considerably improved the usefulness, with the "useful" gradings improving from 50% to 88% across all surgical stages. Accordingly, usefulness improved from 50% to 90%, 67% to 100%, and 40% to 80% during the skin incision and craniotomy, dura incision, and intradural procedure stages, respectively. ARN was useful for superficial tumor resection, but less so for deep-seated tumor resection, except when using the transcortical and interhemispheric approaches. In conclusion, a tablet-type ARN can be useful during skin incisions, craniotomy and dura incisions, superficial tumor resections, and transcortical and interhemispheric approaches for deep-seated tumors. Case selection and presurgical discussions with surgeons were essential for the efficacy of ARN.

Development of a New Image-Guided Neuronavigation System: Mixed-Reality Projection Mapping Is Accurate and Feasible

BACKGROUND

Image-guided systems improve the safety, functional outcome, and overall survival of neurosurgery but require extensive equipment.

OBJECTIVE

To develop an image-guided surgery system that combines the brain surface photographic texture (BSP-T) captured during surgery with 3-dimensional computer graphics (3DCG) using projection mapping.

METHODS

Patients who underwent initial surgery with brain tumors were prospectively enrolled. The texture of the 3DCG (3DCG-T) was obtained from 3DCG under similar conditions as those when capturing the brain surface photographs. The position and orientation at the time of 3DCG-T acquisition were used as the reference. The correct position and orientation of the BSP-T were obtained by aligning the BSP-T with the 3DCG-T using normalized mutual information. The BSP-T was combined with and displayed on the 3DCG using projection mapping. This mixed-reality projection mapping (MRPM) was used prospectively in 15 patients (mean age 46.6 yr, 6 males). The difference between the centerlines of surface blood vessels on the BSP-T and 3DCG constituted the target registration error (TRE) and was measured in 16 fields of the craniotomy area. We also measured the time required for image processing.

RESULTS

The TRE was measured at 158 locations in the 15 patients, with an average of 1.19 ± 0.14 mm (mean ± standard error). The average image processing time was 16.58 min.

CONCLUSION

Our MRPM method does not require extensive equipment while presenting information of patients' anatomy together with medical images in the same coordinate system. It has the potential to improve patient safety.

Patient posture correction and alignment using mixed reality visualization and the HoloLens 2

PURPOSE

The purpose of this study was to develop and preliminarily test a radiotherapy system for patient posture correction and alignment using mixed reality (MixR) visualization. The write-up of this work also provides an opportunity to introduce the concepts and technology of MixR for a medical physics audience who may be unfamiliar with the topic.

METHODS

A MixR application was developed for on optical-see-through head-mounted display (HoloLens 2) allowing a user to simultaneously and directly view a patient and a reference hologram derived from their simulation CT scan. The hologram provides a visual reference for the exact posture needed during treatment and is initialized in relation to the origin of a radiotherapy device using marker-based tracking. The system further provides marker-less tracking that allows the user tofreely navigate the room as they view and align the patient from various angles. The system was preliminarily tested using both a rigid (pelvis) and nonrigid (female mannequin) anthropomorphic phantom. Each phantom was aligned via hologram and accuracy quantified using CBCT and CT.

RESULTS

A fully realized system was developed. Rigid registration accuracy was on the order of 3.0 ± 1.5 mm based on the performance of three users repeating alignment five times each. The lateral direction showed the most variability among users and was associated with the largest off-sets (approximately 2.0 mm). For nonrigid alignment, the MixR setup outperformed a setup based on three-point alignment and setup photos, the latter of which showed a difference in arm position of 2 cm and a torso roll of 6-7°.

CONCLUSIONS

MixR visualization is a rapidly emerging domain that has the potential to significantly impact the field of medicine. The current application is an illustration of this and highlights the advantages of MixR for patient setup in radiation oncology. The key feature of the system is the way in which it transforms nonrigid registration into rigid registration by providing an efficient, portable, and cost-effective mechanism for reproducing patient posture without the use of ionizing radiation. Preliminary estimates of registration accuracy indicate clinical viability and form the foundation for further development and clinical testing.

A workflow to generate patient-specific three-dimensional augmented reality models from medical imaging data and example applications in urologic oncology

Augmented reality (AR) and virtual reality (VR) are burgeoning technologies that have the potential to greatly enhance patient care. Visualizing patient-specific three-dimensional (3D) imaging data in these enhanced virtual environments may improve surgeons' understanding of anatomy and surgical pathology, thereby allowing for improved surgical planning, superior intra-operative guidance, and ultimately improved patient care. It is important that radiologists are familiar with these technologies, especially since the number of institutions utilizing VR and AR is increasing. This article gives an overview of AR and VR and describes the workflow required to create anatomical 3D models for use in AR using the Microsoft HoloLens device. Case examples in urologic oncology (prostate cancer and renal cancer) are provided which depict how AR has been used to guide surgery at our institution.

Marker-less augmented reality based on monocular vision for falx meningioma localization

BACKGROUND

The existing augmented reality (AR) based neuronavigation systems typically require markers and additional tracking devices for model registration, which causes excessive preparatory steps.

METHODS

For fast and accurate intraoperative navigation, this work proposes a marker-less AR system that tracks the head features with the monocular camera. After the semi-automatic initialization process, the feature points between the captured image and the pre-loaded keyframes are matched for obtaining correspondences. The camera pose is estimated by solving the Perspective-n-Point problem.

RESULTS

The localization error of AR visualization on scalp and falx meningioma is 0.417 ± 0.057 and 1.413 ± 0.282 mm, respectively. The maximum localization error is less than 2 mm. The AR system is robust to occlusions and changes in viewpoint and scale.

CONCLUSIONS

We demonstrate that the developed system can successfully display the augmented falx meningioma with enough accuracy and provide guidance for neurosurgeons to locate the tumour in brain.

Augmented Reality-Assisted Craniotomy for Parasagittal and Convexity En Plaque Meningiomas and Custom-Made Cranio-Plasty: A Preliminary Laboratory Report

BACKGROUND

This report discusses the utility of a wearable augmented reality platform in neurosurgery for parasagittal and convexity en plaque meningiomas with bone flap removal and custom-made cranioplasty.

METHODS

A real patient with en plaque cranial vault meningioma with diffuse and extensive dural involvement, extracranial extension into the calvarium, and homogeneous contrast enhancement on gadolinium-enhanced T1-weighted MRI, was selected for this case study. A patient-specific manikin was designed starting with the segmentation of the patient's preoperative MRI images to simulate a craniotomy procedure. Surgical planning was performed according to the segmented anatomy, and customized bone flaps were designed accordingly. During the surgical simulation stage, the VOSTARS head-mounted display was used to accurately display the planned craniotomy trajectory over the manikin skull. The precision of the craniotomy was assessed based on the evaluation of previously prepared custom-made bone flaps.

RESULTS

A bone flap with a radius 0.5 mm smaller than the radius of an ideal craniotomy fitted perfectly over the performed craniotomy, demonstrating an error of less than ±1 mm in the task execution. The results of this laboratory-based experiment suggest that the proposed augmented reality platform helps in simulating convexity en plaque meningioma resection and custom-made cranioplasty, as carefully planned in the preoperative phase.

CONCLUSIONS

Augmented reality head-mounted displays have the potential to be a useful adjunct in tumor surgical resection, cranial vault lesion craniotomy and also skull base surgery, but more study with large series is needed.

Applications of augmented reality in the neurosurgical operating room: A systematic review of the literature

Advancements in imaging techniques are key forces of progress in neurosurgery. The importance of accurate visualization of intraoperative anatomy cannot be overemphasized and is commonly delivered through traditional neuronavigation. Augmented Reality (AR) technology has been tested and applied widely in various neurosurgical subspecialties in intraoperative, clinical use and shows promise for the future. This systematic review of the literature explores the ways in which AR technology has been successfully brought into the operating room (OR) and incorporated into clinical practice. A comprehensive literature search was performed in the following databases from inception-April 2020: Ovid MEDLINE, Ovid EMBASE, and The Cochrane Library. Studies retrieved were then screened for eligibility against predefined inclusion/exclusion criteria. A total of 54 articles were included in this systematic review. The studies were sub- grouped into brain and spine subspecialties and analyzed for their incorporation of AR in the neurosurgical clinical setting. AR technology has the potential to greatly enhance intraoperative visualization and guidance in neurosurgery beyond the traditional neuronavigation systems. However, there are several key challenges to scaling the use of this technology and bringing it into standard operative practice including accurate and efficient brain segmentation of magnetic resonance imaging (MRI) scans, accounting for brain shift, reducing coregistration errors, and improving the AR device hardware. There is also an exciting potential for future work combining AR with multimodal imaging techniques and artificial intelligence to further enhance its impact in neurosurgery.

Augmented Reality with HoloLens® in Parotid Tumor Surgery: A Prospective Feasibility Study

INTRODUCTION

Augmented reality can improve planning and execution of surgical procedures. Head-mounted devices such as the HoloLens® (Microsoft, Redmond, WA, USA) are particularly suitable to achieve these aims because they are controlled by hand gestures and enable contactless handling in a sterile environment.

OBJECTIVES

So far, these systems have not yet found their way into the operating room for surgery of the parotid gland. This study explored the feasibility and accuracy of augmented reality-assisted parotid surgery.

METHODS

2D MRI holographic images were created, and 3D holograms were reconstructed from MRI DICOM files and made visible via the HoloLens. 2D MRI slices were scrolled through, 3D images were rotated, and 3D structures were shown and hidden only using hand gestures. The 3D model and the patient were aligned manually.

RESULTS

The use of augmented reality with the HoloLens in parotic surgery was feasible. Gestures were recognized correctly. Mean accuracy of superimposition of the holographic model and patient's anatomy was 1.3 cm. Highly significant differences were seen in position error of registration between central and peripheral structures (p = 0.0059), with a least deviation of 10.9 mm (centrally) and highest deviation for the peripheral parts (19.6-mm deviation).

CONCLUSION

This pilot study offers a first proof of concept of the clinical feasibility of the HoloLens for parotid tumor surgery. Workflow is not affected, but additional information is provided. The surgical performance could become safer through the navigation-like application of reality-fused 3D holograms, and it improves ergonomics without compromising sterility. Superimposition of the 3D holograms with the surgical field was possible, but further invention is necessary to improve the accuracy.

Role of Three-Dimensional Visualization Modalities in Medical Education

For the past two decades, slide-based presentation has been the method of content delivery in medical education. In recent years, other teaching modalities involving three-dimensional (3D) visualization such as 3D printed anatomical models, virtual reality (VR), and augmented reality (AR) have been explored to augment the education experience. This review article will analyze the use of slide-based presentation, 3D printed anatomical models, AR, and VR technologies in medical education, including their benefits and limitations.

Comparing Computed Tomography-Derived Augmented Reality Holograms to a Standard Picture Archiving and Communication Systems Viewer for Presurgical Planning: Feasibility Study

Background

Picture archiving and communication systems (PACS) are ubiquitously used to store, share, and view radiological information for preoperative planning across surgical specialties. Although traditional PACS software has proven reliable in terms of display accuracy and ease of use, it remains limited by its inherent representation of medical imaging in 2 dimensions. Augmented reality (AR) systems present an exciting opportunity to complement traditional PACS capabilities.

Objective

This study aims to evaluate the technical feasibility of using a novel AR platform, with holograms derived from computed tomography (CT) imaging, as a supplement to traditional PACS for presurgical planning in complex surgical procedures.

Methods

Independent readers measured objects of predetermined, anthropomorphically correlated sizes using the circumference and angle tools of standard-of-care PACS software and a newly developed augmented reality presurgical planning system (ARPPS).

Results

Measurements taken with the standard PACS and the ARPPS showed no statistically significant differences. Bland-Altman analysis showed a mean difference of 0.08% (95% CI –4.20% to 4.36%) for measurements taken with PACS versus ARPPS’ circumference tools and –1.84% (95% CI –6.17% to 2.14%) for measurements with the systems’ angle tools. Lin’s concordance correlation coefficients were 1.00 and 0.98 for the circumference and angle measurements, respectively, indicating almost perfect strength of agreement between ARPPS and PACS. Intraclass correlation showed no statistically significant difference between the readers for either measurement tool on each system.

Conclusions

ARPPS can be an effective, accurate, and precise means of 3D visualization and measurement of CT-derived holograms in the presurgical care timeline.

Virtual Reality in Neurosurgery: "Can You See It?"-A Review of the Current Applications and Future Potential

Virtual reality (VR) technology had its early development in the 1960s in the U.S. Air Force and has since evolved into a budding area of scientific research with many practical medical purposes. From medical education to resident training to the operating room, VR has provided tangible benefits to learners and trainees and has also improved surgery through enhanced preoperative planning and efficiency in the operating room. Neurosurgery is a particularly complex field of medicine, in which VR has blossomed into a tool with great usefulness and promise. In spinal surgery, VR simulation has allowed for the practice of innovative minimally invasive procedures. In cranial surgery, VR has excelled in helping neurosurgeons design unique patient-specific approaches to particularly challenging tumor excisions. In neurovascular surgery, VR has helped trainees practice and perfect procedures requiring high levels of dexterity to minimize intraoperative complications and patient radiation exposure. In peripheral nerve surgery, VR has allowed surgeons to gain increased practice and comfort with complex microsurgeries such as nerve decompression. Overall, VR continues to increase its potential in neurosurgery and is poised to benefit patients in a multitude of ways. Although cost-prohibiting, legal, and ethical challenges surrounding this technology must be considered, future research and more direct quantitative outcome comparisons between standard and VR-supplemented procedures would help provide more direction regarding the feasibility of widespread adoption of VR technology in neurosurgery.

Virtual or Augmented Reality to Enhance Surgical Education and Surgical Planning

Virtual reality and augmented reality technologies have evolved with a growing presence in both clinical care and surgical training.

Presurgical and Intraoperative Augmented Reality in Neuro-Oncologic Surgery: Clinical Experiences and Limitations

Virtual reality (VR) and augmented reality (AR) represent novel adjuncts for neurosurgical planning in neuro-oncology. In addition to established use in surgical and medical training, VR/AR are gaining traction for clinical use preoperatively and intraoperatively. To understand the utility of VR/AR in the clinical setting, we conducted a literature search in Ovid MEDLINE and EMBASE with various search terms designed to capture the use of VR/AR in neurosurgical procedures for resection of cranial tumors. The search retrieved 302 articles, of which 35 were subjected to full-text review; 19 full-text articles were included in the review. Key findings highlighted by the individual authors were extracted and summarized into themes to present the value of VR/AR in the clinical setting. These studies included various VR/AR systems applied to surgeries involving heterogeneous pathologies and outcome measures. Overall, VR/AR were found to be qualitatively advantageous due to enhanced visualization of complex anatomy and improved intraoperative lesion localization. When these technologies were compared with existing neuronavigation systems, quantitative clinical benefits were also reported. The capacity to visualize three-dimensional images superimposed on patient anatomy is a potentially valuable tool in complex neurosurgical environments. Surgical limitations may be addressed through future advances in image registration and tracking as well as intraoperatively acquired imaging with the ability to yield real-time virtual models.

Implementing Virtual and Augmented Reality Tools for Radiology Education and Training, Communication, and Clinical Care

Advances in virtual immersive and augmented reality technology, commercially available for the entertainment and gaming industry, hold potential for education and clinical use in medicine and the field of medical imaging. Radiology departments have begun exploring the use of these technologies to help with radiology education and clinical care. The purpose of this review article is to summarize how three institutions have explored using virtual and augmented reality for radiology.

Augmented Reality in the Health Domain: Projecting Spatial Augmented Reality Visualizations on a Perceiver's Body for Health Communication Effects

An experiment is reported that studied the effects of spatial embodiment in augmented reality on medical attitudes about the self. College students (N = 90) viewed public service announcements (PSAs) with overlaid virtual fetuses and X-rayed images of lungs on various interfaces representing embodiment-a two-dimensional screen, a three-dimensional (3D) mannequin, and the participants' bodies (3D). Results indicated that PSA messages with richer embodied interfaces increase the sense of "being there," also known as spatial presence (SP), in sequential order; this leads to increased negative emotion regarding smoking cigarettes and an increased willingness to engage with a cigarette cessation campaign. When the SP mediates the dual model process, only affective attitudes increase the behavioral intention to engage with the campaign.

Combining intraoperative ultrasound brain shift correction and augmented reality visualizations: a pilot study of eight cases

We present our work investigating the feasibility of combining intraoperative ultrasound for brain shift correction and augmented reality (AR) visualization for intraoperative interpretation of patient-specific models in image-guided neurosurgery (IGNS) of brain tumors. We combine two imaging technologies for image-guided brain tumor neurosurgery. Throughout surgical interventions, AR was used to assess different surgical strategies using three-dimensional (3-D) patient-specific models of the patient's cortex, vasculature, and lesion. Ultrasound imaging was acquired intraoperatively, and preoperative images and models were registered to the intraoperative data. The quality and reliability of the AR views were evaluated with both qualitative and quantitative metrics. A pilot study of eight patients demonstrates the feasible combination of these two technologies and their complementary features. In each case, the AR visualizations enabled the surgeon to accurately visualize the anatomy and pathology of interest for an extended period of the intervention. Inaccuracies associated with misregistration, brain shift, and AR were improved in all cases. These results demonstrate the potential of combining ultrasound-based registration with AR to become a useful tool for neurosurgeons to improve intraoperative patient-specific planning by improving the understanding of complex 3-D medical imaging data and prolonging the reliable use of IGNS.

Editorial: Challenges for the usability of AR and VR for clinical neurosurgical procedures

There are a number of challenges that must be faced when trying to develop AR and VR-based Neurosurgical simulators, Surgical Navigation Platforms, and "Smart OR" systems. Trying to simulate an operating room environment and surgical tasks in Augmented and Virtual Reality is a challenge many are attempting to solve, in order to train surgeons or help them operate. What are some of the needs of the surgeon, and what are the challenges encountered (human computer interface, perception, workflow, etc). We discuss these tradeoffs and conclude with critical remarks.

Neurosurgical Virtual Reality Simulation for Brain Tumor Using High-definition Computer Graphics: A Review of the Literature

Simulation and planning of surgery using a virtual reality model is becoming common with advances in computer technology. In this study, we conducted a literature search to find trends in virtual simulation of surgery for brain tumors. A MEDLINE search for “neurosurgery AND (simulation OR virtual reality)” retrieved a total of 1,298 articles published in the past 10 years. After eliminating studies designed solely for education and training purposes, 28 articles about the clinical application remained. The finding that the vast majority of the articles were about education and training rather than clinical applications suggests that several issues need be addressed for clinical application of surgical simulation. In addition, 10 of the 28 articles were from Japanese groups. In general, the 28 articles demonstrated clinical benefits of virtual surgical simulation. Simulation was particularly useful in better understanding complicated spatial relations of anatomical landmarks and in examining surgical approaches. In some studies, Virtual reality models were used on either surgical navigation system or augmented reality technology, which projects virtual reality images onto the operating field. Reported problems were difficulties in standardized, objective evaluation of surgical simulation systems; inability to respond to tissue deformation caused by surgical maneuvers; absence of the system functionality to reflect features of tissue (e.g., hardness and adhesion); and many problems with image processing. The amount of description about image processing tended to be insufficient, indicating that the level of evidence, risk of bias, precision, and reproducibility need to be addressed for further advances and ultimately for full clinical application.

Augmented Reality in Neurosurgery: A Review of Current Concepts and Emerging Applications

Augmented reality (AR) superimposes computer-generated virtual objects onto the user's view of the real world. Among medical disciplines, neurosurgery has long been at the forefront of image-guided surgery, and it continues to push the frontiers of AR technology in the operating room. In this systematic review, we explore the history of AR in neurosurgery and examine the literature on current neurosurgical applications of AR. Significant challenges to surgical AR exist, including compounded sources of registration error, impaired depth perception, visual and tactile temporal asynchrony, and operator inattentional blindness. Nevertheless, the ability to accurately display multiple three-dimensional datasets congruently over the area where they are most useful, coupled with future advances in imaging, registration, display technology, and robotic actuation, portend a promising role for AR in the neurosurgical operating room.

Utilizing virtual and augmented reality for educational and clinical enhancements in neurosurgery

Neurosurgery has undergone a technological revolution over the past several decades, from trephination to image-guided navigation. Advancements in virtual reality (VR) and augmented reality (AR) represent some of the newest modalities being integrated into neurosurgical practice and resident education. In this review, we present a historical perspective of the development of VR and AR technologies, analyze its current uses, and discuss its emerging applications in the field of neurosurgery.

Medical telerobotic systems: current status and future trends

Teleoperated medical robotic systems allow procedures such as surgeries, treatments, and diagnoses to be conducted across short or long distances while utilizing wired and/or wireless communication networks. This study presents a systematic review of the relevant literature between the years 2004 and 2015, focusing on medical teleoperated robotic systems which have witnessed tremendous growth over the examined period. A thorough insight of telerobotics systems discussing design concepts, enabling technologies (namely robotic manipulation, telecommunications, and vision systems), and potential applications in clinical practice is provided, while existing limitations and future trends are also highlighted. A representative paradigm of the short-distance case is the da Vinci Surgical System which is described in order to highlight relevant issues. The long-distance telerobotics concept is exemplified through a case study on diagnostic ultrasound scanning. Moreover, the present review provides a classification into short- and long-distance telerobotic systems, depending on the distance from which they are operated. Telerobotic systems are further categorized with respect to their application field. For the reviewed systems are also examined their engineering characteristics and the employed robotics technology. The current status of the field, its significance, the potential, as well as the challenges that lie ahead are thoroughly discussed.

A Low-Cost iPhone-Assisted Augmented Reality Solution for the Localization of Intracranial Lesions

Background

Precise location of intracranial lesions before surgery is important, but occasionally difficult. Modern navigation systems are very helpful, but expensive. A low-cost solution that could locate brain lesions and their surface projections in augmented reality would be beneficial. We used an iPhone to partially achieve this goal, and evaluated its accuracy and feasibility in a clinical neurosurgery setting.

Methodology/Principal Findings

We located brain lesions in 35 patients, and using an iPhone, we depicted the lesion’s surface projection onto the skin of the head. To assess the accuracy of this method, we pasted computed tomography (CT) markers surrounding the depicted lesion boundaries on the skin onto 15 patients. CT scans were then performed with or without contrast enhancement. The deviations (D) between the CT markers and the actual lesion boundaries were measured. We found that 97.7% of the markers displayed a high accuracy level (D ≤ 5mm). In the remaining 20 patients, we compared our iPhone-based method with a frameless neuronavigation system. Four check points were chosen on the skin surrounding the depicted lesion boundaries, to assess the deviations between the two methods. The integrated offset was calculated according to the deviations at the four check points. We found that for the supratentorial lesions, the medial offset between these two methods was 2.90 mm and the maximum offset was 4.2 mm.

Conclusions/Significance

This low-cost, image-based, iPhone-assisted, augmented reality solution is technically feasible, and helpful for the localization of some intracranial lesions, especially shallow supratentorial intracranial lesions of moderate size.

Augmented reality in neurosurgery: a systematic review

Neuronavigation has become an essential neurosurgical tool in pursuing minimal invasiveness and maximal safety, even though it has several technical limitations. Augmented reality (AR) neuronavigation is a significant advance, providing a real-time updated 3D virtual model of anatomical details, overlaid on the real surgical field. Currently, only a few AR systems have been tested in a clinical setting. The aim is to review such devices. We performed a PubMed search of reports restricted to human studies of in vivo applications of AR in any neurosurgical procedure using the search terms "Augmented reality" and "Neurosurgery." Eligibility assessment was performed independently by two reviewers in an unblinded standardized manner. The systems were qualitatively evaluated on the basis of the following: neurosurgical subspecialty of application, pathology of treated lesions and lesion locations, real data source, virtual data source, tracking modality, registration technique, visualization processing, display type, and perception location. Eighteen studies were included during the period 1996 to September 30, 2015. The AR systems were grouped by the real data source: microscope (8), hand- or head-held cameras (4), direct patient view (2), endoscope (1), and X-ray fluoroscopy (1) head-mounted display (1). A total of 195 lesions were treated: 75 (38.46 %) were neoplastic, 77 (39.48 %) neurovascular, and 1 (0.51 %) hydrocephalus, and 42 (21.53 %) were undetermined. Current literature confirms that AR is a reliable and versatile tool when performing minimally invasive approaches in a wide range of neurosurgical diseases, although prospective randomized studies are not yet available and technical improvements are needed.

Surgical Outcomes and Predictors of Recurrence in Elderly Patients with Meningiomas

BACKGROUND

Meningiomas are more prevalent with increasing age, and there is no general consensus of surgical management for this group of tumors. The role of resection and the predictors of recurrence of meningiomas in elderly patients have not been well established. The authors reviewed their experience of surgical resection of these tumors in elderly patients and determined the factors for recurrence.

METHODS

Clinical and radiologic information of 81 elderly patients with meningiomas was retrospectively reviewed. Simpson grading scales were used to evaluate the extent of surgical resection. Statistical analysis was conducted using Kaplan-Meier curves and Cox proportional hazards regression.

RESULTS

In this study, the recurrence rate in the grades I-II resection group was 10%, whereas in the grades III-IV resection group, the recurrence rate was 46% (P < 0.001). In Cox regression analysis, Simpson grades I-II/complete resection was revealed as a significant predictor of recurrence-free survival (P = 0.002). The hazard ratio for recurrence after grades III-IV resection was 5 times higher than after grades I-II resection. Location, size of the tumor, medical comorbidities, Karnofsky Performance Status, Geriatric Scoring System (GSS) score >16, and repeated resection were identified as predictors of recurrence-free survival after resection of meningiomas.

CONCLUSIONS

Elderly patients with meningiomas having a GSS score >16 and undergoing Simpson grades I-II/gross total resection have low operative morbidity, mortality, and recurrence rates and are less likely to continue on antiepileptic medications. Although complete tumor resection is the goal, the surgical approach should be tailored to each patient depending on the patient's GSS score.

Systematic review on the effectiveness of augmented reality applications in medical training

Background

Computer-based applications are increasingly used to support the training of medical professionals. Augmented reality applications (ARAs) render an interactive virtual layer on top of reality. The use of ARAs is of real interest to medical education because they blend digital elements with the physical learning environment. This will result in new educational opportunities. The aim of this systematic review is to investigate to which extent augmented reality applications are currently used to validly support medical professionals training.

Methods

PubMed, Embase, INSPEC and PsychInfo were searched using predefined inclusion criteria for relevant articles up to August 2015. All study types were considered eligible. Articles concerning AR applications used to train or educate medical professionals were evaluated.

Results

Twenty-seven studies were found relevant, describing a total of seven augmented reality applications. Applications were assigned to three different categories. The first category is directed toward laparoscopic surgical training, the second category toward mixed reality training of neurosurgical procedures and the third category toward training echocardiography. Statistical pooling of data could not be performed due to heterogeneity of study designs. Face-, construct- and concurrent validity was proven for two applications directed at laparoscopic training, face- and construct validity for neurosurgical procedures and face-, content- and construct validity in echocardiography training. In the literature, none of the ARAs completed a full validation process for the purpose of use.

Conclusion

Augmented reality applications that support blended learning in medical training have gained public and scientific interest. In order to be of value, applications must be able to transfer information to the user. Although promising, the literature to date is lacking to support such evidence.