Integration of indocyanine green videoangiography with operative microscope: augmented reality for interactive assessment of vascular structures and blood flow

Visualization, navigation, augmentation. The ever-changing perspective of the neurosurgeon

Introduction

The evolution of neurosurgery coincides with the evolution of visualization and navigation. Augmented reality technologies, with their ability to bring digital information into the real environment, have the potential to provide a new, revolutionary perspective to the neurosurgeon.

Research question

To provide an overview on the historical and technical aspects of visualization and navigation in neurosurgery, and to provide a systematic review on augmented reality (AR) applications in neurosurgery.

Material and methods

We provided an overview on the main historical milestones and technical features of visualization and navigation tools in neurosurgery. We systematically searched PubMed and Scopus databases for AR applications in neurosurgery and specifically discussed their relationship with current visualization and navigation systems, as well as main limitations.

Results

The evolution of visualization in neurosurgery is embodied by four magnification systems: surgical loupes, endoscope, surgical microscope and more recently the exoscope, each presenting independent features in terms of magnification capabilities, eye-hand coordination and the possibility to implement additional functions. In regard to navigation, two independent systems have been developed: the frame-based and the frame-less systems. The most frequent application setting for AR is brain surgery (71.6%), specifically neuro-oncology (36.2%) and microscope-based (29.2%), even though in the majority of cases AR applications presented their own visualization supports (66%).

Discussion and conclusions

The evolution of visualization and navigation in neurosurgery allowed for the development of more precise instruments; the development and clinical validation of AR applications, have the potential to be the next breakthrough, making surgeries safer, as well as improving surgical experience and reducing costs.

Comprehensive review of surgical microscopes: technology development and medical applications

SIGNIFICANCE

Surgical microscopes provide adjustable magnification, bright illumination, and clear visualization of the surgical field and have been increasingly used in operating rooms. State-of-the-art surgical microscopes are integrated with various imaging modalities, such as optical coherence tomography (OCT), fluorescence imaging, and augmented reality (AR) for image-guided surgery.

AIM

This comprehensive review is based on the literature of over 500 papers that cover the technology development and applications of surgical microscopy over the past century. The aim of this review is threefold: (i) providing a comprehensive technical overview of surgical microscopes, (ii) providing critical references for microscope selection and system development, and (iii) providing an overview of various medical applications.

APPROACH

More than 500 references were collected and reviewed. A timeline of important milestones during the evolution of surgical microscope is provided in this study. An in-depth technical overview of the optical system, mechanical system, illumination, visualization, and integration with advanced imaging modalities is provided. Various medical applications of surgical microscopes in neurosurgery and spine surgery, ophthalmic surgery, ear-nose-throat (ENT) surgery, endodontics, and plastic and reconstructive surgery are described.

RESULTS

Surgical microscopy has been significantly advanced in the technical aspects of high-end optics, bright and shadow-free illumination, stable and flexible mechanical design, and versatile visualization. New imaging modalities, such as hyperspectral imaging, OCT, fluorescence imaging, photoacoustic microscopy, and laser speckle contrast imaging, are being integrated with surgical microscopes. Advanced visualization and AR are being added to surgical microscopes as new features that are changing clinical practices in the operating room.

CONCLUSIONS

The combination of new imaging technologies and surgical microscopy will enable surgeons to perform challenging procedures and improve surgical outcomes. With advanced visualization and improved ergonomics, the surgical microscope has become a powerful tool in neurosurgery, spinal, ENT, ophthalmic, plastic and reconstructive surgeries.

Augmented reality in neurosurgical navigation: a survey

BACKGROUND

Neurosurgery has exceptionally high requirements for minimally invasive and safety. This survey attempts to analyze the practical application of AR in neurosurgical navigation. Also, this survey describes future trends in augmented reality neurosurgical navigation systems.

METHODS

In this survey, we searched related keywords "augmented reality", "virtual reality", "neurosurgery", "surgical simulation", "brain tumor surgery", "neurovascular surgery", "temporal bone surgery", and "spinal surgery" through Google Scholar, World Neurosurgery, PubMed and Science Direct. We collected 85 articles published over the past five years in areas related to this survey.

RESULTS

Detailed study has been conducted on the application of AR in neurosurgery and found that AR is constantly improving the overall efficiency of doctor training and treatment, which can help neurosurgeons learn and practice surgical procedures with zero risks.

CONCLUSIONS

Neurosurgical navigation is essential in neurosurgery. Despite certain technical limitations, it is still a necessary tool for the pursuit of maximum security and minimal intrusiveness. This article is protected by copyright. All rights reserved.

Application of Fluorescein Fluorescence in Vascular Neurosurgery

Background: Fluorescein sodium (FNa) is a fluorescent drug with a long history of use for assessing retinal blood flow in ophthalmology; however, its application in vascular neurosurgery is only now gaining popularity. This review summarizes the current knowledge about using FNa videoangiography in vascular neurosurgery.

Methods: We performed a literature review on the usage of FNa for fluorescent videoangiography procedures in neurosurgery. We analyzed methods of injection, dosages of FNa, visualizing platforms, and interpretation of FNa videoangiography. We also reviewed practical applications of FNa videoangiography during various vascular neurosurgeries.

Results: FNa videoangiography can be performed with intraarterial (intracarotid) or intravenous dye injections. Both methods provide excellent resolution with enhanced fluorescence that shows intravascular blood flow on top of visible surrounding anatomy, and both allow simultaneous purposeful microsurgical manipulations. Although it is invasive, an intracarotid FNa injection results in faster contrast appearance and higher-intensity fluorescence and requires a lower dose per injection (reported range, 1–50 mg) compared with peripheral intravenous FNa injection (reported range, 75–2,000 mg or 1–1.5 mg/kg body weight). Four optical excitation/detection tools for FNa videoangiography have been successfully used: conventional xenon-light operating microscope with a special filter set, pencil-type light-emitting diode probe with a filter set, laser-illumination operating microscope, and an endoscope with a filter set. FNa videoangiography was reported to be feasible and useful in various clinical scenarios, such as examining the feeders and drainers in arteriovenous malformation surgery, checking the patency of a microvascular anastomosis, and assessing blood flow during aneurysm clipping. FNa videoangiography can be repeated during the same procedure and used along with indocyanine green (ICG) videoangiography.

Conclusions: Compared with ICG videoangiography, FNa videoangiography has the advantages of enabling real-time inspection and better visualization at deep locations; however, thick vessel walls limit visualization of FNa in larger vessels. FNa videoangiography is a useful tool in multiple neurovascular scenarios and merits further studies to establish its clinical value.

Development of a Simulation Model for Fluorescence-Guided Brain Tumor Surgery

Objective: Fluorescence dyes are increasingly used in brain tumor surgeries, and thus the development of simulation models is important for teaching neurosurgery trainees how to perform fluorescence-guided operations. We aimed to create a tumor model for fluorescence-guided surgery in high-grade glioma (HGG).

Methods: The tumor model was generated by the following steps: creating a tumor gel with a similar consistency to HGG, selecting fluorophores at optimal concentrations with realistic color, mixing the fluorophores with tumor gel, injecting the gel into fresh pig/sheep brain, and testing resection of the tumor model under a fluorescence microscope. The optimal tumor gel was selected among different combinations of agar and gelatin. The fluorophores included fluorescein, indocyanine green (ICG), europium, chlorin e6 (Ce6), and protoporphyrin IX (PpIX). The tumor model was tested by neurosurgeons and neurosurgery trainees, and a survey was used to assess the validity of the model. In addition, the photobleaching phenomenon was studied to evaluate its influence on fluorescence detection.

Results: The best tumor gel formula in terms of consistency and tactile response was created using 100 mL water at 100°C, 0.5 g of agar, and 3 g of gelatin mixed thoroughly for 3 min. An additional 1 g of agar was added when the tumor gel cooled to 50°C. The optimal fluorophore concentration ranges were fluorescein 1.9 × 10⁻⁴ to 3.8 × 10⁻⁴ mg/mL, ICG 4.9 × 10⁻³ to 9.8 × 10⁻³ mg/mL, europium 7.0 × 10⁻² to 1.4 × 10⁻¹ mg/mL, Ce6 2.2 × 10⁻³ to 4.4 × 10⁻³ mg/mL, and PpIX 1.8 × 10⁻² to 3.5 × 10⁻² mg/mL. No statistical differences among fluorophores were found for face validity, content validity, and fluorophore preference. Europium, ICG, and fluorescein were shown to be relatively stable during photobleaching experiments, while chlorin e6 and PpIX had lower stability.

Conclusions: The model can efficiently highlight the “tumor” with 3 different colors—green, yellow, or infrared green with color overlay. These models showed high face and content validity, although there was no significant difference among the models regarding the degree of simulation and training effectiveness. They are useful educational tools for teaching the key concepts of intra-axial tumor resection techniques, such as subpial dissection and nuances of fluorescence-guided surgery.

Confocal-Assisted Multispectral Fluorescent Microscopy for Brain Tumor Surgery

Optimal surgical therapy for brain tumors is the combination of complete resection with minimal invasion and damage to the adjacent normal tissue. To achieve this goal, we need advanced imaging techniques on a scale from macro- to microscopic resolution. In the last decade, the development of fluorescence-guided surgery has been the most influential breakthrough, marginally improving outcomes in brain tumor surgery. Multispectral fluorescence microscopy (MFL) is a novel imaging technique that allows the overlapping of a fluorescent image and a white light image in real-time, with delivery of the merged image to the surgeon through the eyepieces of a surgical microscope. MFL permits the detection and characterization of brain tumors using fluorescent molecular markers such as 5-aminolevulinic acid (5-ALA) or indocyanine green (ICG), while simultaneously obtaining high definition white light images to create a pseudo-colored composite image in real-time. Limitations associated with the use of MFL include decreased light imaging intensity and decreased levels of magnification that may compromise maximal tumor resection on a cellular scale. Confocal laser endomicroscopy (CLE) is another novel advanced imaging technique that is based on miniaturization of the microscope imaging head in order to provide the possibility of in vivo microscopy at the cellular level. Clear visualization of the cellular cytoarchitecture can be achieved with 400-fold-1,000-fold magnification. CLE allows on the one hand the intra-operative detection and differentiation of single tumor cells (without the need for intra-operative histologic analysis of biopsy specimens) as well as the definition of borders between tumor and normal tissue at a cellular level, dramatically improving the accuracy of surgical resection. The application and implementation of CLE-assisted surgery in surgical oncology increases not only the number of options for real-time diagnostic imaging, but also the therapeutic options by extending the resection borders of cancer at a cellular level and, more importantly, by protecting the functionality of normal tissue in the adjacent areas of the human brain. In this article, we describe our experience using these new techniques of confocal-assisted fluorescent surgery including analysis on the technology, usability, indications, limitations, and further developments.

Intraoperative brain tumor resection with indocyanine green using augmented microscopy

Treatment outcomes for brain cancer have seen dismal improvements over the last two decades as evident in available statistical data. Efforts to address this challenge include development of near-infrared contrast agents for improvements in diagnostic and therapeutic modalities. This creates a need for imaging technologies that can support the intraoperative use of such agents. Here, we report implementation of a recently introduced augmented microscope in combination with indocyanine green (ICG), a near-infrared contrast agent, for surgical image guidance of a glioma resection in a rat model. Luc-C6 cells were implanted in rats in the left-frontal lobe and grown for 22 days. Surgical resection was performed by a neurosurgeon using the augmented microscope with ICG contrast. ICG accumulated in the tumor tissue due to enhanced permeation and retention from the compromised blood-brain barrier. Videos and images were acquired to evaluate image quality and resection margins. The augmented microscope highlighted tumor tissue regions via visualization of ICG fluorescence and was capable of guiding the rat glioma resection.

Laboratory Evaluation of a Robotic Operative Microscope - Visualization Platform for Neurosurgery

Background We assessed a new robotic visualization platform with novel user-control features and compared its performance to the previous model of operative microscope. Methods In a neurosurgery research laboratory, we performed anatomical dissections and assessed robotic, exoscopic, endoscopic, fluorescence functionality. Usability and functionality were tested in the operating room over 1 year. Results The robotic microscope showed higher sensitivity for fluorescein sodium, higher detail in non-fluorescent background, and recorded/presented pictures with color quality similar to observation through the oculars. PpIX visualization was comparable to the previous microscope. Near-infrared indocyanine green imaging 3-step replay allowed for more convenient accurate assessment of blood flow. Point lock and pivot point functions were used in dissections to create 3D virtual reality microsurgical anatomy demonstrations. Pivot point control was particularly useful in deep surgical corridors with dynamic retraction. 3D exoscopic function was successfully used in brain tumor and spine cases. Endoscopic assistance was used for around-the-corner views in minimally invasive approaches. We present illustrative cases highlighting utility and new ways to control the operative microscope. Conclusion Improvements of the robotic visualization platform include intraoperative fluorescence visualization using FNa, integrated micro-inspection tool, improved ocular imaging clarity, and exoscopic mode. New robotic movements positively assist the surgeon and provide improved ergonomics and a greater level of intraoperative comfort, with the potential to increase the viewing quality. New operational modes also allow significant impact for anatomy instruction. With the increasing number and complexity of functions, surgeons should receive additional training in order to avail themselves of the advantages of the numerous novel features.

Real-time, label-free, intraoperative visualization of peripheral nerves and micro-vasculatures using multimodal optical imaging techniques

Accurate, real-time identification and display of critical anatomic structures, such as the nerve and vasculature structures, are critical for reducing complications and improving surgical outcomes. Human vision is frequently limited in clearly distinguishing and contrasting these structures. We present a novel imaging system, which enables noninvasive visualization of critical anatomic structures during surgical dissection. Peripheral nerves are visualized by a snapshot polarimetry that calculates the anisotropic optical properties. Vascular structures, both venous and arterial, are identified and monitored in real-time using a near-infrared laser-speckle-contrast imaging. We evaluate the system by performing in vivo animal studies with qualitative comparison by contrast-agent-aided fluorescence imaging.

Recent Development of Augmented Reality in Surgery: A Review

Introduction

The development augmented reality devices allow physicians to incorporate data visualization into diagnostic and treatment procedures to improve work efficiency, safety, and cost and to enhance surgical training. However, the awareness of possibilities of augmented reality is generally low. This review evaluates whether augmented reality can presently improve the results of surgical procedures.

Methods

We performed a review of available literature dating from 2010 to November 2016 by searching PubMed and Scopus using the terms “augmented reality” and “surgery.” Results. The initial search yielded 808 studies. After removing duplicates and including only journal articles, a total of 417 studies were identified. By reading of abstracts, 91 relevant studies were chosen to be included. 11 references were gathered by cross-referencing. A total of 102 studies were included in this review.

Conclusions

The present literature suggest an increasing interest of surgeons regarding employing augmented reality into surgery leading to improved safety and efficacy of surgical procedures. Many studies showed that the performance of newly devised augmented reality systems is comparable to traditional techniques. However, several problems need to be addressed before augmented reality is implemented into the routine practice.

Dual-Image Videoangiography During Intracranial Microvascular Surgery

OBJECTIVE

Indocyanine green videoangiography (ICG-VA) is a valuable tool to assess vessel and aneurysm patency during neurovascular surgical procedures. However, ICG-VA highlights vascular structures, which appear white over a black background. Anatomic relationships are sometimes difficult to understand at first glance. Dual-image videoangiography (DIVA) enables simultaneous visualization of light and near-infrared fluorescence images of ICG-VA.

METHODS

The DIVA system was mounted on an OPMI Pentero Flow 800 intraoperative microscope. DIVA was used during microsurgical procedures on 5 patients who were operated for aneurysm clipping and superficial temporal artery-middle cerebral artery bypass.

RESULTS

DIVA provides real-time simultaneous visualization of aneurysm and vessels and surrounding structures including brain, nerves, and surgical clips. Although visual contrast between vessels and background is higher with standard black-and-white imaging, DIVA makes it easier to understand anatomic relationships between intracranial structures. DIVA also provides better vision of the depth of field.

CONCLUSIONS

DIVA has the potential to become a widely used intraoperative tool to check patency of intracranial vessels. It should be considered as an adjunct to standard ICG-VA for better understanding of vascular anatomy in relation to surrounding structures and can have an impact on decision making during surgery.

Intraoperative Fluorescence Imaging for Personalized Brain Tumor Resection: Current State and Future Directions

INTRODUCTION

Fluorescence-guided surgery is one of the rapidly emerging methods of surgical "theranostics." In this review, we summarize current fluorescence techniques used in neurosurgical practice for brain tumor patients as well as future applications of recent laboratory and translational studies.

METHODS

Review of the literature.

RESULTS

A wide spectrum of fluorophores that have been tested for brain surgery is reviewed. Beginning with a fluorescein sodium application in 1948 by Moore, fluorescence-guided brain tumor surgery is either routinely applied in some centers or is under active study in clinical trials. Besides the trinity of commonly used drugs (fluorescein sodium, 5-aminolevulinic acid, and indocyanine green), less studied fluorescent stains, such as tetracyclines, cancer-selective alkylphosphocholine analogs, cresyl violet, acridine orange, and acriflavine, can be used for rapid tumor detection and pathological tissue examination. Other emerging agents, such as activity-based probes and targeted molecular probes that can provide biomolecular specificity for surgical visualization and treatment, are reviewed. Furthermore, we review available engineering and optical solutions for fluorescent surgical visualization. Instruments for fluorescent-guided surgery are divided into wide-field imaging systems and hand-held probes. Recent advancements in quantitative fluorescence-guided surgery are discussed.

CONCLUSION

We are standing on the threshold of the era of marker-assisted tumor management. Innovations in the fields of surgical optics, computer image analysis, and molecular bioengineering are advancing fluorescence-guided tumor resection paradigms, leading to cell-level approaches to visualization and resection of brain tumors.

Novel imaging approaches to cerebrovascular disease

Imaging techniques available to the physician treating neurovascular disease have substantially grown over the past several decades. New techniques as well as advances in imaging modalities continuously develop and provide an extensive array of modalities to diagnose, characterize, and understand neurovascular pathology. Modern noninvasive neurovascular imaging is generally based on computed tomography (CT), magnetic resonance (MR) imaging, or nuclear imaging and includes CT angiography, CT perfusion, xenon-enhanced CT, single-photon emission CT, positron emission tomography, magnetic resonance angiography, MR perfusion, functional magnetic resonance imaging with global and regional blood oxygen level dependent imaging, and magnetic resonance angiography with the use of the noninvasive optional vessel analysis software (River Forest, Ill). In addition to a brief overview of the technique, this review article discusses the clinical indications, advantages, and disadvantages of each of those modalities.

Feasibility of simultaneous sodium fluorescein and indocyanine green injection in neurosurgical procedures

OBJECTIVE

The objective of this study is to assess the feasibility of simultaneous Sodium Fluorescein (SF) and Indocyanine Green (ICG) injection during neurosurgical procedures.

PATIENTS AND METHODS

Three patients harboring a high-grade glioma (HGG) were retrospectively identified in the surgical database of the Neurosurgical Unit 2 at the Foundation IRCCS Istituto Neurologico C. Besta in Milan, by having received intraoperatively both SF for tumor resection and ICG for vasculature angiographic studies in the same surgical procedure. We identified 2 males and 1 female (age range 25-60). Lesions were located in the left temporo-polar area and hippocampus (1 case), right superior frontal gyrus (1 case), left supplementary motor area (1 case). All the three lesions showed Magnetic Resonance Imaging (MRI) characteristics of HGG and, for this reason, in all patients a fluorescein-guided tumor removal was proposed. In the same surgical procedure ICG videoangiography was considered necessary in order to study arterial and venous vasculature, given by the strict relation of the tumor with an unexpected Posterior Communicating Artery (PComA) aneurysm in one case and with cortical drainage veins complexes in the other two cases. In all cases a microscope equipped with both YELLOW560 and IR800 integrated filters (Pentero 900, Carl Zeiss, Oberkorchen, Germany) was used. Fluorescein was i.v. injected at a dose of 5mg/kg immediately after patient intubation. ICG was i.v. injected in bolus on demand of the operating surgeon at a dose of 12.5mg.

RESULTS

No side-effects related to simultaneous injection of SF and ICG were identified. In all three cases, the use of SF allowed to better visualize the tumor areas during surgical removal, thus leading to a radical resection until no macroscopic appearance of residual tumor mass and no fluorescence was visible in the surgical cavity. ICG videoangiography confirmed the patency of branches of internal carotid artery after clipping of an unexpected small PComA aneurysm found intraoperatively during tumor removal in one case, while in patient 2 and 3 it allowed to evaluate patency and study flow pattern in cortical drainage veins that were intimately related to the tumors and the way of the surgical approach. Postoperative MRI showed a Gross Total Resection of the tumors in all cases.

CONCLUSIONS

This study showed for the first time the feasibility of intravenous SF injection and ICG videoangiography in the same surgical procedure. The presence of different fluorescence filters on the same surgical microscope allows the surgeon to recognize and safely resect the tumor and simultaneously evaluate local brain vascularization.

Review of fluorescence guided surgery visualization and overlay techniques

In fluorescence guided surgery, data visualization represents a critical step between signal capture and display needed for clinical decisions informed by that signal. The diversity of methods for displaying surgical images are reviewed, and a particular focus is placed on electronically detected and visualized signals, as required for near-infrared or low concentration tracers. Factors driving the choices such as human perception, the need for rapid decision making in a surgical environment, and biases induced by display choices are outlined. Five practical suggestions are outlined for optimal display orientation, color map, transparency/alpha function, dynamic range compression, and color perception check.

Augmented microscopy: real-time overlay of bright-field and near-infrared fluorescence images

Intraoperative applications of near-infrared (NIR) fluorescent contrast agents can be aided by instrumentation capable of merging the view of surgical field with that of NIR fluorescence. We demonstrate augmented microscopy, an intraoperative imaging technique in which bright-field (real) and electronically processed NIR fluorescence (synthetic) images are merged within the optical path of a stereomicroscope. Under luminance of 100,000 lx, representing typical illumination of the surgical field, the augmented microscope detects 189 nM concentration of indocyanine green and produces a composite of the real and synthetic images within the eyepiece of the microscope at 20 fps. Augmentation described here can be implemented as an add-on module to visualize NIR contrast agents, laser beams, or various types of electronic data within the surgical microscopes commonly used in neurosurgical, cerebrovascular, otolaryngological, and ophthalmic procedures.