Computer-vision based analysis of the neurosurgical scene - A systematic review

Introduction

With increasing use of robotic surgical adjuncts, artificial intelligence and augmented reality in neurosurgery, the automated analysis of digital images and videos acquired over various procedures becomes a subject of increased interest. While several computer vision (CV) methods have been developed and implemented for analyzing surgical scenes, few studies have been dedicated to neurosurgery.

Research question

In this work, we present a systematic literature review focusing on CV methodologies specifically applied to the analysis of neurosurgical procedures based on intra-operative images and videos. Additionally, we provide recommendations for the future developments of CV models in neurosurgery.

Material and methods

We conducted a systematic literature search in multiple databases until January 17, 2023, including Web of Science, PubMed, IEEE Xplore, Embase, and SpringerLink.

Results

We identified 17 studies employing CV algorithms on neurosurgical videos/images. The most common applications of CV were tool and neuroanatomical structure detection or characterization, and to a lesser extent, surgical workflow analysis. Convolutional neural networks (CNN) were the most frequently utilized architecture for CV models (65%), demonstrating superior performances in tool detection and segmentation. In particular, mask recurrent-CNN manifested most robust performance outcomes across different modalities.

Discussion and conclusion

Our systematic review demonstrates that CV models have been reported that can effectively detect and differentiate tools, surgical phases, neuroanatomical structures, as well as critical events in complex neurosurgical scenes with accuracies above 95%. Automated tool recognition contributes to objective characterization and assessment of surgical performance, with potential applications in neurosurgical training and intra-operative safety management.

Exploring technology acceptance of head-mounted device-based augmented reality surgical navigation in orthopaedic surgery

BACKGROUND

This study used the Unified Theory of Acceptance and Use of Technology (UTAUT) to investigate the acceptance of HMD-based AR surgical navigation.

METHODS

An experiment was conducted in which participants drilled 12 predefined holes using freehand drilling, proprioceptive control, and AR assistance. Technology acceptance was assessed through a survey and non-participant observations.

RESULTS

Participants' intention to use AR correlated (p < 0.05) with social influence (Spearman's rho (rs) = 0.599), perceived performance improvement (rs = 0.592) and attitude towards AR (rs = 0.542).

CONCLUSIONS

While most participants acknowledged the potential of AR, they also highlighted persistent barriers to adoption, such as issues related to user-friendliness, time efficiency and device discomfort. To overcome these challenges, future AR surgical navigation systems should focus on enhancing surgical performance while minimising disruptions to workflows and operating times. Engaging orthopaedic surgeons in the development process can facilitate the creation of tailored solutions and accelerate adoption.

Non-programmable shunts for communicating hydrocephalus and 3D volumetry: a retrospective analysis

OBJECTIVE

Although the use of different types of valves have been extensively studied in shunt surgeries for communicating hydrocephalus (cHC), a consensus about the valve type remains absent. The objective of this study is to evaluate our results with the primary placement of non-programmable valves (NPV) for this indication.

METHODS

We retrospectively analyzed all first NPV implanted between 2014-2020 for cHC. We studied the revision rate, clinical outcome described by modified Ranking Scale (mRS) and radiological evolution using Evans index (EI) and ventricular volumes 3D-semiautomatic segmentation (vv-3DSAS).

RESULTS

Forty-one patients were shunted for post-hemorrhagic (61%), post-traumatic (24.4%) and tumoral (14.6%) hydrocephalus (HC). Mean age was 65 years (range 25-89yrs). Overall, 59 procedures were performed including 18 revision surgeries in 12 patients (29.3%). Underlying reason for first shunt revision were valve type related: valve dysfunction, overdrainage, underdrainage and non-valve type related: malpositioning, infection, shunt migration. The shunt-related revision rate was 17.1%. Twenty-eight patients (68.3%) had an mRS improvement of 1 or more points. We found a good correlation between ventricle volumes (VV) and EI and a significant reduction in VV measured by EI and vv-3DSAS was observed. However, the mRS improvement was not correlated with a reduction of ventricle volumes.

CONCLUSION

Overall, our results in terms of shunt revisions as well as clinical and radiological evolution are comparable with the literature for NPV. vv-3DSAS can be used and could be useful to detect small changes in VV in patients with cHC.

Neuro-oncological augmented reality planning for intracranial tumor resection

Background

Before starting surgery for the resection of an intracranial tumor, its outlines are typically marked on the skin of the patient. This allows for the planning of the optimal skin incision, craniotomy, and angle of approach. Conventionally, the surgeon determines tumor borders using neuronavigation with a tracked pointer. However, interpretation errors can lead to important deviations, especially for deep-seated tumors, potentially resulting in a suboptimal approach with incomplete exposure. Augmented reality (AR) allows displaying of the tumor and critical structures directly on the patient, which can simplify and improve surgical preparation.

Methods

We developed an AR-based workflow for intracranial tumor resection planning deployed on the Microsoft HoloLens II, which exploits the built-in infrared-camera for tracking the patient. We initially performed a phantom study to assess the accuracy of the registration and tracking. Following this, we evaluated the AR-based planning step in a prospective clinical study for patients undergoing resection of a brain tumor. This planning step was performed by 12 surgeons and trainees with varying degrees of experience. After patient registration, tumor outlines were marked on the patient's skin by different investigators, consecutively using a conventional neuronavigation system and an AR-based system. Their performance in both registration and delineation was measured in terms of accuracy and duration and compared.

Results

During phantom testing, registration errors remained below 2.0 mm and 2.0° for both AR-based navigation and conventional neuronavigation, with no significant difference between both systems. In the prospective clinical trial, 20 patients underwent tumor resection planning. Registration accuracy was independent of user experience for both AR-based navigation and the commercial neuronavigation system. AR-guided tumor delineation was deemed superior in 65% of cases, equally good in 30% of cases, and inferior in 5% of cases when compared to the conventional navigation system. The overall planning time (AR = 119 ± 44 s, conventional = 187 ± 56 s) was significantly reduced through the adoption of the AR workflow (p &lt; 0.001), with an average time reduction of 39%.

Conclusion

By providing a more intuitive visualization of relevant data to the surgeon, AR navigation provides an accurate method for tumor resection planning that is quicker and more intuitive than conventional neuronavigation. Further research should focus on intraoperative implementations.

SURG-37. NEURO-ONCOLOGICAL AUGMENTED REALITY: A MORE INTUITIVE APPROACH TO RESECTION PLANNING

BACKGROUND

When preparing for the resection of an intracranial lesion, its borders and optimal approach are often determined using neuronavigation with a tracked pointer. This can sometimes prove challenging, especially for deep-seated lesions. Augmented reality (AR) can simplify and improve this step by directly displaying the lesion on the patient's skin.

METHODS

A proprietary inside-out infrared tracking solution was developed, allowing for heads-up displaying AR scenes on the Microsoft HoloLens II without the need for an external tracking camera or computer. We included twenty patients with an intracerebral lesion planned for resection. After semi-automatic hologram-to-patient registration, different participants marked the lesion outlines on the patient’s skin, consecutively aided by the Brainlab neuronavigation system and the HoloLens. Each registration on both systems provided a registration transform that was compared for accuracy and consistency. Participant performance was quantified in terms of duration and accuracy for both patient registration and lesion delineation, and compared to expert performance.

RESULTS

When using AR both registration and delineation were significantly faster than with conventional neuronavigation (p = 0.02 and p &lt; 0.001, respectively, and p &lt; 0.001 for the total duration), taking 79.23 ± 17.48 and 39.58 ± 39.10 seconds while neuronavigation required 96.61 ± 24.54 and 90.80 ± 44.09 seconds. AR had a registration offset of 3.3mm and 3.4°, and was more consistent compared to neuronavigation. AR facilitated more accurate and detailed lesion delineation, while neuronavigation often overestimated lesion size.

CONCLUSION

Augmented reality provides a faster and more accurate alternative for resection planning. Lesion delineation was more intuitive while retaining high accuracy. Future research should focus on further intraoperative implementations.

The effect of augmented reality on the accuracy and learning curve of external ventricular drain placement

OBJECTIVE

The traditional freehand technique for external ventricular drain (EVD) placement is most frequently used, but remains the primary risk factor for inaccurate drain placement. As this procedure could benefit from image guidance, the authors set forth to demonstrate the impact of augmented-reality (AR) assistance on the accuracy and learning curve of EVD placement compared with the freehand technique.

METHODS

Sixteen medical students performed a total of 128 EVD placements on a custom-made phantom head, both before and after receiving a standardized training session. They were guided by either the freehand technique or by AR, which provided an anatomical overlay and tailored guidance for EVD placement through inside-out infrared tracking. The outcome was quantified by the metric accuracy of EVD placement as well as by its clinical quality.

RESULTS

The mean target error was significantly impacted by either AR (p = 0.003) or training (p = 0.02) in a direct comparison with the untrained freehand performance. Both untrained (11.9 ± 4.5 mm) and trained (12.2 ± 4.7 mm) AR performances were significantly better than the untrained freehand performance (19.9 ± 4.2 mm), which improved after training (13.5 ± 4.7 mm). The quality of EVD placement as assessed by the modified Kakarla scale (mKS) was significantly impacted by AR guidance (p = 0.005) but not by training (p = 0.07). Both untrained and trained AR performances (59.4% mKS grade 1 for both) were significantly better than the untrained freehand performance (25.0% mKS grade 1). Spatial aptitude testing revealed a correlation between perceptual ability and untrained AR-guided performance (r = 0.63).

CONCLUSIONS

Compared with the freehand technique, AR guidance for EVD placement yielded a higher outcome accuracy and quality for procedure novices. With AR, untrained individuals performed as well as trained individuals, which indicates that AR guidance not only improved performance but also positively impacted the learning curve. Future efforts will focus on the translation and evaluation of AR for EVD placement in the clinical setting.

Augmented Reality-Assisted Neurosurgical Drain Placement (ARANED): Technical Note

BACKGROUND

Many surgical procedures, such as placement of intracranial drains, are currently being performed blindly, relying on anatomical landmarks. As a result, accuracy results still have room for improvement. Neuronavigation could address this issue, but its application in an urgent setting is often impractical. Augmented reality (AR) provided through a head-worn device has the potential to tackle this problem, but its implementation should meet physicians' needs.

METHODS

The Surgical Augmented Reality Assistance (SARA) project aims to develop an AR solution that is suitable for preoperative planning, intraoperative visualisation and navigational support in an everyday clinical setting, using a Microsoft HoloLens.

RESULTS

Proprietary hardware and software adaptations and dedicated navigation algorithms are applied to the Microsoft HoloLens to optimise it specifically for neurosurgical navigation. This includes a pipeline with an additional set of advanced, semi-automated algorithms responsible for image processing, hologram-to-patient registration and intraoperative tracking using infrared depth-sensing. A smooth and efficient workflow while maintaining high accuracy is prioritised. The AR solution provides a fully integrated and completely mobile navigation setup. Initial preclinical and clinical validation tests applying the solution to intracranial drain placement are described.

CONCLUSION

AR has the potential to vastly increase accuracy of everyday procedures that are frequently performed without image guidance, but could still benefit from navigational support, such as intracranial drain placements. Technical development should go hand in hand with preclinical and clinical validation in order to demonstrate improvements in accuracy and clinical outcomes.

Richter Transformation of Chronic Lymphocytic Leukemia: A Review of Fluorodeoxyglucose Positron Emission Tomography-Computed Tomography and Molecular Diagnostics

Chronic lymphocytic leukemia (CLL) is a low-grade B-cell proliferative disease with a generally indolent course. In a few cases, it undergoes transformation and becomes a more aggressive malignancy, such as diffuse large B-cell lymphoma (DLBCL). This process, which is called Richter transformation (RT), is often detected too late and is associated with a poor prognosis. There are multiple molecular diagnostic approaches to detect RT in preexisting CLL. Metabolic imaging using 18-fluorine fluorodeoxyglucose positron emission tomography–computed tomography (18F-FDG PET/CT) can be a very useful tool for early detection of RT and which can hence allow for timely intervention, thereby improving the patient’s chances of survival.

Molecular Imaging in Genetic Medicine

The field of biomedical imaging has made significant advances in recent times. This includes extremely high-resolution anatomic imaging and functional imaging of physiologic and pathologic processes as well as novel modalities in optical imaging to evaluate molecular features within the cellular environment. The latter has made it possible to image phenotypic markers of various genotypes that are implicated in human development, behavior, and disease. This article discusses the role of molecular imaging in genetic and precision medicine.