Augmented reality-assisted roadmaps during periventricular brain surgery

A decade of progress: bringing mixed reality image-guided surgery systems in the operating room

Advancements in mixed reality (MR) have led to innovative approaches in image-guided surgery (IGS). In this paper, we provide a comprehensive analysis of the current state of MR in image-guided procedures across various surgical domains. Using the Data Visualization View (DVV) Taxonomy, we analyze the progress made since a 2013 literature review paper on MR IGS systems. In addition to examining the current surgical domains using MR systems, we explore trends in types of MR hardware used, type of data visualized, visualizations of virtual elements, and interaction methods in use. Our analysis also covers the metrics used to evaluate these systems in the operating room (OR), both qualitative and quantitative assessments, and clinical studies that have demonstrated the potential of MR technologies to enhance surgical workflows and outcomes. We also address current challenges and future directions that would further establish the use of MR in IGS.

Division of the iliac vessels in the anterior intrapelvic approach to acetabular fracture

INTRODUCTION

The modified Stoppa approach is gradually becoming the gold standard in pelvic ring and acetabulum surgery. One of the potential intraoperative complications is vascular injury. The aim of this study was to identify the level of division of common iliac vessels with respect to a bone landmark, their inter-individual variability and their correlation with morphological criteria.

MATERIAL AND METHODS

This was a single-center continuous retrospective study of patients who had preoperative CT angiography for pelvic fracture between February 2017 and May 2018. The level of arterial and venous division and the angle of vein division were measured bilaterally for each patient from the most antero-inferior part of the sacroiliac joint on multiplanar reconstruction and standardized analysis. Relationships with morphological data (age, gender, BMI, height), anterior column fracture and deep venous thrombosis were analyzed.

RESULTS

The right arterial division level was 50±16mm (-2.35; 96) from the landmark and the left arterial division level 44±14mm (0; 80). The right venous division level was 30±12mm (-9; 75) and the left venous division level 30±13mm (-5; 66). The right venous bifurcation angle was 65±18° (22; 119) and the left venous bifurcation angle 68±17° (18; 117). The arterial division level was significantly higher on the right side (p=0.007). There were no significant correlations with morphological data.

CONCLUSION

The great inter-individual variability of iliac vessels should prompt analysis of their morphology on routine imaging when planning pelvic surgery using the modified Stoppa approach, in order to anticipate the risk of bleeding.

LEVEL OF EVIDENCE

IV; cases series.

Corona mortis and pelvic dissection: Understanding the relationship between anatomical structures and bone areas

Pelvic fractures are becoming increasingly frequent. The gold standard for surgical managements remains open procedures. Despite its excellent biomechanically results, it can lead to many complications. Minimally invasive surgery could reduce these complications. For complex pelvic trauma, extraperitoneal endoscopic technique has never been described. The aim of this study is to determine anatomical landmarks which are useful for endoscopic pelvic ring surgery using an extraperitoneal approach. The second objective is to compare this minimally invasive procedure to expose the bone versus a traditional open approach. After preparing the vessels with latex injections, 10 specimens are dissected alternately, using an endoscopic method (MIS) on one side and an open method on the other side. Both procedures are performed on the same subject. The visualized bone areas are drilled with burr holes. The marked surfaces are measured with photogrammetry. Finally, the data are processed (surface analysis). An extraperitoneal endoscopic dissection that follows anatomical landmarks can be performed. Bone area (mm2 ) visualized by endoscopy was 74 ± 14 (59-94) compared to 71 ± 16 (48-94) by open method. Paired t-test was performed with no significant difference between the two methods. Skin and muscular incisions were significantly lower in the MIS group (5.1, IC95% [4.1; 6.1], p < 0.001). An extraperitoneal endoscopic dissection of the pelvis can be performed. We also find no significant difference between our method and an open traditional approach concerning bone exposure. We offer a holistic approach to treat pelvic fractures by identifying key anatomical structures.

Validation of real-time inside-out tracking and depth realization technologies for augmented reality-based neuronavigation

PURPOSE

Concomitant with the significant advances in computing technology, the utilization of augmented reality-based navigation in clinical applications is being actively researched. In this light, we developed novel object tracking and depth realization technologies to apply augmented reality-based neuronavigation to brain surgery.

METHODS

We developed real-time inside-out tracking based on visual inertial odometry and a visual inertial simultaneous localization and mapping algorithm. The cube quick response marker and depth data obtained from light detection and ranging sensors are used for continuous tracking. For depth realization, order-independent transparency, clipping, and annotation and measurement functions were developed. In this study, the augmented reality model of a brain tumor patient was applied to its life-size three-dimensional (3D) printed model.

RESULTS

Using real-time inside-out tracking, we confirmed that the augmented reality model remained consistent with the 3D printed patient model without flutter, regardless of the movement of the visualization device. The coordination accuracy during real-time inside-out tracking was also validated. The average movement error of the X and Y axes was 0.34 ± 0.21 and 0.04 ± 0.08 mm, respectively. Further, the application of order-independent transparency with multilayer alpha blending and filtered alpha compositing improved the perception of overlapping internal brain structures. Clipping, and annotation and measurement functions were also developed to aid depth perception and worked perfectly during real-time coordination. We named this system METAMEDIP navigation.

CONCLUSIONS

The results validate the efficacy of the real-time inside-out tracking and depth realization technology. With these novel technologies developed for continuous tracking and depth perception in augmented reality environments, we are able to overcome the critical obstacles in the development of clinically applicable augmented reality neuronavigation.

Mixed Reality for Cranial Neurosurgical Planning: A Single-Center Applicability Study With the First 107 Subsequent Holograms

BACKGROUND AND OBJECTIVES

Mixed reality (MxR) benefits neurosurgery by improving anatomic visualization, surgical planning and training. We aim to validate the usability of a dedicated certified system for this purpose.

METHODS

All cases prepared with MxR in our center in 2022 were prospectively collected. Holographic rendering was achieved using an incorporated fully automatic algorithm in the MxR application, combined with contrast-based semiautomatic rendering and/or manual segmentation where necessary. Hologram segmentation times were documented. Visualization during surgical preparation (defined as the interval between finalized anesthesiological induction and sterile draping) was performed using MxR glasses and direct streaming to a side screen. Surgical preparation times were compared with a matched historical cohort of 2021. Modifications of the surgical approach after 3-dimensional (3D) visualization were noted. Usability was assessed by evaluating 7 neurosurgeons with more than 3 months of experience with the system using a Usefulness, Satisfaction and Ease of use (USE) questionnaire.

RESULTS

One hundred-seven neurosurgical cases prepared with a 3D hologram were collected. Surgical indications were oncologic (63/107, 59%), cerebrovascular (27/107, 25%), and carotid endarterectomy (17/107, 16%). Mean hologram segmentation time was 39.4 ± 20.4 minutes. Average surgical preparation time was 48.0 ± 17.3 minutes for MxR cases vs 52 ± 17 minutes in the matched 2021 cohort without MxR (mean difference 4, 95% CI 1.7527-9.7527). Based on the 3D hologram, the surgical approach was modified in 3 cases. Good usability was found by 57% of the users.

CONCLUSION

The perioperative use of 3D holograms improved direct anatomic visualization while not significantly increasing intraoperative surgical preparation time. Usability of the system was adequate. Further technological development is necessary to improve the automatic algorithms and reduce the preparation time by circumventing manual and semiautomatic segmentation. Future studies should focus on quantifying the potential benefits in teaching, training, and the impact on surgical and functional outcomes.

Evaluation of augmented reality technology in global urologic surgery

BACKGROUND

The COVID-19 pandemic drastically reduced opportunities for surgical skill sharing between high-income and low to middle-income countries. Augmented reality (AR) technology allows mentors in one country to virtually train a mentee in another country during surgical cases without international travel. We hypothesize that AR technology is an effective live surgical training and mentorship modality.

METHODS

Three senior urologic surgeons in the US and UK worked with four urologic surgeon trainees across the continent of Africa using AR systems. Trainers and trainees individually completed post-operative questionnaires evaluating their experience.

RESULTS

Trainees rated the quality of virtual training as equivalent to in-person training in 83% of cases (N = 5 of 6 responses). Trainers reported the technology's visual quality as "acceptable" in 67% of cases (N = 12 of 18 responses). The audiovisual capabilities of the technology had a "high" impact in the majority of the cases.

CONCLUSION

AR technology can effectively facilitate surgical training when in-person training is limited or unavailable.

Augmented reality in liver surgery

INTRODUCTION

During an operation, augmented reality (AR) enables surgeons to enrich their vision of the operating field by means of digital imagery, particularly as regards tumors and anatomical structures. While in some specialties, this type of technology is routinely ustilized, in liver surgery due to the complexity of modeling organ deformities in real time, its applications remain limited. At present, numerous teams are attempting to find a solution applicable to current practice, the objective being to overcome difficulties of intraoperative navigation in an opaque organ.

OBJECTIVE

To identify, itemize and analyze series reporting AR techniques tested in liver surgery, the objectives being to establish a state of the art and to provide indications of perspectives for the future.

METHODS

In compliance with the PRISMA guidelines and availing ourselves of the PubMed, Embase and Cochrane databases, we identified English-language articles published between January 2020 and January 2022 corresponding to the following keywords: augmented reality, hepatic surgery, liver and hepatectomy.

RESULTS

Initially, 102 titles, studies and summaries were preselected. Twenty-eight corresponding to the inclusion criteria were included, reporting on 183patients operated with the help of AR by laparotomy (n=31) or laparoscopy (n=152). Several techniques of acquisition and visualization were reported. Anatomical precision was the main assessment criterion in 19 articles, with values ranging from 3mm to 14mm, followed by time of acquisition and clinical feasibility.

CONCLUSION

While several AR technologies are presently being developed, due to insufficient anatomical precision their clinical applications have remained limited. That much said, numerous teams are currently working toward their optimization, and it is highly likely that in the short term, the application of AR in liver surgery will have become more frequent and effective. As for its clinical impact, notably in oncology, it remains to be assessed.

Machine Learning-Based Surgical Planning for Neurosurgery: Artificial Intelligent Approaches to the Cranium

Objectives

Artificial intelligence (AI) applications in neurosurgery have an increasing momentum as well as the growing number of implementations in the medical literature. In recent years, AI research define a link between neuroscience and AI. It is a connection between knowing and understanding the brain and how to simulate the brain. The machine learning algorithms, as a subset of AI, are able to learn with experiences, perform big data analysis, and fulfill human-like tasks. Intracranial surgical approaches that have been defined, disciplined, and developed in the last century have become more effective with technological developments. We aimed to define individual-safe, intracranial approaches by introducing functional anatomical structures and pathological areas to artificial intelligence.

Methods

Preoperative MR images of patients with deeply located brain tumors were used for planning. Intracranial arteries, veins, and neural tracts are listed and numbered. Voxel values of these selected regions in cranial MR sequences were extracted and labeled. Tumor tissue was segmented as the target. Q-learning algorithm which is a model-free reinforcement learning algorithm was run on labeled voxel values (on optimal paths extracted from the new heuristic-based path planning algorithm), then the algorithm was assigned to list the cortico-tumoral pathways that aim to remove the maximum tumor tissue and in the meantime that functional anatomical tissues will be least affected.

Results

The most suitable cranial entry areas were found with the artificial intelligence algorithm. Cortico-tumoral pathways were revealed using Q-learning from these optimal points.

Conclusions

AI will make a significant contribution to the positive outcomes as its use in both preoperative surgical planning and intraoperative technique equipment assisted neurosurgery, its use increased.