Practical Application of Augmented/Mixed Reality Technologies in Surgery of Abdominal Cancer Patients

A decade of progress: bringing mixed reality image-guided surgery systems in the operating room

Advancements in mixed reality (MR) have led to innovative approaches in image-guided surgery (IGS). In this paper, we provide a comprehensive analysis of the current state of MR in image-guided procedures across various surgical domains. Using the Data Visualization View (DVV) Taxonomy, we analyze the progress made since a 2013 literature review paper on MR IGS systems. In addition to examining the current surgical domains using MR systems, we explore trends in types of MR hardware used, type of data visualized, visualizations of virtual elements, and interaction methods in use. Our analysis also covers the metrics used to evaluate these systems in the operating room (OR), both qualitative and quantitative assessments, and clinical studies that have demonstrated the potential of MR technologies to enhance surgical workflows and outcomes. We also address current challenges and future directions that would further establish the use of MR in IGS.

More than meets the eye: Augmented reality in surgical oncology

BACKGROUND AND OBJECTIVES

In the field of surgical oncology, there has been a desire for innovative techniques to improve tumor visualization, resection, and patient outcomes. Augmented reality (AR) technology superimposes digital content onto the real-world environment, enhancing the user's experience by blending digital and physical elements. A thorough examination of AR technology in surgical oncology has yet to be performed.

METHODS

A scoping review of intraoperative AR in surgical oncology was conducted according to the guidelines and recommendations of The Preferred Reporting Items for Systematic Review and Meta-analyzes Extension for Scoping Reviews (PRISMA-ScR) framework. All original articles examining the use of intraoperative AR during surgical management of cancer were included. Exclusion criteria included virtual reality applications only, preoperative use only, fluorescence, AR not specific to surgical oncology, and study design (reviews, commentaries, abstracts).

RESULTS

A total of 2735 articles were identified of which 83 were included. Most studies (52) were performed on animals or phantom models, while the remaining included patients. A total of 1112 intraoperative AR surgical cases were performed across the studies. The most common anatomic site was brain (20 articles), followed by liver (16), renal (9), and head and neck (8). AR was most often used for intraoperative navigation or anatomic visualization of tumors or critical structures but was also used to identify osteotomy or craniotomy planes.

CONCLUSIONS

AR technology has been applied across the field of surgical oncology to aid in localization and resection of tumors.

Exploration of the application of augmented reality technology for teaching spinal tumor's anatomy and surgical techniques

Background

Augmented reality (AR) technology is gradually being applied in surgical teaching as an innovative teaching method. Developing innovative teaching methods to replicate clinical theory and practical teaching scenarios, simulate preoperative planning and training for bone tumor surgery, and offer enhanced training opportunities for young physicians to acquire and apply clinical knowledge is a crucial concern that impacts the advancement of the discipline and the educational standards for young orthopedic physicians.

Objective

This study explores the application effect of augmented reality technology in anatomy teaching and surgical clinical teaching for spinal tumor.

Methods

The method utilizes virtual reality and augmented reality technology to present a spinal tumor model and the surgical process of percutaneous vertebroplasty. We conducted a random selection of 12 students forming into the augmented reality teaching group and 13 students forming into the traditional teaching group among the 8-year medical students from Peking Union Medical College and Tsinghua University, ensuring that the age and learning stage of the students in both groups were similar. Two groups of students were taught using traditional teaching methods and augmented reality technology-assisted teaching methods, respectively. A questionnaire survey was conducted after class to assess the quality of course instruction, student motivation in learning, their proficiency in anatomical structures, their comprehension of spinal tumor growth and metastasis, and their understanding and proficiency in percutaneous vertebroplasty.

Results

This study was the first to apply augmented reality technology in teaching, using spinal tumors and percutaneous vertebroplasty as examples, a head-mounted augmented reality device was used to create learning scenarios, presenting the complex three-dimensional spatial structure intuitively. The two groups of students differ significantly in their rating of teaching quality, enthusiasm for learning, knowledge of anatomical features, understanding of spinal trabecular structure, and understanding of steps in percutaneous vertebroplasty. The augmented reality technology-assisted teaching system demonstrates outstanding advantages.

Conclusion

Augmented reality technology has great potential and broad prospects in teaching bone tumors, which can help improve the visualization, interactivity, and three-dimensional spatial sense of medical teaching in spinal tumor. The application and development prospects of using augmented reality technology for anatomy instruction, surgical teaching, and simulation training are extensive.

Experience in the Application of Augmented Reality Technology in the Surgical Treatment of Patients Suffering Primary and Recurrent Pelvic Tumors

Surgical treatment of locally spread tumors in pelvic organs remains an urgent and complicated oncological problem. The recurrence rate after radical treatment ranges from 15.1% to 45.2%. The key to successful and safe surgical intervention lies in meticulous planning and intraoperative navigation, including the utilization of augmented reality (AR) technology. This paper presents the experience of clinically testing an AR technology application algorithm in the surgical treatment of 11 patients. The main stages of the algorithm are described. Radical operations incorporating intraoperative AR technology with favorable outcomes were performed on eight patients. One patient underwent a palliative intervention, while two patients did not undergo surgery. The testing of the algorithm for the application of AR technology in the surgical treatment of primary and recurrent pelvic tumors demonstrated both a technical possibility and reproducibility of this algorithm and the AR technology itself in clinical practice.

The Multidomain Metaverse Cancer Care Digital Platform: Development and Usability Study

BACKGROUND

As cancer treatment methods have diversified and the importance of self-management, which lowers the dependence rate on direct hospital visits, has increased, effective cancer care education and management for health professionals and patients have become necessary. The metaverse is in the spotlight as a means of digital health that allows users to engage in cancer care education and management beyond physical constraints. However, it is difficult to find a multipurpose medical metaverse that can not only be used in the field but also complements current cancer care.

OBJECTIVE

This study aimed to develop an integrated metaverse cancer care platform, Dr. Meta, and examine its usability.

METHODS

We conducted a multicenter, cross-sectional survey between November and December 2021. A descriptive analysis was performed to examine users' experiences with Dr. Meta. In addition, a supplementary open-ended question was used to ask users for their suggestions and improvements regarding the platform.

RESULTS

Responses from 70 Korean participants (male: n=19, 27% and female: n=51, 73%) were analyzed. More than half (n=37, 54%) of the participants were satisfied with Dr. Meta; they responded that it was an interesting and immersive platform (n=50, 72%). Less than half perceived no discomfort when using Dr. Meta (n=34, 49%) and no difficulty in wearing and operating the device (n=30, 43%). Furthermore, more than half (n=50, 72%) of the participants reported that Dr. Meta would help provide non-face-to-face and noncontact services. More than half also wanted to continue using this platform in the future (n=41, 59%) and recommended it to others (n=42, 60%).

CONCLUSIONS

We developed a multidomain metaverse cancer care platform that can support both health professionals and patients in non-face-to-face cancer care. The platform was uniquely disseminated and implemented in multiple regional hospitals and showed the potential to perform successful cancer care.

Use of augmented reality in mental health-related conditions: A systematic review

Objective

Augmented reality (AR) is a relatively new technology that merges virtual and physical environments, augmenting one's perception of reality. AR creates a computer-generated environment that evokes a unique perception of reality, where real and virtual objects are registered with one another, which operates interactively and in real time. Recently, the medical application of AR technology has dramatically increased with other assisted technologies, from training to clinical practice. The ability to manipulate the real environment extensively has given AR interventions an advantage over traditional approaches. In this study, we aim to conduct a systematic review of the use of AR to have a better understanding of how the use of AR may affect patients with mental health-related conditions when combined with gamification.

Method

This systematic review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines by searching Pubmed and Web of Science databases.

Results and Conclusion

We identified 48 relevant studies that fulfill the criteria. The studies were grouped into four categories: Neurodevelopmental disorders, anxiety and phobia, psychoeducation & well-being, and procedural & pain management. Our results revealed the effectiveness of AR in mental health-related conditions. However, the heterogeneity and small sample sizes demonstrate the need for further research with larger sample sizes and high-quality study designs.