Investigating the utility of VR for spatial understanding in surgical planning: evaluation of head-mounted to desktop display

Virtual Reality-Enhanced Assessment of the Anterior Glottic Angle Using Cadaveric Models: A Proof-of-Concept Study

OBJECTIVES

Vocal fold paralysis (VFP) is a complex disorder that affects voice, speech, swallowing, and overall quality of life. Current evaluation methods for determining the position of paralyzed vocal folds lack the objectivity required for personalized interventions and research on innovative treatments for VFP. This study was designed to validate the accuracy and reproducibility of a virtual reality (VR)-based platform to measure the anterior glottic angle (AGA), a critical component in determining the position of a paralyzed vocal fold.

STUDY DESIGN

Retrospective.

METHODS

A retrospective analysis of computed tomography (CT) scans of 39 adult patients was conducted to measure the AGA shortly after death. Two measurement methods were used: 2-dimensional (2D)-CT for direct measurements on 2D images and a 3-dimensional (3D)-VR method utilizing a dedicated platform to create a 3D VR model of the larynx. The AGA measurements conducted by two senior otolaryngologists using the 3D-VR method were compared to the 2D-CT measurements made by one of these same otolaryngologists.

RESULTS

The mean AGA measured by the 3D-VR method was found to be 32.936 ± 6.486° (n = 39), and the measurements were highly correlated (r = 0.9670, P < 0.0001). By contrast, the 2D-CT method without VR yielded a significantly lower mean angle of 23.754° (n = 35) with a higher standard deviation of 10.365°. The 3D-VR method demonstrated excellent reliability for AGA measurements (intraclass correlation coefficient of 0.954).

CONCLUSION

The findings underscore the potential value of using a VR-based platform to improve reproducibility and reduce the variability in measurements of AGA in cases of VFP.

Virtual Reality Helps Describe the Progression of Thyroid Cartilage Calcification

OBJECTIVES

Thyroid cartilage (TC) calcifications may impact surgical planning and clinical management. However, few studies to date have implemented virtual reality (VR) to evaluate these calcifications. This study assessed the feasibility of evaluating TC calcifications in various regions and measuring their volumes through VR models generated from computed tomography scans. We also investigated age and gender-related differences in calcification patterns.

METHODS

Ninety-two participants were categorized into younger, middle-aged, and older age groups. Calcification patterns (degree in Hounsfield units and volume of calcification in cm3) in different TC regions were identified by VR analysis, which enabled comparisons between age groups and genders.

RESULTS

Significant differences in calcification patterns were observed between males and females, particularly in the middle right, middle left, bottom left, and vertex regions. Age-related differences in the vertex region showed increased calcification in the older age group.

CONCLUSION

This study points to the contribution of VR in the evaluation of complex anatomical structures. The findings revealed significant gender and age patterns in TC calcification. These insights can inform surgical planning and highlight the potential of using VR to gain a better understanding of TC calcification clinically.

Metaverse Wearables for Immersive Digital Healthcare: A Review

The recent exponential growth of metaverse technology has been instrumental in reshaping a myriad of sectors, not least digital healthcare. This comprehensive review critically examines the landscape and future applications of metaverse wearables toward immersive digital healthcare. The key technologies and advancements that have spearheaded the metamorphosis of metaverse wearables are categorized, encapsulating all-encompassed extended reality, such as virtual reality, augmented reality, mixed reality, and other haptic feedback systems. Moreover, the fundamentals of their deployment in assistive healthcare (especially for rehabilitation), medical and nursing education, and remote patient management and treatment are investigated. The potential benefits of integrating metaverse wearables into healthcare paradigms are multifold, encompassing improved patient prognosis, enhanced accessibility to high-quality care, and high standards of practitioner instruction. Nevertheless, these technologies are not without their inherent challenges and untapped opportunities, which span privacy protection, data safeguarding, and innovation in artificial intelligence. In summary, future research trajectories and potential advancements to circumvent these hurdles are also discussed, further augmenting the incorporation of metaverse wearables within healthcare infrastructures in the post-pandemic era.

Technological advancements in surgical laparoscopy considering artificial intelligence: a survey among surgeons in Germany

PURPOSE

The integration of artificial intelligence (AI) into surgical laparoscopy has shown promising results in recent years. This survey aims to investigate the inconveniences of current conventional laparoscopy and to evaluate the attitudes and desires of surgeons in Germany towards new AI-based laparoscopic systems.

METHODS

A 12-item web-based questionnaire was distributed to 38 German university hospitals as well as to a Germany-wide voluntary hospital association (CLINOTEL) consisting of 66 hospitals between July and November 2022.

RESULTS

A total of 202 questionnaires were completed. The majority of respondents (88.1%) stated that they needed one assistant during laparoscopy and rated the assistants' skillfulness as "very important" (39.6%) or "important" (49.5%). The most uncomfortable aspects of conventional laparoscopy were inappropriate camera movement (73.8%) and lens condensation (73.3%). Selected features that should be included in a new laparoscopic system were simple and intuitive maneuverability (81.2%), automatic de-fogging (80.7%), and self-cleaning of camera (77.2%). Furthermore, AI-based features were improvement of camera positioning (71.3%), visualization of anatomical landmarks (67.3%), image stabilization (66.8%), and tissue damage protection (59.4%). The reason for purchasing an AI-based system was to improve patient safety (86.1%); the reasonable price was €50.000-100.000 (34.2%), and it was expected to replace the existing assistants' workflow up to 25% (41.6%).

CONCLUSION

Simple and intuitive maneuverability with improved and image-stabilized camera guidance in combination with a lens cleaning system as well as AI-based augmentation of anatomical landmarks and tissue damage protection seem to be significant requirements for the further development of laparoscopic systems.

The Impact of Virtual Interactive Three-Dimensional Model in the Conceptualization of the Anatomy of the Sacrum: A Randomized Controlled Trial

Introduction Virtual interactive three-dimensional model (VI3DM) is an emerging technology with promising futures in medical education. It allows learners to view and interact with the three-dimensional (3D) object in an isolated virtual environment, as well as on screen-based platforms. This technology seems more helpful in understanding the learning objectives that demand high cognitive and visuospatial skills. The sacrum, part of the posterior wall of the bony pelvis, is a structure of interest to medical professionals and forensic experts. Understanding the anatomy and relations of the sacrum demands good spatial understanding. Hypothetically, virtual 3D models should help in learning the anatomy of the sacrum along with its relations and attachments. This study was conducted to find out the effect of low-cost digital 3D models on the anatomical knowledge of the study. Aims and objectives The goal of the work was to identify the role of virtual 3D models in the conceptualization of the anatomy of the sacrum. The study's objectives were to identify the impact of virtual 3D models on students' knowledge of the external features, relations, attachments, and joints formed by the sacrum. Material and methods Two hundred first-year medical students (168 males and 32 females) who participated in the study after providing informed consent were divided into two equal groups, a control group (n=100) and an experimental group (n=100), after matching the age, gender, and anatomical knowledge of the sacrum. We used two-dimensional (2D) images and virtual interactive 3D models of the sacrum as control and intervention, respectively, in this randomized controlled study. We conducted a post-test quiz after the 30-minute session of self-directed learning. Results The mean difference between the post-test score and the pre-test score of the experimental group (4.1±1.6 ) was significantly higher than the difference between the post-test and pre-test scores of the control group (2.5±1.2). The virtual interactive 3D model of the sacrum was significantly effective in the conceptualization of the sacrum anatomy. Conclusion A virtual interactive 3D model is an effective tool to conceptualize the anatomy of the sacrum and can be explored for its use in further complex anatomical structures. Digital 3D models can become a platform for the application of various virtual realities (VR) and artificial intelligences in medical education.

Virtual reality a technological miracle transforming physical rehabilitation: A scoping review

Physical rehabilitation is evolving day by day. In the same way, simulation in rehabilitation is increasing and has now become a cornerstone for rehabilitation programs. Increase in the number of new protocols, clinical methods, and treatment standardization, virtual reality is appearing as a new medium to deliver the simulation. Virtual reality gives the benefits of forming standardized treatment protocols on demand for various conditions repetitively with a cost-effective delivery system. This was an observational retrospective study. The PubMed database was used to obtain the available material related to virtual reality and rehabilitation and was searched using the same keywords. The articles were then sorted as the subject to the recent decade. The basic information was then obtained, which included timespan, sources of the document, average years of publication, document types we collected, and average citation per year per document. Analysis of the literature that was available online related to virtual reality and rehabilitation between 2011 and 2021 generated a list of 813 documents from 275 sources, of which 810 were from journal articles and 3 were book chapters with an average year of publication of 2.16. The highest number of publications was 480 in 2020, followed by 150 in 2019, 95 in 2018, and 28 in 2017. The annual growth rate percentage of scientific publications was 26.1%. Therefore, more studies should be performed on virtual reality.

Three-dimensional virtual reality in surgical planning for breast cancer with reconstruction

Breast surgery is performed to achieve local control in patients with breast cancer. Visualization of the anatomy with a virtual reality software platform reconstructed from magnetic resonance imaging data improves surgical planning with regards to volume and localization of the tumor, lymph nodes, blood vessels, and surrounding tissue to perform oncoplastic tissue rearrangement. We report the use and advantages of virtual reality added to the magnetic resonance imaging assessment in a 36-year-old woman with breast cancer who underwent nipple sparing mastectomy with tissue expander reconstruction.

Toward the validation of VR-HMDs for medical education: a systematic literature review

The latest technological advancements in the domain of virtual reality (VR) have created new opportunities to use VR as a training platform for medical students and practitioners more broadly. Despite the growing interest in the use of VR as a training tool, a commonly identified gap in VR-training for medical education is the confidence in the long-term validity of the applications. A systematic literature review was undertaken to explore the extent of VR (in particular head-mounted displays) applications for medical training with an additional focus on validation measures. The papers included in this review discussed empirical case studies of specific applications; however, these were mostly concerned with human-computer interaction and were polarized between demonstrating that a conceptual technology solution was feasible for simulation or looked at specific areas of VR usability with little discussion on validation measures for long-term training effectiveness and outcomes. The review uncovered a wide range of ad hoc applications and studies in terms of technology vendors, environments, tasks, envisaged users and effectiveness of learning outcomes. This presents decision-making challenges for those seeking to adopt, implement and embed such systems in teaching practice. The authors of this paper then take a wider socio-technical systems perspective to understand how the holistic training system can be engineered and validated effectively as fit for purpose, through distillation of a generic set of requirements from the literature review to aid design specification and implementation, and to drive more informed and traceable validation of these types of systems. In this review, we have identified 92 requirement statements in 11 key areas against which a VR-HMD training system could be validated; these were grouped into design considerations, learning mechanisms and implementation considerations.

Determinants of Learning Anatomy in an Immersive Virtual Reality Environment - A Scoping Review

Given the decline of cadavers as anatomy teaching tools, immersive virtual reality (VR) technology has gained popularity as a potential alternative. To better understand how to maximize the educational potential of VR, this scoping review aimed to identify potential determinants of learning anatomy in an immersive VR environment. A literature search yielded 4523 studies, 25 of which were included after screening. Six common factors were derived from secondary outcomes in these papers: cognitive load, cybersickness, student perceptions, stereopsis, spatial understanding, and interactivity. Further objective research investigating the impact of these factors on anatomy examination performance is required.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s40670-022-01701-y.

Real-time mixed reality display of dual particle radiation detector data

Radiation source localization and characterization are challenging tasks that currently require complex analyses for interpretation. Mixed reality (MR) technologies are at the verge of wide scale adoption and can assist in the visualization of complex data. Herein, we demonstrate real-time visualization of gamma ray and neutron radiation detector data in MR using the Microsoft HoloLens 2 smart glasses, significantly reducing user interpretation burden. Radiation imaging systems typically use double-scatter events of gamma rays or fast neutrons to reconstruct the incidence directional information, thus enabling source localization. The calculated images and estimated 'hot spots' are then often displayed in 2D angular space projections on screens. By combining a state-of-the-art dual particle imaging system with HoloLens 2, we propose to display the data directly to the user via the head-mounted MR smart glasses, presenting the directional information as an overlay to the user's 3D visual experience. We describe an open source implementation using efficient data transfer, image calculation, and 3D engine. We thereby demonstrate for the first time a real-time user experience to display fast neutron or gamma ray images from various radioactive sources set around the detector. We also introduce an alternative source search mode for situations of low event rates using a neural network and simulation based training data to provide a fast estimation of the source's angular direction. Using MR for radiation detection provides a more intuitive perception of radioactivity and can be applied in routine radiation monitoring, education & training, emergency scenarios, or inspections.

The use and future perspective of Artificial Intelligence-A survey among German surgeons

Purpose

Clinical abundance of artificial intelligence has increased significantly in the last decade. This survey aims to provide an overview of the current state of knowledge and acceptance of AI applications among surgeons in Germany.

Methods

A total of 357 surgeons from German university hospitals, academic teaching hospitals and private practices were contacted by e-mail and asked to participate in the anonymous survey.

Results

A total of 147 physicians completed the survey. The majority of respondents (n = 85, 52.8%) stated that they were familiar with AI applications in medicine. Personal knowledge was self-rated as average (n = 67, 41.6%) or rudimentary (n = 60, 37.3%) by the majority of participants. On the basis of various application scenarios, it became apparent that the respondents have different demands on AI applications in the area of "diagnosis confirmation" as compared to the area of "therapy decision." For the latter category, the requirements in terms of the error level are significantly higher and more respondents view their application in medical practice rather critically. Accordingly, most of the participants hope that AI systems will primarily improve diagnosis confirmation, while they see their ethical and legal problems with regard to liability as the main obstacle to extensive clinical application.

Conclusion

German surgeons are in principle positively disposed toward AI applications. However, many surgeons see a deficit in their own knowledge and in the implementation of AI applications in their own professional environment. Accordingly, medical education programs targeting both medical students and healthcare professionals should convey basic knowledge about the development and clinical implementation process of AI applications in different medical fields, including surgery.

Performance Influence Mechanism of Individuals’ Spatial Ability in Teleoperation

BACKGROUND: Teleoperation enables performing tasks in hazardous or inaccessible environments. The relationship between spatial ability and teleoperation performance has been extensively studied; however, relatively few studies have considered examples wherein the specific influence mechanism between the two is examined. This study aims to explore how individuals’ spatial ability affects teleoperation performance.METHODS: Forty subjects completed teleoperated expedition and escape tasks in a virtual unfamiliar environment according to the assigned requirements. After each expedition task, subjects’ mental model about the unfamiliar environment was evaluated. The escape task performance was measured in terms of path length, completion time, and the number of collisions. The impact of spatial ability on escape task performance wherein mental model as a mediator was examined. The Bootstrapping method was used to examine the hypothesis regarding the mediating role of mental model in the influence of spatial ability on teleoperation performance.RESULTS: Subjects with higher spatial ability exhibited significantly better mental models and had fewer collisions. In addition, subjects with better mental models had significantly shorter path lengths and spent marginally less time on escape. In general, the mental model appeared important for path length and completion time, but not collisions.CONCLUSIONS: The combined results of the two tasks preliminarily proved that spatial ability affected path length of the escape task through the mental model after the exploration task. The findings are expected to aid in astronaut selection and teleoperation training for space station missions.Pan D, Liu D, Tian Z, Zhang Y. Performance influence mechanism of individuals’ spatial ability in teleoperation. Aerosp Med Hum Perform. 2022; 93(10):717–724.

Mixed-Reality-Assisted Puncture of the Common Femoral Artery in a Phantom Model

Percutaneous femoral arterial access is daily practice in a variety of medical specialties and enables physicians worldwide to perform endovascular interventions. The reported incidence of percutaneous femoral arterial access complications is 3–18% and often results from suboptimal puncture location due to insufficient visualization of the target vessel. The purpose of this proof-of-concept study was to evaluate the feasibility and the positional error of a mixed-reality (MR)-assisted puncture of the common femoral artery in a phantom model using a commercially available navigation system. In total, 15 MR-assisted punctures were performed. Cone-beam computed tomography angiography (CTA) was used following each puncture to allow quantification of positional error of needle placements in the axial and sagittal planes. Technical success was achieved in 14/15 cases (93.3%) with a median axial positional error of 1.0 mm (IQR 1.3) and a median sagittal positional error of 1.1 mm (IQR 1.6). The median duration of the registration process and needle insertion was 2 min (IQR 1.0). MR-assisted puncture of the common femoral artery is feasible with acceptable positional errors in a phantom model. Future studies should aim to measure and reduce the positional error resulting from MR registration.

New Approach to Accelerated Image Annotation by Leveraging Virtual Reality and Cloud Computing

Three-dimensional imaging is at the core of medical imaging and is becoming a standard in biological research. As a result, there is an increasing need to visualize, analyze and interact with data in a natural three-dimensional context. By combining stereoscopy and motion tracking, commercial virtual reality (VR) headsets provide a solution to this critical visualization challenge by allowing users to view volumetric image stacks in a highly intuitive fashion. While optimizing the visualization and interaction process in VR remains an active topic, one of the most pressing issue is how to utilize VR for annotation and analysis of data. Annotating data is often a required step for training machine learning algorithms. For example, enhancing the ability to annotate complex three-dimensional data in biological research as newly acquired data may come in limited quantities. Similarly, medical data annotation is often time-consuming and requires expert knowledge to identify structures of interest correctly. Moreover, simultaneous data analysis and visualization in VR is computationally demanding. Here, we introduce a new procedure to visualize, interact, annotate and analyze data by combining VR with cloud computing. VR is leveraged to provide natural interactions with volumetric representations of experimental imaging data. In parallel, cloud computing performs costly computations to accelerate the data annotation with minimal input required from the user. We demonstrate multiple proof-of-concept applications of our approach on volumetric fluorescent microscopy images of mouse neurons and tumor or organ annotations in medical images.