Highly immersive virtual reality laparoscopy simulation: development and future aspects

Enhancing endoscopic scene reconstruction with color-aware inverse rendering through neural SDF and radiance fields

Virtual surgical training is crucial for enhancing minimally invasive surgical skills. Traditional geometric reconstruction methods based on medical CT/MRI images often fall short in providing color information, which is typically generated through pseudo-coloring or artistic rendering. To simultaneously reconstruct both the geometric shape and appearance information of organs, we propose a novel organ model reconstruction network called Endoscope-NeSRF. This network jointly leverages neural radiance fields and Signed Distance Function (SDF) to reconstruct a textured geometric model of the organ of interest from multi-view photometric images acquired by an endoscope. The prior knowledge of the inverse correlation between the distance from the light source to the object and the radiance improves the real physical properties of the organ. The dilated mask further refines the appearance and geometry at the organ's edges. We also proposed a highlight adaptive optimization strategy to remove highlights caused by the light source during the acquisition process, thereby preventing the reconstruction results in areas previously affected by highlights from turning white. Finally, the real-time realistic rendering of the organ model is achieved by combining the inverse rendering and Bidirectional Reflectance Distribution Function (BRDF) rendering methods. Experimental results show that our method closely matches the Instant-NGP method in appearance reconstruction, outperforming other state-of-the-art methods, and stands as the superior method in terms of geometric reconstruction. Our method obtained a detailed geometric model and realistic appearance, providing a realistic visual sense for virtual surgical simulation, which is important for medical training.

[Evidence, Availability and Future Visions in Simulation in General and Visceral Surgery]

Practice makes perfect - a saying that everyone has certainly heard. Surgeons of all levels of training can demonstrably practice to some extent on simulators. This training outside the operating theatre and independent of patients makes sense, both ethically and financially. Although the effectiveness of simulation in surgery has been proven several times, simulation training is not a mandatory part of surgical specialist training in Germany. Simulation covers a very wide range in terms of application, effort and costs. This review is intended to give an overview of the systems and their areas of application and the target group. The focus lies on the commonly available systems and possible advantages and disadvantages. Practical skills are in the foreground and all three pillars of general and visceral surgery - conventional techniques, laparoscopy and robotics - are taken into account. However, simulators alone do not achieve cost-benefit effectiveness. The full potential of such an investment can only be exploited with a site-specific, structured training concept in which simulation training according to the post-graduate year and appropriate allocation to surgeries in the operating room are closely interlinked. It should always be possible to train basic skills on site. The significant additional costs for complex simulation systems are possible, depending on the financial resources, or should be purchased in a network or for national courses. The techniques of immersive virtual reality in combination with artificial intelligence and deformation algorithms will certainly play a decisive role for the future of simulation, whereby the use of the available systems must be a primary goal. The integration of simulation into specialist training should be striven for, not least in order to justify the costs.

Toward the validation of VR-HMDs for medical education: a systematic literature review

The latest technological advancements in the domain of virtual reality (VR) have created new opportunities to use VR as a training platform for medical students and practitioners more broadly. Despite the growing interest in the use of VR as a training tool, a commonly identified gap in VR-training for medical education is the confidence in the long-term validity of the applications. A systematic literature review was undertaken to explore the extent of VR (in particular head-mounted displays) applications for medical training with an additional focus on validation measures. The papers included in this review discussed empirical case studies of specific applications; however, these were mostly concerned with human-computer interaction and were polarized between demonstrating that a conceptual technology solution was feasible for simulation or looked at specific areas of VR usability with little discussion on validation measures for long-term training effectiveness and outcomes. The review uncovered a wide range of ad hoc applications and studies in terms of technology vendors, environments, tasks, envisaged users and effectiveness of learning outcomes. This presents decision-making challenges for those seeking to adopt, implement and embed such systems in teaching practice. The authors of this paper then take a wider socio-technical systems perspective to understand how the holistic training system can be engineered and validated effectively as fit for purpose, through distillation of a generic set of requirements from the literature review to aid design specification and implementation, and to drive more informed and traceable validation of these types of systems. In this review, we have identified 92 requirement statements in 11 key areas against which a VR-HMD training system could be validated; these were grouped into design considerations, learning mechanisms and implementation considerations.

Use of Extended Reality in Medical Education: An Integrative Review

Extended reality (XR) has emerged as an innovative simulation-based learning modality. An integrative review was undertaken to explore the nature of evidence, usage, and effectiveness of XR modalities in medical education. One hundred and thirty-three (N = 133) studies and articles were reviewed. XR technologies are commonly reported in surgical and anatomical education, and the evidence suggests XR may be as effective as traditional medical education teaching methods and, potentially, a more cost-effective means of curriculum delivery. Further research to compare different variations of XR technologies and best applications in medical education and training are required to advance the field.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s40670-022-01698-4.

Face, content, and construct validity of the virtual immersive operating room simulator for training laparoscopic procedures

BACKGROUND

The aim of this work is to present the face, content, and construct validation of the virtual immersive operating room simulator (VIORS) for procedural training of surgeons' laparoscopic psychomotor skills and evaluate the immersive training experience.

METHODS

The VIORS simulator consists of an HMD Oculus Rift 2016 with a visor on a 1080 × 1200 pixel OLED screen, two positioning sensors with two adapted controls to simulate laparoscopic instruments, and an acrylic base to simulate the conventional laparoscopic setup. The immersion consists of a 360° virtual operating room environment, based on the EndoSuite at Hospital Infantil de Mexico Federico Gomez, which reproduces a configuration of equipment, instruments, and common distractions in the operating room during a laparoscopic cholecystectomy procedure. Forty-five surgeons, residents, and medicine students participated in this study: 27 novices, 13 intermediates, and 5 experts. They completed a questionnaire on the realism and operating room immersion, as well as their capabilities for laparoscopic procedural training, scored in the 5-point Likert scale. The data of instrument movement were recorded and analyzed using 13 movement analysis parameters (MAPs). The experience during training with VIORS was evaluated through NASA-TLX.

RESULTS

The participants were enthusiastic about the immersion and sensation levels of the VIORS simulator, with positive scores on the realism and its capabilities for procedural training using VIORS. The results proved that the VIORS simulator was able to differentiate between surgeons with different skill levels. Statistically significant differences were found in nine MAPs, demonstrating their construct validity for the objective assessment of the procedural laparoscopic performance. At cognitive level, the inversion experience proves a moderate mental workload when the laparoscopic procedure is carried out.

CONCLUSION

The VIORS simulator has been successfully presented and validated. The VIORS simulator is a useful and effective device for the training of procedural laparoscopic psychomotor skills.

Investigating the effectiveness of immersive VR skill training and its link to physiological arousal

This paper details the motivations, design, and analysis of a study using a fine motor skill training task in both VR and physical conditions. The objective of this between-subjects study was to (a) investigate the effectiveness of immersive virtual reality for training participants in the 'buzz-wire' fine motor skill task compared to physical training and (b) investigate the link between participants' arousal with their improvements in task performance. Physiological arousal levels in the form of  electro-dermal activity (EDA) and ECG (Electrocardiogram) data were collected from 87 participants, randomly distributed across the two conditions. Results indicated that VR training is as good as, or even slightly better than, training in physical training in improving task performance. Moreover, the participants in the VR condition reported an increase in self-efficacy and immersion, while marginally significant differences were observed in the presence and the temporal demand (retrieved from NASA-TLX measurements). Participants in the VR condition showed on average less arousal than those in the physical condition. Though correlation analyses between performance metrics and arousal levels did not depict any statistically significant results, a closer examination of EDA values revealed that participants with lower arousal levels during training, across conditions, demonstrated better improvements in performance than those with higher arousal. These findings demonstrate the effectiveness of VR in training and the potential of using arousal and training performance data for designing adaptive VR training systems. This paper also discusses implications for researchers who consider using biosensors and VR for motor skill experiments.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10055-022-00699-3.

How Can We Improve Gynecological Surgery Webinars during the COVID-19 Pandemic?

The COVID-19 pandemic prevented doctors from attending surgical meetings or conferences where they learned surgical skills from others and shared surgical experiences. It also resulted in the rapid use of webinars in obstetrics and gynecology meetings. While webinars or virtual meetings enable distance learning and replace face-to-face meetings using various teleconferencing software programs, many attendees are not satisfied and find it difficult to learn surgical techniques using commercially available telecommunication programs. Therefore, dedicated webinars are necessary to present emerging surgical technologies, satisfy the attendees, and achieve a successful outcome. This article reviews the existing telecommunication programs, new presentation technologies, and proposed webinars developments to improve its delivery of surgical techniques and training during the COVID-19 pandemic and in the future.

Virtual Reality Therapy in Palliative Care: A Case Series

OBJECTIVES

Virtual reality (VR) opens a variety of therapeutic options to improve symptom burden in patients with advanced disease. Until to date, only few studies have evaluated the use of VR therapy in the context of palliative care. This case series aims to evaluate the feasibility and acceptability of VR therapy in a population of palliative care patients.

METHODS

In this single-site case series, we report on six palliative care patients undergoing VR therapy. The VR therapy consisted of a one-time session ranging between 20 to 60 minutes depending on the patient's needs and the content chosen for the VR sessions. A semi-structured survey was conducted and the Edmonton Symptom Assessment System (ESAS) and the Distress Thermometer were performed pre- and post-intervention.

RESULTS

Overall, VR therapy was well accepted by all patients. Five out of six patients reported having appreciated VR therapy. There were individual differences of perceived effects using VR therapy. The semi-structured survey revealed that some patients felt a temporary detachment from their body and that patients were able to experience the VR session as a break from omnipresent worries and the hospital environment ("I completely forgot where I am"). There was a considerable reduction in the total ESAS score post-treatment (T0 ESASTot = 27.2; T1 ESASTot = 18.8) and a slightly reduction in distress (T0 DTTot = 4.4; T1 DTTot = 3.8). However, two patients were more tired after the intervention.Significance of Results: Our preliminary results demonstrate that VR therapy is acceptable, feasible and safe for use within a palliative care population and appears to be a viable treatment option. Clinical trials are both warranted and necessary to confirm any therapeutic effects of VR therapy, as is the need to tailor VR systems better for use in palliative care settings.

VolumetricOR: A New Approach to Simulate Surgical Interventions in Virtual Reality for Training and Education

BACKGROUND

Surgical training is primarily carried out through observation during assistance or on-site classes, by watching videos as well as by different formats of simulation. The simulation of physical presence in the operating theatre in virtual reality might complement these necessary experiences. A prerequisite is a new education concept for virtual classes that communicates the unique workflows and decision-making paths of surgical health professions (i.e. surgeons, anesthesiologists and surgical assistants) in an authentic and immersive way. For this project, media scientists, designers and surgeons worked together to develop the foundations for new ways of conveying knowledge using virtual reality in surgery.

MATERIALS AND METHOD

A technical workflow to record and present volumetric videos of surgical interventions in a photorealistic virtual operating room was developed. Situated in the virtual reality demonstrator called VolumetricOR, users can experience and navigate through surgical workflows as if they are physically present. The concept is compared with traditional video-based formats of digital simulation in surgical training.

RESULTS

VolumetricOR let trainees experience surgical action and workflows (a) three-dimensionally, (b) from any perspective and (c) in real scale. This improves the linking of theoretical expertise and practical application of knowledge and shifts the learning experience from observation to participation.

DISCUSSION

Volumetric training environments allow trainees to acquire procedural knowledge before going to the operating room and could improve the efficiency and quality of the learning and training process for professional staff by communicating techniques and workflows when the possibilities of training on-site are limited.

VR-based training of craniotomy for intracranial aneurysm surgery

PURPOSE

Intracranial aneurysms can be treated micro-surgically. This procedure involves an appropriate head position of the patient and a proper craniotomy. These steps enable a proper access, facilitating the subsequent steps. To train the access planning process, we propose a VR-based training system.

METHOD

We designed and implemented an immersive VR access simulation, where the user is surrounded by a virtual operating room, including medical equipment and virtual staff. The patient's head can be positioned via hand rotation and an arbitrary craniotomy contour can be drawn. The chosen access can be evaluated by exposing the aneurysm using a microscopic view.

RESULTS

The evaluation of the simulation took place in three stages: testing the simulation using the think-aloud method, conducting a survey and examining the precision of drawing the contour. Although there are differences between the virtual interactions and their counterparts in reality, the participants liked the immersion and felt present in the operating room. The calculated surface dice similarity coefficient, Hausdorff distance and feedback of the participants show that the difficulty of drawing the craniotomy is appropriate.

CONCLUSION

The presented training simulation for head positioning and access planning benefits from the immersive environment. Thus, it is an appropriate training for novice neurosurgeons and medical students with the goal to improve anatomical understanding and to become aware of the importance of the right craniotomy hole.

Immersive 360° videos in health and social care education: a scoping review

BACKGROUND

Research on the pedagogical use of immersive 360° videos is a rapidly expanding area within health and social care education. Despite this interest, there is a paucity of empirical data on its application.

METHOD

A scoping review methodology framework was used to search for relevant articles published between 1970 and July 2021. Six databases were used to identify studies using immersive 360° videos for training and education purposes within health and social care: PubMed, Ovid Medline, Psych Info, Psych Articles, Cochrane Database and Embase. Research questions included: Is there any evidence that immersive 360° videos increase learning outcomes and motivation to learn in health and social care education? What are the key pedagogical concepts and theories that inform this area of research? What are the limitations of using immersive 360° videos within health and social education? The four dimensions contained within Keller's ARCS model (attention, relevance, confidence and satisfaction) frame the results section.

RESULTS

Fourteen studies met our inclusion criteria. Learning outcomes confirm that immersive 360° videos as a pedagogical tool: increases attention, has relevance in skill enhancement, confidence in usability and user satisfaction. In particular, immersive 360° videos has a positive effect on the user's emotional response to the learning climate, which has a significant effect on users' motivation to learn. There was a notable lack of pedagogical theory within the studies retrieved and a general lack of clarity on learning outcomes.

CONCLUSION

Studies examining the effectiveness of such interventions remains weak due to smaller sample sizes, lack of randomised control trials, and a gap in reporting intervention qualities and outcomes. Nevertheless, 360° immersive video is a viable alternative to VR and regular video, it is cost-effective, and although more robust research is necessary, learning outcomes are promising.

FUTURE DIRECTIONS

Future research would do well to focus on interactivity and application of pedagogical theory within immersive 360° videos experiences. We argue that more and higher quality research studies, beyond the scope of medical education, are needed to explore the acceptability and effective implementation of this technology.

Virtual embolization for treatment support of intracranial AVMs using an interactive desktop and VR application

PURPOSE

The treatment of intracranial arteriovenous malformations (AVM) is challenging due to their complex anatomy. For this vessel pathology, arteries are directly linked to veins without a capillary bed in between. For endovascular treatment, embolization is carried out, where the arteries that supply the AVM are consecutively blocked. A virtual embolization could support the medical expert in treatment planning.

METHOD

We designed and implemented an immersive VR application that allows the visualization of the simulated blood flow by displaying millions of particles. Furthermore, the user can interactively block or unblock arteries that supply the AVM and analyze the altered blood flow based on pre-computed simulations.

RESULTS

In a pilot study, the application was successfully adapted to three patient-specific cases. We performed a qualitative evaluation with two experienced neuroradiologist who regularly conduct AVM embolizations. The feature of virtually blocking or unblocking feeders was rated highly beneficial, and a desire for the inclusion of quantitative information was formulated.

CONCLUSION

The presented application allows for virtual embolization and interactive blood flow visualization in an immersive virtual reality environment. It could serve as useful addition for treatment planning and education in clinical practice, supporting the understanding of AVM topology as well as understanding the influence of the AVM's feeding arteries.

How COVID-19 kick-started online learning in medical education-The DigiMed study

BACKGROUND

The coronavirus disease 2019 (COVID-19) pandemic led to far-reaching restrictions of social and professional life, affecting societies all over the world. To contain the virus, medical schools had to restructure their curriculum by switching to online learning. However, only few medical schools had implemented such novel learning concepts. We aimed to evaluate students' attitudes to online learning to provide a broad scientific basis to guide future development of medical education.

METHODS

Overall, 3286 medical students from 12 different countries participated in this cross-sectional, web-based study investigating various aspects of online learning in medical education. On a 7-point Likert scale, participants rated the online learning situation during the pandemic at their medical schools, technical and social aspects, and the current and future role of online learning in medical education.

RESULTS

The majority of medical schools managed the rapid switch to online learning (78%) and most students were satisfied with the quantity (67%) and quality (62%) of the courses. Online learning provided greater flexibility (84%) and led to unchanged or even higher attendance of courses (70%). Possible downsides included motivational problems (42%), insufficient possibilities for interaction with fellow students (67%) and thus the risk of social isolation (64%). The vast majority felt comfortable using the software solutions (80%). Most were convinced that medical education lags behind current capabilities regarding online learning (78%) and estimated the proportion of online learning before the pandemic at only 14%. In order to improve the current curriculum, they wish for a more balanced ratio with at least 40% of online teaching compared to on-site teaching.

CONCLUSION

This study demonstrates the positive attitude of medical students towards online learning. Furthermore, it reveals a considerable discrepancy between what students demand and what the curriculum offers. Thus, the COVID-19 pandemic might be the long-awaited catalyst for a new "online era" in medical education.

On the Use of Virtual Reality for Medical Imaging Visualization

Advanced visualization of medical imaging has been a motive for research due to its value for disease analysis, surgical planning, and academical training. More recently, attention has been turning toward mixed reality as a means to deliver more interactive and realistic medical experiences. However, there are still many limitations to the use of virtual reality for specific scenarios. Our intent is to study the current usage of this technology and assess the potential of related development tools for clinical contexts. This paper focuses on virtual reality as an alternative to today's majority of slice-based medical analysis workstations, bringing more immersive three-dimensional experiences that could help in cross-slice analysis. We determine the key features a virtual reality software should support and present today's software tools and frameworks for researchers that intend to work on immersive medical imaging visualization. Such solutions are assessed to understand their ability to address existing challenges of the field. It was understood that most development frameworks rely on well-established toolkits specialized for healthcare and standard data formats such as DICOM. Also, game engines prove to be adequate means of combining software modules for improved results. Virtual reality seems to remain a promising technology for medical analysis but has not yet achieved its true potential. Our results suggest that prerequisites such as real-time performance and minimum latency pose the greatest limitations for clinical adoption and need to be addressed. There is also a need for further research comparing mixed realities and currently used technologies.

Assessment of Wheelchair Propulsion Performance in an Immersive Virtual Reality Simulator

Maneuvering a wheelchair is an important necessity for the everyday life and social activities of people with a range of physical disabilities. However, in real life, wheelchair users face several common challenges: articulate steering, spatial relationships, and negotiating obstacles. Therefore, our research group has developed a head-mounted display (HMD)-based intuitive virtual reality (VR) stimulator for wheelchair propulsion. The aim of this study was to investigate the feasibility and efficacy of this VR stimulator for wheelchair propulsion performance. Twenty manual wheelchair users (16 men and 4 women) with spinal cord injuries ranging from T8 to L2 participated in this study. The differences in wheelchair propulsion kinematics between immersive and non-immersive VR environments were assessed using a 3D motion analysis system. Subjective data of the HMD-based intuitive VR stimulator were collected with a Presence Questionnaire and individual semi-structured interview at the end of the trial. Results indicated that propulsion performance was very similar in terms of start angle (p = 0.34), end angle (p = 0.46), stroke angle (p = 0.76), and shoulder movement (p = 0.66) between immersive and non-immersive VR environments. In the VR episode featuring an uphill journey, an increase in propulsion speed (p < 0.01) and cadence (p < 0.01) were found, as well as a greater trunk forward inclination (p = 0.01). Qualitative interviews showed that this VR simulator made an attractive, novel impression and therefore demonstrated the potential as a tool for stimulating training motivation. This HMD-based intuitive VR stimulator can be an effective resource to enhance wheelchair maneuverability experiences.

Teaching with Disruptive Technology: The Use of Augmented, Virtual, and Mixed Reality (HoloLens) for Disease Education

Modern technologies are often utilised in schools or universities with a variety of educational goals in mind. Of particular interest is the enhanced interactivity and engagement offered by mixed reality devices such as the HoloLens, as well as the ability to explore anatomical models of disease using augmented and virtual realities. As the students are required to learn an ever-increasing number of diseases within a university health science or medical degree, it is crucial to consider which technologies provide value to educators and students. This chapter explores the opportunities for using modern disruptive technologies to teach a curriculum surrounding disease. For relevant examples, a focus will be placed on asthma as a respiratory disease which is increasing in prevalence, and stroke as a neurological and cardiovascular disease. The complexities of creating effective educational curricula around these diseases will be explored, along with the benefits of using augmented reality and mixed reality as viable teaching technologies in a range of use cases.

Immersive Video Modeling Versus Traditional Video Modeling for Teaching Central Venous Catheter Insertion to Medical Residents

Background 

Central Venous Catheter (CVC) placement is a common critical care procedure. Simulated practice has been shown to reduce its iatrogenic complications. Video modeling (VM) is an instructional adjunct that improves the quality and success of CVC insertion. Immersive VM can improve recall and skill translation, but its role in teaching medical procedures is not established.

Research question/hypothesis

We hypothesized that, relative to traditional VM, immersive VM would decrease cognitive load and enhance ultrasound-guided CVC insertion skill acquisition.

Methods

Thirty-two resident physicians from four specialties were randomized into traditional (control) or immersive VM (intervention) groups for three CVC training sessions. Cognitive load was quantified via NASA Task Load Index (TLX). Mean (± standard deviations) values were compared using two-tailed t-tests. Skill acquisition was quantified by procedural time and the average 5-point [EM1] [TB2] entrustment score of three expert raters.

Results

Overall entrustment scores improved from the first (3.44±0.98) to the third (4.06±1.23; p<0.002) session but were not significantly different between the control and intervention groups. There were no significant differences between NASA TLX scores or procedural time.

Conclusion

We found no significant difference in entrustment, cognitive load, or procedural time. Immersive VM was not found to be superior to traditional VM for teaching CVC insertion.

[Surgical Education in the Digital Age - Virtual Reality, Augmented Reality and Robotics in the Medical School]

Hintergrund Die digitale Transformation der Medizin verändert den Beruf des Arztes. Augmented und Virtual Reality (AR/VR) und die Robotik erfahren einen zunehmenden Einsatz in unterschiedlichen klinischen Kontexten und bedürfen einer begleitenden Aus- und Weiterbildung, die bereits im Medizinstudium beginnen muss. Hier besteht aktuell eine große Diskrepanz zwischen dem hohen Bedarf und der Anzahl an wissenschaftlich überprüften Konzepten. Ziel der vorliegenden Arbeit war die Konzeptionierung und strukturierte Evaluation eines neu entwickelten Lern-/Lehrkonzepts zur digitalen Transformation mit Fokus auf die chirurgische Lehre.

Methoden 35 Studierende haben in 3 Kursen des Blended-Learning-Curriculums „Medizin im digitalen Zeitalter“ teilgenommen. Das 4. Modul dieses Kurses thematisiert Virtual Reality, Augmented Reality und Robotik in der Chirurgie. Es gliedert sich in die folgenden Kursteile: (1) immersive Simulation einer laparoskopischen Cholezystektomie, (2) leberchirurgische Operationsplanung mittels AR/VR, (3) Basisfertigkeiten am VR-Simulator für robotische Chirurgie, (4) kollaborative OP Planung im virtuellen Raum und (5) Expertengespräch. Nach Abschluss des Gesamtcurriculums erfolgte eine qualitative und quantitative Evaluation des Kurskonzepts mittels semistrukturierter Interviews sowie anhand von standardisierten Prä-post-Evaluationsfragebögen.

Ergebnisse Im qualitativen Auswertungsverfahren der Interviews wurden 79 Textaussagen 4 Schwerpunktkategorien zugewiesen. Den größten Anteil (35%) nahmen hierbei Äußerungen zum „Expertengespräch“ ein, das von den Studierenden als elementarer Teil des Kurskonzepts gewertet wurde. Darüber hinaus empfanden die Studierenden den Kurs als horizonterweiterndes „Lernerlebnis“ (29% der Aussagen) mit einem hohen „Praxisbezug“ (27%). Die quantitative Studierendenevaluation zeigt eine positive Entwicklung für die Teilkompetenzen Wissen und Fertigkeiten sowie eine Tendenz zu einer positiven Haltung nach Kursabschluss.

Schlussfolgerung Die chirurgische Lehre ist zur Entwicklung digitaler Kompetenzen prädestiniert. Dabei muss die Geschwindigkeit des Veränderungsprozesses der digitalen Transformation im chirurgischen Fachgebiet beachtet und im curricularen Konzept verankert werden.

Evaluation of Laparoscopy Virtual Reality Training on the Improvement of Trainees' Surgical Skills

Background and objectives: The primary objective was to evaluate the benefit of training with virtual reality simulation. The secondary objective was to describe the short-term skill acquisition obtained by simulation training and to determine the factors affecting its magnitude. Materials and Methods: We prospectively performed a three-stage evaluation: face, constructive, and predictive to evaluate the training with a laparoscopic simulator with haptic feedback. The participants (n = 63) were divided according to their level of experience into three groups: 16% residents; 46% specialists and 38% were consultants. Results: Face evaluation demonstrates the acceptance of the design and realism of the tasks; it showed a median score of eight (IQR 3) on a Likert scale and 54% of participants (n = 34) gave the tissue feedback a moderate rating. Constructive evaluation demonstrates the improvement of the participants in the training session and the ability of the designed task to distinguish the experienced from the inexperienced surgeon based on the performance score, at task I (transfer of pegs) and II (laparoscopic salpingectomy). There was an improvement in both tasks with a significant increase in score and reduction in time. The study showed that those with a high score at the pre-test recorded a high score post-test, showing a significant pair-wise comparison (Z) and correlation (p) showing a significant statistical significance (p < 0.001). The predictive evaluation demonstrates the beneficiary effect of training four weeks afterward on the practice of surgeons addressed with five questions. It showed an improvement regarding implementation into daily routine, performance of procedure, suturing, shortening of the operative time, and complication management. Conclusions: Virtual reality simulation established high ratings for both realism and training capacity, including clinical relevance, critical relevance, and maintaining training enthusiasm.

Toward interprofessional team training for surgeons and anesthesiologists using virtual reality

Purpose

In this work, a virtual environment for interprofessional team training in laparoscopic surgery is proposed. Our objective is to provide a tool to train and improve intraoperative communication between anesthesiologists and surgeons during laparoscopic procedures.

Methods

An anesthesia simulation software and laparoscopic simulation software are combined within a multi-user virtual reality (VR) environment. Furthermore, two medical training scenarios for communication training between anesthesiologists and surgeons are proposed and evaluated. Testing was conducted and social presence was measured. In addition, clinical feedback from experts was collected by following a think-aloud protocol and through structured interviews.

Results

Our prototype is assessed as a reasonable basis for training and extensive clinical evaluation. Furthermore, the results of testing revealed a high degree of exhilaration and social presence of the involved physicians. Valuable insights were gained from the interviews and the think-aloud protocol with the experts of anesthesia and surgery that showed the feasibility of team training in VR, the usefulness of the system for medical training, and current limitations.

Conclusion

The proposed VR prototype provides a new basis for interprofessional team training in surgery. It engages the training of problem-based communication during surgery and might open new directions for operating room training.

Electronic supplementary material

The online version of this article (10.1007/s11548-020-02276-y) contains supplementary material, which is available to authorized users.

Training of patient handover in virtual reality

Patient handover is an important part for information transfer between medical professionals in a clinical setting. Yet, in current medical education, these conversations are only trained sparsely, since they are costly to perform as they take place in offsite courses and are led by experts over several days. Virtual reality (VR)-based training courses could increase the availability of training, by eliminating travel costs and reducing the time-commitment of the teaching experts. This work presents a VR prototype of a multi-user training and examination application for patient handover. To ensure a similar interaction quality to its current real world counterpart, this work used omni-directional video recordings to create a realistic setting and compared different projection methods. A pilot study highlighted distinct use-cases of sphere and mesh projections to visualize the recordings. The results suggest enhanced spatial presence relating to the usage of omni-directional videos in VR-applications.

Approximate Depth Shape Reconstruction for RGB-D Images Captured from HMDs for Mixed Reality Applications

Depth sensors are important in several fields to recognize real space. However, there are cases where most depth values in a depth image captured by a sensor are constrained because the depths of distal objects are not always captured. This often occurs when a low-cost depth sensor or structured-light depth sensor is used. This also occurs frequently in applications where depth sensors are used to replicate human vision, e.g., when using the sensors in head-mounted displays (HMDs). One ideal inpainting (repair or restoration) approach for depth images with large missing areas, such as partial foreground depths, is to inpaint only the foreground; however, conventional inpainting studies have attempted to inpaint entire images. Thus, under the assumption of an HMD-mounted depth sensor, we propose a method to inpaint partially and reconstruct an RGB-D depth image to preserve foreground shapes. The proposed method is comprised of a smoothing process for noise reduction, filling defects in the foreground area, and refining the filled depths. Experimental results demonstrate that the inpainted results produced using the proposed method preserve object shapes in the foreground area with accurate results of the inpainted area with respect to the real depth with the peak signal-to-noise ratio metric.

Virtual Reality Single-Port Sleeve Gastrectomy Training Decreases Physical and Mental Workload in Novice Surgeons: An Exploratory Study

BACKGROUND

Novice surgeons experience high levels of physical and mental workload during the early stages of their curriculum and clinical practice. Laparoscopic sleeve gastrectomy is the first bariatric procedure worldwide. Feasibility and safety of single-port sleeve gastrectomy (SPSG) has been demonstrated. An immersive virtual reality (VR) simulation was developed to provide a repetitive exercise to learn this novel technique. The primary objective of this study was to evaluate the impact of the VR training tool on mental and physical workload in novice surgeons. The secondary objective included an evaluation of the VR simulator.

METHODS

A monocentric-controlled trial was conducted. Ten participants were divided into two groups, the VR group and the control group (without VR training). Surgery residents participated in a first real case of SPSG and a second case 1 month later. The VR group underwent a VR training between the two surgeries. Mental and physical loads were assessed with self-assessment questionnaires: NASA-TLX, Borg scale, and manikin discomfort test. The VR simulator was evaluated through presence, cybersickness, and usability questionnaires.

RESULTS

This study showed a decrease of the mental demand and effort dimensions of NASA-TLX between the first and the second surgery in the VR group (P < .05). During the second surgery, a marginally significant difference was shown concerning the mental demand between the two groups. Postural discomfort of the VR group decreased with practice (P < .01), mainly between the first and the second surgery (P < .05). Furthermore, participants characterized the VR simulator as realistic, usable, and very useful to learned surgery.

CONCLUSION

This exploratory study showed an improvement in mental and physical workload when novice surgeons trained with VR (repetitive practice, gesture improvement, reduction of stress, etc.). Virtual reality appears to be a promising perspective for surgical training.

Virtual reality exposure therapy for neuro-psychomotor recovery in adults: a systematic review

OBJECTIVE

To analyse the virtual reality applications in the subjects' neuro-psychomotor functions rehabilitation with motor and/or neuropsychiatric impairment.

METHODS

The search was carried out in nine databases (Scopus, PubMed, Web of Science, ScienceDirect, Cochrane Library, CINAHL, PsycINFO, LILACS, and SciELO), from December 2017 to March 2019. An additional manual search was performed, taking into consideration references of the included papers, through the same eligibility criteria. The methodological quality of the included papers was evaluated using the Physiotherapy Evidence Database (PEDro).

RESULTS

One hundred and twenty-two papers were selected by the initial screening, but only 33 studies participated in the final inclusion in the study (11 clinical trials and 14 experimental studies). The data extracted were: the proposed objective, the sample population, the neurological condition treated, the VR modality used in the procedures and the study intervention period.

CONCLUSIONS

Virtual reality supports the rehabilitation process of neuro-psychomotor functions, allowing potential gains in the patients' recovery. Therefore, its development facilitates its availability and access in the future.Implications for rehabilitationIt has minimal adverse effects during the virtual therapies performance, such as the presence of vertigo related to cybersickness conditions, suggesting virtual reality as a safe rehabilitation tool, compared to other therapies.Virtual reality use is useful and effective in helping the rehabilitation process of motor, cognitive and psychosocial functions.It plays a role as an adjunctive and complementary therapy in the neuro-psychomotor rehabilitation process to obtain a clinically significant result.

Virtual Reality: Endless Potential in Pediatric Palliative Care: A Case Report

Pediatric palliative care deals with the physical, psychosocial, and spiritual concerns of patients and their families. And to do this, clinicians must use all the tools at their disposal, including pharmacological and nonpharmacological modalities. Virtual reality is quickly becoming a useful tool in many areas of medicine, including surgical planning, simulation training, rehabilitation, and pain prevention and treatment. Recently it has been used in the adult palliative care population, for symptom management, and memory and legacy creation. We present a case report for, what we believe to be, the first time in the pediatric palliative care population.

Implementing Virtual and Augmented Reality Tools for Radiology Education and Training, Communication, and Clinical Care

Advances in virtual immersive and augmented reality technology, commercially available for the entertainment and gaming industry, hold potential for education and clinical use in medicine and the field of medical imaging. Radiology departments have begun exploring the use of these technologies to help with radiology education and clinical care. The purpose of this review article is to summarize how three institutions have explored using virtual and augmented reality for radiology.

Simulation in endoscopy: Practical educational strategies to improve learning

In gastrointestinal endoscopy, simulation-based training can help endoscopists acquire new skills and accelerate the learning curve. Simulation creates an ideal environment for trainees, where they can practice specific skills, perform cases at their own pace, and make mistakes with no risk to patients. Educators also benefit from the use of simulators, as they can structure training according to learner needs and focus solely on the trainee. Not all simulation-based training, however, is effective. To maximize benefits from this instructional modality, educators must be conscious of learners' needs, the potential benefits of training, and associated costs. Simulation should be integrated into training in a manner that is grounded in educational theory and empirical data. In this review, we focus on four best practices in simulation-based education: deliberate practice with mastery learning, feedback and debriefing, contextual learning, and innovative educational strategies. For each topic, we provide definitions, supporting evidence, and practical tips for implementation.

Applying Modern Virtual and Augmented Reality Technologies to Medical Images and Models

Recent technological innovations have created new opportunities for the increased adoption of virtual reality (VR) and augmented reality (AR) applications in medicine. While medical applications of VR have historically seen greater adoption from patient-as-user applications, the new era of VR/AR technology has created the conditions for wider adoption of clinician-as-user applications. Historically, adoption to clinical use has been limited in part by the ability of the technology to achieve a sufficient quality of experience. This article reviews the definitions of virtual and augmented reality and briefly covers the history of their development. Currently available options for consumer-level virtual and augmented reality systems are presented, along with a discussion of technical considerations for their adoption in the clinical environment. Finally, a brief review of the literature of medical VR/AR applications is presented prior to introducing a comprehensive conceptual framework for the viewing and manipulation of medical images in virtual and augmented reality. Using this framework, we outline considerations for placing these methods directly into a radiology-based workflow and show how it can be applied to a variety of clinical scenarios.

Towards Virtual VATS, Face, and Construct Evaluation for Peg Transfer Training of Box, VR, AR, and MR Trainer

The aim of this study is to develop and assess the peg transfer training module face, content and construct validation use of the box, virtual reality (VR), cognitive virtual reality (CVR), augmented reality (AR), and mixed reality (MR) trainer, thereby to compare advantages and disadvantages of these simulators. Training system (VatsSim-XR) design includes customized haptic-enabled thoracoscopic instruments, virtual reality helmet set, endoscope kit with navigation, and the patient-specific corresponding training environment. A cohort of 32 trainees comprising 24 novices and 8 experts underwent the real and virtual simulators that were conducted in the department of thoracic surgery of Yunnan First People's Hospital. Both subjective and objective evaluations have been developed to explore the visual and haptic potential promotions in peg transfer education. Experiments and evaluation results conducted by both professional and novice thoracic surgeons show that the surgery skills from experts are better than novices overall, AR trainer is able to provide a more balanced training environments on visuohaptic fidelity and accuracy, box trainer and MR trainer demonstrated the best realism 3D perception and surgical immersive performance, respectively, and CVR trainer shows a better clinic effect that the traditional VR trainer. Combining these in a systematic approach, tuned with specific fidelity requirements, medical simulation systems would be able to provide a more immersive and effective training environment.

A Process for Digitizing and Simulating Biologically Realistic Oligocellular Networks Demonstrated for the Neuro-Glio-Vascular Ensemble

One will not understand the brain without an integrated exploration of structure and function, these attributes being two sides of the same coin: together they form the currency of biological computation. Accordingly, biologically realistic models require the re-creation of the architecture of the cellular components in which biochemical reactions are contained. We describe here a process of reconstructing a functional oligocellular assembly that is responsible for energy supply management in the brain and creating a computational model of the associated biochemical and biophysical processes. The reactions that underwrite thought are both constrained by and take advantage of brain morphologies pertaining to neurons, astrocytes and the blood vessels that deliver oxygen, glucose and other nutrients. Each component of this neuro-glio-vasculature ensemble (NGV) carries-out delegated tasks, as the dynamics of this system provide for each cell-type its own energy requirements while including mechanisms that allow cooperative energy transfers. Our process for recreating the ultrastructure of cellular components and modeling the reactions that describe energy flow uses an amalgam of state-of the-art techniques, including digital reconstructions of electron micrographs, advanced data analysis tools, computational simulations and in silico visualization software. While we demonstrate this process with the NGV, it is equally well adapted to any cellular system for integrating multimodal cellular data in a coherent framework.

Artificial Versus Video-Based Immersive Virtual Surroundings: Analysis of Performance and User’s Preference

Introduction. Immersive virtual reality (VR) laparoscopy simulation connects VR simulation with head-mounted displays to increase presence during VR training. The goal of the present study was the comparison of 2 different surroundings according to performance and users’ preference. Methods. With a custom immersive virtual reality laparoscopy simulator, an artificially created VR operating room (AVR) and a highly immersive VR operating room (IVR) were compared. Participants (n = 30) performed 3 tasks (peg transfer, fine dissection, and cholecystectomy) in AVR and IVR in a crossover study design. Results. No overall difference in virtual laparoscopic performance was obtained when comparing results from AVR with IVR. Most participants preferred the IVR surrounding (n = 24). Experienced participants (n = 10) performed significantly better than novices (n = 10) in all tasks regardless of the surrounding ( P < .05). Participants with limited experience (n = 10) showed differing results. Presence, immersion, and exhilaration were significantly higher in IVR. Two thirds assumed that IVR would have a positive influence on their laparoscopic simulator use. Conclusion. This first study comparing AVR and IVR did not reveal differences in virtual laparoscopic performance. IVR is considered the more realistic surrounding and is therefore preferred by the participants.