Evaluating the Use of Mixed Reality to Teach Gross and Microscopic Respiratory Anatomy

Dissecting human anatomy learning process through anatomical education with augmented reality: AEducAR 2.0, an updated interdisciplinary study

Anatomical education is pivotal for medical students, and innovative technologies like augmented reality (AR) are transforming the field. This study aimed to enhance the interactive features of the AEducAR prototype, an AR tool developed by the University of Bologna, and explore its impact on human anatomy learning process in 130 second-year medical students at the International School of Medicine and Surgery of the University of Bologna. An interdisciplinary team of anatomists, maxillofacial surgeons, biomedical engineers, and educational scientists collaborated to ensure a comprehensive understanding of the study's objectives. Students used the updated version of AEducAR, named AEducAR 2.0, to study three anatomical topics, specifically the orbit zone, facial bones, and mimic muscles. AEducAR 2.0 offered two learning activities: one explorative and one interactive. Following each activity, students took a test to assess learning outcomes. Students also completed an anonymous questionnaire to provide background information and offer their perceptions of the activity. Additionally, 10 students participated in interviews for further insights. The results demonstrated that AEducAR 2.0 effectively facilitated learning and students' engagement. Students totalized high scores in both quizzes and declared to have appreciated the interactive features that were implemented. Moreover, interviews shed light on the interesting topic of blended learning. In particular, the present study suggests that incorporating AR into medical education alongside traditional methods might prove advantageous for students' academic and future professional endeavors. In this light, this study contributes to the growing research emphasizing the potential role of AR in shaping the future of medical education.

Delivering clinical tutorials to medical students using the Microsoft HoloLens 2: A mixed-methods evaluation

BACKGROUND

Mixed reality offers potential educational advantages in the delivery of clinical teaching. Holographic artefacts can be rendered within a shared learning environment using devices such as the Microsoft HoloLens 2. In addition to facilitating remote access to clinical events, mixed reality may provide a means of sharing mental models, including the vertical and horizontal integration of curricular elements at the bedside. This study aimed to evaluate the feasibility of delivering clinical tutorials using the Microsoft HoloLens 2 and the learning efficacy achieved.

METHODS

Following receipt of institutional ethical approval, tutorials on preoperative anaesthetic history taking and upper airway examination were facilitated by a tutor who wore the HoloLens device. The tutor interacted face to face with a patient and two-way audio-visual interaction was facilitated using the HoloLens 2 and Microsoft Teams with groups of students who were located in a separate tutorial room. Holographic functions were employed by the tutor. The tutor completed the System Usability Scale, the tutor, technical facilitator, patients, and students provided quantitative and qualitative feedback, and three students participated in semi-structured feedback interviews. Students completed pre- and post-tutorial, and end-of-year examinations on the tutorial topics.

RESULTS

Twelve patients and 78 students participated across 12 separate tutorials. Five students did not complete the examinations and were excluded from efficacy calculations. Student feedback contained 90 positive comments, including the technology's ability to broadcast the tutor's point-of-vision, and 62 negative comments, where students noted issues with the audio-visual quality, and concerns that the tutorial was not as beneficial as traditional in-person clinical tutorials. The technology and tutorial structure were viewed favourably by the tutor, facilitator and patients. Significant improvement was observed between students' pre- and post-tutorial MCQ scores (mean 59.2% Vs 84.7%, p < 0.001).

CONCLUSIONS

This study demonstrates the feasibility of using the HoloLens 2 to facilitate remote bedside tutorials which incorporate holographic learning artefacts. Students' examination performance supports substantial learning of the tutorial topics. The tutorial structure was agreeable to students, patients and tutor. Our results support the feasibility of offering effective clinical teaching and learning opportunities using the HoloLens 2. However, the technical limitations and costs of the device are significant, and further research is required to assess the effectiveness of this tutorial format against in-person tutorials before wider roll out of this technology can be recommended as a result of this study.

The HoloLens in medicine: A systematic review and taxonomy

The HoloLens (Microsoft Corp., Redmond, WA), a head-worn, optically see-through augmented reality (AR) display, is the main player in the recent boost in medical AR research. In this systematic review, we provide a comprehensive overview of the usage of the first-generation HoloLens within the medical domain, from its release in March 2016, until the year of 2021. We identified 217 relevant publications through a systematic search of the PubMed, Scopus, IEEE Xplore and SpringerLink databases. We propose a new taxonomy including use case, technical methodology for registration and tracking, data sources, visualization as well as validation and evaluation, and analyze the retrieved publications accordingly. We find that the bulk of research focuses on supporting physicians during interventions, where the HoloLens is promising for procedures usually performed without image guidance. However, the consensus is that accuracy and reliability are still too low to replace conventional guidance systems. Medical students are the second most common target group, where AR-enhanced medical simulators emerge as a promising technology. While concerns about human-computer interactions, usability and perception are frequently mentioned, hardly any concepts to overcome these issues have been proposed. Instead, registration and tracking lie at the core of most reviewed publications, nevertheless only few of them propose innovative concepts in this direction. Finally, we find that the validation of HoloLens applications suffers from a lack of standardized and rigorous evaluation protocols. We hope that this review can advance medical AR research by identifying gaps in the current literature, to pave the way for novel, innovative directions and translation into the medical routine.

Use of 3-Dimensional Modeling and Augmented/Virtual Reality Applications in Microsurgical Neuroanatomy Training

BACKGROUND

Understanding the microsurgical neuroanatomy of the brain is challenging yet crucial for safe and effective surgery. Training on human cadavers provides an opportunity to practice approaches and learn about the brain's complex organization from a surgical view. Innovations in visual technology, such as virtual reality (VR) and augmented reality (AR), have immensely added a new dimension to neuroanatomy education. In this regard, a 3-dimensional (3D) model and AR/VR application may facilitate the understanding of the microsurgical neuroanatomy of the brain and improve spatial recognition during neurosurgical procedures by generating a better comprehension of interrelated neuroanatomic structures.

OBJECTIVE

To investigate the results of 3D volumetric modeling and AR/VR applications in showing the brain's complex organization during fiber dissection.

METHODS

Fiber dissection was applied to the specimen, and the 3D model was created with a new photogrammetry method. After photogrammetry, the 3D model was edited using 3D editing programs and viewed in AR. The 3D model was also viewed in VR using a head-mounted display device.

RESULTS

The 3D model was viewed in internet-based sites and AR/VR platforms with high resolution. The fibers could be panned, rotated, and moved freely on different planes and viewed from different angles on AR and VR platforms.

CONCLUSION

This study demonstrated that fiber dissections can be transformed and viewed digitally on AR/VR platforms. These models can be considered a powerful teaching tool for improving the surgical spatial recognition of interrelated neuroanatomic structures. Neurosurgeons worldwide can easily avail of these models on digital platforms.

Use of Extended Reality in Medical Education: An Integrative Review

Extended reality (XR) has emerged as an innovative simulation-based learning modality. An integrative review was undertaken to explore the nature of evidence, usage, and effectiveness of XR modalities in medical education. One hundred and thirty-three (N = 133) studies and articles were reviewed. XR technologies are commonly reported in surgical and anatomical education, and the evidence suggests XR may be as effective as traditional medical education teaching methods and, potentially, a more cost-effective means of curriculum delivery. Further research to compare different variations of XR technologies and best applications in medical education and training are required to advance the field.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s40670-022-01698-4.

Student Engagement Using HoloLens Mixed-Reality Technology in Human Anatomy Laboratories for Osteopathic Medical Students: an Instructional Model

Mixed-reality technology is a powerful tool used in healthcare and medical education to engage students in life-like scenarios. This blend of virtual and augmented reality images incorporates virtual projections with the real environment to allow real-time observation and interaction [1]. While this immersive technology offers advantages over cadaver dissections, it creates new challenges to keeping students engaged [2, 3]. Student engagement improves students' commitment to learning, critical thinking, and motivation and results in successful course outcomes [4, 5]. This paper provides an activity model using the HoloLens mixed-reality technology to deliver human gross anatomy laboratory sessions to first-year osteopathic medical students. The activity was designed using Gagne's model for instructional design and team-based learning to create an active learning model, which targets the behavioral, emotional, and cognitive dimensions of student engagement [6, 7]: behavioral engagement through autonomy and time on task, emotional engagement through providing the guiding exploration and narrative flow to accompany students' visual experience, and cognitive engagement by incorporating team-based learning (TBL) and case-based learning (CBL). The instructional model also answers the call for a new type of virtual reality instructor and pedagogical strategy that addresses the unique challenges and increases student engagement with this new technology. The effectiveness of this classroom activity was assessed by observing students for indicators or behaviors of student engagement, which are discussed. Further studies are required to measure the extent to which these indicators were exhibited and compare student engagement with this mixed-reality to didactic cadaver-based laboratory sessions.

Application of HoloLens-based augmented reality and three-dimensional printed anatomical tooth reference models in dental education

Tooth anatomy is fundamental knowledge used in everyday dental practice to reconstruct the occlusal surface during cavity fillings. The main objective of this project was to evaluate the suitability of two types of anatomical tooth reference models used to support reconstruction of the occlusal anatomy of the teeth: (1) a three-dimensional (3D)-printed model and (2) a model displayed in augmented reality (AR) using Microsoft HoloLens. The secondary objective was to evaluate three aspects impacting the outcome: clinical experience, comfort of work, and other variables. The tertiary objective was to evaluate the usefulness of AR in dental education. Anatomical models of crowns of three different molars were made using cone beam computed tomography image segmentation, printed with a stereolithographic 3D-printer, and then displayed in the HoloLens. Each participant reconstructed the occlusal anatomy of three teeth. One without any reference materials and two with an anatomical reference model, either 3D-printed or holographic. The reconstruction work was followed by the completion of an evaluation questionnaire. The maximum Hausdorff distances (Hmax) between the superimposed images of the specimens after the procedures and the anatomical models were then calculated. The results showed that the most accurate but slowest reconstruction was achieved with the use of 3D-printed reference models and that the results were not affected by other aspects considered. For this method, the Hmax was observed to be 630 μm (p = 0.004). It was concluded that while AR models can be helpful in dental anatomy education, they are not suitable replacements for physical models.

The use of mixed reality technology for the objective assessment of clinical skills: a validation study

BACKGROUND

Mixed Reality technology may provide many advantages over traditional teaching methods. Despite its potential, the technology has yet to be used for the formal assessment of clinical competency. This study sought to collect validity evidence and assess the feasibility of using the HoloLens 2 mixed reality headset for the conduct and augmentation of Objective Structured Clinical Examinations (OSCEs).

METHODS

A prospective cohort study was conducted to compare the assessment of undergraduate medical students undertaking OSCEs via HoloLens 2 live (HLL) and recorded (HLR), and gold-standard in-person (IP) methods. An augmented mixed reality scenario was also assessed.

RESULTS

Thirteen undergraduate participants completed a total of 65 OSCE stations. Overall inter-modality correlation was 0.81 (p = 0.01), 0.98 (p = 0.01) and 0.82 (p = 0.01) for IP vs. HLL, HLL vs. HLR and IP vs. HLR respectively. Skill based correlations for IP vs. HLR were assessed for history taking (0.82, p = 0.01), clinical examination (0.81, p = 0.01), procedural (0.88, p = 0.01) and clinical skills (0.92, p = 0.01), and assessment of a virtual mixed reality patient (0.74, p = 0.01). The HoloLens device was deemed to be usable and practical (Standard Usability Scale (SUS) score = 51.5), and the technology was thought to deliver greater flexibility and convenience, and have the potential to expand and enhance assessment opportunities.

CONCLUSIONS

HoloLens 2 is comparable to traditional in-person examination of undergraduate medical students for both live and recorded assessments, and therefore is a valid and robust method for objectively assessing performance. The technology is in its infancy, and users need to develop confidence in its usability and reliability as an assessment tool. However, the potential to integrate additional functionality including holographic content, automated tracking and data analysis, and to facilitate remote assessment may allow the technology to enhance, expand and standardise examinations across a range of educational contexts.

Technological resources for teaching and learning about human anatomy in the medical course: Systematic review of literature

The consolidation of technology as an alternative strategy to cadaveric dissection for teaching anatomy in medical courses was accelerated by the recent Covid-19 pandemic, which caused the need for social distance policies and the closure of laboratories and classrooms. Consequently, new technologies were created, and those already been developed started to be better explored. However, information about many of these instruments and resources is not available to anatomy teachers. This systematic review presents the technological means for teaching and learning about human anatomy developed and applied in medical courses in the last ten years, besides the infrastructure necessary to use them. Studies in English, Portuguese, and Spanish were searched in MEDLINE, Scopus, ERIC, LILACS, and SciELO databases, initially resulting in a total of 875 identified articles, from which 102 were included in the analysis. They were classified according to the type of technology used: three-dimensional (3D) printing (n = 22), extended reality (n = 49), digital tools (n = 23), and other technological resources (n = 8). It was made a detailed description of technologies, including the stage of the medical curriculum in which it was applied, the infrastructure utilized, and which contents were covered. The analysis shows that between all technologies, those related to the internet and 3D printing are the most applicable, both in student learning and the financial cost necessary for its structural implementation.

Augmented reality in robotic assisted orthopaedic surgery: A pilot study

BACKGROUND

The research and development of augmented-reality (AR) technologies in surgical applications has seen an evolution of the traditional user-interfaces (UI) utilised by clinicians when conducting robot-assisted orthopaedic surgeries. The typical UI for such systems relies on surgeons managing 3D medical imaging data in the 2D space of a touchscreen monitor, located away from the operating site. Conversely, AR can provide a composite view overlaying the real surgical scene with co-located virtual holographic representations of medical data, leading to a more immersive and intuitive operator experience.

MATERIALS AND METHODS

This work explores the integration of AR within an orthopaedic setting by capturing and replicating the UI of an existing surgical robot within an AR head-mounted display worn by the clinician. The resulting mixed-reality workflow enabled users to simultaneously view the operating-site and real-time holographic operating informatics when carrying out a robot-assisted patellofemoral-arthroplasty (PFA). Ten surgeons were recruited to test the impact of the AR system on procedure completion time and operating surface roughness.

RESULTS AND DISCUSSION

The integration of AR did not appear to require subjects to significantly alter their surgical techniques, which was demonstrated by non-significant changes to the study's clinical metrics, with a statistically insignificant mean increase in operating time (+0.778 s, p = 0.488) and a statistically insignificant change in mean surface roughness (p = 0.274). Additionally, a post-operative survey indicated a positive consensus on the usability of the AR system without incurring noticeable physical distress such as eyestrain or fatigue.

CONCLUSIONS

Overall, these study results demonstrated a successful integration of AR technologies within the framework of an existing robot-assisted surgical platform with no significant negative effects in two quantitative metrics of surgical performance, and a positive outcome relating to user-centric and ergonomic evaluation criteria.