A Novel Three-Dimensional Interactive Virtual Face to Facilitate Facial Anatomy Teaching Using Microsoft HoloLens

Augmented and Virtual Reality Applications in Facial Plastic Surgery: A Scoping Review

OBJECTIVES

Augmented reality (AR) and virtual reality (VR) are emerging technologies with wide potential applications in health care. We performed a scoping review of the current literature on the application of augmented and VR in the field of facial plastic and reconstructive surgery (FPRS).

DATA SOURCES

PubMed and Web of Science.

REVIEW METHODS

According to PRISMA guidelines, PubMed and Web of Science were used to perform a scoping review of literature regarding the utilization of AR and/or VR relevant to FPRS.

RESULTS

Fifty-eight articles spanning 1997-2023 met the criteria for review. Five overarching categories of AR and/or VR applications were identified across the articles: preoperative, intraoperative, training/education, feasibility, and technical. The following clinical areas were identified: burn, craniomaxillofacial surgery (CMF), face transplant, face lift, facial analysis, facial palsy, free flaps, head and neck surgery, injectables, locoregional flaps, mandible reconstruction, mandibuloplasty, microtia, skin cancer, oculoplastic surgery, rhinology, rhinoplasty, and trauma.

CONCLUSION

AR and VR have broad applications in FPRS. AR for surgical navigation may have the most emerging potential in CMF surgery and free flap harvest. VR is useful as distraction analgesia for patients and as an immersive training tool for surgeons. More data on these technologies' direct impact on objective clinical outcomes are still needed.

LEVEL OF EVIDENCE

N/A Laryngoscope, 134:2568-2577, 2024.

Delivering clinical tutorials to medical students using the Microsoft HoloLens 2: A mixed-methods evaluation

BACKGROUND

Mixed reality offers potential educational advantages in the delivery of clinical teaching. Holographic artefacts can be rendered within a shared learning environment using devices such as the Microsoft HoloLens 2. In addition to facilitating remote access to clinical events, mixed reality may provide a means of sharing mental models, including the vertical and horizontal integration of curricular elements at the bedside. This study aimed to evaluate the feasibility of delivering clinical tutorials using the Microsoft HoloLens 2 and the learning efficacy achieved.

METHODS

Following receipt of institutional ethical approval, tutorials on preoperative anaesthetic history taking and upper airway examination were facilitated by a tutor who wore the HoloLens device. The tutor interacted face to face with a patient and two-way audio-visual interaction was facilitated using the HoloLens 2 and Microsoft Teams with groups of students who were located in a separate tutorial room. Holographic functions were employed by the tutor. The tutor completed the System Usability Scale, the tutor, technical facilitator, patients, and students provided quantitative and qualitative feedback, and three students participated in semi-structured feedback interviews. Students completed pre- and post-tutorial, and end-of-year examinations on the tutorial topics.

RESULTS

Twelve patients and 78 students participated across 12 separate tutorials. Five students did not complete the examinations and were excluded from efficacy calculations. Student feedback contained 90 positive comments, including the technology's ability to broadcast the tutor's point-of-vision, and 62 negative comments, where students noted issues with the audio-visual quality, and concerns that the tutorial was not as beneficial as traditional in-person clinical tutorials. The technology and tutorial structure were viewed favourably by the tutor, facilitator and patients. Significant improvement was observed between students' pre- and post-tutorial MCQ scores (mean 59.2% Vs 84.7%, p < 0.001).

CONCLUSIONS

This study demonstrates the feasibility of using the HoloLens 2 to facilitate remote bedside tutorials which incorporate holographic learning artefacts. Students' examination performance supports substantial learning of the tutorial topics. The tutorial structure was agreeable to students, patients and tutor. Our results support the feasibility of offering effective clinical teaching and learning opportunities using the HoloLens 2. However, the technical limitations and costs of the device are significant, and further research is required to assess the effectiveness of this tutorial format against in-person tutorials before wider roll out of this technology can be recommended as a result of this study.

3D human anatomy augmentation over a mannequin for the training of nursing skills

BACKGROUND

The in-depth understanding of human anatomy is the foundation for safety in nursing practice. Augmented reality is an emerging technology that can be used for integrative learning in nursing education.

OBJECTIVE

The study aimed to develop a human anatomy-based skill training system and pilot test its usability and feasibility.

METHODS

Twenty-seven nursing students participated in 3D anatomy-based skill training for intramuscular injection and Levin tube feeding using HoloLens 2. Various user interfaces including pictures, videos, animation graphics, and annotation boxes assisted users with a comprehensive understanding of the step-by-step procedures for these techniques. A one-group pre-post test was conducted to observe changes in skill performance competency, usability, and learning satisfaction.

RESULTS

After study participation, a statistically significant improvement in skill performance competency (p< 0.05) was observed. The usability results showed that students were satisfied with the usefulness of the program (9.55 ± 0.49) and scored highly for the intention to participate in other educational programs (9.62 ± 0.59). A high level of learning satisfaction was achieved (9.55 ± 0.49), with positive responses in fostering students' engagement and excitement in the application of cutting-edge technology.

CONCLUSION

The 3D anatomy-based nursing skill training demonstrated good potential to improve learning outcomes and facilitate engagement in self-directed practice. This can be integrated into undergraduate nursing education as an assistant teaching tool, contributing to the combination of knowledge and practice.

The application of augmented reality in plastic surgery training and education: A narrative review

Continuing problems with fewer training opportunities and a greater awareness of patient safety have led to a constant search for an alternative technique to bridge the existing theory-practice gap in plastic surgery training and education. The current COVID-19 epidemic has aggravated the situation, making it urgent to implement breakthrough technological initiatives currently underway to improve surgical education. The cutting edge of technological development, augmented reality (AR), has already been applied in numerous facets of plastic surgery training, and it is capable of realizing the aims of education and training in this field. In this article, we will take a look at some of the most important ways that AR is now being used in plastic surgery education and training, as well as offer an exciting glimpse into the potential future of this field thanks to technological advancements.

The impact of enhancing publicity and commemoration of body donors at Zhengzhou University, China

As a fundamental subject of medical education, human anatomy plays a critical role in the development of medical science. However, because of multiple factors including cultural conservativism and limited social understanding, China is facing a particularly severe shortage of bodies donated for anatomy education. Zhengzhou University (ZZU) has continued to uphold whole-body dissection as the preferred method for medical students to learn anatomy. For this study, records of registered individuals (who have signed a body donation agreement) and donors (whose bodies have been received) from 2001 to 2020 were collected and analyzed. The aim of this study was to explore the factors influencing the body donation program (BDP) at ZZU, and then reinforce the social understanding for the BDP. The results showed a significant increase in the numbers of both registered individuals and donors since 2015, which is the year the publicity and commemoration in honor of donors were increased. There were no significant differences between the biological male and female sexes in the registered individuals, but the number of male donors (12.85 ± 10.86, per year) was significantly higher than that of female donors (4.75 ± 4.53, per year). The current donor profile at ZZU is male in his 60/70s, while the profile of registered individuals is male or female in their 60s. Strengthening the publicity and commemoration in honor of donors may contribute to the implementation of BDPs.

The HoloLens in medicine: A systematic review and taxonomy

The HoloLens (Microsoft Corp., Redmond, WA), a head-worn, optically see-through augmented reality (AR) display, is the main player in the recent boost in medical AR research. In this systematic review, we provide a comprehensive overview of the usage of the first-generation HoloLens within the medical domain, from its release in March 2016, until the year of 2021. We identified 217 relevant publications through a systematic search of the PubMed, Scopus, IEEE Xplore and SpringerLink databases. We propose a new taxonomy including use case, technical methodology for registration and tracking, data sources, visualization as well as validation and evaluation, and analyze the retrieved publications accordingly. We find that the bulk of research focuses on supporting physicians during interventions, where the HoloLens is promising for procedures usually performed without image guidance. However, the consensus is that accuracy and reliability are still too low to replace conventional guidance systems. Medical students are the second most common target group, where AR-enhanced medical simulators emerge as a promising technology. While concerns about human-computer interactions, usability and perception are frequently mentioned, hardly any concepts to overcome these issues have been proposed. Instead, registration and tracking lie at the core of most reviewed publications, nevertheless only few of them propose innovative concepts in this direction. Finally, we find that the validation of HoloLens applications suffers from a lack of standardized and rigorous evaluation protocols. We hope that this review can advance medical AR research by identifying gaps in the current literature, to pave the way for novel, innovative directions and translation into the medical routine.

Can mixed reality technologies teach surgical skills better than traditional methods? A prospective randomised feasibility study

BACKGROUND

Basic surgical skills teaching is often delivered with didactic audio-visual content, and new digital technologies may allow more engaging and effective ways of teaching to be developed. The Microsoft HoloLens 2 (HL2) is a multi-functional mixed reality headset. This prospective feasibility study sought to assess the device as a tool for enhancing technical surgical skills training.

METHODS

A prospective randomised feasibility study was conducted. 36 novice medical students were trained to perform a basic arteriotomy and closure using a synthetic model. Participants were randomised to receive a structured surgical skills tutorial via a bespoke mixed reality HL2 tutorial (n = 18), or via a standard video-based tutorial (n = 18). Proficiency scores were assessed by blinded examiners using a validated objective scoring system and participant feedback collected.

RESULTS

The HL2 group showed significantly greater improvement in overall technical proficiency compared to the video group (10.1 vs. 6.89, p = 0.0076), and a greater consistency in skill progression with a significantly narrower range of scores (SD 2.48 vs. 4.03, p = 0.026). Participant feedback showed the HL2 technology to be more interactive and engaging with minimal device related problems experienced.

CONCLUSIONS

This study has demonstrated that mixed reality technology may provide a higher quality educational experience, improved skill progression and greater consistency in learning when compared to traditional teaching methodologies for basic surgical skills. Further work is required to refine, translate, and evaluate the scalability and applicability of the technology across a broad range of skills-based disciplines.

Use of Extended Reality in Medical Education: An Integrative Review

Extended reality (XR) has emerged as an innovative simulation-based learning modality. An integrative review was undertaken to explore the nature of evidence, usage, and effectiveness of XR modalities in medical education. One hundred and thirty-three (N = 133) studies and articles were reviewed. XR technologies are commonly reported in surgical and anatomical education, and the evidence suggests XR may be as effective as traditional medical education teaching methods and, potentially, a more cost-effective means of curriculum delivery. Further research to compare different variations of XR technologies and best applications in medical education and training are required to advance the field.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40670-022-01698-4.

The use of mixed reality technology for the objective assessment of clinical skills: a validation study

BACKGROUND

Mixed Reality technology may provide many advantages over traditional teaching methods. Despite its potential, the technology has yet to be used for the formal assessment of clinical competency. This study sought to collect validity evidence and assess the feasibility of using the HoloLens 2 mixed reality headset for the conduct and augmentation of Objective Structured Clinical Examinations (OSCEs).

METHODS

A prospective cohort study was conducted to compare the assessment of undergraduate medical students undertaking OSCEs via HoloLens 2 live (HLL) and recorded (HLR), and gold-standard in-person (IP) methods. An augmented mixed reality scenario was also assessed.

RESULTS

Thirteen undergraduate participants completed a total of 65 OSCE stations. Overall inter-modality correlation was 0.81 (p = 0.01), 0.98 (p = 0.01) and 0.82 (p = 0.01) for IP vs. HLL, HLL vs. HLR and IP vs. HLR respectively. Skill based correlations for IP vs. HLR were assessed for history taking (0.82, p = 0.01), clinical examination (0.81, p = 0.01), procedural (0.88, p = 0.01) and clinical skills (0.92, p = 0.01), and assessment of a virtual mixed reality patient (0.74, p = 0.01). The HoloLens device was deemed to be usable and practical (Standard Usability Scale (SUS) score = 51.5), and the technology was thought to deliver greater flexibility and convenience, and have the potential to expand and enhance assessment opportunities.

CONCLUSIONS

HoloLens 2 is comparable to traditional in-person examination of undergraduate medical students for both live and recorded assessments, and therefore is a valid and robust method for objectively assessing performance. The technology is in its infancy, and users need to develop confidence in its usability and reliability as an assessment tool. However, the potential to integrate additional functionality including holographic content, automated tracking and data analysis, and to facilitate remote assessment may allow the technology to enhance, expand and standardise examinations across a range of educational contexts.

Current and potential use of fresh frozen cadaver in surgical training and anatomical education

As surgical procedures continue to be more complex, the need for more effective training in anatomy has increased. The study of anatomy plays a significant role in the understanding of the human body as well as in basic and advanced clinical training. Among the different cadaver models, fresh frozen cadavers (FFCs) are known for their realistic tissue quality. The purpose of this article was to review and summarize the preparation procedures for and reported cases involving FFCs. PubMed, Scopus, Medline, and Web of Science were searched for relevant studies. The preparation procedures were divided into five steps: washing, irrigation, freezing, defrosting, and arterial infusion. Not all steps were reported to be mandatory, but omitting one or more could result in a loss of quality. FFCs were reported to be used for various purposes: undergraduate education, general surgery training, vascular surgery training, minimal access surgery (laparoscopic surgery) training, and microsurgery training. In all categories, expert opinions and statistical analyses indicated successful outcomes. The reasons for high satisfaction with FFCs included realistic texture, capability of reenacting actual operations, and accuracy of anatomical locations. The results also revealed the importance and advantages of the dissection courses in surgical training. Since the direct comparison between cadaver models is insufficient, future studies regarding this topic are deemed necessary. In addition, it would be advantageous to develop methods to improve FFC quality, or ideas to optimize this model for certain purposes.

Augmented Reality in Surgery: A Scoping Review

Augmented reality (AR) is an innovative system that enhances the real world by superimposing virtual objects on reality. The aim of this study was to analyze the application of AR in medicine and which of its technical solutions are the most used. We carried out a scoping review of the articles published between 2019 and February 2022. The initial search yielded a total of 2649 articles. After applying filters, removing duplicates and screening, we included 34 articles in our analysis. The analysis of the articles highlighted that AR has been traditionally and mainly used in orthopedics in addition to maxillofacial surgery and oncology. Regarding the display application in AR, the Microsoft HoloLens Optical Viewer is the most used method. Moreover, for the tracking and registration phases, the marker-based method with a rigid registration remains the most used system. Overall, the results of this study suggested that AR is an innovative technology with numerous advantages, finding applications in several new surgery domains. Considering the available data, it is not possible to clearly identify all the fields of application and the best technologies regarding AR.

Emerging simulation technologies in global craniofacial surgical training

The last few decades have seen an exponential growth in the development and adoption of novel technologies in medical and surgical training of residents globally. Simulation is an active and innovative teaching method, and can be achieved via physical or digital models. Simulation allows the learners to repeatedly practice without the risk of causing any error in an actual patient and enhance their surgical skills and efficiency. Simulation may also allow the clinical instructor to objectively test the ability of the trainee to carry out the clinical procedure competently and independently prior to trainee's completion of the program. This review aims to explore the role of emerging simulation technologies globally in craniofacial training of students and residents in improving their surgical knowledge and skills. These technologies include 3D printed biomodels, virtual and augmented reality, use of google glass, hololens and haptic feedback, surgical boot camps, serious games and escape games and how they can be implemented in low and middle income countries. Craniofacial surgical training methods will probably go through a sea change in the coming years, with the integration of these new technologies in the surgical curriculum, allowing learning in a safe environment with a virtual patient, through repeated exercise. In future, it may also be used as an assessment tool to perform any specific procedure, without putting the actual patient on risk. Although these new technologies are being enthusiastically welcomed by the young surgeons, they should only be used as an addition to the actual curriculum and not as a replacement to the conventional tools, as the mentor-mentee relationship can never be replaced by any technology.