Filling the Core EPA 10 assessment void: A framework for individual assessment of Core Entrustable professional activity 10 competencies in medical students

Evaluation of high-fidelity and virtual reality simulation platforms for assessing fourth-year medical students' encounters with patients in need of urgent or emergent care

BACKGROUND

Medical students in the U.S. must demonstrate urgent and emergent care competence before graduation. Urgent and emergent care competence involves recognizing, evaluating and initiating management of an unstable patient. High-fidelity (HF) simulation can improve urgent and emergent care skills, but because it is resource intense, alternative methods are needed.

STUDY OBJECTIVE

Our primary purpose was to use program evaluations to compare medical student experiences with HF and virtual reality (VR) simulations as assessment platforms for urgent and emergent care skills.

METHODS

During their emergency medicine clerkship, students at The Ohio State University College of Medicine must demonstrate on HF manikins, competence in recognizing and initiating care of a patient requiring urgent or emergent care. Students evaluated these simulations on a five-point quality scale and answered open-ended questions about simulation strengths and weaknesses. Faculty provided feedback on student competence in delivering urgent or emergent care. In 2022, we introduced VR as an alternative assessment platform. We used Wilcoxon Signed Ranks and Boxplots to compare ratings of HF to VR and McNemar Test to compare competence ratings. Comments were analyzed with summative content analysis or thematic coding.

RESULTS

We received at least one evaluation survey from 160 of 216 (74.1%) emergency medicine clerkship students. We were able to match 125 of 216 (57.9%) evaluation surveys for students who completed both. Average ratings of HF simulations were 4.6 of 5, while ratings of VR simulations were slightly lower at 4.4. Comments suggested that feedback from both simulation platforms was valued. Students described VR as novel, immersive, and good preparation for clinical practice. Constructive criticism identified the need for additional practice in the VR environment. Student performance between platforms was significantly different with 91.7% of students achieving competence in HF, but only 65.5% in VR (p≤.001, odds-ratio = 5.75).

CONCLUSION

VR simulation functions similarly to HF for formative assessment of urgent and emergent care competence. However, using VR simulation for summative assessment of urgent and emergent care competence must be considered with caution because students require considerable practice and acclimation to the virtual environment.

Using Virtual Reality to Teach Medical Students Cross-Coverage Skills

BACKGROUND

Recognizing and attempting management of patients with urgent or emergent conditions is one of the Association of American Medical Colleges Core Entrustable Professional Activities (#10) and a skill desired of new interns. However, given the acuity of these patient conditions, medical students often struggle to gain experience in these situations. Virtual reality could help fill this void while lowering costs and resources compared with high-fidelity simulation.

METHODS

We converted a high-fidelity chest pain simulation case to virtual reality format utilizing short video clips filmed with a 360-degree camera and superimposed menus of options at decision points. This virtual reality simulation was offered to fourth-year medical students during their transition to residency course in the spring of 2023. Students were offered a post-survey on the simulation.

RESULTS

There were 47 fourth-year students that completed the virtual reality simulation; 41 completed the post-survey (response rate 87.2%). Over 90% of the students agreed or strongly agreed with the following statements: the virtual reality simulation was a valuable part of the transition to residency course, the virtual reality case was similar to what they will face as an intern, and they would like to have more virtual reality simulations earlier in the fourth year; 85.4% agreed or strongly agreed that the virtual reality simulation helped prepare them for the first few days of intern year.

CONCLUSIONS

We demonstrated that virtual reality is an acceptable, cost-effective, and feasible modality to teach medical students how to recognize and attempt management of urgent clinical situations (Core Entrustable Professional Activity 10).