Capturing growth curves of medical students' clinical skills performance

INTRODUCTION

A benefit of a milestone or Entrustable Professional Activity (EPA) assessment framework is the ability to capture longitudinal performance with growth curves using multi-level modelling (MLM). Growth curves can inform curriculum design and individualised learning. Residency programmes have found growth curves to vary by resident and by milestone. Only one study has analysed medical students' growth curves for EPAs. Analysis of EPA growth curves is critical because no change in performance raises concerns for EPAs as an assessment framework.

METHODS

Spencer Fox Eccles School of Medicine-University of Utah students' workplace-based assessment ratings for 7 EPAs were captured at 3 time-points in years 3-4 of AY2017-2018 to AY2020-2021. MLM was used to capture EPA growth curves and determine if variation in growth curves was explained by internal medicine (IM) clerkship order.

FINDINGS

A curvilinear slope significantly captured 256 students' average ratings overtime for EPA1a-history-taking, EPA2-clinical reasoning, EPA3-diagnostics, EPA5-documentation and EPA6-presentation, and a linear slope significantly captured EPA9-teamwork ratings, p ≤ 0.001. Growth curves were steepest for EPA2-clinical reasoning and EPA3-diagnostics. Growth curves varied by students, p < 0.05 for all EPA ratings, but IM clerkship rotation order did not significantly explain the variance, p > 0.05.

DISCUSSION

The increase in ratings from Year 3 to Year 4 provides validity evidence for use of EPAs in an assessment framework. Students may benefit from more curriculum/skills practice for EPA2-clinical reasoning and EPA3-diagnostics prior to year 3. Variation in student's growth curves is important for coaching and skill development; a one size fits all approach may not suffice.

Comparison of internal medicine applicant and resident characteristics with performance on ACGME milestones

Internal medicine (IM) residency programs select applicants based on several metrics. Factors predicting success during residency are unclear across studies. To identify whether specific applicant or resident factors are associated with IM resident performance using ACGME milestones. We tested for associations between applicant factors available prior to the start of IM residency and resident factors measured during IM residency training, and resident performance on ACGME milestones across three consecutive years of IM training between 2015-2020. Univariable and multivariable linear regression modeling was used to test associations. Eighty-nine categorical IM residents that completed 3 consecutive years of training were included. Median age was 28 years (IQR 27-29) and 59.6% were male. Mean ACGME milestone scores increased with each post-graduate year (PGY) from 3.36 (SD 0.19) for PGY-1, to 3.80 (SD 0.15) for PGY-2, to 4.14 (SD 0.15) for PGY-3. Univariable modeling suggested referral to the clinical competency committee (CCC) for professionalism concerns was negatively associated with resident performance during each PGY. No applicant or resident factors included in the final multivariable regression models (age at starting residency, USMLE Step scores, interview score, rank list position, ITE scores) were associated with ACGME milestone scores for PGY-1 and PGY-2. Referral to the CCC for professionalism was negatively associated with resident performance during PGY-3. Residency selection factors did not predict resident milestone evaluation scores. Referral to the CCC was associated with significantly worse resident evaluation scores, suggesting professionalism may correlate with clinical performance.

Impact of Opt-Out Therapy Appointments on Resident Physicians' Mental Health, Well-Being, Stigma, and Willingness to Engage

Background Health care professionals report higher levels of mental health symptoms, pandemic-related stress, personal health concerns, and reduced proactive coping, especially in recent years marked by the COVID-19 pandemic. As physician utilization rates of mental health and well-being services remain low, the need for preemptive care is crucial. Objective The present study sought to ascertain satisfaction, value, and attitude toward future mental health services among resident physicians. Methods Throughout the 2020-2021 academic year, resident physicians within 8 training programs at a large academic training hospital were offered single opt-out mental health appointments from hospital-funded, graduate medical education wellness staff at no cost to the resident. Appointments were conducted virtually during protected work time. A survey was sent to participants within 2 weeks following their appointment. Results A total of 153 residents (postgraduate years 1 to 7) were offered one-time opt-out appointments. Overall, 91 (59%) residents attended their opt-out appointments. Survey response rate was 57% (n=52). Respondents reported high levels of satisfaction (96%, 50 of 52), felt the appointment was worth their time (96%, 50 of 52), and felt that the opt-out appointments demonstrated training programs cared about their well-being (94%, 49 of 52). Nearly all residents (98%, 51 of 52) recommended appointments be offered to future residents. Most respondents (80%, 42 of 52) indicated that appointments increased their willingness to engage in mental health services. Conclusions Opt-out appointments increased resident willingness to engage in mental health services, positive attitudes toward future mental health services, perceived training program's care about their well-being, and reduced perceived mental illness stigma.

Medical Student Experiential Learning in Telesimulation

OBJECTIVES

Telesimulation utilizes telecommunication technology to engage learners in simulation while in different physical locations. Despite this potential advantage, understanding of the student experience and assessment of student learning in telesimulation activities is limited. This study evaluates medical student emotional experience and self-identified learning in telesimulation through the Kolb experiential learning framework and qualitative analysis.

METHODS

Fourth-year medical students enrolled in the Spencer Fox Eccles School of Medicine at the University of Utah participated in 3 telesimulation activities as part of a required internal medicine course. Students were surveyed regarding their satisfaction with the activity (N = 114) and responded to questions about their emotional experience and self-identified areas of learning. Free-text responses were analyzed using qualitative content analysis to identify themes until thematic saturation (N = 66).

RESULTS

Students were highly satisfied with telesimulation, with greater than 90% of students expressing a positive view of simulation realism, debrief quality, and group size. Themes of anxiety and uncertainty, confidence versus incompetence, team dynamics, fun, and difficult patient interaction were identified regarding the emotional experience. Themes of communication and teamwork, managing emotions, information gathering, differential diagnosis, resource reference, executing treatment, and medical knowledge were identified regarding student-identified learning.

CONCLUSION

In this analysis of medical student experiences with telesimulation, we found students have rich emotional, cognitive, and behavioral experiences and self-identify learning across a variety of domains. Our findings support further study of telesimulation for medical student learning and demonstrate how assessment of outcomes via Kolb framework, using the learner's reflective observation and self-identified learning, may help better define learning outcomes from simulation.