Understanding resident wellness: A path analysis of the clinical learning environment at three institutions

PURPOSE

The clinical learning environment (CLE) affects resident physician well-being. This study assessed how aspects of the learning environment affected the level of resident job stress and burnout.

MATERIALS AND METHODS

Three institutions surveyed residents assessing aspects of the CLE and well-being via anonymous survey in fall of 2020 during COVID. Psychological safety (PS) and perceived organizational support (POS) were used to capture the CLE, and the Mini-Z Scale was used to assess resident job stress and burnout. A total of 2,196 residents received a survey link; 889 responded (40% response rate). Path analysis explored both direct and indirect relationships between PS, POS, resident stress, and resident burnout.

RESULTS

Both POS and PS had significant negative relationships with experiencing a great deal of job stress; the relationship between PS and stress was noticeably stronger than POS and stress (POS: B= -0.12, p=.025; PS: B= -0.37, p<.001). The relationship between stress and residents' level of burnout was also significant (B = 0.38, p<.001). The overall model explained 25% of the variance in resident burnout.

CONCLUSIONS

Organizational support and psychological safety of the learning environment is associated with resident burnout. It is important for educational leaders to recognize and mitigate these factors.

Understanding Academic Self-Concept and Performance in Medical Education

PROBLEM

Medical students' academic self-concept (ASC) is an important factor in better understanding noncognitive mediators of performance in medical school. However, research is limited on ASC in medical students across multiple phases of undergraduate medical education curriculum. This pilot study explored the relationship between ASC and academic performance across different phases of a U.S. medical school curriculum, specifically at the end of the second (preclinical) and third (clinical) years.

APPROACH

Medical students across 2 cohorts at Virginia Commonwealth University School of Medicine, Richmond, Virginia, were surveyed using an ASC confidence subscale in 2019. Multiple linear regression analysis was conducted using medical student ASC scores in preclinical (n = 190) and clinical (n = 149) phases and performance data. Clinical performance was calculated through a weighted mean of clerkship grades based on the number of weeks for each clerkship.

OUTCOMES

Preclinical performance was related to ASC, gender, and performance after year 1. ASC scores varied significantly by gender in the preclinical cohort ( P < .01), with men reporting higher ASC than women (mean [SD], 2.94 [0.41] vs 2.78 [0.38]). Significant gender differences in performance were found at the end of year 3 ( P < .01), with women performing more favorably compared with men (mean [SD], 94.1 [59.04] vs 124.24 [64.54]). The relationship between ASC and performance at the end of year 2 suggested students with higher ASC perform better during their preclinical phase.

NEXT STEPS

This pilot study supports future scholarship in 2 areas: (1) identification and assessment of additional factors that influence the relationship between ASC and academic performance across the entire undergraduate medical education curriculum and (2) development and implementation of evidence-based interventions to support student ASC and performance and enhance the learning environment. Analyzing longitudinal trends across multiple cohorts will drive evidence-based interventions at learner and programmatic levels.

A mixed methods exploration of the emergence of goal orientation in medical students' individualized learning plans

PURPOSE

Goal orientation (GO) describes an individual's approach to different achievement situations. Understanding the motivations and approach to achieving goals of medical students is vital with the increasing emphasis on self-directed learning. The purpose of this study was to identify themes in self-improvement reflections that relate to each GO dimension (learning, performance-prove, and performance-avoid).

METHODS

A sequential explanatory mixed methods design was used. GO data was used to categorize students into groups aligning with the GO dimension identified in the previous stage of quantitative analysis. Individualized learning plans (ILPs) for each GO dimension group were coded inductively to identify emergent themes related to goal setting and achievement.

RESULTS

The learning GO group was the largest of the three GOs. Five themes were identified from inductive analysis: importance of practice, identifying elements that helped, identifying structural barriers, opportunities for improvement, and acknowledging experience. While these themes occur across GO, patterns exist within their ILPs based on GO.

CONCLUSIONS

We identified common themes for motivations of medical students, and these motivations might differ depending on their GO. Further exploration into the themes over the course of their training will provide additional insights on what factors may be involved in student motivations towards learning and achievement. Educators can use this information to individualize feedback and students can better understand their motivations towards achieving goals.

Rapid Feedback: Assessing Pre-clinical Teaching in the Era of Online Learning

INTRODUCTION

Medical schools vary in their approach to providing feedback to faculty. The purpose of this study was to test the effects of rapid student feedback in a course utilizing novel virtual learning methods.

METHODS

Second-year medical students were supplied with an optional, short questionnaire at the end of each class session and asked to provide feedback within 48 h. At the close of each survey, results were emailed to faculty. After the course, students and faculty were asked to rate the effectiveness of this method. This study did not affect administration of the usual end-of-course summative evaluations.

RESULTS

Ninety-one percent of students who participated noted increased engagement in the feedback process, but only 18% on average chose to participate. Faculty rated rapid feedback as more actionable than summative feedback (67%), 50% rated it as more specific, and 42% rated it as more helpful. Some wrote that comments were too granular, and others noted a negative personal emotional response.

CONCLUSION

Rapid feedback engaged students, provided actionable feedback, and increased communication between students and instructors, suggesting that this approach added value. Care must be taken to reduce the student burden and support relational aspects of the process.

Examining Bloom's Taxonomy in Multiple Choice Questions: Students' Approach to Questions

BACKGROUND

Analytic thinking skills are important to the development of physicians. Therefore, educators and licensing boards utilize multiple-choice questions (MCQs) to assess these knowledge and skills. MCQs are written under two assumptions: that they can be written as higher or lower order according to Bloom's taxonomy, and students will perceive questions to be the same taxonomical level as intended. This study seeks to understand the students' approach to questions by analyzing differences in students' perception of the Bloom's level of MCQs in relation to their knowledge and confidence.

METHODS

A total of 137 students responded to practice endocrine MCQs. Participants indicated the answer to the question, their interpretation of it as higher or lower order, and the degree of confidence in their response to the question.

RESULTS

Although there was no significant association between students' average performance on the content and their question classification (higher or lower), individual students who were less confident in their answer were more than five times as likely (OR = 5.49) to identify a question as higher order than their more confident peers. Students who responded incorrectly to the MCQ were 4 times as likely to identify a question as higher order than their peers who responded correctly.

CONCLUSIONS

The results suggest that higher performing, more confident students rely on identifying patterns (even if the question was intended to be higher order). In contrast, less confident students engage in higher-order, analytic thinking even if the question is intended to be lower order. Better understanding of the processes through which students interpret MCQs will help us to better understand the development of clinical reasoning skills.

Versatility in multiple mini-interview implementation: Rater background does not significantly influence assessment scoring

The medical school admissions process seeks to assess a core set of cognitive and non-cognitive competencies that reflect professional readiness and institutional mission alignment. The standardized format of multiple mini-interviews (MMIs) can enhance assessments, and thus many medical schools have switched to this for candidate interviews. However, because MMIs are resource-intensive, admissions deans use a variety of interviewers from different backgrounds/professions. Here, we analyze the MMI process for the 2018 admissions cycle at the VCU School of Medicine, where 578 applicants were interviewed by 126 raters from five distinct backgrounds: clinical faculty, basic science faculty, medical students, medical school administrative staff, and community members. We found that interviewer background did not significantly influence MMI evaluative performance scoring, which eliminates a potential concern about the consistency and reliability of assessment.

Developing Comprehensive Strategies to Evaluate Medical School Curricula

Evaluation of medical school curriculum is important to document outcomes, effectiveness of learning, engagement in quality improvement, and to meet accreditation compliance. This monograph provides a roadmap and resource for medical schools to meaningfully evaluate their curriculum based on specific metrics. The method of evaluation includes an examination of Kirkpatrick's levels of outcomes including reactions, learning, behavior, and impact. It is important that student outcomes are mapped in relation to curricular objectives. There are specific outcomes that may be utilized to determine if the curriculum has met the institution's goals. The first is comparison to national metrics (United States Medical Licensing Examinations and American Association of Medical Colleges Graduation Questionnaire). Second, medical schools collect internal program metrics, which include specific student performance metrics, such as number of students graduating, attrition, and matching to specialty. Further, schools may examine student performance and surveys in the preclerkship and clinical phases (e.g., grades, failing courses, survey responses about the curriculum), including qualitative responses on surveys or focus groups. As the learning environment is critical to learning, a deep dive to understand the environment and mistreatment may be important for program evaluation. This may be performed by specifically examining the Graduation Questionnaire, internal surveys, and mistreatment reporting. Finally, there are numerous attitudinal instruments that may help medical schools understand their students' development at one point or over time. These include measurements of stress, wellness, burnout, lifelong learning, and attitudes toward patient safety. Together, examining the composite of outcomes helps to understand and improve the medical school curriculum.

461. Electronic Hand Hygiene Compliance Monitoring Systems: Not All Are Created Equal

Background

While direct observation is considered the gold standard for hand hygiene (HH) surveillance, there is a growing interest in the implementation of electronic monitoring systems, which claim to accurately capture individual-level HH performance.

Methods

Two types of electronic hand hygiene monitoring systems (EHHMS) were trialed at an 865-bed, academic medical center over an 18-month period. Each type of EHHMS was piloted in two inpatient units, and hospital employees who had contact with patients and/or the patient environment were eligible to participate. In each trial, participants received standard training and were then asked to wear EHHMS badges while continuing their normal workflow. Methods of assessment included regular review of EHHMS reports, an inter-rater reliability analysis to compare EHHMS to direct observation by trained HH observer, and a qualitative electronic survey to assess the acceptability of EHHMS. HH compliance goal was set at 90%.

Results

In the first pilot, 279 employees volunteered to trial Type A EHHMS for 14 weeks, with an overall HH compliance of 30% (87,688 opportunities). In the second pilot, 169 employees volunteered to trial Type B EHHMS for 12 weeks, with an overall HH compliance of 93% (363,272 opportunities). Voluntary survey response rate for Type A was 32% (90/279) and for Type B was 40% (67/169). The majority of respondents consistently used EHHMS in daily workflow (Type A: 82%, 68/83) (Type B: 82%, 55/67) and most did not felt apprehensive about using the EHHMS (Type A: 19%, 16/83) (Type B: 22%, 15/67).

Inter-rater reliability assessment of piloted EHHMS

Conclusion

Type B EHHMS captured our healthcare workers’ HH performance during clinical workflow with a greater accuracy and more HH events than Type A. EHHMS may provide an alternative method to capture HH compliance in the healthcare setting. Hospitals considering the use of an EHHMS should assess the technology’s ability to accurately capture HH performance in the clinical workflow prior full housewide implementation.

Disclosures

All authors: No reported disclosures.