The Mindful Manager: Validation of a Rounding Leadership Instrument for Residents

Designing a Checklist for Directly Observing Use of One-Minute Preceptor Steps on Inpatient Rounds: A Pilot Study

BACKGROUND

"One-minute preceptor" (OMP) is a well-established educational technique; however, primary literature on OMP lacks a tool to assess behavioral change after delivery of curricula.Primary aim of this pilot study was to design a checklist for direct observation of teachers using OMP on general medicine rounds and obtain inter-rater reliability evidence for the checklist.

METHODS

This study pilots an internally designed 6-item checklist to assess change in directly observed behavior. We describe the process of developing the checklist and training the observers. We calculated a percent agreement and Cohen's kappa to assess inter-rater reliability.

RESULTS

Raters had a high percent agreement ranging from 0.8 to 0.9 for each step of OMP. Cohen's kappa ranged from 0.49 to 0.77 for the five OMP steps. The highest kappa obtained was for getting a commitment (κ = 0.77) step, whereas the lowest agreement was for correcting mistakes (κ = 0.49).

CONCLUSION

We showed a percent agreement ≥0.8 and moderate agreement based on Cohen's kappa with most steps of OMP on our checklist. A reliable OMP checklist is an important step in further improving the assessment and feedback of resident teaching skills on general medicine wards.

A Resident-as-Leader Curriculum for Managing Inpatient Teams

PROBLEM

Leading inpatient teams is a foundational clinical responsibility of resident physicians and leadership is a core competency for inpatient physicians, yet few training programs have formal leadership curricula to realize this clinical skill.

INTERVENTION

We implemented a 4-module curriculum for PGY1 internal medicine residents. The program focused on the managerial skills necessary for daily clinical leadership, followed by clinical coaching. Interns were first introduced to foundational concepts and then given the opportunity to apply those concepts to real-world practice followed by clinical coaching.

CONTEXT

Using direct-observations and a previously published checklist for rounds leadership, this study sought to evaluate the workplace behavior change for novice residents leading inpatient teams for the first time. We conducted a prospective cohort study (March 2016 and August 2018) of internal medicine residents at a large tertiary academic medical center in Boston, MA. Trained faculty raters performed direct observations of clinical rounding experiences using the checklist and compared the findings to historical and internal controls. Questionnaires were distributed pre- and post- curriculum to assess satisfaction and readiness to lead a team.

IMPACT

We trained 65 PGY1 residents and raters conducted 140 direct observations - 36 in the intervention group and 104 among historical controls. The unadjusted mean score in rounds leadership skills for the intervention group was 19.0 (SD = 5.1) compared to 16.2 (SD = 6.2) for historical controls. Adjusting for repeated measures, we found significant improvement in mean scores for behaviors linked to the curricular objectives (p = 0.008) but not for general behaviors not covered by the curriculum (p = 0.2).

LESSONS LEARNED

A formal curriculum to train residents as leaders led to behavior change in the workplace in domains essential to rounds leadership. We also found that the curriculum was highly regarded in that all interns indicated they would recommend the curriculum to a peer. Moreover, the program may have assuaged some anxiety during the transition to junior year as 90% of interns surveyed felt more ready to start PGY2 year than historical trainings. We learned that while a robust, multi-faceted modular curriculum and clinical coaching successfully resulted in behavior change, the resources required to manage this program are significant and difficult to sustain. Future iterations could include asynchronous material and potentially peer-observation of rounds leadership to reduce the burden on faculty and program curricular time.