"Doing it Right" Overnight: a Multi-perspective Qualitative Study Exploring Senior Medical Resident Overnight Call

Development of a Tool to Assess Surgical Resident Competence On-Call: The Western University Call Assessment Tool (WUCAT)

BACKGROUND

A central tenet of competency-based medical education is the formative assessment of trainees. There are currently no assessments designed to examine resident competence on-call, despite the on-call period being a significant component of residency, characterized by less direct supervision compared to daytime. The purpose of this study was to design a formative on-call assessment tool and collect valid evidence on its application.

METHODS

Nominal group technique was used to identify critical elements of surgical resident competence on-call to inform tool development. The tool was piloted over six months in the Division of Plastic & Reconstructive Surgery at our institution. Quantitative and qualitative evidence was collected to examine tool validity.

RESULTS

A ten-item tool was developed based on the consensus group results. Sixty-three assessments were completed by seven staff members on ten residents during the pilot. The tool had a reliability coefficient of 0.67 based on a generalizability study and internal item consistency was 0.92. Scores were significantly associated with years of training. We found the tool improved the quantity and structure of feedback given and that the tool was considered feasible and acceptable by both residents and staff members.

CONCLUSIONS

The Western University Call Assessment Tool (WUCAT) has multiple sources of evidence supporting its use in assessing resident competence on-call.

Understanding the near-peer relationship: resident perspectives around a novel on-call workplace-based assessment

BACKGROUND

Workplace-based assessment (WBA) is a critical component of competency-based medical education (CBME), though literature on WBA for overnight call is limited. We evaluated a WBA tool completed by supervising subspecialty trainees on paediatric residents during subspecialty overnight call, for usefulness facilitating feedback/coaching in this setting.

METHODS

Web-based surveys were sent to residents pre- and post-WBA tool implementation monthly for four months (August-December 2018), exploring feedback frequency, Likert-scaled opinions of tool feasibility/usefulness facilitating feedback, and qualitative experiences. Assessor comments were categorized as actionable/non-actionable. Quantitative data was summarized using descriptive statistics. Qualitative data was coded to identify themes.

RESULTS

Total response rates averaged 41% (total 25 responses, average five respondents/12 residents on-call each month). Post-implementation (n = 16 responses), a non-sustained trend of increased Medical Expert feedback was observed. Residents were generally divided or disagreed on tool usefulness facilitating feedback and feasibility. Comments contained actionable feedback in < 10% of completed WBAs. Qualitative analysis revealed barriers to tool-facilitated coaching including: feedback quality and setting/environment, role of senior near-peer as assessor, interpersonal burden in encounters, and tool-specific issues.

CONCLUSIONS

Increasing frequency of WBA tool completion is not sufficient to achieve CBME goals. Factors impacting feedback/coaching within the resident/near-peer dyad must be addressed.

An act of performance: Exploring residents' decision-making processes to seek help

CONTEXT

Residents are expected to ask for help when feeling insufficiently confident or competent to act in patients' best interests. While previous studies focused on the perspective of supervisor-resident relationships in residents' help-seeking decisions, attention for how the workplace environment and, more specifically, other health care team members influence these decisions is limited. Using a sociocultural lens, this study aimed to explore how residents' decision-making processes to seek help are shaped by their workplace environment.

METHODS

Through a constructivist grounded theory methodology, we purposively and theoretically sampled 18 residents: 9 juniors (postgraduate year 1/2) and 9 seniors (postgraduate year 5/6) at Amsterdam University Medical Centers. Using semi-structured interviews, participating residents' decision-making processes to seek help during patient care delivery were explored. Data collection and analysis were iterative; themes were identified using constant comparative analysis.

RESULTS

Residents described their help-seeking decision-making processes as an 'act of performance': they considered how asking for help could potentially impact their assessments. They described this act of performance as the product of an internal 'balancing act' with at its core the non-negotiable priority for providing safe and high-quality patient care. With this in mind, residents weighed up demonstrating the ability to work independently, maintaining credibility and becoming an accepted member of the health care team when deciding to seek help. This 'balancing act' was influenced by sociocultural characteristics of the learning environment, residents' relationships with supervisors and the perceived approachability of other health care team members.

CONCLUSIONS

This study suggests that sociocultural forces influence residents to experience help-seeking as an act of performance. Especially, a safe learning environment resulting from constructive relationships with supervisors and the approachability of other health care team members lowered the barriers to seek help. Supervisors could address these barriers by having regular conversations with residents about when to seek help.