"It's Time to Do It, to Make It a Major Part of Medicine": Faculty Experience With a Shift Toward an Antioppressive Medical School Curriculum

PURPOSE

Health inequities compel medical educators to transform curricula to prepare physicians to improve the health of diverse populations. This mandate requires curricular focus on antioppression, which is a change for faculty who learned and taught under a different paradigm. This study used the Concerns-Based Adoption Model (CBAM) to explore faculty perceptions of and experiences with a shift to a curriculum that prioritizes antioppressive content and process.

METHOD

In this qualitative study, authors interviewed faculty course directors and teachers at the University of California, San Francisco School of Medicine from March 2021 to January 2022. Questions addressed faculty experience and understanding regarding the curriculum shift toward antioppression, perceptions of facilitators and barriers to change, and their interactions with colleagues and learners about this change. Using the CBAM components as sensitizing concepts, the authors conducted thematic analysis.

RESULTS

Sixteen faculty participated. Their perceptions of their experience with the first year of an antioppression curriculum initiative were characterized by 3 broad themes: (1) impetus for change, (2) personal experience with antioppressive curricular topics, and (3) strategies necessary to accomplish the change. Faculty described 3 driving forces for the shift toward antioppressive curricula: moral imperative, response to national and local events, and evolving culture of medicine. Despite broad alignment with the change, faculty expressed uncertainties on 3 subthemes: uncertainty about what is an antioppressive curriculum, the scientific perspective, and fear. Faculty also reflected on primary facilitators and barriers to accomplishing the change.

CONCLUSIONS

The shift to an antioppressive curriculum compels faculty to increase their knowledge and skills and adopt a critical, self-reflective lens on the interplay of medicine and oppression. This study's findings can inform faculty development efforts and highlight curricular leadership and resources needed to support faculty through this type of curricular change.

The Role of Faculty Development in Advancing Change at the Organizational Level

ABSTRACT

While the traditional goal of faculty development (FD) has been to enhance individual growth and development, this goal may no longer suffice to address the compelling challenges faculty members are facing, such as increasing workloads, emotional well-being, and institutional support for education. Addressing these challenges will require change at the organizational level. The purpose of this perspective is to articulate a vision for FD programming that describes ways in which FD leaders, together with other educational leaders, can bring about change at the organizational level to support excellence and innovation in health professions education. To impact the organization at large, the authors propose a model that includes four major goals: (1) promoting individual and group development, through educational and leadership development programs, coaching and mentoring, and advanced degrees; (2) advocating for infrastructure and resources, including academies of medical educators, educational scholarship units, educational awards, and intramural funding for educational innovation and scholarship; (3) influencing policies and procedures, by engaging educators on key committees, reviewing appointment and promotion criteria, defining educator roles and portfolios, and valuing diversity, equity and inclusion; and (4) contributing to organization-wide initiatives, such as addressing "hot button" issues, identifying value factors that support investments in FD and medical education, and enhancing the visibility of educators. In this model, the four goals are dynamically interconnected and can impact the culture of the organization. For each goal, the authors offer evidence-informed actions that FD leaders, along with other educational leaders, can adopt to improve the organizational culture and inspire institutionally relevant actions. Since each institution is unique, the options are illustrative and not prescriptive. The intent is to provide examples of how FD leaders and programs can enhance the educational mission through broader engagement with their institutions.

Assessing supervisor versus trainee viewpoints of entrustment through cognitive and affective lenses: an artificial intelligence investigation of bias in feedback

The entrustment framework redirects assessment from considering only trainees' competence to decision-making about their readiness to perform clinical tasks independently. Since trainees and supervisors both contribute to entrustment decisions, we examined the cognitive and affective factors that underly their negotiation of trust, and whether trainee demographic characteristics may bias them. Using a document analysis approach, we adapted large language models (LLMs) to examine feedback dialogs (N = 24,187, each with an associated entrustment rating) between medical student trainees and their clinical supervisors. We compared how trainees and supervisors differentially documented feedback dialogs about similar tasks by identifying qualitative themes and quantitatively assessing their correlation with entrustment ratings. Supervisors' themes predominantly reflected skills related to patient presentations, while trainees' themes were broader-including clinical performance and personal qualities. To examine affect, we trained an LLM to measure feedback sentiment. On average, trainees used more negative language (5.3% lower probability of positive sentiment, p < 0.05) compared to supervisors, while documenting higher entrustment ratings (+ 0.08 on a 1-4 scale, p < 0.05). We also found biases tied to demographic characteristics: trainees' documentation reflected more positive sentiment in the case of male trainees (+ 1.3%, p < 0.05) and of trainees underrepresented in medicine (UIM) (+ 1.3%, p < 0.05). Entrustment ratings did not appear to reflect these biases, neither when documented by trainee nor supervisor. As such, bias appeared to influence the emotive language trainees used to document entrustment more than the degree of entrustment they experienced. Mitigating these biases is nonetheless important because they may affect trainees' assimilation into their roles and formation of trusting relationships.

Student challenges during third-year perioperative clerkships through the lens of faculty and residents: A qualitative study

PURPOSE

Studies of medical students suggest they often find the transition from the pre-clinical curriculum to clinical rotations particularly challenging during perioperative clerkships. Educators could add a new perspective into students' clerkship experiences and potential interventions to improve them. The purpose of this study was to examine the educator perspective on students' experiences in perioperative clerkships. The findings could inform potential curricular interventions to facilitate student transition from a didactic environment into perioperative clerkships.

METHODS

Semi-structured qualitative interviews were conducted with 16 faculty and residents in the departments of anaesthesia, obstetrics and gynaecology (OBGYN), and general surgery across multiple clinical teaching sites at one institution. Interview questions explored their perceptions of the challenges students face during their transition into perioperative clerkships and probed thoughts on curriculum interventions they believed would be the most beneficial. Interviews were recorded, transcribed and analysed thematically.

FINDINGS

Three themes were identified. Faculty and residents perceive that student experiences on perioperative clerkships are shaped by (1) students' ability to adapt to the specialty and operating room norms on these clerkships, (2) students' understanding of how they can meaningfully contribute to the clinical team, and (3) dedicated teaching time constraints. Interventions were suggested to address educator expectations and student gaps, such as implementing a pre-clerkship orientation across anaesthesia, general surgery and OBGYN.

CONCLUSIONS

To facilitate the medical student transition to perioperative clerkships, interventions should aid students in adapting to clerkship norms for these specialties and clarifying their role and expectations within the care team.

Aligning structures with values to sustain health professions education research

This commentary highlights the role of leaders for sustaining health professions education research; in doing so, it advocates for identifying explicit values, derived through dialogue with stakeholders, and aligning investments with those values.

Home practice for robotic surgery: a randomized controlled trial of a low-cost simulation model

Pre-operative simulated practice allows trainees to learn robotic surgery outside the operating room without risking patient safety. While simulation practice has shown efficacy, simulators are expensive and frequently inaccessible. Cruff (J Surg Educ 78(2): 379-381, 2021) described a low-cost simulation model to learn hand movements for robotic surgery. Our study evaluates whether practice with low-cost home simulation models can improve trainee performance on robotic surgery simulators. Home simulation kits were adapted from those described by Cruff (J Surg Educ 78(2): 379-381, 2021). Hand controllers were modified to mimic the master tool manipulators (MTMs) on the da Vinci Skills Simulator (dVSS). Medical students completed two da Vinci exercises: Sea Spikes 1 (SS1) and Big Dipper Needle Driving (BDND). They were subsequently assigned to either receive a home simulation kit or not. Students returned two weeks later and repeated SS1 and BDND. Overall score, economy of motion, time to completion, and penalty subtotal were collected, and analyses of covariance were performed. Semi-structured interviews assessed student perceptions of the robotic simulation experience. Thirty-three medical students entered the study. Twenty-nine completed both sessions. The difference in score improvement between the experimental and control groups was not significant. In interviews, students provided suggestions to increase fidelity and usefulness of low-cost robotic home simulation. Low-cost home simulation models did not improve student performance on dVSS after two weeks of at-home practice. Interview data highlighted areas to focus future simulation efforts. Ongoing work is necessary to develop low-cost solutions to facilitate practice for robotic surgery and foster more inclusive and accessible surgical education.

Investigating feedback orientation in medical learners

BACKGROUND/PURPOSE

Feedback processes in health professions education (HPE) are not always successful. While recommendations to improve feedback provision dominate the literature, studying specific learner attributes that impact feedback uptake may also improve feedback processes. Feedback orientation is a concept from management science involving four dimensions of learner attributes and attitudes that impact their feedback uptake: utility, accountability, social awareness, and feedback self-efficacy. Feedback orientation may represent a valuable concept in HPE. We aimed to understand medical learners' feedback orientation at different stages in their development.

METHODS

We used the Feedback Orientation Scale, a 20-item survey instrument, for a cross-sectional analysis of feedback orientation in medical students and Internal Medicine residents at one large academic center. We performed descriptive statistics and analysis of variance for data analysis.

RESULTS

We found the same factors (dimensions) to feedback orientation in our population as in management science. Overall feedback orientation scores were high and were largely consistent across trainee levels. Utility was the domain that was highest across learners, whereas feedback self-efficacy was lowest.

CONCLUSIONS

Feedback orientation represents a useful concept to explore medical learners' attitudes toward feedback's role in their development. The four domains can help guide further nuanced feedback research and application.[Box: see text].

Impact of an Innovative Endowed Chair Program on Medical Educator Recipients

PURPOSE

Rarely do faculty members receive endowed chairs as recognition for their work as educators. In addition to the title, endowed chairholders have traditionally received discretionary income to pursue value-added work. This study assessed the impact on recipients of receiving an endowed chair for education.

METHOD

The authors conducted a qualitative thematic analysis between 2018 and 2020, interviewing University of California, San Francisco, School of Medicine chairholders who had completed at least one 5-year term. Authors double-coded all transcripts, reconciled codes, applied social cognitive career theory during analysis, and identified themes through an iterative consensus-building approach.

RESULTS

Twenty-three of 24 (96%) eligible faculty members from 16 departments participated. Themes identified were symbolism, resources, education and educator credibility, development, and impact. The chair was a symbol that brought recognition, indicated quality, and amplified visibility and status within the institution and externally. Receiving an endowed chair conferred credibility on recipients and empowered them in the educational domain. The resources allowed chairholders the flexibility to undertake activities that were of value to them, to mentees, and to the organization. Holding the chair facilitated professional development for self and others. Chair recipients reported impact that persisted long after their term(s) concluded. A model of impact emerged, suggesting that simply possessing the chair title led to visibility and gravitas, which, combined with resources, allowed the holder to leverage opportunities in education.

CONCLUSIONS

The endowed chair is an important strategy for career development in education for the chairholder and enhances the position of education institutionally. Having a plan sharpens the focus on activities, results, and impact.

Faculty motivations for leading clinical clerkship electives: A qualitative study

PURPOSE

Faculty are motivated to pursue clinician-educator careers out of a sense of purpose, duty, connectedness, satisfaction, and mastery. Yet, many suffer from burnout due to a lack of funding, resources, and competing clinical demands. Reasons for clinician-educator participation in unfunded educational leadership positions are underexplored. This study examined faculty members' reasons for volunteering and remaining as clerkship elective directors, an unfunded leadership position.

METHODS

In this qualitative study, the authors conducted 17 semi-structured interviews with clerkship elective directors in March 2021. The authors conducted a thematic analysis of deidentified transcripts using motivation theories as a lens.

RESULTS

Directors' motivations to engage in this unfunded educational leadership position stemmed from their existing clinician-educator identity and a sense of purpose and duty. Directors are sustained by the satisfaction derived from witnessing the positive impact they have on learners' career development and skills building, the impact of learners on the clinical environment, as well as personal benefits in the mastery of educator skills and enhanced visibility as educators.

CONCLUSIONS

Unfunded educational leadership positions can advance clinician-educators' commitment to learners and alter the learning environment. Strategies for faculty recruitment and retention in unfunded leadership positions include ensuring meaningful contact with learners, as well as opportunities for personal career development through skills building and enhanced visibility through recognition.

Residents' Experiences of Negative Emotions toward Patients: Challenges to their Identities

PhenomenonMedical learners are more likely than practicing physicians to experience negative emotions toward some patients whom they find challenging, and medical students experience such emotions related to their identity as learners. Little is known about experiences of residents, who are further along in their physician identity formation and have greater autonomy and competence. We explored and characterized how residents understand their experiences of the phenomenon of feeling negative emotions toward patients in relation to their identities as residents. Approach: In 2018, 305 final-year obstetrics and gynecology residents were invited to participate in interviews, which we conducted until reaching theoretical sufficiency. In semi-structured interviews conducted by phone, we probed interactions when residents felt negative emotions toward patients, including reasons for their feelings related to their professional identities, strategies, and curricular desires. The authors coded data and identified patterns using thematic analysis. Findings: Nineteen residents were interviewed by phone. Residents experienced negative emotions toward patients because of challenges to their identities as: physicians - wanting respect and specific unexpected patient behaviors; learners - desiring complete autonomy and experiencing challenges with attending physicians; teachers - wanting to be a role model and protect junior learners; and workers - trying to complete tasks. Among the strategies used to manage feelings toward patients, they struggled with "venting", or complaining about patients, which was not always helpful and residents recognized as perceived negatively by students. They desired curricular support for these interactions such as debriefs and other supported reflection, faculty modeling, and communication skills training. Insights: Like medical students and physicians in practice, residents experience negative emotions toward patients, often because of and made more difficult by their identities as physicians, learners, teachers, and workers. Educators should support residents' reflections about these interactions, model compassionate behavior when feeling challenged by patients, and address unhealthy coping strategies.

"What is the mechanism?": Cues, barriers, and opportunities to discuss foundational science during internal medicine rounds

BACKGROUND

Repeated application of foundational science (FS) during medical reasoning results in encapsulation of knowledge needed to develop clinical expertise. Despite proven benefit of educating learners using a FS framework to anchor clinical decision making, how FS is integrated on clinical rotations has not been well characterized. This study examines how and when FS discussion occurs on internal medicine teaching rounds.

MATERIAL AND METHODS

We performed a convergent mixed method study. Six internal medicine teams at a quaternary hospital were observed during rounds and team members interviewed. Transcripts were analyzed using thematic analysis. Descriptive statistics provided a summary of the observations.

RESULTS

Our study revealed that rounds used a teacher-centered model where FS knowledge was transmitted as pearls external to the clinical context. FS content arose primarily when the patient was complex. Barriers preventing FS discussion were lack of time and perceived lack of personal FS knowledge.

CONCLUSION

Our study describes scenarios that commonly elicit discussion of FS on inpatient medicine rounds highlighting a 'transmission' model of FS knowledge. We suggest a learner-centered model that engages students in the practice of integrating FS into clinical reasoning.

Moving toward Mastery: Changes in Student Perceptions of Clerkship Assessment with Pass/Fail Grading and Enhanced Feedback

ProblemClerkship grades contribute to a summative assessment culture in clerkships and can therefore interfere with students' learning. For example, by focusing on summative, tiered clerkship grades, students often discount accompanying feedback that could inform future learning. This case report seeks to explore whether an assessment system intervention which eliminated tiered grades and enhanced feedback was associated with changes in student perceptions of clerkship assessment and perceptions of the clinical learning environment. Intervention: In January 2019, our institution eliminated tiered clerkship grading (honors/pass/fail) for medical students during the core clerkship year and implemented pass/fail clerkship grading along with required twice weekly, work-based assessments for formative feedback. Context: In this single institution, cross-sectional survey study, we collected data from fourth-year medical students one year after an assessment system intervention. The intervention entailed changing from honors/pass/fail to pass/fail grading in all eight core clerkships and implementing an electronic system to record twice-weekly real-time formative work-based assessments. The survey queried student perceptions on the fairness and accuracy of grading and the clinical learning environment-including whether clerkships were mastery- or performance-oriented. We compared responses from students one year after the assessment intervention to those from the class one year before the intervention. Comparisons were made using unpaired, two-tailed t-tests or chi-squared tests as appropriate with Cohen's d for effect size estimation for score differences. Content analysis was used to analyze responses from two open-ended questions about feedback and grading. Impact: Survey response rates were similar before and after intervention (76% (127/168) vs. 72% (118/163), respectively) with no between-group differences in demographics. The after-intervention group showed statistically significant increases in the following factors: "grades are transparent and fair" (Cohen's d = 0.80), "students receive useful feedback" (d = 0.51), and "resident evaluation procedures are fair" (d = 0.40). After-intervention respondents perceived the clerkship learning environment to be more mastery-oriented (d = 0.52), less performance approach-oriented (d = 0.63), and less performance avoid-oriented (d = 0.49). There were no statistical differences in the factors "attending evaluation procedures are fair," "evaluations are accurate," "evaluations are biased," or "perception of stereotype threat." Open-ended questions revealed student recommendations to improve clerkship summary narratives, burden of work-based assessment, and in-person feedback. Lessons Learned: After an assessment system change to pass/fail grading with work-based assessments, we observed moderate to large improvements in student perceptions of clerkship grading and the mastery orientation of the learning environment. Our intervention did not improve perceptions around bias in assessment in clerkships. Other medical schools may consider similar interventions to begin to address student concerns with clerkship assessment and promote a more adaptive learning environment.

How Do Clinical Electives during the Clerkship Year Influence Career Exploration? A Qualitative Study

Problem: Although many students begin medical school with some idea of their specialty interest, up to 80% of these students choose a different specialty by their final year. This pivot tends to happen in the clerkship year, when students are immersed in the clinical environment, gaining a practical understanding of the day-to-day work in different fields. Yet, in this year students have limited experiences with specialties. Clinical electives during the clerkship year may aid students in career development. The authors examined student career exploration through the lens of social cognitive career theory (SCCT). SCCT posits three variables that influence career development: personal goals, self-efficacy, and understanding outcome expectations. With this framework, the authors sought to understand how a program of clinical electives during the clerkship year influences students' perceptions of their career exploration. We aimed to: (1) describe an innovative clerkship elective program designed for career exploration, and (2) explore how this influenced students' career exploration using qualitative analysis. Intervention: Beginning in 2018, students at our institution were required to participate in three 2-week clinical electives during their clerkship year, called Clinical Immersive Experiences (CIExes). CIExes were categorized into four different types: apprenticeship, clinical skills building, integrative (multi-disciplinary), or subspecialty. Authors invited fourth year students to participate in interviews (January to March 2019) about how they selected electives and how these electives contributed to their career exploration. Interviews continued until reaching information sufficiency. Authors coded and analyzed transcripts using template analysis. Context: This curricular intervention took place in the context of large-scale curricular redesign. Students began clerkships partway into their second year of medical school. The family and community medicine clerkship, which was previously a 6-week core clerkship, was changed to a longitudinal format, thus freeing up 6 weeks for electives. Other core clerkships included anesthesia (2 weeks), medicine (8 weeks), neurology (4 weeks), obstetrics and gynecology (6 weeks), pediatrics (6 weeks), psychiatry (4 weeks), and surgery (8 weeks). Impact: From 15 student interviews, we identified three major themes. First, CIExes facilitated personalized career exploration. All students felt that at least one elective helped them solidify their decision about a specialty choice. Second, CIExes promoted focused learning and skills development that complemented core rotations. They noted the benefit of positive relationships with supervisors, particularly attendings, during these electives. Third, students highlighted how these electives fostered a positive learning environment and enhanced wellbeing. SCCT clarified how the CIEx program helped students advance their personal goals, self-efficacy, and outcome expectations during a pivotal time in medical school. Lessons Learned: We learned that from the student perspective, the inclusion of clinical electives in the clerkship year benefited students' career exploration by helping them develop and refine their career goals, increase self-efficacy, and test outcome expectations in a meaningful way as anticipated from SCCT. In addition, we found that CIExes created a positive learning environment that allowed deep relationships to develop in fields of interest and that supported a strong sense of wellbeing. Supplemental data for this article is available online at https://doi.org/10.1080/10401334.2021.1891545.

Exploring how feedback reflects entrustment decisions using artificial intelligence

CONTEXT

Clinical supervisors make judgements about how much to trust learners with critical activities in patient care. Such decisions mediate trainees' opportunities for learning and competency development and thus are a critical component of education. As educators apply entrustment frameworks to assessment, it is important to determine how narrative feedback reflecting entrustment may also address learners' educational needs.

METHODS

In this study, we used artificial intelligence (AI) and natural language processing (NLP) to identify characteristics of feedback tied to supervisors' entrustment decisions during direct observation encounters of clerkship medical students (3328 unique observations). Supervisors conducted observations of students and collaborated with them to complete an entrustment-based assessment in which they documented narrative feedback and assigned an entrustment rating. We trained a deep neural network (DNN) to predict entrustment levels from the narrative data and developed an explainable AI protocol to uncover the latent thematic features the DNN used to make its prediction.

RESULTS

We found that entrustment levels were associated with level of detail (specific steps for performing clinical tasks), feedback type (constructive versus reinforcing) and task type (procedural versus cognitive). In justifying both high and low levels of entrustment, supervisors detailed concrete steps that trainees performed (or did not yet perform) competently.

CONCLUSIONS

Framing our results in the factors previously identified as influencing entrustment, we find a focus on performance details related to trainees' clinical competency as opposed to nonspecific feedback on trainee qualities. The entrustment framework reflected in feedback appeared to guide specific goal-setting, combined with details necessary to reach those goals. Our NLP methodology can also serve as a starting point for future work on entrustment and feedback as similar assessment datasets accumulate.

Does being a coach benefit clinician-educators? A mixed methods study of faculty self-efficacy, job satisfaction and burnout

INTRODUCTION

Coaching is a growing clinician-educator role. Self-efficacy is a powerful faculty motivator that is associated positively with job satisfaction and negatively with burnout. This study examines self-efficacy, job satisfaction, and burnout in coaches and other clinician-educators.

METHODS

We conducted a mixed methods study using a quantitative survey followed by qualitative interviews of faculty at the University of California, San Francisco. Coaches (funded 20% full-time equivalents), faculty with other funded education positions ("funded"), and faculty without funded education positions ("unfunded") completed a 48-item survey addressing self-efficacy (teaching, professional development, and scholarship), job satisfaction, and burnout. Data were analyzed using analysis of variance followed by post-hoc tests and chi-square tests. To elaborate quantitative results, we conducted qualitative interviews of 15 faculty and analyzed data using framework analysis.

RESULTS

202 of 384 faculty (52.6%) responded to the survey; 187 complete surveys were analyzed. Teaching self-efficacy was similar across groups. Coaches and funded educators had significantly higher professional development self-efficacy and job satisfaction than unfunded educators. Burnout was more prevalent in coaches and unfunded educators. Qualitative analysis yielded three themes: sources of reward, academic identity, and strategies to mitigate burnout. Educator roles provide reward that enhances self-efficacy and job satisfaction but also generate competing demands. Coaches cited challenges in forming professional identities and working with struggling learners.

DISCUSSION

The coaching role provides faculty with benefits similar to other funded educator roles, but the particular demands of the coach role may contribute to burnout.

The Cognitive Load of Inpatient Consults: Development of the Consult Cognitive Load Instrument and Initial Validity Evidence

PURPOSE

Fellows and residents provide inpatient consultations. Though consults vary considerably, measuring the associated cognitive load (CL) is key to guiding faculty on how to optimize learning during consults. However, existing CL instruments, such as the unidimensional Paas scale, cannot separate the 3 components of CL and may miss the nuances of consult CL. Therefore, the authors developed the Consult Cognitive Load (CCL) instrument to measure the 3 CL components during consults.

METHOD

In 2018-2019, the authors developed the CCL at the University of California, San Francisco, using Wilson's constructive approach to measurement. To generate content and response process validity evidence, the authors consulted the literature and experts to generate construct maps, items, and a scoring rubric and conducted cognitive interviews. They administered the CCL to internal medicine and psychiatry trainees across 5 University of California campuses and used Rasch family and linear regression models to assess internal structure validity and relationships to key predictor variables. They compared the CCL with the Paas scale using Wright maps and used latent correlations to support separating CL into 3 components.

RESULTS

Analysis revealed appropriate fit statistics, appropriate mean respondent location increases across all levels, threshold banding, and expected relationships with key predictor variables. The CCL provided more coverage of the 3 CL components compared with the Paas scale. Correlations among the 3 CL components were not strong, suggesting that the CCL offers more nuance than a unidimensional measure of CL in the context of consults.

CONCLUSIONS

This study generated initial validity evidence to support the CCL's use as a measure of consult CL and supports measuring the 3 CL components separately rather than as a single construct in the context of consults. Learners and faculty could compare learner CCL scores with reference scores to promote reflection, metacognition, and coaching.

Emotion in remediation: A scoping review of the medical education literature

OBJECTIVES

Remediation can be crucial and high stakes for medical learners, and experts agree it is often not optimally conducted. Research from other fields indicates that explicit incorporation of emotion improves education because of emotion's documented impacts on learning. Because this could present an important opportunity for improving remediation, we aimed to investigate how the literature on remediation interventions in medical education discusses emotion.

METHODS

The authors used Arksey and O'Malley's framework to conduct a scoping literature review of records describing remediation interventions in medical education, using PubMed, CINAHL Complete, ERIC, Web of Science and APA PsycInfo databases, including all English-language publications through 1 May 2020 meeting search criteria. They included publications discussing remediation interventions either empirically or theoretically, pertaining to physicians or physician trainees of any level. Two independent reviewers used a standardised data extraction form to report descriptive information; they reviewed included records for the presence of mentions of emotion, described the mentions and analysed results thematically.

RESULTS

Of 1644 records, 199 met inclusion criteria and were reviewed in full. Of those, 112 (56%) mentioned emotion in some way; others focused solely on cognitive aspects of remediation. The mentions of emotion fell into three themes based on when the emotion was cited as present: during regular coursework or practice, upon referral for remediation and during remediation. One-quarter of records (50) indicated potential intentional incorporation of emotion into remediation programme design, but they were non-specific as to how emotions related to the learning process itself.

CONCLUSION

Even though emotion is omnipresent in remediation, medical educators frequently do not factor emotion into the design of remediation approaches and rarely explicitly utilise emotion to improve the learning process. Applications from other fields may help medical educators leverage emotion to improve learning in remediation, including strategies to frame and design remediation.

Exploring the relationship between emotion and cognitive load types during patient handovers

Cognitive Load Theory has emerged as an important approach to improving instruction in the health professions workplace, including patient handovers. At the same time, there is growing recognition that emotion influences learning through numerous cognitive processes including motivation, attention, working memory, and long-term memory. This study explores how emotion influences the cognitive load experienced by trainees performing patient handovers. From January to March 2019, 693 (38.7%) of 1807 residents and fellows from a 24-hospital health system in New York city completed a survey after performing a handover. Participants rated their emotional state and cognitive load. The survey included questions about features of the learner, task, and instructional environment. The authors used factor analysis to identify the core dimensions of emotion. Regression analyses explored the relationship between the emotion factors and cognitive load types. Two emotion dimensions were identified representing invigoration and tranquility. In regression analyses, higher levels of invigoration, tranquility, and their interaction were independently associated with lower intrinsic load and extraneous load. The interaction of invigoration and tranquility predicted lower germane load. The addition of the emotion variables to multivariate models including other predictors of cognitive load types significantly increased the amount of variance explained. The study provides a model for measuring emotions in workplace learning. Because emotion appears to have a significant influence on cognitive load types, instructional designers should consider strategies that help trainees regulate emotion in order to reduce cognitive load and improve learning and performance.

The Cognitive Load of Inpatient Consults: A Convergent Parallel Mixed Methods Study Using the Consult Cognitive Load Instrument

PURPOSE

Consultation is crucial for patient care and a primary responsibility of fellows. Understanding the cognitive load associated with the complex skill of consultation would enhance fellow learning. The authors aimed to determine themes describing the fellow experience during inpatient consults specifically, align these themes with Consult Cognitive Load (CCL) scores, and identify strategies to manage cognitive load to enhance fellow education and performance, and, consequently, patient care.

METHOD

The authors studied 16 fellows using mixed methods. Fellows who accepted an invitation completed an inpatient consult followed by the CCL, a measure of cognitive load during consults, and an interview. Three authors conducted a thematic analysis. Member checks and triangulation supported theme trustworthiness. Subsequently, 3 authors rated the extent and cognitive demand of each theme expressed in each transcript. The authors measured interrater reliability and used Spearman correlation to describe the association of these ratings with CCL scores. The authors examined themes to identify strategies that educators might use.

RESULTS

Analysis revealed 4 themes: "nature and scope," which conceptually aligned with intrinsic load (IL); "leveraging resources," which had elements of both IL and extraneous load (EL); "extraneous factors," which aligned with EL; and "drivers," which aligned with germane load (GL). Interrater reliability for extent and demand ratings ranged from 0.57 to 0.79. The correlation between "nature and scope" and IL was 0.37, "extraneous factors" and EL 0.71, and "drivers" and GL 0.32. "Leveraging resources" did not correlate with IL (0.06) or EL (-0.09). Potential strategies based on themes included offering level-appropriate assistance to match IL, focusing the fellow's attention to reduce EL, and providing succinct teaching to promote GL.

CONCLUSIONS

This study provided deep insight into the fellow consult experience and suggested strategies for educators to guide consult learning. The theme "leveraging resources" merits further exploration.

Near-Peer Learning During the Surgical Clerkship: A Way to Facilitate Learning After a 15-Month Preclinical Curriculum

OBJECTIVES

To investigate the performance and perspectives of third-year medical students (MS3s) participating in near-peer learning (NPL) sessions during their core surgical clerkship following a 15-month preclerkship curriculum.

DESIGN

An evaluation study of 7 NPL sessions developed and implemented by fourth-year medical students (MS4s) held from March 2019 to February 2020. MS4s taught 1-2 sessions per rotation that included test taking strategies, illness script development, radiology review, case-based multiple-choice questions, and rapid review. Participants completed a questionnaire with 11 seven-point Likert and open-ended questions after each session. Analyses included quantitative comparison of shelf score averages between NPL participants and nonparticipants and qualitative content analysis for open-ended questions.

SETTING

Surgical clerkship at the University of California, San Francisco.

PARTICIPANTS

Forty-eight (32%) MS3s participated, with an average attendance of 10 students per rotation. Thirty-three (69%) participants completed the questionnaire.

RESULTS

MS3s enjoyed the session (6.9 ± 0.4), improved their knowledge (6.8 ± 0.5), and felt more prepared for the surgery shelf examination (6.5 ± 0.6). MS4 leaders found that MS3s always wanted radiology review, and their interest in test taking strategies and illness script development declined across the clerkship year. Participants had lower shelf exam scores compared to nonparticipants (68.1 vs 71.4, respectively; p = 0.04, ES = 0.03). Shelf exam scores increased over time in both cohorts. Each group had 2 shelf exam failures. Qualitative analysis suggests that MS3s appreciated the NPL's tailored approach and exam demystification, with a desire for increased NPL integration into the clerkship.

CONCLUSION

Students participating in NPL were satisfied with the sessions. Participants may have been students who struggled as indicated by shelf exam scores and appreciated the support. The shift in preferred topics across the blocks reflects the students' development during clerkships. Near-peer teachers should adjust sessions over time to fit students' evolving needs.

Evidence for validity for the Cognitive Load Inventory for Handoffs

CONTEXT

Patient handovers remain a significant patient safety challenge. Cognitive load theory (CLT) can be used to identify the cognitive mechanisms for handover errors. The ability to measure cognitive load types during handovers could drive the development of more effective curricula and protocols. No such measure currently exists.

METHODS

The authors developed the Cognitive Load Inventory for Handoffs (CLIH) using a multi-step process, including expert interviews to enhance content validity and talk-alouds to optimise response process validity. The final version contained 28 items. From January to March 2019, we administered a cross-sectional survey to 1807 residents and fellows from a large health care system in the USA. Participants completed the CLIH following a handover. Exploratory factor analysis of data from one-third of respondents identified high-performing items; confirmatory factor analysis of data from the remaining sample assessed model fit. Model fit was evaluated using the comparative fit index (CFI) (>0.90), Tucker-Lewis index (TFI) (>0.80), standardised root mean square residual (SRMR) (<0.08) and root mean square of error of approximation (RMSEA) (<0.08).

RESULTS

Participants included 693 trainees (38.4%) (231 in the exploratory study and 462 in the confirmatory study). Eleven items were removed during exploratory factor analysis. Confirmatory factor analysis of the 16 remaining items (five for intrinsic load, seven for extraneous load and four for germane load) supported a three-factor model and met criteria for good model fit: the CFI was 0.95, TFI was 0.93, RMSEA was 0.074 and SRMR was 0.07. The factor structure was comparable for gender and role. Intrinsic, extraneous and germane load scales had high internal consistency. With one exception, scale scores were associated, as hypothesised, with postgraduate level and clinical setting.

CONCLUSIONS

The CLIH measures three types of cognitive load during patient handovers. Evidencefor validity is provided for the CLIH's content, response process, internal structure and association with other variables. This instrument can be used to determine the relative drivers of cognitive load during handovers in order to optimize handover instruction and protocols.

"There Is a Lot of Change Afoot": A Qualitative Study of Faculty Adaptation to Elimination of Tiered Grades With Increased Emphasis on Feedback in Core Clerkships

PURPOSE

Medical schools responding to challenges with fairness, equity, learning environments, and student wellness have reconsidered clerkship grades. This study used the Concerns-Based Adoption Model (CBAM) to explore how faculty responded to a change in the approach to assessment from focus on grades toward focus on feedback.

METHOD

This qualitative study used an inductive approach to analyze data from semistructured interviews with teaching faculty and education leaders at University of California, San Francisco, School of Medicine 6 months following the elimination of traditional tiered clerkship grades. Participants were recruited in 2019 using purposive sampling. Interview questions addressed participants' perceptions of the new approach to assessment and grading that emphasizes assessment for learning, including advantages and concerns. The authors analyzed data using thematic analysis informed by sensitizing concepts from CBAM.

RESULTS

Nineteen faculty participated (11 medicine, 8 surgery). Faculty expressed optimism about the impact of the change on their clinical educator roles and sought faculty development to enhance feedback and assessment skills. Perceived benefits to students addressed learning and wellness, with concern for students' motivation and professional development. Faculty shared uncertainty about their roles and larger systemic consequences involving the residency match and institutional reputation. Participants acknowledged that traditional grading is imperfect, yet felt a departure from this system represented significant culture change.

CONCLUSIONS

Faculty perceptions of the assessment and grade change in 2 large departments demonstrated tensions around grading. Their descriptions of change as an ongoing process aligned with CBAM and required them to consider new approaches and perspectives. While discourse about assessment and grading frequently focuses on learners, these findings support institutional consideration of the impact of changes in assessment on teaching faculty.

Educator Identity Formation: A Faculty Development Workshop

INTRODUCTION

Health professions faculty members often struggle deciding on career paths balancing their identities as clinicians, educators, and scholars. Identity formation research has identified three major influences: context, roles, and agency. Identity influences career decisions and, in turn, affects engagement in medical education and faculty development. We designed a single-session workshop to foster educator identity formation.

METHODS

The workshop varied from 1 to 3 hours. It explored how identity develops and considered how self, role, and context could be shaped to grow and sustain identity. Participants used a handout called Identity Quakes to indicate satisfaction with their support, engagement, and empowerment. The workshop employed direct instruction to provide language and tools to scaffold conversation and self-reflection leading to future plans aligning participants' professional identities and roles with resources to support further development.

RESULTS

From 2016 to 2020, we offered the workshop to faculty members from diverse professions, including medicine, nursing, dentistry, physical therapy, and others, on 11 occasions (locally, nationally, and internationally) with audiences of 15-200 participants. At offerings that collected evaluations, the workshop received high ratings of 4.61-4.90 (very good-excellent) on a 5-point scale.

DISCUSSION

This single-session workshop is a valuable opportunity to reflect on identity, which faculty members rarely get to do formally. The Identity Quakes handout prompts participants to challenge their assumptions about their professional identities and roles, employ their agency/choice, and consider future career choices.

How do attending physicians describe cognitive overload among their workplace learners?

OBJECTIVES

Cognitive load theory (CLT) focuses on the limited bandwidth of working memory. Core to CLT is the concept of cognitive overload, which occurs when working memory demands exceed working memory capacity, and learning and performance suffer. Within health professions education (HPE), workplace learning settings are very complex, placing learners at high risk of cognitive overload. Although continuous monitoring of physiologic parameters can indicate states of high cognitive load, how to practically identify cognitively overloaded learners within everyday workplace settings is not well understood. We sought to characterise how attending physicians described their perceiving of cognitive overload among learners in two different workplace settings: the gastrointestinal endoscopy suite and the intensive care unit.

METHODS

We performed a secondary qualitative analysis of transcripts of interviews with workplace teachers that had been carried out during two previous studies. These studies had addressed different objectives but both were informed by CLT. Each included questions that prompted participants to reflect on how they perceived cognitive overload to manifest among learners in the workplace. To investigate the phenomenon of cognitive overload, we developed a new codebook and performed content analysis.

RESULTS

We analysed 42 interview transcripts (22 endoscopists, 12 hospitalists, eight intensivists). Participants described four behaviours they had witnessed among learners they thought were cognitively overloaded: poor performance on workplace tasks; non-verbal physical manifestations (including posture, eye and body movements and autonomic functions); verbal utterances (words and sounds), and interpersonal interactions with team members. Endoscopists often described individually oriented examples, whereas intensivists and hospitalists tended to frame examples within an interpersonal context.

CONCLUSIONS

We identified four overarching ways in which HPE workplace teachers perceived learners as appearing to be cognitively overloaded. Workplace teachers and learners should be mindful of and watch for these signs, which may signal states of cognitive overload. Earlier recognition of cognitive overload may facilitate timely action to reduce cognitive overload and promote learning.

How Medical Educators Can Foster Equity and Inclusion in Their Teaching: A Faculty Development Workshop Series

PROBLEM

Diversity, equity, and inclusion are increasingly highlighted in medical education, but bias continues to disproportionately impact learners from racial and ethnic groups traditionally underrepresented in medicine. Many faculty struggle to engage with these concepts in their teaching and fear making mistakes, but few opportunities exist for faculty to develop their skills.

APPROACH

To advance the goal of fostering an equitable educational environment, the authors envisioned a series of faculty development workshops to cultivate faculty skills and comfort with incorporating equity and inclusion in their teaching, assessment, and curriculum development work. The authors conducted a needs assessment and followed Kern's 6-step process for curriculum development to build the workshop series.

OUTCOMES

Using local resources and expertise, the authors built a workshop series that culminated in a certificate in Teaching for Equity and Inclusion. The development process took 24 months; 22 faculty volunteered to design and teach 7 new workshops. Initial workshop offerings have reached 101 participants, and 120 faculty members have enrolled in the certificate. The workshops have been well received and are rated on par with or more highly than other faculty development offerings at the University of California, San Francisco.

NEXT STEPS

Future directions include assessing the impact of the certificate on participants and workshop faculty, creating opportunities for workshop faculty to share best practices, and exploring alternative models for participation. The authors' experience demonstrates the feasibility of creating a self-sustaining program that will advance faculty competence in the critical areas of diversity, equity, and inclusion.

Faculty and Resident Engagement With a Workplace-Based Assessment Tool: Use of Implementation Science to Explore Enablers and Barriers

PURPOSE

Implementation of workplace-based assessment programs has encountered significant challenges. Faculty and residents alike often have a negative view of these programs as "tick-box" or "jump through the hoops" exercises. A number of recommendations have been made to address these challenges. To understand the experience with a workplace-based assessment tool that follows many of these recommendations, the authors conducted a qualitative study using the Consolidated Framework for Implementation Research (CFIR) to identify enablers and barriers to engagement with the tool.

METHOD

The Psychopharmacotherapy-Structured Clinical Observation (P-SCO) is a direct observation tool designed to assess resident performance during a psychiatric medication management visit. From August 2017 to February 2018, the P-SCO was implemented in the outpatient continuity clinics for second- and third-year residents at Zucker Hillside Hospital/Northwell Health. In February and March 2019, the authors conducted semistructured interviews of participating faculty and residents. Interview guides based on the CFIR were used to capture the enablers and barriers to engagement. Interview transcripts were independently coded. Codes were then organized into themes relevant to the domains of the CFIR.

RESULTS

Ten faculty and 10 residents were interviewed. Overall, participants had a positive experience with the P-SCO. Enabling factors for faculty and residents included the ongoing training, design features of the P-SCO, predisposing beliefs, dedicated faculty time, and the perception that the P-SCO improved verbal feedback quality. Barriers for faculty included checklist length and discomfort with feedback that threatens identity, and barriers for residents included faculty variability in timeliness and quality of feedback and minimal review of the feedback after initial receipt.

CONCLUSIONS

This study demonstrates that the negative experience of faculty and residents with workplace-based assessment tools shown in prior studies can be overcome, at least in part, when specific implementation strategies are pursued. The findings provide guidance for future research and implementation efforts.

Using Cognitive Load Theory to Improve Teaching in the Clinical Workplace

INTRODUCTION

Cognitive load theory (CLT) views working memory as the primary bottleneck for learning, as it is limited in both capacity and retention. CLT delineates three types of activities that impose on working memory: intrinsic load, germane load, and extraneous load. These three constructs have practical ramifications for direct teaching, learning environments, and curricular design. CLT could help educators across health professions improve quality of teaching, especially in demanding and unpredictable workplace environments. However, few educational resources exist to familiarize clinical workplace educators with CLT.

METHODS

We developed a 2-hour workshop focused on CLT's core concepts and practical applications, targeted at health professions' workplace educators. It featured large-group, small-group, and individual reflective activities. An end-of-workshop survey was administered, and a follow-up survey was sent to participants 2 months after the workshop.

RESULTS

A total of 134 educators attended the first two offerings of the workshop in two different states. Participants considered CLT as relevant to a variety of workplace teaching settings and activities. Participants' self-assessed familiarity with CLT on a 0-100 scale increased from a mean of 36 (SD = 26) before the workshop to 59 (SD = 17) after the workshop. At follow-up, participants scored an average of 85% on content knowledge questions. Approximately half of respondents to the follow-up survey stated they had made or planned to make specific changes to their workplace teaching leveraging tenets of CLT.

DISCUSSION

The workshop conveyed CLT concepts and primed participants to independently craft CLT-based interventions for their own teaching practices.

Exploring Residents' Experience of Career Development Scholarship Tracks: A Qualitative Case Study Using Social Cognitive Career Theory

Problem: Trainees enter graduate medical education with professional aspirations that often extend beyond the role of clinician to roles such as educator, innovator, leader, advocate, or researcher. Many residency programs have implemented academic tracks to support career development in these areas. With the exception of research tracks, these tracks generally do not include significant longitudinal protected time and often rely upon 'extra-curricular' effort and possess insufficient structure, mentorship, and accountability. Most prior studies of non-research scholarship tracks have not been theory driven and do not explore in depth the experience of residents who participate. Approach: To address this gap in the literature, we conducted a qualitative case study informed by Social Cognitive Career Theory to explore the professional identity development of residents who participated in a non-research scholarship track that incorporates recommended best practices. The track, Pathways to Expertise Program, incorporates features of successful research tracks: protected time, longitudinal experience, mentorship, platforms for recognition, and accountability. Participants from the first three cohorts were interviewed at the time of their graduation (2017-2019). Semi-structured interviews were conducted, transcribed, and independently coded. Social Cognitive Career Theory informed the organization of codes into themes. Context: The Pathways to Expertise Program was implemented in a psychiatry residency training program in a large urban academic teaching hospital. Impact: Fifteen residents entered Pathways to Expertise Program during the study period and all 15 participated in the study. Fourteen completed the program and presented their projects at the department grand rounds. For dissemination, 12 presented their project findings at one or more national meetings in the form of a poster (20 distributed across 11 residents), workshop (six distributed across four residents), or presentation (two across two residents). Six residents accounted for a total of seven first author publications in peer reviewed journals. All participants described how their self-efficacy increased as a result of new skills (e.g., content, methodology, and scientific communication), mentorship (e.g., content and process guidance), peer and broader support (e.g., small group supervision), persuasive communications (e.g., recognition both locally and nationally), and positive emotional reactions (e.g., triumph). The residents also described expecting compelling benefits (e.g., stronger application for fellowship and expanded career opportunities). Participants indicated that the experience influenced their career goals and how they perceived their professional identities. Lessons Learned: These findings suggest that a longitudinal academic track that incorporates features of successful research tracks (protected time, mentorship, peer support, and accountability for deliverables) can be instrumental in forming and maturing professional identities for non-clinical roles. These tracks can accomplish several important goals, including enhancing resilience via identity formation around passion and purpose and meeting society's need for physicians who are engaged in inquiry and innovation. Implications for the design of academic tracks in general are explored.

Overcoming the Challenges of Direct Observation and Feedback Programs: A Qualitative Exploration of Resident and Faculty Experiences

Problem: Prior studies have reported significant negative attitudes amongst both faculty and residents toward direct observation and feedback. Numerous contributing factors have been identified, including insufficient time for direct observation and feedback, poorly understood purpose, inadequate training, disbelief in the formative intent, inauthentic resident-patient clinical interactions, undermining of resident autonomy, lack of trust between the faculty-resident dyad, and low-quality feedback information that lacks credibility. Strategies are urgently needed to overcome these challenges and more effectively engage faculty and residents in direct observation and feedback. Otherwise, the primary goals of supporting both formative and summative assessment will not be realized and the viability of competency-based medical education will be threatened. Intervention: Toward this end, recent studies have recommended numerous strategies to overcome these barriers: protected time for direct observation and feedback; ongoing faculty and resident training on goals and bidirectional, co-constructed feedback; repeated direct observations and feedback within a longitudinal resident-supervisor relationship; utilization of assessment tools with evidence for validity; and monitoring for engagement. Given the complexity of the problem, it is likely that bundling multiple strategies together will be necessary to overcome the challenges. The Direct Observation Structured Feedback Program (DOSFP) incorporated many of the recommended features, including protected time for direct observation and feedback within longitudinal faculty-resident relationships. Using a qualitative thematic approach the authors conducted semi-structured interviews, during February and March, 2019, with 10 supervisors and ten residents. Participants were asked to reflect on their experiences. Interview guide questions explored key themes from the literature on direct observation and feedback. Transcripts were anonymized. Two authors independently and iteratively coded the transcripts. Coding was theory-driven and differences were discussed until consensus was reached. The authors then explored the relationships between the codes and used a semantic approach to construct themes. Context: The DOSFP was implemented in a psychiatry continuity clinic for second and third year residents. Impact: Faculty and residents were aligned around the goals. They both perceived the DOSFP as focused on growth rather than judgment even though residents understood that the feedback had both formative and summative purposes. The DOSFP facilitated educational alliances characterized by trust and respect. With repeated practice within a longitudinal relationship, trainees dropped the performance orientation and described their interactions with patients as authentic. Residents generally perceived the feedback as credible, described feedback quality as high, and valued the two-way conversation. However, when receiving feedback with which they did not agree, residents demurred or, at most, would ask a clarifying question, but then internally discounted the feedback. Lessons Learned: Direct observation and structured feedback programs that bundle recent recommendations may overcome many of the challenges identified by previous research. Yet, residents discounted disagreeable feedback, illustrating a significant limitation and the need for other strategies that help residents reconcile conflict between external data and one's self-appraisal.

A mobile app to capture EPA assessment data: Utilizing the consolidated framework for implementation research to identify enablers and barriers to engagement

INTRODUCTION

Mobile apps that utilize the framework of entrustable professional activities (EPAs) to capture and deliver feedback are being implemented. If EPA apps are to be successfully incorporated into programmatic assessment, a better understanding of how they are experienced by the end-users will be necessary. The authors conducted a qualitative study using the Consolidated Framework for Implementation Research (CFIR) to identify enablers and barriers to engagement with an EPA app.

METHODS

Structured interviews of faculty and residents were conducted with an interview guide based on the CFIR. Transcripts were independently coded by two study authors using directed content analysis. Differences were resolved via consensus. The study team then organized codes into themes relevant to the domains of the CFIR.

RESULTS

Eight faculty and 10 residents chose to participate in the study. Both faculty and residents found the app easy to use and effective in facilitating feedback immediately after the observed patient encounter. Faculty appreciated how the EPA app forced brief, distilled feedback. Both faculty and residents expressed positive attitudes and perceived the app as aligned with the department's philosophy. Barriers to engagement included faculty not understanding the EPA framework and scale, competing clinical demands, residents preferring more detailed feedback and both faculty and residents noting that the app's feedback should be complemented by a tool that generates more systematic, nuanced, and comprehensive feedback. Residents rarely if ever returned to the feedback after initial receipt.

DISCUSSION

This study identified key enablers and barriers to engagement with the EPA app. The findings provide guidance for future research and implementation efforts focused on the use of mobile platforms to capture direct observation feedback.

Learning Challenges, Teaching Strategies, and Cognitive Load: Insights From the Experience of Seasoned Endoscopy Teachers

PURPOSE

Learners of medical procedures must develop, refine, and apply schemas for both cognitive and psychomotor constructs, which may strain working memory capacity. Procedures with limitations in visual and tactile information may add risk of cognitive overload. The authors sought to elucidate how experienced procedural teachers perceived learners' challenges and their own teaching strategies in the exemplar setting of gastrointestinal endoscopy.

METHOD

The authors interviewed 22 experienced endoscopy teachers in the United States, Canada, and the Netherlands between May 2016 and March 2019 and performed thematic analysis using template analysis method. Interviews addressed learner challenges and teaching strategies from the teacher participants' perspectives. Cognitive load theory informed data interpretation and analysis.

RESULTS

Participants described taking steps to "diagnose" trainee ability and identify struggling trainees. They described learning challenges related to trainees (performance over mastery goal orientation, low self-efficacy, lack of awareness), tasks (psychomotor challenges, mental model development, tactile understanding), teachers (teacher-trainee relationship, inadequate teaching, teaching variability), and settings (internal/external distractions, systems issues). Participants described employing strategies that could match intrinsic load to learners' levels (teaching along developmental continuum, motor instruction, technical assistance/takeover), minimize extraneous load (optimize environment, systems solutions, emotional support, define expectations), and optimize germane load (promote mastery, teach schemas, stop and focus).

CONCLUSIONS

Participants provided insight into possible challenges while learning complex medical procedures with limitations in sensory channels, as well as teaching strategies that may address these challenges at individual and systems levels. Using cognitive load theory, the authors provide recommendations for procedural teachers.

Enhancing Operative Feedback: A Descriptive Trajectory for Surgical Development in Otolaryngology

OBJECTIVE

Developmental frameworks are narrative descriptions of learner performance that may be useful to provide operative feedback tools in surgical education. The authors previously proposed the Technician, Anatomist, Anticipator, Strategist, Executive (TAASE) framework utilizing data collected from general surgeons and conjectured its applicability in other surgical specialties. Here, we construct a developmental framework in otolaryngology using similar techniques, then compare the results to TAASE.

DESIGN

Ten otolaryngology educators participated in semi-structured, audio-recorded interviews, to explore how otolaryngology faculty characterize surgical learning at different levels of training. Researchers analyzed the transcripts using a thematic analysis approach to build a developmental framework. Results were then qualitatively compared to the TAASE trajectory.

SETTING

Single tertiary academic medical center.

PARTICIPANTS

Faculty within the Department of Otolaryngology-Head and Neck Surgery.

RESULTS

Otolaryngology faculty characterized surgical development as an integrated trajectory of technical and cognitive skills progressing from specific operative tasks to a global understanding of patients and procedures largely consistent with stages outlined in the Technician, Anatomist, Anticipator, Strategist, and Executive model. A new theme identified was learner emotions, which may impact learning and hinder or enhance progression along the trajectory.

CONCLUSIONS

Otolaryngology and general surgeons agree upon a common arc to operative maturity reflected by the TAASE trajectory. The TAASE framework may be useful as a tool for intraoperative assessment in otolaryngology to help promote quality feedback. Further research is needed to understand how emotions may influence operative skill development.

In Pursuit of Honors: A Multi-Institutional Study of Students' Perceptions of Clerkship Evaluation and Grading

PURPOSE

To examine medical students' perceptions of the fairness and accuracy of core clerkship assessment, the clerkship learning environment, and contributors to students' achievement.

METHOD

Fourth-year medical students at 6 institutions completed a survey in 2018 assessing perceptions of the fairness and accuracy of clerkship evaluation and grading, the learning environment including clerkship goal structures (mastery- or performance-oriented), racial/ethnic stereotype threat, and student performance (honors earned). Factor analysis of 5-point Likert items (1 = strongly disagree, 5 = strongly agree) provided scale scores of perceptions. Using multivariable regression, investigators examined predictors of honors earned. Qualitative content analysis of responses to an open-ended question yielded students' recommendations to improve clerkship grading.

RESULTS

Overall response rate was 71.1% (666/937). Students believed that being liked and particular supervisors most influenced final grades. Only 44.4% agreed that grading was fair. Students felt the clerkship learning environment promoted both mastery and performance avoidance behaviors (88.0% and 85.6%, respectively). Students from backgrounds underrepresented in medicine were more likely to experience stereotype threat vulnerability (55.7% vs 10.9%, P < .0005). Honors earned was positively associated with perceived accuracy of grading and interest in competitive specialties while negatively associated with stereotype threat. Students recommended strategies to improve clerkship grading: eliminating honors, training evaluators, and rewarding improvement on clerkships.

CONCLUSIONS

Participants had concerns around the fairness and accuracy of clerkship evaluation and grading and potential bias. Students expressed a need to redefine the culture of assessment on core clerkships to create more favorable learning environments for all students.

Integrating Robotic Technology Into Resident Training: Challenges and Recommendations From the Front Lines

PURPOSE

To develop recommendations for improving the integration of robotic technology into today's apprentice-based resident training.

METHOD

During a national meeting in 2017, 24 robotic surgeons were interviewed about their experiences integrating robotic technology into resident training. Qualitative thematic analysis of interview notes and recordings revealed themes related to challenges and recommendations.

RESULTS

Four themes emerged, each corresponding to a general recommendation for integrating robotic technology into training. The first, surgical techniques versus tools, contrasts faculty's sequential mastery-surgical techniques first, then the robotic tool-with residents' simultaneous learning. The recommendation is to create separate learning opportunities for focused skill acquisition. The second theme, timing of exposure to the robotic tool, describes trainees' initial focus on tool use for basic surgical steps. The recommendation is to increase access to basic robotic cases. The third theme covers the relationship of laparoscopic and robotic surgery. The recommendation is to emphasize similar and dissimilar features during all minimally invasive surgical cases. The fourth theme, use of the dual console (which enables two consoles to operate the robot, the primary determines the secondary's functionality), highlights the unique teaching opportunities this console creates. The recommendation is for surgeons to give verbal guidance so residents completely understand surgical techniques.

CONCLUSIONS

Surgical educators should consider technique versus tool, timing of exposure to the tool, overlapping and varying features of robotic and laparoscopic surgery, and use of the dual console as they develop curricula to ensure thorough acquisition and synthesis of all elements of robotic surgery.

Trainee perception of cognitive load during observed faculty staff teaching of procedural skills

OBJECTIVES

Although teachers impact learners' cognitive load, how specific teaching activities affect intrinsic, germane and extraneous load during procedural skills training is unknown. We sought to characterise teaching activities used in the exemplar procedural setting of colonoscopy, and to explore how they were enacted and how learners perceived them as affecting intrinsic, germane and extraneous cognitive load.

METHODS

We observed 10 colonoscopies performed by eight different gastroenterology fellows and supervised by 10 different attending physicians at two hospitals, and recorded the teaching activities observed, as well as details of when they were used and how they were enacted. After the colonoscopy, each fellow completed the Cognitive Load Inventory for Colonoscopy to quantify intrinsic, germane and extraneous load. We then interviewed each fellow to determine how he or she perceived teaching as affecting cognitive load. Qualitative data were subjected to content analysis. Instances of germane load-promoting activities were correlated with measured germane load.

RESULTS

We observed 515 instances of teaching activities. The intensity of teaching varied substantially, ranging from 0.7 to 3.3 activities per minute, as did the pattern of teaching activities used by different attending physicians. Little teaching occurred immediately before or after a procedure. Fellows usually perceived teaching as affecting cognitive load in ways that promoted learning, particularly by reducing intrinsic load and increasing germane load. Fellows strongly perceived that the provision of autonomy promoted germane load. Conversely, fellows perceived that excessive teaching increased extraneous load. Instances of germane load-promoting teaching activities correlated moderately with measured germane load.

CONCLUSIONS

Teaching in the exemplar procedural setting of colonoscopy affected learners' cognitive load in mostly beneficial ways, yet even 'good' teaching activities had detrimental effects when used excessively. Teachers of procedures should consider learner experience, task complexity and environmental factors to modulate the modality, content and intensity of teaching to promote balanced cognitive load and learning. Teaching more reservedly during the procedure and taking advantage of pre- and post-procedure opportunities may help.

Strengthening Teachers' Professional Identities Through Faculty Development

Although medical schools espouse a commitment to the educational mission, faculty members often struggle to develop and maintain their identities as teachers. Teacher identity is important because it can exert a powerful influence on career choice, academic roles and responsibilities, and professional development opportunities. However, most faculty development initiatives focus on knowledge and skill acquisition rather than the awakening or strengthening of professional identity. The goal of this Perspective is to highlight the importance of faculty members' professional identities as teachers, explore how faculty development programs and activities can support teachers' identities, and describe specific strategies that can be used in professional development. These strategies include the embedding of identity and identity formation into existing offerings by asking questions related to identity, incorporating identity in longitudinal programs, building opportunities for community building and networking, promoting reflection, and capitalizing on mentorship. Stand-alone faculty development activities focusing on teachers' identities can also be helpful, as can a variety of approaches that advocate for organizational change and institutional support. To achieve excellence in teaching and learning, faculty members need to embrace their identities as teachers and be supported in doing so by their institutions and by faculty development.

Cognitive load theory for training health professionals in the workplace: A BEME review of studies among diverse professions: BEME Guide No. 53

AIM

Cognitive load theory (CLT) is of increasing interest to health professions education researchers. CLT has intuitive applicability to workplace settings, yet how CLT should inform teaching, learning, and research in health professions workplaces is unclear.

METHOD

To map the existing literature, we performed a scoping review of studies involving cognitive load, mental effort and/or mental workload in professional workplace settings within and outside of the health professions. We included actual and simulated workplaces and workplace tasks.

RESULT

Searching eight databases, we identified 4571 citations, of which 116 met inclusion criteria. Studies were most often quantitative. Methods to measure cognitive load included psychometric, physiologic, and secondary task approaches. Few covariates of cognitive load or performance were studied. Overall cognitive load and intrinsic load were consistently negatively associated with the level of experience and performance. Studies consistently found distractions and other aspects of workplace environments as contributing to extraneous load. Studies outside the health professions documented similar findings to those within the health professions, supporting relevance of CLT to workplace learning.

CONCLUSION

The authors discuss implications for workplace teaching, curricular design, learning environment, and metacognition. To advance workplace learning, the authors suggest future CLT research should address higher-level questions and integrate other learning frameworks.

Performance Assessment of Pharmacotherapy: Results from a Content Validity Survey of the Psychopharmacotherapy-Structured Clinical Observation (P-SCO) Tool

OBJECTIVE

The Psychopharmacotherapy-Structured Clinical Observation (P-SCO) tool is designed to assess performance of a medication management visit and to enhance feedback. Prior research indicated that the P-SCO was feasible to implement in a resident clinic and generated behaviorally specific, high-quality feedback. This research also highlighted problems with some of the instrument's items. This study seeks to improve the items.

METHODS

The authors initially revised the P-SCO items based on the problems identified by a prior study. Next, these items were iteratively modified by experts in clinical pharmacotherapy and educational assessment. Forty-five items emerged. Finally, faculty attending an annual department education retreat rated each item on its relevance (4-point scale) and provided comments on how the item might be revised. For final inclusion, an item must have met a quantitative threshold (i.e., content validity index equal to or greater than 0.8 and the lower end of the asymmetric confidence interval equal to or greater than 3.0) and received comments that were supportive.

RESULTS

Forty-one of the 45 items had strong quantitative support. However, the comments endorsed lumping a number of items in order to decrease overlap between items and to shorten the instrument. This process resulted in the further elimination of 15 items.

CONCLUSIONS

The revised 26-item P-SCO builds upon prior evidence of feasibility and utility and now possesses additional evidence of content validity. The use of the tool should enhance feedback and improve the capacity of educational programs to assess performance.

Evidence for the Validity of the Psychopharmacotherapy-Structured Clinical Observation Tool: Results of a Factor and Time Series Analysis

OBJECTIVE

The Psychopharmacotherapy-Structured Clinical Observation (P-SCO) is a direct observation tool designed to assess resident performance of a medication visit. This study examines two dimensions of validity for the P-SCO: internal structure and how scores correlate with another variable associated with competence (experience).

METHODS

The faculty completed 601 P-SCOs over 4 years. Multilevel exploratory factor analysis was performed with minimum thresholds for eigenvalue (≥ 1.0) and proportion of variance explained (≥ 5.0%). Internal reliability was assessed with Cronbach alpha. To examine how scores changed with experience, mean ratings (1-4 scale) were calculated for each factor by quarter of the academic year. Separate linear mixed models were also performed.

RESULTS

The analysis yielded three factors that explained 50% of the variance and demonstrated high internal reliability: affective tasks (alpha = 0.90), cognitive tasks (alpha = 0.84), and hard tasks (alpha = 0.74). Items within "hard tasks" were assessment of substance use, violence risk, and adherence, and inquiry about interactions with other providers. Monitoring adverse effects did not load on the hard task factor but also had overall low mean ratings. Compared to the first quarter, fourth quarter scores for affective tasks (b = 0.54, p < 0.01) and hard tasks (b = 0.46, p = 0.02) were significantly improved while cognitive tasks had a non-significant increase. For the hard tasks, the proportion of residents with a low mean rating improved but was still over 30% during the fourth quarter.

CONCLUSIONS

The results provide evidence for the validity of the P-SCO with respect to its internal structure and how scores correlate with experience. Curricular implications are explored, especially for the tasks that were hard to learn.