Coaching by design: exploring a new approach to faculty development in a competency-based medical education curriculum

Implementation and Outcomes of an Academic Peer Coaching Program for Pediatric Residents

Introduction Academic coaching fosters self-directed learning and is growing in popularity within residency programs. Implementation is often limited by available faculty time and funding. Peer coaching is an emerging alternative but is not well studied. This study aims to demonstrate the acceptability, feasibility, and efficacy of a resident peer coaching program. Methodology In the 2021-2022 academic year, within a large pediatric residency program, we selected and trained senior residents as coaches and interns who opted in as coachees. Coaching dyads began meeting in the fall and worked toward individualized goals throughout the year; control interns participated in routine didactics. Outcomes included Accreditation Council for Graduate Medical Education (ACGME) milestone scores and a self-assessment survey (SAS). Results We enrolled 15/42 (36%) interns as coachees, with the remaining 27 (64%) as controls. Narrative feedback from coaches and coachees was overall positive, and time commitment was feasible for program staff (10-12 hours/month), coaches (three to four hours/month), and coachees (one to two hours/month) with minimal financial needs. Post-intervention, more coachees than controls scored ≥4.0 on ACGME milestones systems-based practice 3 (SBP3; 3/15, 20%, vs. 2/27, 7%), SBP4 (4/15, 27%, vs. 5/27, 19%), and practice-based learning and improvement 1 (4/15, 27%, vs. 3/27, 11%). SAS response rate was 8/15 (53%) for coachees and 5/27 (19%) for controls. More coachees than controls reported baseline difficulty with time management often (3/8, 38%, vs. 1/5, 20%); only coachees improved post-intervention, with 0/8 (0%) having difficulty often versus 2/5 (40%) of controls. Conclusions Resident peer coaching is acceptable and feasible to implement. Coachees reported more improvement in time management than controls, and ACGME milestone scores suggest improved use of evidence-based medicine and interprofessional care coordination among coachees.

'We're like Spider-Man; with great power comes great responsibility': Coaches' experiences supporting struggling medical students

BACKGROUND

Medical students can experience a range of academic and non-academic struggles. Coaching is a valuable strategy to support learners, but coaches describe working with struggling learners as taxing. Transformative learning theory (TLT) provides insights into how educators grow from challenging experiences to build resilience. This study explores how coaches evolve as educators through supporting struggling students.

METHODS

This qualitative study grounded in an interpretivist paradigm used interviews of longitudinal medical student coaches at two academic institutions. Interviews, using TLT as a sensitizing concept, explored coaches' experience coaching struggling learners. We performed thematic analysis.

RESULTS

We interviewed 15 coaches. Coaches described supporting students through multi-faceted struggles which often surprised the coach. Three themes characterized coaches' experiences: personal responsibility, emotional response, and personal learning. Coaches shouldered high personal responsibility for learners' success. For some, this burden felt emotional, raised parental instincts and questions about maintaining boundaries with learners. Coaches evolved their coaching approach, challenged biases, and built skills. Coaches learned to better appreciate the learner point of view and employ resources to support students.

DISCUSSION

Through navigating learner struggles, educators can gain self-efficacy, learn to understand learners' perspectives, and evolve their coaching approach to lessen their personal emotional burden through time.

Roles and competencies of medical faculty members: A competency framework

PURPOSE

Medical education requires competent faculty members with the ability to create change in medical education. The focus on teacher competency are emerged as the results of medical education movements toward learner competency. The purpose of this study was to identify medical faculties competencies in their main roles and to provide a competency framework.

METHOD

The integrative review approach was utilized for identifying competencies and expert opinions was conducted to assigned competencies to roles and academic ranks. The search strategies were conducted in online databases including PubMed, Scopus, WOS and Eric to detect studies published from May 2003 to May 2023.

RESULTS

The identified competencies were classified to four themes including (1) essential personal competencies, (2) technical/functional competencies, (3) enabling competencies, and (4) meta-competencies.

CONCLUSION

According to the results, a competency framework was proposed which is a valuable tool for various important decisions related to faculty, such as promotions and tenurship.

Competency-Based Education: The Need to Debunk Misconceptions and Develop a Common Language

Competency-based education (CBE) dates back to the early 1900s. A renewed interest in CBE in health professions education has been sparked by the recommendations that emerged from the 2017 Josiah Macy Jr Foundation conference on CBE. These recommendations urge changes in structure and pedagogy to address gaps between societal needs, health care practice, and health professions education. Despite widespread interest, there is significant variability in what constitutes CBE. This variability has challenged the development of an evidence base in the literature and has led to widespread misconceptions regarding CBE. Implementation science principles outline the need to develop a common language to inform effective and sustained implementation of an innovation. By misusing or overusing similar words and using different concepts interchangeably without first developing a common, widely understood language around CBE, we cannot expect that CBE will be among the tools successful in closing gaps between health care practice and education.

Twelve tips for designing and implementing an academic coaching program

Coaching has become increasingly popular as a mechanism to support learning across the health professions education (HPE) continuum. While there is a growing body of literature in this area, there is minimal guidance related to the design and implementation of academic coaching in health professional courses. This paper seeks to contribute to this literature by presenting guidance for academic developers who are considering introducing academic coaching into a health professional course. The 12 tips are based on the authors' collective experiences of designing and implementing academic coaching in university medical courses in Australia and the UK. Although focused on medical education, this paper is intended to have applicability across the health professions, and potentially across university and postgraduate training contexts. Together, the tips offer a strategic and operational framework to guide the design and implementation of academic coaching initiatives in health professions education.

Competency-based faculty development: applying transformations from lessons learned in competency-based medical education

Faculty development in medical education is often delivered in an ad hoc manner instead of being a deliberately sequenced program matched to data-informed individual needs. In this article, the authors, all with extensive experience in Faculty Development (FD), present a competency-based faculty development (CBFD) framework envisioned to enhance the impact of FD. Steps and principles in the CBFD framework reflect the lessons learned from competency-based medical education (CBME) with its foundational goal to better train physicians to meet societal needs. The authors see CBFD as a similar framework, this one to better train faculty to meet educational needs. CBFD core elements include articulated competencies for the varied educational roles faculty fulfill, deliberately designed curricula structured to build those competencies, and an assessment program and process to support individualized faculty learning and professional growth. The framework incorporates ideas about where and how CBFD should be delivered, the use of coaching to promote reflection and identity formation and the creation of communities of learning. As with CBME, the CBFD framework has included the important considerations of change management, including broad stakeholder engagement, continuous quality improvement and scholarship. The authors have provided examples from the literature as well as challenges and considerations for each step.

A Portfolio Coach-Informed Professional Development Framework

INTRODUCTION

A majority of US medical schools have incorporated faculty coach-supported educational portfolios into the curriculum. Existing research describes coach professional development, competencies, and program perceptions. However, limited research exists on how programs address coach professional development needs. Our sequential objectives were to (1) explore faculty coach professional development experiences within medical student coaching programs and (2) develop a preliminary framework for medical faculty coach professional development.

METHODS

Faculty portfolio coaches who completed 4 years of a longitudinal coaching program were recruited to complete a semi-structured exit interview. Interviews were transcribed using detailed transcription. Two analysts inductively generated a codebook of parent and child codes to identify themes. They compared themes to the professional development model proposed by O'Sullivan and Irby.

RESULTS

Of the 25 eligible coaches, 15 completed the interview. Our team organized themes into two broad domains paralleling the established model: program-specific professional development and career-relevant professional development. Four program-specific professional development themes emerged: doing; modeling; relating; and hosting. Three career-relevant professional development themes emerged: advancement; meaning; and understanding. We then applied themes within each domain to propose strategies to optimize coach professional development and develop a framework modeled after O'Sullivan and Irby.

DISCUSSION

To our knowledge, we propose the first portfolio coach-informed framework for professional development. Our work builds on established standards, expert opinion, and research responsible for portfolio coach professional development and competencies. Allied health institutions with portfolio coaching programs can apply the framework for professional development innovation.

"It's All About the Trust And Building A Foundation:" Evaluation of a Longitudinal Medical Student Coaching Program

Coaching is increasingly implemented in medical education to support learners' growth, learning, and wellbeing. Data demonstrating the impact of longitudinal coaching programs are needed.

We developed and evaluated a comprehensive longitudinal medical student coaching program designed to achieve three aims for students: fostering personal and professional development, advancing physician skills with a growth mindset, and promoting student wellbeing and belonging within an inclusive learning community. We also sought to advance coaches' development as faculty through satisfying education roles with structured training. Students meet with coaches weekly for the first 17 months of medical school for patient care and health systems skills learning, and at least twice yearly throughout the remainder of medical school for individual progress and planning meetings and small-group discussions about professional identity. Using the developmental evaluation framework, we iteratively evaluated the program over the first five years of implementation with multiple quantitative and qualitative measures of students' and coaches' experiences related to the three aims.

The University of California, San Francisco, School of Medicine, developed a longitudinal coaching program in 2016 for medical students alongside reform of the four-year curriculum. The coaching program addressed unmet student needs for a longitudinal, non-evaluative relationship with a coach to support their development, shape their approach to learning, and promote belonging and community.

In surveys and focus groups, students reported high satisfaction with coaching in measures of the three program aims. They appreciated coaches' availability and guidance for the range of academic, personal, career, and other questions they had throughout medical school. Students endorsed the value of a longitudinal relationship and coaches' ability to meet their changing needs over time. Students rated coaches' teaching of foundational clinical skills highly. Students observed coaches learning some clinical skills with them - skills outside a coach's daily practice. Students also raised some concerns about variability among coaches. Attention to wellbeing and belonging to a learning community were program highlights for students. Coaches benefited from relationships with students and other coaches and welcomed the professional development to equip them to support all student needs.

Students perceive that a comprehensive medical student coaching program can achieve aims to promote their development and provide support. Within a non-evaluative longitudinal coach relationship, students build skills in driving their own learning and improvement. Coaches experience a satisfying yet challenging role. Ongoing faculty development within a coach community and funding for the role seem essential for coaches to fulfill their responsibilities.

It Takes a Village: Optimal Graduate Medical Education Requires a Deliberately Developmental Organization

Coaching is proposed as a means of improving the learning culture of medicine. By fostering trusting teacher-learner relationships, learners are encouraged to embrace feedback and make the most of failure. This paper posits that a cultural shift is necessary to fully harness the potential of coaching in graduate medical education. We introduce the deliberately developmental organization framework, a conceptual model focusing on three core dimensions: developmental communities, developmental aspirations, and developmental practices. These dimensions broaden the scope of coaching interactions. Implementing this organizational change within graduate medical education might be challenging, yet we argue that embracing deliberately developmental principles can embed coaching into everyday interactions and foster a culture in which discussing failure to maximize learning becomes acceptable. By applying the dimensions of developmental communities, aspirations, and practices, we present a six-principle roadmap towards transforming graduate medical education training programs into deliberately developmental organizations.

Perioperative Reflection: An Understudied Yet Common Practice Among Surgical Residents

OBJECTIVE

Directed reflection, a descriptive, analytical, and critical reasoning process, has been implemented in several professional fields as a method of developing expertise. In surgery, investigation of reflection has been limited. To date, we are unaware of any study that has sought to define or characterize the independent reflective practices of surgical trainees. In this small pilot study, we characterize the independent operative reflection activities among surgical residents at a single academic institution.

DESIGN

An unvalidated survey was developed through cognitive interviews and pilot testing to assess 5 aspects of operative reflection. An IRB approved electronic survey was developed and distributed to all general surgery residents at an academic institution. Responses were divided by post-graduate year into junior residents (JR; PGY 1-3) and senior residents (SR; PGY 4-5).

RESULTS

A response rate of 46% (23/50) was attained. All respondents self-identified as operative reflectors. Three primary methods of reflection were identified: mental, written, and oral. Mental reflection was the primary reflection method of 75% of our respondents as identified by step-by-step mental rehearsal, internal narration, and question series. Reflection was practiced everyday by 42% of respondents. Topics of reflection focused on weaknesses (50%), self-improvement (35%) and judgment (15%). Several respondents (57%) indicated that reflection-based learning tools would be useful. No statistically significant changes in the method or frequency were established between junior and senior residents (p = 0.378, p = 0.464, respectively). Variation in mental reflection activities among JR and SR groups were found (p = 0.029, p = 0.00004, respectively).

CONCLUSIONS

The enduring role of reflection in experiential learning has been established in many professional fields but remains under-utilized in surgical education. Our study suggests surgical residents reflect independently of curriculum in individual and categorizable ways. Further investigation is required to adequately characterize reflective habits of the surgical resident and evaluate its impact on competency.

Optimal training in communication model (OPTiCOM): A programmatic roadmap

OBJECTIVES

Teaching primary care residents patient communication skills is essential, complex, and impeded by barriers. We find no models guiding faculty how to train residents in the workplace that integrate necessary system components, the science of physician-patient communication training and competency-based medical education. The aim of this project is to create such a model.

METHODS

We created OPTiCOM using four steps: (1) communication educator interviews, analysis and theme development; (2) initial model construction; (3) model refinement using expert feedback; (4) structured literature review to validate, refine and finalize the model.

RESULTS

Our model contains ten interdependent building blocks organized into four developmental tiers. The Foundational value tier has one building block Naming relationship as a core value. The Expertize and resources tier includes four building blocks addressing: Curricular expertize, Curricular content, Leadership, and Time. The four building blocks in the Application and development tier are Observation form, Faculty development, Technology, and Formative assessment. The Language and culture tier identifies the final building block, Culture promoting continuous improvement in teaching communication.

CONCLUSIONS

OPTiCOM organizes ten interdependent systems building blocks to maximize and sustain resident learning of communication skills. Practice Implications Residency faculty can use OPTiCOM for self-assessment, program creation and revision.

Video-based coaching for surgical residents: a systematic review and meta-analysis

BACKGROUND

Video-based coaching (VBC) is used to supplement current teaching methods in surgical education and may be useful in competency-based frameworks. Whether VBC can effectively improve surgical skill in surgical residents has yet to be fully elucidated. The objective of this study is to compare surgical residents receiving and not receiving VBC in terms of technical surgical skill.

METHODS

The following databases were searched from database inception to October 2021: Medline, EMBASE, Cochrane Central Register of Controlled Trials (CENTRAL), and PubMed. Articles were included if they were randomized controlled trials (RCTs) comparing surgical residents receiving and not receiving VBC. The primary outcome, as defined prior to data collection, was change in objective measures of technical surgical skill following implementation of either VBC or control. A pairwise meta-analyses using inverse variance random effects was performed. Standardized mean differences (SMD) were used as the primary outcome measure to account for differences in objective surgical skill evaluation tools.

RESULTS

From 2734 citations, 11 RCTs with 157 residents receiving VBC and 141 residents receiving standard surgical teaching without VBC were included. There was no significant difference in post-coaching scores on objective surgical skill evaluation tools between groups (SMD 0.53, 95% CI 0.00 to 1.01, p = 0.05, I² = 74%). The improvement in scores pre- and post-intervention was significantly greater in residents receiving VBC compared to those not receiving VBC (SMD 1.62, 95% CI 0.62 to 2.63, p = 0.002, I² = 85%). These results were unchanged with leave-one-out sensitivity analysis and subgroup analysis according to operative setting.

CONCLUSION

VBC can improve objective surgical skills in surgical residents of various levels. The benefit may be most substantial for trainees with lower baseline levels of objective skill. Further studies are required to determine the impact of VBC on competency-based frameworks.

Understanding the near-peer relationship: resident perspectives around a novel on-call workplace-based assessment

BACKGROUND

Workplace-based assessment (WBA) is a critical component of competency-based medical education (CBME), though literature on WBA for overnight call is limited. We evaluated a WBA tool completed by supervising subspecialty trainees on paediatric residents during subspecialty overnight call, for usefulness facilitating feedback/coaching in this setting.

METHODS

Web-based surveys were sent to residents pre- and post-WBA tool implementation monthly for four months (August-December 2018), exploring feedback frequency, Likert-scaled opinions of tool feasibility/usefulness facilitating feedback, and qualitative experiences. Assessor comments were categorized as actionable/non-actionable. Quantitative data was summarized using descriptive statistics. Qualitative data was coded to identify themes.

RESULTS

Total response rates averaged 41% (total 25 responses, average five respondents/12 residents on-call each month). Post-implementation (n = 16 responses), a non-sustained trend of increased Medical Expert feedback was observed. Residents were generally divided or disagreed on tool usefulness facilitating feedback and feasibility. Comments contained actionable feedback in < 10% of completed WBAs. Qualitative analysis revealed barriers to tool-facilitated coaching including: feedback quality and setting/environment, role of senior near-peer as assessor, interpersonal burden in encounters, and tool-specific issues.

CONCLUSIONS

Increasing frequency of WBA tool completion is not sufficient to achieve CBME goals. Factors impacting feedback/coaching within the resident/near-peer dyad must be addressed.

"The Most Crushing Thing": Understanding Resident Assessment Burden in a Competency-Based Curriculum

BACKGROUND

Competency-based medical education (CBME) was expected to increase the workload of assessment for graduate training programs to support the development of competence. Learning conditions were anticipated to improve through the provision of tailored learning experiences and more frequent, low-stakes assessments. Canada has adopted an approach to CBME called Competence by Design (CBD). However, in the process of implementation, learner anxiety and assessment burden have increased unexpectedly. To mitigate this unintended consequence, we need a stronger understanding of how resident assessment burdens emerge and function.

OBJECTIVE

This study investigates contextual factors leading to assessment burden on residents within the framework of CBD.

METHODS

Residents were interviewed about their experiences of assessment using constructivist grounded theory. Participants (n=21) were a purposive sample from operative and perioperative training programs, recruited from 6 Canadian medical schools between 2019 and 2020. Self-determination theory was used as a sensitizing concept to categorize findings on types of assessment burden.

RESULTS

Nine assessment burdens were identified and organized by threats to psychological needs for autonomy, relatedness, and competence. Burdens included: missed opportunities for self-regulated learning, lack of situational control, comparative assessment, lack of trust, constraints on time and resources, disconnects between teachers and learners, lack of clarity, unrealistic expectations, and limitations of assessment forms for providing meaningful feedback.

CONCLUSIONS

This study contributes a contextual understanding of how assessment burdens emerged as unmet psychological needs for autonomy, relatedness, and competence, with unintended consequences for learner well-being and intrinsic motivation.

Community, Coaching, and Action Research: Promoting Ambitious Teaching in Medical Education

Ambitious teaching is an instructional approach enacted through central tasks of teaching that involves a fundamental shift in mindset and practice. In this approach, the teacher facilitates student learning in the context of authentic, interactive experiences by eliciting student thinking and adapting instruction accordingly. We designed the Medical Educator-Excellence in Teaching (MEET) program to promote ambitious teaching in medical education. Here, we describe the structure of MEET, the framework that informed our work, and program evaluation data. We propose MEET as a model of educator development that promotes ambitious teaching through development of educator community, focused coaching, and inquiry into practice.

Perceptions of the multidisciplinary operative team on intraoperative telecoaching among surgeons

Telecoaching, intraoperative coaching through videoconference, has been suggested as a tool to overcome logistical barriers with in-person coaching. However, little is known about the operative team's perception of telecoaching and its unique set of challenges. This qualitative study explores the perceptions of the multidisciplinary operative team on surgical telecoaching. A telecoaching program between peer surgeons was implemented using the Karl Storz Visitor1 remote presence system (Karl Storz, Germany). Semistructured interviews were conducted with the 12 operative team members present during 2 telecoaching sessions completed during the study period. Twelve participants were interviewed. The 4 central themes that emerged from the data were effective communication and collaboration, improving performance, operating room workflow, and culture and optics. Participating surgeon mentees reported that the session met expectations and learning goals and revealed concerns about negative perception of their autonomy and expertise by colleagues and patients. Conversely, team members unanimously reported a positive impression of surgeon mentees for taking additional measures to improve their performance and for patient outcomes. The operative team members reported that telecoaching was conducive to their own learning and relevant for complex cases. Considerations for future implementation of telecoaching include robust privacy standards for patients and staff, strong internet connectivity, coordinating with the operative team, and space constraints. Operative team participants viewed the intervention favorably and identified practical considerations for its continued use in an operating room environment. However, more work is needed on surgical culture as a contributor to low adoption and its impact on coaching programming activity.

The Construction and Application of E-Learning Curricula Evaluation Metrics for Competency-Based Teacher Professional Development

Today, students at universities in advanced countries typically enroll in colleges, such as the College of Education, which offer interdisciplinary programs for undergraduates in their first and second years, allowing them to explore personal interests, experience educational research fields, complete their integrated curricula, and then choose a major in their third year. To cooperate with the government’s epidemic prevention policies and measures in the post-COVID-19 era, the trend of e-learning and distance teaching has accelerated the establishment of integrated online curricula with interdisciplinary programs for undergraduates in the College of Education to facilitate effective future teacher professional development (TPD). Therefore, it is very important to construct e-learning curricula evaluation metrics for competency-based teacher professional development (CB-TPD) and to implement them in teaching practice. This research used social network analysis (SNA) methods, approaches, and theoretical concepts, such as affiliation networks and bipartite graphs comprised of educational occupational titles and common professional competencies (i.e., Element Name and ID), as well as knowledge, skills, abilities, and other characteristics (KSAOs), from the U.S. occupational information network (O*NET) 26.1 OnLine database, to collect data on the occupations of educational professionals. This study also used Gephi network analysis and visualization software to carry out descriptive statistics of keyword co-occurrences to measure their centrality metrics, including weighted degree centrality, degree centrality, betweenness centrality, and closeness centrality, and to verify their importance and ranking in professional competency in eight categories of educational professionals (i.e., three categories of special education teachers and five categories of teachers, except special education). The analysis of the centrality metrics identified the educational common professional competency (ECPC) keyword co-occurrences, which were then used to design, develop, and apply e-learning curricula evaluation metrics for CB-TPD. The results of this study can be used as a reference for conducting related academic research and cultivating educational professionals’ online curricula, including ECPC keywords, integrated curricula design and the development of transdisciplinary programs, and teacher education, as well as to facilitate the construction and application of future e-learning curricula evaluation metrics for CB-TPD.

A Pilot Study of the Generalizability of Preclinical Entrustment Assessments in Undergraduate Medical Education

PURPOSE

The reproducibility and consistency of assessments of entrustable professional activities (EPAs) in undergraduate medical education (UME) have been identified as potential areas of concern. EPAs were designed to facilitate workplace-based assessments by faculty with a shared mental model of a task who could observe a trainee complete the task multiple times. In UME, trainees are frequently assessed outside the workplace by faculty who only observe a task once.

METHOD

In November 2019, the authors conducted a generalizability study (G-study) to examine the impact of student, faculty, case, and faculty familiarity with the student on the reliability of 162 entrustment assessments completed in a preclerkship environment. Three faculty were recruited to evaluate 18 students completing 3 standardized patient (SP) cases. Faculty familiarity with each student was determined. Decision studies were also completed. Secondary analysis of the relationship between student performance and entrustment (scoring inference) compared average SP checklist scores and entrustment scores.

RESULTS

G-study analysis revealed that entrustment assessments struggled to achieve moderate reliability. The student accounted for 30.1% of the variance in entrustment scores with minimal influence from faculty and case, while the relationship between student and faculty accounted for 26.1% of the variance. G-study analysis also revealed a difference in generalizability between assessments by unfamiliar (φ = 0.75) and familiar (φ = 0.27) faculty. Subanalyses showed that entrustment assessments by familiar faculty were moderately correlated to average SP checklist scores (r = 0.44, P < .001), while those by unfamiliar faculty were weakly correlated (r = 0.16, P = .13).

CONCLUSIONS

While faculty and case had a limited impact on the generalizability of entrustment assessments made outside the workplace in UME, faculty who were familiar with a student's ability had a notable impact on generalizability and potentially on the scoring validity of entrustment assessments, which warrants further study.

A Systems Thinking approach for the creation of effective competency-based medical education programs

Successful and sustainable implementation of Competency-based Medical Education (CBME) programs is a significant and daunting challenge facing medical education worldwide. Our manuscript endorses for the first time, Systems Thinking as a concept for transforming and redesigning CBME programs employing the full 7-system elements as advocated by the Biomatrix Systems Theory. The majority of internationally recommended actions and processes for such an endeavor are highlighted, each within its system element. New innovative ideas such as having competency-structured clinical training activities as well as re-writing medical textbooks following a novel competency-based roadmap for their disease monographs etc. are also highlighted. Furthermore, the need for innovative partnerships as well as novel medical rotations that may facilitate the creation of "master clinicians" are also stressed.

Feedback and coaching

If used thoughtfully and with intent, feedback and coaching will promote learning and growth as well as personal and professional development in our learners. Feedback is an educational tool as well as a social interaction between learner and supervisor, in the context of a respectful and trusting relationship. It challenges the learner's thinking and supports the learner's growth. Coaching is an educational philosophy dedicated to supporting learners' personal and professional development and growth and supporting them to reach their potential. In clinical education, feedback is most effective when it is explicitly distinguished from summative assessment. Importantly, feedback should be about firsthand observed behaviors (which can be direct or indirect) and not about information which comes from a third party. Learners are more receptive to feedback if it comes from a source that they perceive as credible, and with whom they have developed rapport. The coaching relationship between learner and supervisor should also be built on mutual trust and respect. Coaching can be provided in the moment (feedback on everyday clinical activities that leads to performance improvement, even with short interaction with a supervisor) and over time (a longer term relationship with a supervisor in which there is reflection on the learner's development and co-creation of new learning goals). Feedback and coaching are most valuable when the learner and teacher exhibit a growth mindset. At the organizational level, it is important that both the structures and training are in place to ensure a culture of effective feedback and coaching in the clinical workplace.Conclusions: Having a thoughtful and intentional approach to feedback and coaching with learners, as well as applying evidence-based principles, will not only contribute in a significant way to their developmental progression, but will also provide them with the tools they need to have the best chance of achieving competence throughout their training. What is Known: • Feedback and coaching are key to advancing the developmental progression of trainees as they work towards achieving competence. • Feedback is not a one-way delivery of specific information from supervisor to trainee, but rather a social interaction between two individuals in which trust and respect play a key role. • Provision of effective feedback may be hampered by confusing formative (supporting trainee learning and development) and summative (the judgment that is made about a trainee's level of competence) purposes. What is New: • Approaches to both the provision of feedback/coaching and the assessment of competence must be developed in parallel to ensure success in clinical training programs. • Faculty development is essential to provide clinical teachers with the skills to provide effective feedback and coaching. • Coaching's effectiveness relies on nurturing strong trainee-supervisor relationships, ensuring high-quality feedback, nourishing a growth mindset, and encouraging an institutional culture that embraces feedback and coaching.

Emotion in remediation: A scoping review of the medical education literature

OBJECTIVES

Remediation can be crucial and high stakes for medical learners, and experts agree it is often not optimally conducted. Research from other fields indicates that explicit incorporation of emotion improves education because of emotion's documented impacts on learning. Because this could present an important opportunity for improving remediation, we aimed to investigate how the literature on remediation interventions in medical education discusses emotion.

METHODS

The authors used Arksey and O'Malley's framework to conduct a scoping literature review of records describing remediation interventions in medical education, using PubMed, CINAHL Complete, ERIC, Web of Science and APA PsycInfo databases, including all English-language publications through 1 May 2020 meeting search criteria. They included publications discussing remediation interventions either empirically or theoretically, pertaining to physicians or physician trainees of any level. Two independent reviewers used a standardised data extraction form to report descriptive information; they reviewed included records for the presence of mentions of emotion, described the mentions and analysed results thematically.

RESULTS

Of 1644 records, 199 met inclusion criteria and were reviewed in full. Of those, 112 (56%) mentioned emotion in some way; others focused solely on cognitive aspects of remediation. The mentions of emotion fell into three themes based on when the emotion was cited as present: during regular coursework or practice, upon referral for remediation and during remediation. One-quarter of records (50) indicated potential intentional incorporation of emotion into remediation programme design, but they were non-specific as to how emotions related to the learning process itself.

CONCLUSION

Even though emotion is omnipresent in remediation, medical educators frequently do not factor emotion into the design of remediation approaches and rarely explicitly utilise emotion to improve the learning process. Applications from other fields may help medical educators leverage emotion to improve learning in remediation, including strategies to frame and design remediation.

Acceptance and validity of the methods used to implement a competency based medical education programme in an Intensive Care Department of a teaching referral center

OBJECTIVES

1. To determine the satisfaction of tutors and residents with a specific methodology used to implement CoBaTrICE. 2. To determine the reliability and validity of the global rating scales designed ad hoc to assess the performance of the residents for training purposes.

DESIGN

Prospective cohort study.

PARTICIPANTS

All the residents and tutors of the ICU Department of the Hospital Universitario y Politécnico la Fe de Valencia.

INTERVENTION

CoBaTrICE implementation started in March 2016, it was based on: (1) Training the tutors in feedback techniques; (2) Performing multiple objective and structured work based assessments to achieve the competences of the program; and (3) The use of an electronic portfolio to promote learning reflection and to collect the evidence that learning was taking place.

METHODS

The acceptance of CoBaTrICE was explored through a satisfaction survey conducted after 9 months of implementation of the training program. The 15 residents and 5 tutors of the ICU Department were asked about the methodology of the formative assessments, the quality of the feedback, self-learning regulation and the electronic portfolio usefulness. The validity of the global rating scales was assessed through the tests alfa de Cronbach, reliability and generalizability indexes, and intraclass correlation coefficient.

RESULTS

The implementation of CoBaTrICE was satisfactory in all the dimensions studied. The global rating scales used for formative purposes showed reliability and validity.

CONCLUSIONS

The methodology used to implement CoBaTrICE was highly valued by tutors and residents. The global rating scales used for formative purposes showed reliability and validity.

Developing a dashboard for faculty development in competency-based training programs: a design-based research project

BACKGROUND

Canadian specialist residency training programs are implementing a form of competency-based medical education (CBME) that requires frequent assessments of entrustable professional activities (EPAs). Faculty struggle to provide helpful feedback and assign appropriate entrustment scores. CBME faculty development initiatives rarely incorporate teaching metrics. Dashboards could be used to visualize faculty assessment data to support faculty development.

METHODS

Using a design-based research process, we identified faculty development needs related to CBME assessments and designed a dashboard containing elements (data, analytics, and visualizations) meeting these needs. Data was collected within the emergency medicine residency program at the University of Saskatchewan through interviews with program leaders, faculty development experts, and faculty participating in development sessions. Two investigators thematically analyzed interview transcripts to identify faculty needs that were audited by a third investigator. The needs were described using representative quotes and the dashboard elements designed to address them.

RESULTS

Between July 1, 2019 and December 11, 2020 we conducted 15 interviews with nine participants (two program leaders, three faculty development experts, and four faculty members). Three needs emerged as themes from the analysis: analysis of assessments, contextualization of assessments, and accessible reporting. We addressed these needs by designing an accessible dashboard to present contextualized quantitative and narrative assessment data for each faculty member.

CONCLUSIONS

We identified faculty development needs related to EPA assessments and designed dashboard elements to meet them. The resulting dashboard was used for faculty development sessions. This work will inform the development of CBME assessment dashboards for faculty.

Scoping reviews in medical education: A scoping review

OBJECTIVES

Over the last two decades, the number of scoping reviews in core medical education journals has increased by 4200%. Despite this growth, research on scoping reviews provides limited information about their nature, including how they are conducted or why medical educators undertake this knowledge synthesis type. This gap makes it difficult to know where the field stands and may hamper attempts to improve the conduct, reporting and utility of scoping reviews. Thus, this review characterises the nature of medical education scoping reviews to identify areas for improvement and highlight future research opportunities.

METHOD

The authors searched PubMed for scoping reviews published between 1/1999 and 4/2020 in 14 medical education journals. The authors extracted and summarised key bibliometric data, the rationales given for conducting a scoping review, the research questions and key reporting elements as described in the PRISMA-ScR. Rationales and research questions were mapped to Arksey and O'Malley's reasons for conducting a scoping review.

RESULTS

One hundred and one scoping reviews were included. On average, 10.1 scoping reviews (SD = 13.1, median = 4) were published annually with the most reviews published in 2019 (n = 42). Authors described multiple reasons for undertaking scoping reviews; the most prevalent being to summarise and disseminate research findings (n = 77). In 11 reviews, the rationales for the scoping review and the research questions aligned. No review addressed all elements of the PRISMA-ScR, with few authors publishing a protocol (n = 2) or including stakeholders (n = 20). Authors identified shortcomings of scoping reviews, including lack of critical appraisal.

CONCLUSIONS

Scoping reviews are increasingly conducted in medical education and published by most core journals. Scoping reviews aim to map the depth and breadth of emerging topics; as such, they have the potential to play a critical role in the practice, policy and research of medical education. However, these results suggest improvements are needed for this role to be fully realised.

Driving lesson or driving test? : A metaphor to help faculty separate feedback from assessment

Although there is consensus in the medical education world that feedback is an important and effective tool to support experiential workplace-based learning, learners tend to avoid the feedback associated with direct observation because they perceive it as a high-stakes evaluation with significant consequences for their future. The perceived dominance of the summative assessment paradigm throughout medical education reduces learners' willingness to seek feedback, and encourages supervisors to mix up feedback with provision of 'objective' grades or pass/fail marks. This eye-opener article argues that the provision and reception of effective feedback by clinical supervisors and their learners is dependent on both parties' awareness of the important distinction between feedback used in coaching towards growth and development (assessment for learning) and reaching a high-stakes judgement on the learner's competence and fitness for practice (assessment of learning). Using driving lessons and the driving test as a metaphor for feedback and assessment helps supervisors and learners to understand this crucial difference and to act upon it. It is the supervisor's responsibility to ensure that supervisor and learner achieve a clear mutual understanding of the purpose of each interaction (i.e. feedback or assessment). To allow supervisors to use the driving lesson-driving test metaphor for this purpose in their interactions with learners, it should be included in faculty development initiatives, along with a discussion of the key importance of separating feedback from assessment, to promote a feedback culture of growth and support programmatic assessment of competence.

Improving Medical Education Through Targeted Coaching

Medical education is shifting from a lecture-based teaching format toward an emphasis on active learning. Educators need support to build a collaborative learning environment where students interact with ideas and each other. In order to support medical educators who are new to teaching in the paradigm of active learning, we argue that targeted, educative coaching, rooted in educational theory, is needed. Coaching is a common practice in learning to teach and is not a new concept in medical education. What is new is targeted coaching situated within an assisted performance framework for the purpose of learning to teach effectively. Targeted coaching includes using coaching practices such as observing, debriefing, and co-planning in purposeful, educative ways. Assisted performance involves supporting the medical educator by identifying teaching skills to be learned, practicing those skills with support, and eventually preparing for unassisted performance. The purpose of this article is to describe a model of professional development that provides targeted support for medical teachers. In this paper, we outline a model of educator development and provide two stories of our model in practice, to further demonstrate how assisted performance and core coaching practices can be used together to impact teaching.

In-the-Moment Feedback and Coaching: Improving R2C2 for a New Context

BACKGROUND

The R2C2, a 4-phase feedback and coaching model, builds relationships, explores reactions, determines content and coaches for change, and facilitates formal feedback conversations between clinical supervisors/preceptors and residents. Formal discussions about performance are typically based on collated information from daily encounter sheets, objective structured clinical examinations, multisource feedback, and other data. This model has not been studied in settings where brief feedback and coaching conversations occur immediately after a specific clinical experience.

OBJECTIVE

We explored how supervisors adapt the R2C2 model for in-the-moment feedback and coaching and developed a guide for its use in this context.

METHODS

Eleven purposefully selected supervisors were interviewed in 2018 to explore where they used the R2C2 model, how they adapted it for in-the-moment conversations, and phrases used corresponding to each phase that could guide design of a new R2C2 in-the-moment model.

RESULTS

Participants readily adapted the model to varied feedback situations; each of the 4 phases were relevant for conversations. Phase-specific phrases that could enable effective coaching conversations in a limited amount of time were identified. Data facilitated a revision of the original R2C2 model for in-the-moment feedback and coaching conversations and design of an accompanying trifold brochure to enable its effective use.

CONCLUSIONS

The R2C2 in-the-moment model offers a systematic approach to feedback and coaching that builds on the original model, yet addresses time constraints and the need for an iterative conversation between the reaction and content phases. The model enables supervisors to coach and co-create an action plan with residents to improve performance.

Team up! Linking teamwork effectiveness of clinical teaching teams to residents' experienced learning climate

Background: Supportive learning climates are key to ensure high-quality residency training. Clinical teachers, collaborating as teaching team, have an important role in maintaining such climates since they are responsible for residency training. Successful residency training is dependent on effective teamwork within teaching teams. Still, it remains unclear whether this team effort benefits residents' perceptions of the learning climate. We, therefore, investigated to what extent teamwork effectiveness within teaching teams is associated with (1) the overall learning climate, and (2) its affective, cognitive and instrumental facets?Methods: This study used a web-based platform to collect data in clinical departments in the Netherlands from January 2014 to May 2017. Teamwork effectiveness was measured with the TeamQ questionnaire, administered amongst clinical teachers. The learning climate was measured with the D-RECT, applied amongst residents. Associations were analyzed using multilevel models and multivariate general linear models.Results: Teamwork effectiveness was positively associated with the overall learning climate as well as with the affective and the instrumental facets of the learning climate. No significant associations were found with the cognitive facet.Conclusion: Effective teamwork within teaching teams benefits learning climates in postgraduate medical education. Therefore, departments aiming to improve their learning climate should target teamwork within teaching teams.