The importance of faculty development in the transition to competency-based medical education

The identification of requirements for competency development during work-integrated learning in healthcare education

BACKGROUND

Work-integrated learning (WIL) is widely accepted and necessary to attain the essential competencies healthcare students need at their future workplaces. Yet, competency-based education (CBE) remains complex. There often is a focus on daily practice during WIL. Hereby, continuous competency development is at stake. Moreover, the fact that competencies need to continuously develop is often neglected.

OBJECTIVES

To ultimately contribute to the optimization of CBE in healthcare education, this study aimed at examining how competency development during WIL in healthcare education could be optimized, before and after graduation.

METHODS

Fourteen semi-structured interviews with 16 experts in competency development and WIL were carried out. Eight healthcare disciplines were included namely associate degree nursing, audiology, family medicine, nursing (bachelor), occupational therapy, podiatry, pediatrics, and speech therapy. Moreover, two independent experts outside the healthcare domain were included to broaden the perspectives on competency development. A qualitative research approach was used based on an inductive thematic analysis using Nvivo12© where 'in vivo' codes were clustered as sub-themes and themes.

RESULTS

The analysis revealed eight types of requirements for effective and continuous competency development, namely requirements in the context of (1) competency frameworks, (2) reflection and feedback, (3) assessment, (4) the continuity of competency development, (5) mentor involvement, (6) ePortfolios, (7) competency development visualizations, and (8) competency development after graduation. It was noteworthy that certain requirements were fulfilled in one educational program whereas they were absent in another. This emphasizes the large differences in how competence-based education is taking shape in different educational programs and internship contexts. Nevertheless, all educational programs seemed to recognize the importance of ongoing competency development.

CONCLUSION

The results of this study indicate that identifying and meeting the requirements for effective and continuous competency development is essential to optimize competency development during practice in healthcare education.

Strategies to Enable Transformation in Medical Education: Faculty and Trainee Development in Competence By Design

Transformative changes in health professions education need to incorporate effective faculty development, but few very large-scale faculty development designs have been described. The Royal College of Physicians and Surgeons of Canada's Competence by Design project was launched to transform the delivery of postgraduate medical education in Canada using a competency-based model. In this paper we outline the goals, principles, and rationale of the Royal College's national strategy for faculty and resident development initiatives to support the implementation of Competence by Design. We describe the activities and resources for both faculty and trainees that facilitated the redesign of training programs for each specialty and subspecialty at the national level, as well as supporting the implementation of the redesign at the local level. This undertaking was not without its challenges: we thus reflect on those challenges, enablers, and the lessons learned, and discuss a continuous quality improvement approach that was taken to iteratively inform the implementation process moving forward.

Using a rapid-cycle approach to evaluate implementation of competency-based medical education in ophthalmology

OBJECTIVE

As competency-based medical education is being implemented across Canada, there is an increasing need to evaluate the progress to date, including identification of strengths and weaknesses, to inform program development. Ophthalmology is preparing for a national launch in coming years. The purpose of this study was to describe key stakeholders' lived experiences in the competency-based medical education foundation-of-discipline stage in one ophthalmology department.

DESIGN

Using a case-study approach, a qualitative rapid-cycle evaluation was conducted during the 2018-2019 academic year.

PARTICIPANTS

Residents, faculty, academic advisors, competence committee members, the program director, the program administrator, and the educational consultant were invited to participate in the program evaluation.

METHODS

The rapid-cycle evaluation consisted of 2 evaluation cycles, with the first round of interviews and focus groups occurring in October 2018 and the second round in March 2019. Recommendations were implemented in November 2019 and June 2019. All data were analyzed thematically using NVivo.

RESULTS

Three main themes emerged across all data sets: developing a shared understanding (e.g., role expectations and changes to assessment), refining assessment processes and tools (e.g., the need for streamlining and clarification), and feedback (e.g., perceived benefits and value of narrative comments).

CONCLUSIONS

Exploring lived experiences in this study resulted in positive and immediate improvements to the residency program. The recommendations and approach will be useful to other Canadian departments and institutions as they prepare for Competence by Design.

Competency-based faculty development: applying transformations from lessons learned in competency-based medical education

Faculty development in medical education is often delivered in an ad hoc manner instead of being a deliberately sequenced program matched to data-informed individual needs. In this article, the authors, all with extensive experience in Faculty Development (FD), present a competency-based faculty development (CBFD) framework envisioned to enhance the impact of FD. Steps and principles in the CBFD framework reflect the lessons learned from competency-based medical education (CBME) with its foundational goal to better train physicians to meet societal needs. The authors see CBFD as a similar framework, this one to better train faculty to meet educational needs. CBFD core elements include articulated competencies for the varied educational roles faculty fulfill, deliberately designed curricula structured to build those competencies, and an assessment program and process to support individualized faculty learning and professional growth. The framework incorporates ideas about where and how CBFD should be delivered, the use of coaching to promote reflection and identity formation and the creation of communities of learning. As with CBME, the CBFD framework has included the important considerations of change management, including broad stakeholder engagement, continuous quality improvement and scholarship. The authors have provided examples from the literature as well as challenges and considerations for each step.

The Effect of a One-Day Workshop on the Quality of Framing Multiple Choice Questions in Physiology in a Medical College in India

Background Multiple choice questions (MCQs) are commonly used in medical exams for more objectivity in assessment. However, the quality of the questions should be optimum for a proper assessment of the students. A faculty development program (FDP) may improve the quality of MCQs. The effect of a one-day workshop on framing MCQ as a part of a FDP has not been explored in our institution. Aim This study aimed to evaluate the quality of MCQ in the subject of physiology before and after a one-day workshop on framing MCQ as a part of a FDP. Methods This was a retrospective study conducted in the Department of Physiology, All India Institute of Medical Sciences, Deoghar, Jharkhand, India. A one-day workshop on framing MCQ as a part of a FDP was conducted in March 2022. We took 100 MCQs and responses from the students from examinations conducted before the workshop and 100 MCQs and responses from the students after the workshop. In pre-workshop and post-workshop, the same five faculties framed the questions. Post-validation item analysis including difficulty index (DIFI), discrimination index (DI), distractor effectiveness (DE), and Kuder-Richardson Formula 20 (KR-20) for internal consistency was calculated. Results Pre-workshop and post-workshop quality of the MCQ remain equal in terms of DIFI (chi-square {3} = 2.42, P = 0.29), DI (chi-square {3} = 2.44, P = 0.49), and DE (chi-square {3} = 4.97, P = 0.17). The KR-20 in pre-workshop and post-workshop was 0.65 and 0.87, respectively. Both had acceptable internal consistency. Conclusion The one-day workshop on framing MCQs as a part of a FDP did not have a significant impact on the quality of the MCQs as measured by the three indices of item quality but did improve the internal consistency of the MCQs. Further educational programs and research are required to find out what measures can improve the quality of MCQs.

Global obstetric anaesthesia: bridging the gap in maternal health care inequities through partnership in education

Maternal mortality rates are unacceptably high globally. Low- and middle-income countries (LMICs) face challenges of an inadequate anaesthesia workforce, under-resourced healthcare systems and sub-optimal access to labour and delivery care, all of which negatively impact maternal and neonatal outcomes. In order to effect the changes in surgical-obstetric-anaesthesia workforce numbers advocated by the Lancet Commission on Global Surgery to support the UN sustainable development goals, mass training and upskilling of both physician and non-physician anaesthetists is imperative. The implementation of outreach programmes and partnerships across organisations and countries has already been shown to improve the provision of safe care to mothers and their babies, and these efforts should be continued. Short subspecialty courses and simulation training are two cornerstones of modern obstetric anaesthesia training in poorly resourced environments. This review discusses the challenges to accessing quality maternal healthcare in LMICs and the use of education, outreach, partnership and research to protect the most vulnerable women from coming to harm in the peripartum period.

Competency-based medical education in the United States: What the otolaryngologist needs to know

Competency-based medical education (CBME) is an outcomes-focused approach to educating medical professionals that will be central to future efforts to improve resident training in otolaryngology. The transition to CBME for otolaryngology in the United States will require the development of specialty-specific assessments and benchmarks, the financial and administrative support for implementation, the professional development of faculty and learners, and the cooperation of all major stakeholders in graduate medical education. In this article, we describe the need for evidence-based innovation in surgical training, the history of CBME in the United States, and the progress towards defining "entrustable professional activities" as the building blocks of assessments for CBME. We explore what such a paradigm shift in surgical education could mean for academic otolaryngologists by examining innovative educational practices in other surgical specialties and discussing foreseeable challenges in implementation for the American healthcare system.

Exploring Perceptions of Competency-Based Medical Education in Undergraduate Medical Students and Faculty: A Program Evaluation

BACKGROUND

There is limited work exploring competency-based medical education (CBME) in undergraduate medical education. We aimed to assess medical students' and faculty's perception of CBME in the undergraduate medicine setting after its implementation at our institution through a Content, Input, Process, Product (CIPP) program evaluation model.

METHODS

We explored the rationale for the transition to a CBME curriculum (Content), the changes to the curriculum and the teams involved in the transition (Input), medical students' and faculty's perception of the current CBME curriculum (Process), and benefits and challenges of implementing undergraduate CBME (Product). A cross-sectional online survey was delivered over 8-weeks in October 2021 to medical students and faculty as part of the Process and Product evaluation.

RESULTS

Medical students displayed greater optimism towards CBME, compared to faculty, in terms of its role in medical education (p<0.05). Faculty were less certain about how CBME was currently implemented (p<0.05), as well as how feedback to students should be delivered (p<0.05). Students and faculty agreed on perceived benefits to CBME implementation. Faculty time commitment to teaching and logistical concerns were reported as perceived challenges.

CONCLUSION

Education leaders must prioritize faculty engagement and continued professional development of faculty to facilitate the transition. This program evaluation identified strategies to aid the transition to CBME in the undergraduate setting.

Impact and effectiveness of a mandatory competency-based simulation program for pediatric emergency medicine faculty

Introduction

Pediatric emergency medicine physicians struggle to maintain their critical procedural and resuscitation skills. Continuing professional development programs incorporating simulation and competency-based standards may help ensure skill maintenance. Using a logic model framework, we sought to evaluate the effectiveness of a mandatory annual competency-based medical education (CBME) simulation program.

Methods

The CBME program, evaluated from 2016 to 2018, targeted procedural, point-of-care ultrasound (POCUS) and resuscitation skills. Delivery of educational content included a flipped-classroom website, deliberate practice, mastery-based learning, and stop-pause debriefing. Participants' competence was assessed using a 5-point global rating scale (GRS; 3 = competent, 5 = mastery). Statistical process control charts were used to measure the effect of the CBME program on team performance during in situ simulations (ISS), measured using the Team Emergency Assessment Measure (TEAM) scale. Faculty completed an online program evaluation survey.

Results

Forty physicians and 48 registered nurses completed at least one course over 3 years (physician mean ± SD 2.2 ± 0.92). Physicians achieved competence on 430 of 442 stations (97.3%). Mean ± SD GRS scores for procedural, POCUS, and resuscitation stations were 4.34 ± 0.43, 3.96 ± 0.35, and 4.17 ± 0.27, respectively. ISS TEAM scores for "followed standards and guidelines" improved significantly. No signals of special cause variation emerged for the other 11 TEAM items, indicating skills maintenance. Physicians rated CBME training as highly valuable (mean question scores 4.15-4.85/5). Time commitment and scheduling were identified as barriers to participation.

Conclusions

Our mandatory simulation-based CBME program had high completion rates and very low station failures. The program was highly rated and faculty improved or maintained their ISS performance across TEAM scale domains.

Defining new roles and competencies for administrative staff and faculty in the age of competency-based medical education

PURPOSE

These authors sought to define the new roles and competencies required of administrative staff and faculty in the age of CBME.

METHOD

A modified Delphi process was used to define the new CBME roles and competencies needed by faculty and administrative staff. We invited international experts in CBME (volunteers from the ICBME Collaborative email list), as well as faculty members and trainees identified via social media to help us determine the new competencies required of faculty and administrative staff in the CBME era.

RESULTS

Thirteen new roles were identified. The faculty-specific roles were: National Leader/Facilitator in CBME; Institutional/University lead for CBME; Assessment Process & Systems Designer; Local CBME Leads; CBME-specific Faculty Developers or Trainers; Competence Committee Chair; Competence Committee Faculty Member; Faculty Academic Coach/Advisor or Support Person; Frontline Assessor; Frontline Coach. The staff-specific roles were: Information Technology Lead; CBME Analytics/Data Support; Competence Committee Administrative Assistant.

CONCLUSIONS

The authors present a new set of faculty and staff roles that are relevant to the CBME context. While some of these new roles may be incorporated into existing roles, it may be prudent to examine how best to ensure that all of them are supported within all CBME contexts in some manner.

Radical reform of the undergraduate medical education program in a developing country: the Egyptian experience

Medical educators are in a continuous quest to close the gap between the needs of medical practice and the rising expectations of the communities in their countries. During the past two decades, competency-based medical education has been evolving as an appealing strategy to close this gap. In 2017, the Egyptian medical education authorities mandated all medical schools to change their curricula to comply with revised national academic reference standards, which changed from outcome-based to competency-based. In parallel, they also changed the timeline of all medical programs for six years of studentship and one-year internship to five years and two years, respectively. This substantial reform involved the assessment of the existing situation, an awareness campaign for the proposed changes and an extensive national faculty development program. Monitoring the implementation of this substantial reform was performed through surveys, field visits and meetings with students, teaching staff and program directors. In addition to the expected challenges, the COVID-19-associated restrictions presented a significant further challenge during the implementation of this reform. This article presents the rationale for and steps of this reform, the challenges faced and how they were addressed.

Coaching in Postgraduate Competency-Based Medical Education: a Qualitative Exploration of Three Models

OBJECTIVE

As postgraduate medical education increasingly transitions to competency-based models, there is a growing need for faculty to help residents process increasing amounts of assessment data. It has been recommended that a designated resident advisor or coach take on this faculty role, but the literature surrounding coaching in medical education is sparse. The authors evaluated the implementation of different coaching models in a postgraduate psychiatry program to identify drivers and barriers to effective coaching.

METHODS

The authors conducted semi-structured interviews in September 2019 with focus groups of residents and faculty to understand their experiences of coaching under different models. They identified major themes through a qualitative analysis of the transcribed focus groups, which took place from September to December 2020.

RESULTS

The authors identified four key themes associated with the implementation of coaching within a competency-based framework, namely role ambiguity, educational alliance, the "idealized coach," and burden.

CONCLUSIONS

While these findings highlight the barriers that can interfere with effective coaching, particularly in the context of widespread curriculum change, they also illuminate opportunities for the coaching role moving forward. Thus, they offer valuable guidance for present and upcoming competency-based programs as they implement coaching and seek to optimize the learning experience for residents.

Optimal training in communication model (OPTiCOM): A programmatic roadmap

OBJECTIVES

Teaching primary care residents patient communication skills is essential, complex, and impeded by barriers. We find no models guiding faculty how to train residents in the workplace that integrate necessary system components, the science of physician-patient communication training and competency-based medical education. The aim of this project is to create such a model.

METHODS

We created OPTiCOM using four steps: (1) communication educator interviews, analysis and theme development; (2) initial model construction; (3) model refinement using expert feedback; (4) structured literature review to validate, refine and finalize the model.

RESULTS

Our model contains ten interdependent building blocks organized into four developmental tiers. The Foundational value tier has one building block Naming relationship as a core value. The Expertize and resources tier includes four building blocks addressing: Curricular expertize, Curricular content, Leadership, and Time. The four building blocks in the Application and development tier are Observation form, Faculty development, Technology, and Formative assessment. The Language and culture tier identifies the final building block, Culture promoting continuous improvement in teaching communication.

CONCLUSIONS

OPTiCOM organizes ten interdependent systems building blocks to maximize and sustain resident learning of communication skills. Practice Implications Residency faculty can use OPTiCOM for self-assessment, program creation and revision.

Exploring the Inner Lens: Examining Oncology Consultants' Use of Competency and Assessment Frameworks in Clinical Teaching Settings

Medical education in oncology has adopted a competency-based medical education (CBME) approach in Canada. Oncologist competencies are arranged within assessment frameworks, but it is unknown whether oncologists routinely use assessment frameworks when determining what to teach oncology trainees. Understanding oncologists' actual approaches to assessment and teaching as a form of enacted CBME could inform and focus faculty development efforts. Using a participatory action research approach, oncology faculty were interviewed regarding their teaching practices and use of assessment frameworks. Faculty suggestions regarding challenges in setting teaching goals and completing assessment within the CBME paradigm were solicited. Furthermore, teaching objectives developed by faculty in-house were assessed regarding the domains of competence targeted. Thirty-one of the 40 faculty members in the Department of Oncology, Western University, agreed to participate. Trainee factors (training level, areas of interest), faculty personal values and experience, the teaching setting, and some components of frameworks (medical expert, communicator, advocate, and professional) were articulated as determinants of teaching goals. Of the 1117 in-house objectives reviewed, 76% focused on the medical role. Faculty held widely disparate views on assessment framework features which would enable teaching and assessment. Oncology faculty members used a multifaceted approach to determine what to teach trainees. This approach contains elements both of the conventional apprenticeship models under which many faculty members themselves were trained and of the CBME paradigm. Future research should focus on exploring ways to resolve the tension between established individual teaching approaches and nationally standardized competency frameworks which are being implemented.

Understanding the near-peer relationship: resident perspectives around a novel on-call workplace-based assessment

BACKGROUND

Workplace-based assessment (WBA) is a critical component of competency-based medical education (CBME), though literature on WBA for overnight call is limited. We evaluated a WBA tool completed by supervising subspecialty trainees on paediatric residents during subspecialty overnight call, for usefulness facilitating feedback/coaching in this setting.

METHODS

Web-based surveys were sent to residents pre- and post-WBA tool implementation monthly for four months (August-December 2018), exploring feedback frequency, Likert-scaled opinions of tool feasibility/usefulness facilitating feedback, and qualitative experiences. Assessor comments were categorized as actionable/non-actionable. Quantitative data was summarized using descriptive statistics. Qualitative data was coded to identify themes.

RESULTS

Total response rates averaged 41% (total 25 responses, average five respondents/12 residents on-call each month). Post-implementation (n = 16 responses), a non-sustained trend of increased Medical Expert feedback was observed. Residents were generally divided or disagreed on tool usefulness facilitating feedback and feasibility. Comments contained actionable feedback in < 10% of completed WBAs. Qualitative analysis revealed barriers to tool-facilitated coaching including: feedback quality and setting/environment, role of senior near-peer as assessor, interpersonal burden in encounters, and tool-specific issues.

CONCLUSIONS

Increasing frequency of WBA tool completion is not sufficient to achieve CBME goals. Factors impacting feedback/coaching within the resident/near-peer dyad must be addressed.

Fundamental Teaching Activities in Family Medicine Framework: Analysis of Awareness and Utilization

INTRODUCTION

In 2015, the College of Family Physicians of Canada, in performing their commitment to supporting its members in their educational roles, created the Family Medicine Framework (FTA). It was designed to assist family medicine educators with an understanding of the core activities of educators: precepting, coaching, and teaching within or beyond clinical settings. Given that an examination of member awareness of FTA has not been previously undertaken, our primary objective was to conduct an evaluation on its utility and application.

METHODS

In partnership with College of Family Physicians of Canada Faculty Development Education Committee members, we used a practical participatory evaluation approach to conduct a two-phase mixed-methods evaluation of the FTA. We distributed an electronic survey in French and English languages to Canadian faculty development, program, and site directors in family medicine. We then conducted follow-up interviews with self-selected participants.

RESULTS

Of the target populations, 12/15 (80%) faculty development directors (FDDs), 12/18 (66.7%) program directors, and 34/174 (19.5%) site directors completed the electronic survey. Subsequently, 6 FDDs, 3 program directors, and 3 site directors completed an interview (n = 12). Findings indicate that awareness of the FTA was highest among FDDs. Facilitators who encourage teachers to use the FTA and barriers for low uptake were also identified.

DISCUSSION

This evaluation illuminated that varied levels of awareness of the FTA may contribute to the low uptake among education leaders. We also suggest future research to address possible barriers that hinder effective applications of the FTA in faculty development initiatives.

Faculty Competencies: An Exploration of Feasibility and Acceptance

INTRODUCTION

A uniform method of iterative professional development for medical educators does not exist in the United States graduate medical education system. The Society of Teachers of Family Medicine Faculty Competencies Special Project Team, a subgroup of the Faculty Development Collaborative, sought to create a competency-based assessment framework for medical educators. This paper describes the feasibility and acceptance of a draft competencies resource using a survey.

METHODS

A mixed-methods, ten-question survey to assess the feasibility and acceptance of the draft competencies resource was created and distributed to medical educators through educational contacts from October 2019 to November 2019.

RESULTS

Eighty-six surveys were completed. Of the 86 respondents, 48 (55%) answered all the survey questions. Thematic analysis for acceptance of the draft yielded three groups, the accepting, neutral, and nonacceptance groups. Each group had distinct characteristics regarding the likelihood of accepting and using the draft competencies.

CONCLUSIONS

The draft competencies are thought to be feasible, with overall acceptance in the current form. Further research will guide revisions of the competency resource before final distribution.

A Scalable and Effective Course Design for Teaching Competency-Based Euthanasia Communication Skills in Veterinary Curricula

Veterinary staff must be able to navigate end-of-life care with sensitivity and skill to create the best possible outcome for the patient, client, and veterinary team collectively. Despite the clear importance of euthanasia communication and procedural skills in veterinary practice, recent graduates of veterinary programs identified gaps between skills deemed important in clinical practice and skills emphasized in the curriculum. Little time is allocated to euthanasia procedural or communication training across the board in US veterinary programs. Thus, it is of paramount importance to establish intentional and well-designed instruction and assessment of euthanasia communication skills for veterinary trainees. A course on veterinary euthanasia communication skills was designed to emphasize themes and topics essential for a competent veterinarian. Through course evaluations, students expressed the sentiments that this course improved their euthanasia communication skills, that euthanasia communication skills are essential for their careers, and that the course content should be integrated into the core curriculum. This article presents a scaffold for the instruction and assessment of veterinary euthanasia communication skills in alignment with a competency-based veterinary education (CBVE) framework and outlines specific learning interventions used in the course that are scalable and may be extracted and incorporated into existing courses.

Implementation of Competency-Based Medical Education in a Canadian Medical Oncology Training Program: a First Year Retrospective Review

As part of a university-wide initiative, competency-based medical education (CBME) was implemented in the Medical Oncology training program at Queen's University in July 2017. Stages, entrustable professional activities (EPAs), and required training experiences established by the Royal College of Physicians and Surgeons of Canada (RCPSC) national subspecialty committee were adopted. Entrada (Elentra), the electronic portfolio developed at Queen's University, was used for assessment collection. Between July 2017 and December 2018, participating faculty members completed 157 assessments. Eighty-nine percent were EPA assessments with a median of 16 assessments per faculty member (range 1-40). Ninety-five percent of assessments included written "Comments" or "Next steps" with 56% of all assessments including specific or actionable feedback. Discussions between the program director, residents, program administrator, CBME education consultant, and CBME lead led to the identification of 9 lessons learned during implementation. These centered on (1) faculty and resident development and engagement; (2) sharing the work of CBME; (3) collaboration and communication; (4) global assessment; (5) assessment plan challenges; (6) burden of CBME; (7) limitations of e-portfolio; (8) importance of early tracking of resident progress; and (9) culture change. This article describes the experience of the authors and considers strategies that may be helpful to programs implementing CBME in their teaching and learning environment.

Competency-Based Medical Education: Are Canadian Pediatric Anesthesiologists Ready?

Background

With the introduction of Competency-Based Medical Education (CBME), the Canadian Pediatric Anesthesia Society (CPAS) surveyed its members to assess their awareness of and prior experience with CBME concepts and evaluation tools, and identify methods for faculty development of CBME teaching strategies for pediatric anesthesia residents and fellows.

Methods

An online survey was sent to CPAS members. Outcomes included respondents’ previous exposure to CBME and the educational support they had received in anticipation of the curriculum. Questions used multi-item Likert scales and a general feedback question. 

Results

The response rate was 39% (60/155). Eighty-eight percent of respondents spent ≥50% of their time practicing pediatric anesthesia; 78% and 45% spent at least a quarter of their time teaching residents and fellows respectively. Eighty-three percent were familiar with CBME concepts, and 58% were familiar with Milestones, Competencies, and Entrustable Professional Activities (EPAs). However, 64% had not received any formal training and 52% had not used any CBME evaluation tools. Learning preferences included small group discussions (72%), lectures with questions and answers (Q&A) (62%), seminars (50%), and workshops (50%).

Conclusions

Despite widespread awareness of CBME concepts, there is a need to educate Canadian pediatric anesthesiologists regarding CBME evaluation tools. Faculty development support will increase the utilization of these tools in teaching practice.

Feedback and coaching

If used thoughtfully and with intent, feedback and coaching will promote learning and growth as well as personal and professional development in our learners. Feedback is an educational tool as well as a social interaction between learner and supervisor, in the context of a respectful and trusting relationship. It challenges the learner's thinking and supports the learner's growth. Coaching is an educational philosophy dedicated to supporting learners' personal and professional development and growth and supporting them to reach their potential. In clinical education, feedback is most effective when it is explicitly distinguished from summative assessment. Importantly, feedback should be about firsthand observed behaviors (which can be direct or indirect) and not about information which comes from a third party. Learners are more receptive to feedback if it comes from a source that they perceive as credible, and with whom they have developed rapport. The coaching relationship between learner and supervisor should also be built on mutual trust and respect. Coaching can be provided in the moment (feedback on everyday clinical activities that leads to performance improvement, even with short interaction with a supervisor) and over time (a longer term relationship with a supervisor in which there is reflection on the learner's development and co-creation of new learning goals). Feedback and coaching are most valuable when the learner and teacher exhibit a growth mindset. At the organizational level, it is important that both the structures and training are in place to ensure a culture of effective feedback and coaching in the clinical workplace.Conclusions: Having a thoughtful and intentional approach to feedback and coaching with learners, as well as applying evidence-based principles, will not only contribute in a significant way to their developmental progression, but will also provide them with the tools they need to have the best chance of achieving competence throughout their training. What is Known: • Feedback and coaching are key to advancing the developmental progression of trainees as they work towards achieving competence. • Feedback is not a one-way delivery of specific information from supervisor to trainee, but rather a social interaction between two individuals in which trust and respect play a key role. • Provision of effective feedback may be hampered by confusing formative (supporting trainee learning and development) and summative (the judgment that is made about a trainee's level of competence) purposes. What is New: • Approaches to both the provision of feedback/coaching and the assessment of competence must be developed in parallel to ensure success in clinical training programs. • Faculty development is essential to provide clinical teachers with the skills to provide effective feedback and coaching. • Coaching's effectiveness relies on nurturing strong trainee-supervisor relationships, ensuring high-quality feedback, nourishing a growth mindset, and encouraging an institutional culture that embraces feedback and coaching.

Implementing competency-based medical education curriculum in undergraduate psychiatric training in India: Opportunities and challenges

Medical education in India is undergoing a landmark transformation under the National Medical Commission implementing competency-based medical education (CBME). The CBME approach intends to ensure that medical graduates acquire the competencies needed to fulfill the health needs of the patient and society. This outcome-based approach shifts the focus from the traditional knowledge-based training to skill-based training valued on attitude, ethics, and communication (AETCOM) competencies. CBME thus aims to create medical professionals capable of providing holistic care with compassion and excellence embracing the global trends. The opportunity posed by the CBME should be utilized to sensitize and create interest among the learners about the science and scope of psychiatry. However, there are many challenges in the successful implementation of CBME, which have to be identified and addressed on time for serving the purpose. A basic tenet in CBME is to continue training until the desired competencies are achieved; in other words, to de-emphasize time-based learning. Moreover, the current COVID 19 pandemic is posing a significant influence on the execution of CBME implemented in August 2019. Online platforms could have several advantages in assisting the implementation of CBME; they provide an alternative to continue teaching-learning and assessment during these times and allow learners with the flexibility to learn at their own pace. In this article, we discuss the opportunities, including digital platforms and challenges to be overcame as well as the need for training the faculty toward assimilating the curriculum in the undergraduate psychiatric training.

Acceptance and validity of the methods used to implement a competency based medical education programme in an Intensive Care Department of a teaching referral center

OBJECTIVES

1. To determine the satisfaction of tutors and residents with a specific methodology used to implement CoBaTrICE. 2. To determine the reliability and validity of the global rating scales designed ad hoc to assess the performance of the residents for training purposes.

DESIGN

Prospective cohort study.

PARTICIPANTS

All the residents and tutors of the ICU Department of the Hospital Universitario y Politécnico la Fe de Valencia.

INTERVENTION

CoBaTrICE implementation started in March 2016, it was based on: (1) Training the tutors in feedback techniques; (2) Performing multiple objective and structured work based assessments to achieve the competences of the program; and (3) The use of an electronic portfolio to promote learning reflection and to collect the evidence that learning was taking place.

METHODS

The acceptance of CoBaTrICE was explored through a satisfaction survey conducted after 9 months of implementation of the training program. The 15 residents and 5 tutors of the ICU Department were asked about the methodology of the formative assessments, the quality of the feedback, self-learning regulation and the electronic portfolio usefulness. The validity of the global rating scales was assessed through the tests alfa de Cronbach, reliability and generalizability indexes, and intraclass correlation coefficient.

RESULTS

The implementation of CoBaTrICE was satisfactory in all the dimensions studied. The global rating scales used for formative purposes showed reliability and validity.

CONCLUSIONS

The methodology used to implement CoBaTrICE was highly valued by tutors and residents. The global rating scales used for formative purposes showed reliability and validity.

Leading from Behind: An Educational Intervention to Address Faculty and Learner Preparedness for Competence By Design in Psychiatry

Purpose

Residency training programs across Canada are beginning to implement the Royal College of Physicians and Surgeons of Canada’s new Competence By Design (CBD) framework in medical education. The objective of the current research was to assess faculty members’ and learners’ understanding of, and preparedness for, the national shift to CBD in psychiatry before and after an educational intervention.

Methods

The current research implemented a pre-test/post-test design to investigate faculty members’ and learners’ perceptions and attitudes towards competency-based medical education (CBME) and CBD before and after a one-hour educational session delivered by an expert on CBME.

Results

Of the 104 session attendees, 83 (79.8%) completed the pre-survey and 80 (76.9%) completed the post-survey. Both groups reported a moderate level of baseline knowledge of CBME and CBD. Knowledge of CBME improved significantly for both faculty members (p = 0.03) and learners (p < 0.01) after the education session; however, only learners showed a significant increase in knowledge of the CBD framework following the education session (p < 0.01). Further, only learners demonstrated a significant increase in perceived preparedness for CBD following the session (p = 0.02).

Conclusion

Overall, a brief, one-hour education session was at least somewhat effective at improving knowledge and preparedness for psychiatry’s transition to CBD. In order to facilitate the transition to CBD and to assist in the rollout of future policy changes, psychiatry departments should provide both faculty members and learners with educational sessions and resources prior to the policy implementation.

Developing a dashboard for faculty development in competency-based training programs: a design-based research project

BACKGROUND

Canadian specialist residency training programs are implementing a form of competency-based medical education (CBME) that requires frequent assessments of entrustable professional activities (EPAs). Faculty struggle to provide helpful feedback and assign appropriate entrustment scores. CBME faculty development initiatives rarely incorporate teaching metrics. Dashboards could be used to visualize faculty assessment data to support faculty development.

METHODS

Using a design-based research process, we identified faculty development needs related to CBME assessments and designed a dashboard containing elements (data, analytics, and visualizations) meeting these needs. Data was collected within the emergency medicine residency program at the University of Saskatchewan through interviews with program leaders, faculty development experts, and faculty participating in development sessions. Two investigators thematically analyzed interview transcripts to identify faculty needs that were audited by a third investigator. The needs were described using representative quotes and the dashboard elements designed to address them.

RESULTS

Between July 1, 2019 and December 11, 2020 we conducted 15 interviews with nine participants (two program leaders, three faculty development experts, and four faculty members). Three needs emerged as themes from the analysis: analysis of assessments, contextualization of assessments, and accessible reporting. We addressed these needs by designing an accessible dashboard to present contextualized quantitative and narrative assessment data for each faculty member.

CONCLUSIONS

We identified faculty development needs related to EPA assessments and designed dashboard elements to meet them. The resulting dashboard was used for faculty development sessions. This work will inform the development of CBME assessment dashboards for faculty.

Implementing Systematic Faculty Development to Support an EPA-Based Program of Assessment: Strategies, Outcomes, and Lessons Learned

Problem: Development of a novel, competency-based program of assessment requires creation of a plan to measure the processes that enable successful implementation. The principles of implementation science outline the importance of considering key drivers that support and sustain transformative change within an educational program. The introduction of Entrustable Professional Activities (EPAs) as a framework for assessment has underscored the need to create a structured plan to prepare assessors to engage in a new paradigm of assessment. Although approaches to rater training for workplace-based assessments have been described, specific strategies to prepare assessors to apply standards related to the level of supervision a student needs have not been documented. Intervention: We describe our systematic approach to prepare assessors, faculty and postgraduate trainees, to complete EPA assessments for medical students during the clerkship phase of our curriculum. This institution-wide program is designed to build assessors' skills in direct observation of learners during authentic patient encounters. Assessors apply new knowledge and practice skills in using established performance expectations to determine the level of supervision a learner needs to perform clinical tasks. Assessors also learn to provide feedback and narrative comments to coach students and promote their ongoing clinical development. Data visualizations for assessors facilitate reinforcement of the tenets learned during training. Collaborative learning and peer feedback during faculty development sessions promote the formation of a community of practice among assessors. Context: Faculty development for assessors was implemented in advance of implementation of the EPA program. Assessors in the program include residents/fellows who work closely with students, faculty with discipline-specific expertise and a group of experienced clinicians who were selected to serve as experts in competency-based EPA assessments, the Master Assessors. Training focused on creating a shared understanding about the application of criteria used to evaluate student performance. EPA assessments based on the AAMC's Core Entrustable Professional Activities for Entering Residency, were completed in nine core clerkships. EPA assessments included a supervision rating based on a modified scale for use in undergraduate medical education. Impact: Data from EPA assessments completed during the first year of the program were analyzed to evaluate the effectiveness of the faculty development activities implemented to prepare assessors to consistently apply standards for assessment. A systematic approach to training and attention to critical drivers that enabled institution-wide implementation, led to consistency in the supervision rating for students' first EPA assessment completed by any type of assessor, ratings by assessors done within a specific clinical context, and ratings assigned by a group of specific assessors across clinical settings. Lessons learned: A systematic approach to faculty development with a willingness to be flexible and reach potential participants using existing infrastructure, can facilitate assessors' engagement in a new culture of assessment. Interaction among participants during training sessions not only promotes learning but also contributes to community building. A leadership group responsible to oversee faculty development can ensure that the needs of stakeholders are addressed and that a change in assessment culture is sustained.

The critical role of infrastructure and organizational culture in implementing competency-based education and individualized pathways in undergraduate medical education

In 2010, several key works in medical education predicted the changes necessary to train modern physicians to meet current and future challenges in health care, including the standardization of learning outcomes paired with individualized learning processes. The reframing of a medical expert as a flexible, adaptive team member and change agent, effective within a larger system and responsive to the community's needs, requires a new approach to education: competency-based medical education (CBME). CBME is an outcomes-based developmental approach to ensuring each trainee's readiness to advance through stages of training and continue to grow in unsupervised practice. Implementation of CBME with fidelity is a complex and challenging endeavor, demanding a fundamental shift in organizational culture and investment in appropriate infrastructure. This paper outlines how member schools of the American Medical Association Accelerating Change in Medical Education Consortium developed and implemented CBME, including common challenges and successes. Critical supporting factors include adoption of the master adaptive learner construct, longitudinal views of learner development, coaching, and a supportive learning environment.

Teaching residents how to break bad news: piloting a resident-led curriculum and feedback task force as a proof-of-concept study

Background

Breaking bad news (BBN) is a critically important skill set for residents. Limited formal supervision and unpredictable timing of bad news delivery serve as barriers to the exchange of meaningful feedback.

Purpose of study

The goal of this educational innovation was to improve internal medicine residents’ communication skills during challenging BBN encounters. A formal BBN training programme and innovative on-demand task force were part of this two-phase project.

Study design

Internal medicine residents at a large academic medical centre participated in an interactive workshop focused on BBN. Workshop survey results served as a needs assessment for the development of a novel resident-led BBN task force. The task force was created to provide observations at the bedside and feedback after BBN encounters. Training of task force members incorporated video triggers and a feedback checklist. Inter-rater reliability was analysed prior to field testing, which provided data on real-world implementation challenges.

Results

148 residents were trained during the 2-hour communications skills workshop. Based on survey results, 73% (108 of 148) of the residents indicated enhanced confidence in BBN after participation. Field testing of the task force on a hospital ward revealed potential workflow barriers for residents requesting observations and prompted troubleshooting. Solutions were implemented based on field testing results.

Conclusions

A trainee-led BBN task force and communication skills workshop is offered as an innovative model for improving residents’ interpersonal and communication skills in BBN. We believe the model is both sustainable and reproducible. Lessons learnt are offered to aid in implementation in other settings.

Development and implementation of a novel, mandatory competency-based medical education simulation program for pediatric emergency medicine faculty

Background

Maintaining acute care physician competence is critically important. Current maintenance of certification (MOC) programs has started to incorporate simulation-based education (SBE). However, competency expectations have not been defined. This article describes the development of a mandatory annual SBE, competency-based simulation program for technical and resuscitation skills for pediatric emergency medicine (PEM) physicians.

Methods

The competency-based medical education (CBME) program was introduced in 2016. Procedural skill requirements were based on a needs assessment derived from Royal College PEM training guidelines. Resuscitation scenarios were modified versions of pre-existing in-situ mock codes or critical incident cases. All full-time faculty were required to participate annually in both sessions. Delivery of educational content included a flipped classroom website, deliberate practice, and stop-pause debriefing. All stations required competency checklists and global rating scales.

Results

Between 2016 and 2018, 40 physicians and 48 registered nurses attended these courses. Overall course evaluations in 2018 were 4.92/5 and 4.93/5. Barriers to implementation include the need for many simulation education experts, time commitment, and clinical scheduling during course events.

Conclusion

We have developed a mandatory simulation-based, technical, and resuscitation CBME program for PEM faculty. This simulation-based CBME program could be adapted to other acute care disciplines. Further research is required to determine if these skills are enhanced both in a simulated and real environment and if there is an impact on patient outcomes.

Supplementary Information

The online version contains supplementary material available at 10.1186/s41077-021-00170-4.

"PLEASE Complete Your Evaluations!" Strategies to Engage Faculty in Competency-Based Assessments

Competency-based assessments (CBAs) have gained traction in graduate medical education and inform important learner outcomes through the continuum of medical training. Active participation in new CBAs presents challenges to faculty working in a busy clinical environment. As such, the implementation of new CBAs can be approached with intention to foster acceptance and engagement with new evaluations. This paper describes strategies utilized to implement CBAs among clinician educators during a national assessment pilot. Our methods are grounded in educational, psychological, business, ecological, communication, and information technology theory. Our primary interventions included creating a multilevel vision, engaging a dedicated work group, incorporating quality improvement methodology, and integrating technology to successfully implement the assessments. These practical and effective interventions may also be applied to the implementation of other educational innovations.

Living in a World of Change: Bridging the Gap From Competency-Based Medical Education Theory to Practice in Canada

Within graduate medical education, many educators are experiencing a climate of significant change. One transformation, competency-based medical education (CBME), is occurring simultaneously across much of the world, and implementation will require navigating numerous tensions and paradoxes. Successful transformation requires many types of power and is most likely to happen when the medical education community of professionals is engaged in designing, experimenting, acting, and sensemaking together.In this complex climate, the craft of change facilitators and community leaders is needed more than ever. National top-down policies and structures, while important, are not sufficient. The operationalization of new advances is best done when local leaders are afforded room to shape their local context. An evidence-based approach to thinking about the transformative change associated with CBME needs to be adopted. In this age of entrustment, 3 priorities are paramount: (1) engage, entrust, and empower professionals with increasing shared ownership of the innovation; (2) better prepare education professionals in leadership and transformational change techniques in the complex system of medical education; and (3) leverage the wider community of practice to maximize local CBME customization. These recommendations, although based largely on the Canadian experience, are intended to inform CBME transformation in any context.

Collaborative Development of a Shared Framework for Competency-Based Veterinary Education

Competency-based medical education is an educational innovation implemented in health professions worldwide as a means to ensure graduates meet patient and societal needs. The focus on student-centered education and programmatic outcomes offers a series of benefits to learners, institutions and society. However, efforts to establish a shared, comprehensive competency-based framework in veterinary education have lagged. This article reports on the development and outcome of a competency-based veterinary education (CBVE) framework created through multi-institutional collaboration with international input from veterinary educators and veterinary educational leaders. The CBVE Framework is designed to reflect the competencies expected of new graduates from member institutions of the Association of American Veterinary Medical Colleges (AAVMC). The CBVE Framework consists of nine domains of competence and 32 competencies, each supplemented with illustrative sub-competencies to guide veterinary schools in implementing competency-based education in their local context. The nine domains of competence are: clinical reasoning and decision-making; individual animal care and management; animal population care and management; public health; communication; collaboration; professionalism and professional identity; financial and practice management; and scholarship. Developed through diverse input to facilitate broad adoption, the CBVE Framework provides the foundation for competency-based curricula and outcomes assessment in veterinary education internationally. We believe that other groups seeking to design a collective product for broad adoption might find useful the methods used to develop the CBVE Framework, including establishing expertise diversity within a small-to-medium size working group, soliciting progressive input and feedback from stakeholders, and engaging in consensus building and critical reflection throughout the development process.

Taking stock of what is known about faculty development in competency-based medical education: A scoping review paper

Purpose: The primary objective was to inventory what is currently known about faculty development (FD) for competency-based medical educations (CBME) and identify gaps in the literature.Methods: A scoping review methodology was employed. Inclusion criteria for article selection were established with two reviewers completing a full-text analysis. Quality checks were included, along with iterative consultation on data collection and consensus decision making via a grounded theory approach.Results: The review identified 19 articles published between 2009 and 2018. Most articles (N = 15) offered suggestions as to what should happen with FD in CBME, but few (N = 4) adopted an experimental design. Six main themes were identified with three main features of FD noted across themes: (1) The importance of direct and timely feedback to faculty members on their teaching and assessment skills. (2) The role of establishing shared mental models for CBME curricula. (3) That FD is thought of longitudinally, not as a one-time bolus.Conclusion: This work illustrates that there is limited, high quality research in FD for CBME. Future FD activities should consider employing a longitudinal and multi-modal program format that includes feedback for the faculty participants on their teaching and assessments skills, including the development of faculty coaching skills.

Evaluation of a faculty development workshop aimed at development and implementation of a competency-based curriculum for medical undergraduates

Introduction

Faculty development to implement competency-based medical education (CBME) is urgently needed as the Medical Council of India has implemented the competency-based curriculum this year onwards.

Objectives

To evaluate a 2-day faculty development workshop in terms of: (a) increase in knowledge about CBME terminology and concepts, (b) self-reported capacity to develop and implement a competency-based module in their respective disciplines, and (c) satisfaction of the participants.

Methodology

A single arm interventional study using mixed methods was carried out in which faculty members were purposively identified and requested to volunteer for a two-day faculty development workshop on the development and implementation of CBME. The workshop was evaluated (open-ended and Likert scored items) by the participants for self-reported gain in knowledge, gain in their confidence to develop and implement CBME, and level of satisfaction with respect to the components of the workshop. Quantitative data was analyzed by Wilcoxon signed rank test and Kruskal-Wallis test. Qualitative data was analyzed by doing content analysis and emerging themes have been presented.

Results

Eleven faculty members attended a capacity building workshop for developing and implementing a competency-based curriculum for medical undergraduates. There was a significant improvement in their self-reported knowledge and attitude regarding the competency-based curriculum. New concepts learned fell into four domains: modification of the existing curriculum toward competency-based curriculum, knowledge of CBME, teaching-learning and assessment methods, and beneficial to the students. The participants were highly satisfied with the workshop in its current form.

Conclusion

A two-day faculty development workshop can increase the knowledge and understanding of competency-based medical education and can be an important first step in the journey to more specialized training.

Improved cognitive apprenticeship clinical teaching after a faculty development program

BACKGROUND

While it is well known that the cognitive apprenticeship is an effective workplace-based teaching approach for clinical teachers, the effects of faculty development (FD) have not been analyzed from that perspective. The purpose of this study was to investigate self-assessment by clinical teachers of their educational perceptions and behaviors after a FD program using the cognitive apprenticeship model.

METHODS

Board-certified pediatricians who participated in a 3-day FD program on practical clinical teaching were asked to complete questionnaires. Fifty participants completed two questionnaires prior to and 3 and 6 months after the FD program: the first was on the participants' general perceptions and behaviors in relation to their own clinical education and the second was a self-assessment using the Maastricht Clinical Teaching Questionnaire (MCTQ) that was developed based on the cognitive apprenticeship model.

RESULTS

The general survey demonstrated that 78% of the participants experienced positive changes in their educational perceptions 6 months after FD. Self-assessment using the MCTQ showed that the scores in the categories of "articulation," "exploration," and "safe learning environment" remained significantly improved 6 months after the FD program.

CONCLUSIONS

The participants' self-perceived improvement in behaviors was sustainable for 6 months after participation the FD program. The results of the MCTQ show that through their experiences in the FD program, the participants seemingly transformed their clinical teaching to become interactive facilitators, encouraging self-directed learning. Our results also suggest that the MCTQ can be used for self-assessment of clinical teachers and to enhance the effectiveness of the FD program.

Factors to be considered in designing a faculty development program for medical education: local experience from the Western region of Saudi Arabia

Background

Among the different aims of medical education, the provision of society with skilled, professional, and knowledgeable healthcare workers who maintain and develop their expertise over a lifetime career is important. The achievement of this goal is linked with the professional development of both faculty members and healthcare workers. This study aims to measure the perception of faculty members regarding their views about the goals of faculty development programs, practices and activities, and factors that determine their achievement.

Methods

A cross-sectional survey was conducted in multiple universities in the Western region of Saudi Arabia. The participants were given a pre-designed self-administered questionnaire generated from literature. The survey questionnaire consisted of three sections that were designed to assess the faculty members’ perception on the faculty development program.

Results

A total of 210 faculty members participated in the study. The most important perceived goal was to motivate teachers to become better teachers. The most important perceived practice was establishing a positive climate for teaching and learning. The most important perceived factor was skilled and dedicated staff support.

Conclusion

The results of this study demonstrate that faculty members have positive perceptions regarding all aspects of faculty development programs. This study will raise awareness regarding the importance of faculty development programs in sustaining educational vitality. We recommend the implementation and maintenance of comprehensive faculty development programs in Saudi universities.

Nationwide introduction of a new competency framework for undergraduate medical curricula: a collaborative approach

Switzerland recently introduced PROFILES, a revised version of its national outcomes reference framework for the undergraduate medical curriculum. PROFILES is based on a set of competencies adapted from the CanMEDS framework and nine entrustable professional activities (EPAs) that students have to be able to perform autonomously in the context of a predefined list of clinical situations. The nationwide implementation of such a competency- and EPA-based approach to medical education is a complex process that represents an important change to the organisation of undergraduate training in the various medical schools. At the same time, the concepts underlying PROFILES also have to be reflected at the level of the Federal Licencing Examination (FLE) and the national accreditation process. The vice-deans for education mandated a Swiss Working Group for PROFILES Implementation (SWGPI) to elaborate a guide presenting the principles and best practices based on the current scientific literature, to ensure the coherence between the future developments of the medical curricula and the evolution of the FLE, and to propose a coordinated research agenda to evaluate the implementation process. On the basis of the literature and analysis of our national context, we determined the key elements important for a successful implementation. They can be grouped into several areas including curricular design and governance, the assessment system and entrustment process, faculty development and change management. We also identified two dimensions that will be of particular importance to create synergies and facilitate exchange between the medical schools: a systematic approach to curriculum mapping and the longitudinal integration of an e-portfolio to support the student learning process. The nationwide collaborative approach to define strategies and conditions for the implementation of a new reference framework has allowed to develop a shared understanding of the implications of PROFILES, to promote the establishment of Swiss mapping and e-portfolio communities, and to establish the conditions necessary for ensuring the continuous alignment of the FLE with the evolving medical curricula.

Effectiveness and Needs Assessment of Faculty Development Programme for Medical Education: Experience from Saudi Arabia

Objectives

Faculty members are the most important resource in any institution of higher education as medical education has been, and continues to be, a priority for medical colleges in Saudi Arabia. This study aimed to assess faculty members' perceptions of faculty development programmes (FDPs) in supporting important goals in medical education. In addition, this study aimed to assess faculty members' perceived needs.

Methods

This cross-sectional study was conducted between August 2016 and August 2017 and involved participants from six universities in Saudi Arabia's Western Province. The survey consisted of 31 items designed to assess FDP effectiveness and 49 items designed to assess needs in FDPs.

Results

A total of 210 faculty members participated in the study (response rate = 52.5%) and identified 49 needs. Faculty members perceived personal improvement in delivering medical education and the provision of greater educational involvement as the most effective considerations in an FDP. The respondents considered 13 needs to be of utmost importance; the remaining were considered important.

Conclusion

This study assessed and identified faculty needs and important skills to consider when establishing an FDP. Furthermore, it provided information addressing the needs of, or gaps between, current and desired conditions in medical education in Saudi Arabia. The study also identified the most important elements (i.e. personal improvement) of faculty-perceived effectiveness for a successful FDP in medical education.

Objective Assessment of the Entrustable Professional Activity Handover in Undergraduate and Postgraduate Surgical Learners

OBJECTIVE

This study used a virtual patient simulation (VPS) to quantifiably and objectively assess undergraduate (UG) to postgraduate (PG) medical learners' acquisition of the entrustable professional activity (EPA) "handover," focusing particularly on the transition to residency. This EPA is critical because it is part of a core competency for UG and PG training in both the United States and Canada, and is essential for patient safety and comprehensive professional communication.

DESIGN

Data were collected from 3 separate groups of participants: 2 UG cohorts from an earlier study, as well as a PG cohort at the beginning of residency. All participants completed the same trauma VPS, which required a free text summary statement that was used as a surrogate for an oral handover. These were collected and scored independently, using previously developed validated rubrics, one procedural and the second semantic.

SETTING

All study participants were from one site. The VPS case was completed online.

PARTICIPANTS

Two different UG groups, one designated junior (N = 52), was studied at the beginning of their clerkship year, a second group, designated senior (N = 30), was studied at the end of their clerkship year. These groups were compared to a third group of PG learners (N = 31) during the initial 2 weeks of their residency. Informed consent was obtained from all participants.

RESULTS

A procedural rubric assessed learners' cognitive knowledge of trauma care-management. A semantic rubric assessed their use of the professional language necessary for a safe and succinct clinical handover communication. An Analysis of Variance comparing scores on the procedural rubric was highly significant with Tukey LSD tests indicating that all 3 groups were significantly different. Students increased their scores on the procedural rubric at each stage of their training. A parallel Analysis of Variance comparing students' scores on the semantic rubric revealed no significant increase in scores, indicating that students did not improve in their capacity to communicate professionally as they progressed through their training.

CONCLUSIONS

Taken together, these results demonstrate that training was successful in teaching cognitive-based procedures, but not effective in teaching professional communication, which is critical to the EPA handover. Greater emphasis needs to be placed on ensuring the acquisition of professional communication skills throughout the continuum of UG and PG clinical activities. Faculty development should serve as a support to assist medical educators to address this requirement. These results also demonstrate that VPS with associated objective and validated rubrics can be used as an assessment methodology to quantifiably measure learner performance with respect to the EPA handover. A similar strategy should be considered across the UG and PG continuum for other EPAs and could form the nexus for further research.

Competency-Based Medical Education implementation: Are we transforming the culture of assessment?

Purpose: Adopting CBME is challenging in medicine. It mandates a change in processes and approach, ultimately a change in institutional culture with stakeholders ideally embracing and valuing the new processes. Adopting the transformational change model, this study describes the shift in assessment culture by Academic Advisors (AAs) and preceptors over three years of CBME implementation in one Department of Family Medicine. Methods: A qualitative grounded theory method was used for this two-part study. Interviews were conducted with 12 AAs in 2013 and nine AAs in 2016 using similar interview questions. Data were analyzed through a constant comparative method. Results: Three overarching themes emerged from the data: (1) specific identified shifts in assessment culture, (2) factors supporting the shifts in culture, and (3) outcomes related to the culture shift. Conclusions: In both parts of the study, participants noted that assessment took more time and effort. In Part 2, however, the effort was mitigated by a sense of value for all stakeholders. With support from the mandate of regulatory bodies, local leadership, department, faculty development and an electronic platform, a cultural transformation occurred in assessment that enhanced learning and teaching, use of embedded standards for performance decisions, and tracking and documentation performance.

Faculty perspectives on the transition to competency-based medical education in anesthesia

PURPOSE

Canadian residency programs are transitioning from time-based to competency-based medical education (CBME). The anesthesia department at Dalhousie University enrolled its first CBME cohort in 2016, one year prior to national anesthesia rollout. Early implementation allowed a unique opportunity to examine faculty anesthesiologists' experiences with the transition.

METHODS

Using Rogers' Diffusion of Innovations (DOI) theory, we conducted a qualitative interview study. In-depth interviews were held with faculty members (n = 12) at varying stages of innovation adoption (e.g., innovators/early adopters, early/late majority, and laggards) at two time points: onset of CBME and one year later. Interview data were analyzed based on the DOI promoting factors: relative advantage, compatibility, complexity, trialability, and observability.

RESULTS

Relative advantage: Early adopters believed CBME had benefits over the traditional curriculum, while laggards viewed the change as an unproven paradigm shift. CBME was compatible with the values of early adopters, who appreciated resident accountability for learning. Trialability, the degree to which an intervention can be trialed and modified, arose with the early/late majority group, who described an organic process of adaptation over the year. All groups mentioned the need for observable results. Innovators and early adopters were confident CBME would improve learner experiences. Early/late majority noted expedited skill acquisition and improved quality of feedback. Laggards believed observable results would take many years to emerge, if ever. The early/late majority group showed the most progress toward adoption over the study time period, moving from skeptical optimism to active investment.

CONCLUSION

Targeted interventions for faculty uptake should emphasize the trialability and observable results achieved over time. These efforts may have the greatest impact in the early/late majority group.

Considerations that will determine if competency-based assessment is a sustainable innovation

Educational assessment for the health professions has seen a major attempt to introduce competency based frameworks. As high level policy developments, the changes were intended to improve outcomes by supporting learning and skills development. However, we argue that previous experiences with major innovations in assessment offer an important road map for developing and refining assessment innovations, including careful piloting and analyses of their measurement qualities and impacts. Based on the literature, numerous assessment workshops, personal interactions with potential users, and our 40 years of experience in implementing assessment change, we lament the lack of a coordinated approach to clarify and improve measurement qualities and functionality of competency based assessment (CBA). To address this worrisome situation, we offer two roadmaps to guide CBA's further development. Initially, reframe and address CBA as a measurement development opportunity. Secondly, using a roadmap adapted from the management literature on sustainable innovation, the medical assessment community needs to initiate an integrated plan to implement CBA as a sustainable innovation within existing educational programs and self-regulatory enterprises. Further examples of down-stream opportunities to refocus CBA at the implementation level within faculties and within the regulatory framework of the profession are offered. In closing, we challenge the broader assessment community in medicine to step forward and own the challenge and opportunities to reframe CBA as an innovation to improve the quality of the clinical educational experience. The goal is to optimize assessment in health education and ultimately improve the public's health.

On-site Observational Learning in Faculty Development: Impact of an International Program on Clinical Teaching in Medicine

INTRODUCTION

Local faculty development programs may not suffice for physicians to learn about global standards in clinical teaching. This report describes the evaluation of a 1-week international faculty development program in Canada, incorporating on-site observations of clinical teaching with guided reflection, for Japanese clinical teachers. Participants' perceptions of learning processes and outcomes are described.

METHODS

Twenty-nine Japanese clinical teachers from 9 different teaching hospitals located in the Gifu province participated in this program from 2014 to 2017. The program evaluation consisted of end-of-program reflection reports and semistructured interviews conducted 3 months after the visit, which were thematically analyzed.

RESULTS

Three themes related to learning processes emerged: the benefits of observing clinical teaching in a familiar specialty but unfamiliar context; the value of reflection in linking previous experiences with observations; and the role of group interaction and verbalization in creating a sense of community. Three additional themes related to learning outcomes emerged: enhanced knowledge about teaching, with a renewed emphasis on the value of learner respect and trust; increased confidence in teaching competence; and a sense of frustration related to difficulty in changing the status quo.

DISCUSSION

An international faculty development program incorporating on-site observational learning in hospital settings with guided reflections can be effective in learning about clinical teaching; follow-up activities in the home country are also recommended. The key elements of this program, including international experiences, on-site observational learning, and guided reflections, should be considered more frequently in faculty development programs in medicine.

Coaching by design: exploring a new approach to faculty development in a competency-based medical education curriculum

As curricula move from a time-based system to a competency-based medical education system, faculty development will be required. Faculty will be asked to engage in the observation, assessment and feedback of tasks in the form of educational coaching. Faculty development in coaching is necessary, as the processes and tools for coaching learners toward competence are evolving with a novel assessment system. Here, we provide a scoping review of coaching in medical education. Techniques and content that could be included in the curricular design of faculty development programming for coaching (faculty as coach) are discussed based on current educational theory. A novel model of coaching for faculty (faculty as coachee) has been developed and is described by the authors. Its use is proposed for continuing professional development.

Resident and attending perceptions of direct observation in internal medicine: a qualitative study

OBJECTIVES

Direct observation is the foundation of assessment and learning in competency-based medical education (CBME). Despite its importance, there is significant uncertainty about how to effectively implement frequent and high-quality direct observation. This is particularly true in specialties where observation of non-procedural skills is highly valued and presents unique challenges. It is therefore important to understand perceptions of direct observation to ensure successful acceptance and implementation. In this study, we explored perceptions of direct observation in internal medicine.

METHODS

We interviewed internal medicine attending physicians (n = 9) and residents (n = 8) at the University of Toronto, purposively sampled for diversity. Using a constructivist grounded theory approach, constant comparative analysis was performed to develop a framework to understand perceptions of direct observation on the clinical teaching units.

RESULTS

Participants articulated a narrow perception of what constitutes direct observation, in contrast to their own descriptions of skills that were observed. This resulted in the perception that certain valuable skills that participants felt were routinely observed were nonetheless not 'directly observable', such as clinical reasoning, observed through case presentations and patient care discussions. Differentiating direct observation from informal observation led to overestimation of the time and resource requirements needed to enhance direct observation, which contributed to scepticism and lack of engagement related to CBME implementation.

CONCLUSIONS

In an internal medicine training programme, perceptions of what constitutes direct observation can lead to under-recognition and hinder acceptance in workplace-based assessment and learning. Our results suggest a reframing of 'direct observation' for residents and attending physicians, by explicitly identifying desired skills in non-procedurally-based specialties. These findings may help CBME-based training programmes improve the process of direct observation, leading to enhanced assessment and learning.

Exploring Faculty Approaches to Feedback in the Simulated Setting: Are They Evidence Informed?

INTRODUCTION

Feedback in clinical education and after simulated experiences facilitates learning. Although evidence-based guidelines for feedback exist, faculty experience challenges in applying the guidelines. We set out to explore how faculty approach feedback and how these approaches align with current recommendations.

METHODS

There is strong evidence for the following four components of feedback: feedback as a social interaction, tailoring content, providing specific descriptions of performance, and identifying actionable items. Faculty preceptors participated in feedback simulations followed by debriefing. The simulations were video recorded, transcribed, and analyzed qualitatively using template analysis to examine faculty approaches to feedback relative to evidence-informed recommendations.

RESULTS

Recorded encounters involving 18 faculty and 11 facilitators yielded 111 videos. There was variability in the extent to which feedback approaches aligned with recommended practices. Faculty behaviors aligned with recommendations included a conversational approach, flexibly adapting feedback techniques to resident context, offering rich descriptions of observations with specific examples and concrete suggestions, achieving a shared understanding of strengths and gaps early on to allow sufficient time for problem-solving, and establishing a plan for ongoing development. Behaviors misaligned with guidelines included prioritizing the task of feedback over the relationship, lack of flexibility in techniques applied, using generic questions that did not explore residents' experiences, and ending with a vague plan for improvement.

CONCLUSIONS

Faculty demonstrate variability in feedback skills in relation to recommended practices. Simulated feedback experiences may offer a safe environment for faculty to further develop the skills needed to help residents progress within competency-based medical education.

Faculty Development Revisited: A Systems-Based View of Stakeholder Development to Meet the Demands of Entrustable Professional Activity Implementation

In 2015, the Association of American Medical Colleges implemented an interinstitutional pilot of 13 core entrustable professional activities (EPAs) for entering residency, activities that entering residents should be expected to perform with indirect supervision. The pilot included a concept group on faculty development; this group previously offered a shared mental model focused on the development of faculty who devote their efforts to clinical teaching and assessment for learning and entrustment decision making. In this article, the authors draw from the literature of competency-based education to propose what is needed in overall approaches to faculty development to prepare institutions for undergraduate EPA implementation.Taking a systems-based view that defines the necessary tasks of EPA implementation, the authors move beyond the variably used term "faculty" and enumerate a comprehensive list of institutional stakeholders who can meaningfully support and/or engage in the relationships and organizational processes required for EPA learning and assessment. They consider each group's responsibilities and development needs according to five domains delineated by Steinert: teaching improvement, leadership and management, research-building capacity, academic career building, and organizational change.The authors argue that the EPA framework addresses barriers posed with the use of a competency-based framework. By facilitating the communication required for organizational change, enabling valid assessment with comprehensive yet feasible levels of faculty development, and incorporating all relevant data on student professional behavior into summative assessment decisions, EPAs may offer a clearer path toward the goal of competency-based education.

[Educational assessment of the first computerized national ranking exam in France in 2016: Opportunities for improvement]

In June 2016, 8124 medical students in their sixth year of graduation passed the first computerized national ranking exam (CNRE) in France after which they will have to choose what medical specialty they will be practicing all their life. We conducted the first educational assessment of this CNRE according to two criteria: the relevance of the questions and the cognitive domain mainly required to answer these questions. We propose two improvements for the future CNRE: promote student reasoning in the multiple choices questions, reduce to 10 the number of multiple choice questions in the progressive clinical cases and increase by 9 their total number (from 18 to 27), and use a majority of mini-clinical cases for isolated multiple choice questions in order to focus students on reasoning instead of simple knowledge restitution.