Is transferring an educational innovation actually a process of transformation?

Insights From the Transition to Competency-Based Medical Education in Neurology Programs

Canadian neurology residency programs recently transitioned to Competency-Based Medical Education (CBME). Iterative evaluation is required to optimize CBME implementation. This study aimed to examine the variability and challenges in uptake of CBME in neurology residency programs and identify its benefits and pitfalls. Neurology residents and faculty participated in respective anonymous surveys. Common barriers to uptake were identified from both perspectives. Orientation to CBME was adequate, but workload was increased and contributed to burnout for faculty and residents. It is premature to draw conclusions regarding benefits of CBME. Future research considerations include standardization of entrustment scales and reduction of stakeholder burden.

Exploring stakeholder perspectives regarding the implementation of competency-based medical education: a qualitative descriptive study

Introduction

Competency-based medical education (CBME) offers perceived advantages and benefits for postgraduate medical education (PGME) and the training of competent physicians. The purpose of our study was to gain insights from those involved in implementing CBME in two residency programs to inform ongoing implementation practices.

Methods

We conducted a qualitative descriptive study to explore the perspectives of multiple stakeholders involved in the implementation of CBME in two residency programs (the first cohort) to launch the Royal College's Competence by Design model at one Canadian university. Semi-structured interviews were conducted with 17 participants across six stakeholder groups including residents, department chairs, program directors, faculty, medical educators, and program administrators. Data collection and analysis were iterative and reflexive to enhance the authenticity of the results.

Results

The participants' perspectives organized around three key themes including: a) contextualizing curriculum and assessment practices with educational goals of CBME, b) coordinating new administrative requirements to support implementation, and c) adaptability toward a competency-based program structure, each with sub-themes.

Conclusion

By eliciting the perspectives of different stakeholder groups who experienced the implementation processes, we developed a common understanding regarding facilitators and challenges for program directors, program administrators and educational leaders across PGME. Results from our study contribute to the scholarly conversation regarding the key aspects related to CBME implementation and serve to inform its ongoing development and application in various educational contexts.

Prueba de concordancia de guiones para entrenar el razonamiento clínico en estudiantes de fonoaudiología

La prueba de concordancia de guiones (PCG) ha sido utilizada en el entrenamiento y evaluación del razonamiento clínico (RC) como una estrategia innovadora en la formación de profesionales. Sin embargo, no se dispone de evidencia de su aplicación en el pregrado de fonoaudiología. El objetivo de esta investigación fue analizar el desempeño y la percepción de estudiantes de fonoaudiología con respecto al uso de scripts. Se diseñó un piloto pre-experimental y multicéntrico, complementado con tres grupos focales. Las variables cuantitativas continuas fueron resumidas a través de medias y desviación estándar. La comparación entre grupos se ejecutó con Anova one way y la prueba post hoc de Bonferroni, considerando un nivel de significancia p<.05. La fase cualitativa incorporó un análisis de contenido mediante la codificación abierta de textos y la identificación e interpretación de familias de significado emergentes. El rendimiento promedio de los estudiantes fue de 4.03 (DS= 0.35), observándose un incremento en el rendimiento de RC durante el semestre (p= 0.03). La percepción de los estudiantes resulto positiva y se identificó cuatro familias de significado relacionadas con: razonamiento clínico, oportunidades de mejora implementación de la estrategia y retroalimentación docente. A modo de conclusión, la incorporación de scripts en estudiantes de pregrado de fonoaudiología es factible, incrementa el rendimiento y apoya el desarrollo del RC.

Stakeholders' perception on the implementation of Developmental Progress Assessment: using the Theoretical Domains Framework to document behavioral determinants

BACKGROUND

The widespread implementation of longitudinal assessment (LA) to document trainees' progression to independent practice rests more on speculative rather than evidence-based benefits. We aimed to document stakeholders' knowledge of- and attitudes towards LA, and identify how the supports and barriers can help or hinder the uptake and sustainable use of LA.

METHODS

We interviewed representatives from four stakeholder groups involved in LA. The interview protocols were based on the Theoretical Domains Framework (TDF), which contains a total of 14 behaviour change determinants. Two team members coded the interviews deductively to the TDF, with a third resolving differences in coding. The qualitative data analysis was completed with iterative consultations and discussions with team members until consensus was achieved. Saliency analysis was used to identify dominant domains.

RESULTS

Forty-one individuals participated in the study. Three dominant domains were identified. Participants perceive that LA has more positive than negative consequences and requires substantial ressources. All the elements and characteristics of LA are present in our data, with differences between stakeholders.

CONCLUSION

Going forward, we could develop and implement tailored and theory driven interventions to promote a shared understanding of LA, and maintain potential positive outcomes while reducing negative ones. Furthermore, ressources to support LA implementation need to be addressed to facilitate its uptake.

Implementation of competence committees during the transition to CBME in Canada: A national fidelity-focused evaluation

PURPOSE

This study evaluated the fidelity of competence committee (CC) implementation in Canadian postgraduate specialist training programs during the transition to competency-based medical education (CBME).

METHODS

A national survey of CC chairs was distributed to all CBME training programs in November 2019. Survey questions were derived from guiding documents published by the Royal College of Physicians and Surgeons of Canada reflecting intended processes and design.

RESULTS

Response rate was 39% (113/293) with representation from all eligible disciplines. Committee size ranged from 3 to 20 members, 42% of programs included external members, and 20% included a resident representative. Most programs (72%) reported that a primary review and synthesis of resident assessment data occurs prior to the meeting, with some data reviewed collectively during meetings. When determining entrustable professional activity (EPA) achievement, most programs followed the national specialty guidelines closely with some exceptions (53%). Documented concerns about professionalism, EPA narrative comments, and EPA entrustment scores were most highly weighted when determining resident progress decisions.

CONCLUSIONS

Heterogeneity in CC implementation likely reflects local adaptations, but may also explain some of the variable challenges faced by programs during the transition to CBME. Our results offer educational leaders important fidelity data that can help inform the larger evaluation and transformation of CBME.

Vitalizing the evaluation of curricular implementation: a framework for attending to the "how and whys" of curriculum evolution

As curricular reforms are implemented, there is often urgency among scholars to swiftly evaluate curricular outcomes and establish whether desired impacts have been realized. Consequently, many evaluative studies focus on summative program outcomes without accompanying evaluations of implementation. This runs the risk of Type III errors, whereby outcome evaluations rest on unverified assumptions about the appropriate implementation of prescribed curricular activities. Such errors challenge the usefulness of the evaluative studies, casting doubt on accumulated knowledge about curricular innovations, and posing problems for educational systems working to mobilize scarce resources. Unfortunately, however, there is long-standing inattention to the evaluation of implementation in health professions education (HPE). To address this, we propose an accessible framework that provides substantive guidance for evaluative research on implementation of curricular innovations. The Prescribed-Intended-Enacted-Sustainable (PIES) framework that is articulated in this paper, introduces new concepts to HPE-with a view to facilitating more nuanced examination of the evolution of curricula as they are implemented. Critically, the framework is theoretically grounded, integrating evaluation and implementation science as well as education theory. It outlines when, how, and why evaluators need to direct attention to curricular implementation, providing guidance on how programs can map out meaningful evaluative research agendas. Ultimately, this work is intended to support evaluators and educators, seeking to design evaluation studies that provide more faithful, useful representations of the intricacies of curricular change implementation.

[Teaching in a pandemic: the transformation of teaching and clinical supervision]

INTRODUCTION

The COVID-19 pandemic is an event that deeply impacts our personal, professional, and collective lives. How do we teach in these times of great upheaval? What are the main changes that have occurred?

METHOD

Using the Cartel logic, four professors and a qualitative researcher carried out an autoethnographic research aimed at documenting the main changes that have occurred in the teaching of family medicine in their respective practices located in four different academic family medicine groups at the University of Sherbrooke.

RESULTS

Five key moments in teaching that occurred during a pandemic were identified: a) the declaration of a pandemic, b) the approach with the graduating/advanced cohort of residents, c) the anticipation and preparation for the arrival of new residents, d) arrival of first year residents and e) adaptation to the second wave. For each moment, we present the issues encountered in our care and teaching practices under three transversal relational axes: the relationship of humans to their cultural context, the patient-doctor relationship, and the teacher-resident relationship.

CONCLUSION

Our analysis shows that the transmission of medical knowledge and the art of medicine cannot take place without specific attention to the overall cultural context, the contextual relationship of clinical care, and the teaching relationship. Our study also makes it possible to recommend the opening of spaces for reflection and dialogue in our teaching environments.

Rethinking implementation science for health professions education: A manifesto for change

Implementation science approaches the challenges of translating evidence into practice as a matter of scientific inquiry. This conceptual paper uses an implementation science lens to examine the ways in which evidence from health professions education research is brought to bear on decision-making. The authors describe different decision-making contexts and the kinds of evidence they consider, and from this, they outline ways in which research findings might be better presented to support their translation into policy and practice. Reflecting on the nature of decision-making in health professions education and how decisions are made and then implemented in different health professions education contexts, the authors argue that researchers should align their work with the decision-making contexts that are most likely to make use of them. These recommendations reflect implementation science principles of packaging and disseminating evidence in ways that are meaningful for key stakeholders, that stem from co-creation of knowledge, that require or result in meaningful partnerships, and that are context specific and relevant.

Benefits of EPAs at risk? The influence of the workplace environment on the uptake of EPAs in EPA-based curricula

INTRODUCTION

Entrustable Professional Activities (EPAs) have been applied differently in many postgraduate medical education (PGME) programmes, but the reasons for and the consequences of this variation are not well known. Our objective was to investigate how the uptake of EPAs is influenced by the workplace environment and to what extent the benefits of working with EPAs are at risk when the uptake of EPAs is influenced. This knowledge can be used by curriculum developers who intend to apply EPAs in their curricula.

METHOD

For this qualitative study, we selected four PGME programmes: General Practice, Clinical Geriatrics, Obstetrics & Gynaecology, and Radiology & Nuclear Medicine. A document analysis was performed on the national training plans, supported by the AMEE Guide for developing EPA-based curricula and relevant EPA-based literature. Interviews were undertaken with medical specialists who had specific involvement in the development of the curricula. Content analysis was employed and illuminated the possible reasons for variation in the uptake of EPAs.

RESULTS

An important part of the variation in the uptake of EPAs can be explained by environmental factors, such as patient population, the role of the physician in the health-care system, and the setup of local medical care institutions where the training programme takes place. The variation in uptake of EPAs is specifically reflected in the number and breadth of the EPAs, and in the way the entrustment decision is executed within the PGME programme.

DISCUSSION

Due to variation in uptake of EPAs, the opportunities for trainees to work independently during the training programme might be challenging. EPAs can be implemented in the curriculum of PGME programmes in a meaningful way, but only if the quality of an EPA is assessed, future users are involved in the development, and the key feature of EPAs (the entrustment decision) is retained.

Program directors' reflections on national policy change in medical education: insights on decision-making, accreditation, and the CanMEDS framework

BACKGROUND

Outcomes of national policy change impact all levels of the organizational hierarchy. The medical education literature is sparse on how reflections from program directors (PDs) on past large-scale policy changes can inform future policy initiatives. To fill this gap, we conducted a national survey on PDs' perceptions of, and reflections on, decision-making in medical education, accreditation procedures, and the CanMEDS framework implementation.

METHODS

The survey was distributed to former Canadian specialty medicine PDs (N = 684). Descriptive analysis was performed on quantitative data, thematic analysis was performed on qualitative comments, and comparisons between the quantitative and qualitative findings were performed to identify areas of convergence and/or divergence.

RESULTS

A total of 265 (38.7%) former PDs participated. Quantitative analysis revealed that 52.8% of respondents did not feel involved in decision-making regarding policy changes, 45.1% of respondents did not feel prepared to assess the CanMEDS Roles, and PDs were divided on the reasonableness of accreditation documentation. Qualitative analysis produced four themes: communication, resources, expectations of outcomes, and buy-in. Nine sub-themes were also identified. A high level of convergence was identified across the content, with only four areas of divergence identified.

CONCLUSIONS

Our findings have the potential to inform future policy and/or accreditation changes. Without the lens of those charged with overseeing the implementation, policy evaluation and quality improvement will remain uninformed. PDs, therefore, bring unique insights into our understanding of national policy changes, and without the voices of these frontline implementers, the true success of policy change implementation will be hindered.

A layered analysis of self-explanation and structured reflection to support clinical reasoning in medical students

Self-explanation and structured reflection have been studied independently with results suggesting that both learning interventions can effectively support medical students' clinical reasoning development. Given this evidence, medical schools may want/begin to implement these interventions in their curricula. Implementing educational interventions requires educators to maintain the core philosophy and principles of the interventions intact while adjusting implementation techniques to the specificities of individual learning contexts. Educational scholars have yet to explicitly articulate the philosophy, principles and techniques of self-explanation and structured reflection. Without such descriptions, educators risk failing to realize self-explanation's and structured reflection's effect to support students' clinical reasoning skill development in their implementations. Relying on the layered analysis approach, we articulate the philosophy, principles and techniques of self-explanation and structured reflection. This description is framed within the context of an actual implementation to illustrate the philosophies underpinning self-explanation and structured reflection, the principles that realize those philosophies, and the techniques that can be used to enact those principles. Building on the similarities between self-explanation and structured reflection, while also harnessing their differences, we identify why and how these interventions can be combined in a single implementation, while preserving their philosophies and principles. The layered analysis of self-explanation and structured reflection offers essential insights into the underpinnings of these interventions. They are articulated in this manuscript in hopes that other scholars will continue to refine these descriptions thereby facilitating effective use of self-explanation and structured reflection for clinical reasoning development.

Collaborative knotworking - transforming clinical teaching practice through faculty development

BACKGROUND

Faculty development is important for advancing teaching practice in health professions education. However, little is known regarding how faculty development outcomes are achieved and how change in practice may happen through these activities. In this study, we explored how clinical educators integrated educational innovations, developed within a faculty development programme, into their clinical workplaces. Thus, the study seeks to widen the understanding of how change following faculty development unfolds in clinical systems.

METHODS

The study was inspired by case study design and used a longitudinal faculty development programme as a case offering an opportunity to study how participants in faculty development work with change in practice. The study applied activity theory and its concept of activity systems in a thematic analysis of focus group interviews with 14 programme attendees. Participants represented two teaching hospitals, five clinical departments and five different health professions.

RESULTS

We present the activity systems involved in the integration process and the contradiction that arose between them as the innovations were introduced in the workplace. The findings depict how the faculty development participants and the clinicians teaching in the workplace interacted to overcome this contradiction through iterative processes of negotiating a mandate for change, reconceptualising the innovation in response to workplace reactions, and reconciliation as temporary equilibria between the systems.

CONCLUSION

The study depicts the complexities of how educational change is brought about in the workplace after faculty development. Based on our findings and the activity theoretical concept of knotworking, we suggest that these complex processes may be understood as collaborative knotworking between faculty development participants and workplace staff through which both the output from faculty development and the workplace practices are transformed. Increasing our awareness of these intricate processes is important for enhancing our ability to make faculty development reach its full potential in bringing educational change in practice.

Useful to whom? Evaluation utilisation theory and boundaries for programme evaluation scope

CONTEXT

Programme evaluation is perpetually mandated in health professions education. Correspondingly, there has been an expansion in prescriptive methodological guides about 'how' to engage in various best practices in evaluation. However, what has gained less attention is an examination of the 'value' that different stakeholders seek to gain from programme evaluation. Evaluation utilisation theory and research can help us understand the diversity in both the driving forces for and the impact of programme evaluation. Awareness of the heterogeneity of evaluation utilisation priorities has implications for evaluation practices, including both methodological choices and understanding of the impact of programme evaluation in our field.

METHODS

In this article, I expound on the concept of evaluation utilisation by drawing on evaluation theory and research. 'Evaluation utilisation' refers to the application of programme evaluation processes and findings to influence thinking and action. Herein, different forms of 'evaluation utilisation' (including instrumental, conceptual, process and persuasive (symbolic and legitimative) utilisation) are discussed, as well as the related concept of 'evaluation misuse'. Furthermore, how the prioritisation of different forms of 'evaluation utilisation' can influence the scope and impact of evaluation scholarship are also discussed.

CONCLUSIONS

Programme evaluation is a form of inquiry that requires more than the exercise of robust methodological techniques. Rather, it necessitates attention to the, sometimes divergent, priorities of different stakeholder groups. Although there is scant research on evaluation practices in health professions education, evaluation utilisation theory can inform critical examination of evaluation practices and impact in our field. Critically, understanding this body of work can help inform those engaged in evaluation about what they are (or should be) prioritising when they conduct programme evaluation, and better align evaluation methodologies with their scholarly, curricular and administrative intentions. Implications for future research and high-quality, transparent evaluation scholarship are presented.

Implementing an established musculoskeletal educational curriculum in a new context: a study of effectiveness and feasibility

Background: Musculoskeletal (MSK) problems are common, yet many primary care (PC) providers feel inadequately trained to manage these conditions. Previous studies describe successful MSK educational innovations at single sites, but none have reported on subsequent attempts to replicate or adapt these innovations to new contexts. This article presents a study of a national Veterans Affairs MSK training program modified to fit an existing PC educational program.Objectives: (1) To evaluate the effectiveness and feasibility of an adapted MSK curriculum in a new context. (2) To provide a model for adaptation studies in health professions education.Design: A national MSK shoulder and knee curriculum was adapted for San Francisco VA PC trainees, which included a small-group workshop and workplace learning within a newly-created MSK clinic. Effectiveness was evaluated by assessments of trainee confidence in exam and injection skills (via 5-point Likert scale) and faculty-observed performance of knee and shoulder exams (reported as percent of maximum possible score). Feasibility was evaluated by determining acceptability of the program to PC trainees (via 5-point Likert scale) and ability to implement the curriculum using local resources.Results: 52 trainees completed the training during a 2-year period. Trainees' confidence in MSK exam skills improved from 3.3 to 4.5 for shoulder, and from 3.5 to 4.6 for knee. Confidence performing joint injections improved from 2.6 to 4.2 (shoulder) and 2.5 to 4.5 (knee) (p < 0.001 for all). Observed performance improved markedly - from 50% to 92% for shoulder, and 57% to 90% for knee. Feasibility was evident in high acceptability (5.0 for MSK clinic, and 4.9 for workshops), and successful and sustained implementation.Conclusions: Adapting an established MSK curriculum to a new context was effective and feasible. This may serve as a more efficient model for improving trainee education than de novo curriculum design at individual sites.

Translating outcome frameworks to assessment programmes: Implications for validity

OBJECTIVES

Competency-based medical education (CBME) requires that educators structure assessment of clinical competence using outcome frameworks. Although these frameworks may serve some outcomes well (e.g. represent eventual practice), translating these into workplace-based assessment plans may undermine validity and, therefore, trustworthiness of assessment decisions due to a number of competing factors that may not always be visible or their impact knowable. Explored here is the translation process from outcome framework to formative and summative assessment plans in postgraduate medical education (PGME) in three Canadian universities.

METHODS

We conducted a qualitative study involving in-depth semi-structured interviews with leaders of PGME programmes involved in assessment and/or CBME implementation, with a focus on their assessment-based translational activities and evaluation strategies. Interviews were informed by Callon's theory of translation. Our analytical strategy involved directed content analysis, allowing us to be guided by Kane's validity framework, whilst still participating in open coding and analytical memo taking. We then engaged in axial coding to systematically explore themes across the dataset, various situations and our conceptual framework.

RESULTS

Twenty-four interviews were conducted involving 15 specialties across three universities. Our results suggest: (i) using outcomes frameworks for assessment is necessary for good assessment but are also viewed as incomplete constructs; (ii) there are a number of social and practical negotiations with competing factors that displace validity as a core influencer in assessment planning, including implementation, accreditation and technology; and (iii) validity exists as threatened, uncertain and assumed due to a number of unchecked assumptions and reliance on surrogates.

CONCLUSIONS

Translational processes in CBME involve negotiating with numerous influencing actors and institutions that, from an assessment perspective, provide challenges for assessment scientists, institutions and educators to contend with. These processes are challenging validity as a core element of assessment designs. Educators must reconcile these influences when preparing for or structuring validity arguments.

The development of competency frameworks in healthcare professions: a scoping review

Competency frameworks serve various roles including outlining characteristics of a competent workforce, facilitating mobility, and analysing or assessing expertise. Given these roles and their relevance in the health professions, we sought to understand the methods and strategies used in the development of existing competency frameworks. We applied the Arksey and O'Malley framework to undertake this scoping review. We searched six electronic databases (MEDLINE, CINAHL, PsycINFO, EMBASE, Scopus, and ERIC) and three grey literature sources (greylit.org, Trove and Google Scholar) using keywords related to competency frameworks. We screened studies for inclusion by title and abstract, and we included studies of any type that described the development of a competency framework in a healthcare profession. Two reviewers independently extracted data including study characteristics. Data synthesis was both quantitative and qualitative. Among 5710 citations, we selected 190 for analysis. The majority of studies were conducted in medicine and nursing professions. Literature reviews and group techniques were conducted in 116 studies each (61%), and 85 (45%) outlined some form of stakeholder deliberation. We observed a significant degree of diversity in methodological strategies, inconsistent adherence to existing guidance on the selection of methods, who was involved, and based on the variation we observed in timeframes, combination, function, application and reporting of methods and strategies, there is no apparent gold standard or standardised approach to competency framework development. We observed significant variation within the conduct and reporting of the competency framework development process. While some variation can be expected given the differences across and within professions, our results suggest there is some difficulty in determining whether methods were fit-for-purpose, and therefore in making determinations regarding the appropriateness of the development process. This uncertainty may unwillingly create and legitimise uncertain or artificial outcomes. There is a need for improved guidance in the process for developing and reporting competency frameworks.

The use of BEME reviews in the medical education literature

INTRODUCTION

Knowledge syntheses in medical education are intended to promote the translation to, and mobilization of, research knowledge into practice. Despite the effort invested in conducting them, how these knowledge syntheses are used is unclear. This study aimed to explore how knowledge syntheses published by the Best Evidence Medical Education Collaboration (BEME) have been used in a cross-section of published literature.

METHODS

Citation patterns for BEME reviews were explored using data drawn from Web of Science and Scopus, and a sub-sample of citing papers.

RESULTS

Bibliometric data on 3419 papers citing 29 BEME reviews were analysed. More detailed data were extracted from a random sample of 629 full-text papers.

DISCUSSION

BEME reviews were most often positioned to consolidate and summarize the current state of knowledge on a particular topic and to identify gaps in the literature; they were also used to justify current research, and less frequently to contextualize and explain results, or direct future areas of research. Their use to identify instruments or methodological approaches was relatively absent.

CONCLUSION

While BEME reviews are primarily used to justify and support other studies, the current literature does not demonstrate their translation to educational practice.

A Core Components Framework for Evaluating Implementation of Competency-Based Medical Education Programs

PURPOSE

The rapid adoption of competency-based medical education (CBME) provides an unprecedented opportunity to study implementation. Examining "fidelity of implementation"-that is, whether CBME is being implemented as intended-is hampered, however, by the lack of a common framework. This article details the development of such a framework.

METHOD

A two-step method was used. First, a perspective indicating how CBME is intended to bring about change was described. Accordingly, core components were identified. Drawing from the literature, the core components were organized into a draft framework. Using a modified Delphi approach, the second step examined consensus amongst an international group of experts in CBME.

RESULTS

Two different viewpoints describing how a CBME program can bring about change were found: production and reform. Because the reform model was most consistent with the characterization of CBME as a transformative innovation, this perspective was used to create a draft framework. Following the Delphi process, five core components of CBME curricula were identified: outcome competencies, sequenced progression, tailored learning experiences, competency-focused instruction, and programmatic assessment. With some modification in wording, consensus emerged amongst the panel of international experts.

CONCLUSIONS

Typically, implementation evaluation relies on the creation of a specific checklist of practices. Given the ongoing evolution and complexity of CBME, this work, however, focused on identifying core components. Consistent with recent developments in program evaluation, where implementation is described as a developmental trajectory toward fidelity, identifying core components is presented as a fundamental first step toward gaining a more sophisticated understanding of implementation.

Leasing a medical curriculum: What's it worth?

Introduction: The early part of this century saw an unprecedented growth in number and size of Australian medical schools. There was some partnering of the new schools with existing programs. Griffith, Deakin and Curtin Universities leased an established curriculum from Flinders University. Nature and rationale for curriculum leasing: The new schools had short startup times and leasing a curriculum enabled them to appoint key staff, develop facilities and meet accreditation requirements in a timely way. However, the lease arrangements were costly and the curriculum was largely determined before the Dean and key staff appointments. Outcomes of leasing: There was differential adoption of the leased curriculum. The first two years of the courses at Flinders were transferred with little change. The final two years of predominantly clinical studies were developed differently. This is explained through Michael Fullan's work on context in educational change. The context of the clinical years of the courses involved negotiations with local health services and other schools using those health services. The advantage of the leasing arrangements was that the new schools could proceed through early development and accreditation, while having time and opportunity to negotiate a clinical curriculum that engaged local health services and fulfilled the new schools' missions.

Learning Theory and Educational Intervention: Producing Meaningful Evidence of Impact Through Layered Analysis

Like evidence-based medicine, evidence-based education seeks to produce sound evidence of impact that can be used to intervene successfully in the future. The function of educational innovations, however, is much less well understood than the physical mechanisms of action of medical treatments. This makes production, interpretation, and use of educational impact evidence difficult. Critiques of medical education experiments highlight a need for such studies to do a better job of deepening understanding of learning in context; conclusions that "it worked" often precede scrutiny of what "it" was. The authors unpack the problem of representing educational innovation in a conceptually meaningful way. The more fundamental questions of "What is the intended intervention?" and "Did that intervention, in fact, occur?" are proposed as an alternative to the ubiquitous evaluative question of "Did it work?" The authors excavate the layers of intervention-techniques at the surface, principle in the middle, and philosophy at the core-and propose layered analysis as a way of examining an innovation's intended function in context. The authors then use problem-based learning to illustrate how layered analysis can promote meaningful understanding of impact through specification of what was tried, under what circumstances, and what happened as a result. Layered analysis should support innovation design and evaluation by illuminating what principled adaptation of educational technique to local context could look like. It also promotes theory development by enabling more precise description of the learning conditions at work in a given implementation and how they may evolve with broader adoption.

Where philosophy meets culture: exploring how coaches conceptualise their roles

CONTEXT

Although conceptually attractive, coaching in medicine remains ill-defined, with little examination of the transferability of coaching principles from other fields. Here we explore how coaching is enacted both within and outside of medicine; we aim to understand both the elements required for coaching to be useful and the factors that may influence its translation to the medical education context.

METHODS

In this constructivist grounded theory study, we interviewed 24 individuals across three groups: physicians who consider themselves coaches in clinical learning settings (n = 8), physicians with experience as sports, arts or business coaches (n = 10), and sports coaches without medical backgrounds (n = 6). Data collection and analysis were conducted iteratively using constant comparison to identify themes and explore their relationships.

RESULTS

We identified a shared philosophy of coaching, comprising three core elements that our participants endorsed regardless of the coaching context: (i) mutual engagement, with a shared orientation towards growth and development; (ii) ongoing reflection involving both learners and coaches, and (iii) an embrace of failure as a catalyst for learning. Enacting these features appeared to be influenced by culture, which affected how coaching was defined and developed, how the coaching role was positioned within the learning context, and how comfortably vulnerability could be expressed. Participants struggled to clearly define the coaching role in medicine, instead acknowledging that the lines between educational roles were often blurred. Further, the embrace of failure appeared challenging in medicine, where showing vulnerability was perceived as difficult for both learners and teachers.

CONCLUSIONS

Medical education's embrace of coaching should be informed by an understanding of both coach and learner behaviours that need to be encouraged and trained, and the cultural and organisational supports that are required to foster success.

Using Instrument-Guided Team Reflection and Debriefing to Cultivate Teamwork Knowledge, Skills, and Attitudes in Pre-Clerkship Learning Teams

Drawing on the science of teamwork and the science of learning, we designed an instrument-guided team reflection and debriefing activity to foster teamwork knowledge, skills, and attitudes (KSAs) in medical students. We then embedded this activity within and between a biweekly series of pre-clerkship Team-Based Learning sessions with the goal of encouraging medical students to cultivate a practical and metacognitive appreciation of eight foundational teamwork KSAs that are applicable to both healthcare teams and classroom learning teams. On evaluations, 144 learners from a class of 156 reported increased appreciation for and team improvement with these teamwork KSAs.

Embracing standardisation and contextualisation in medical education

CONTEXT

The tensions that emerge between the universal and the local in a global world require continuous negotiation. However, in medical education, standardization and contextual diversity tend to operate as separate philosophies, with little attention to the interplay between them.

METHODS

The authors synthesise the literature related to the intersections and resulting tensions between standardization and contextual diversity in medical education. In doing so, the authors analyze the interplay between these competing concepts in two domains of medical education (admissions and competency-based medical education), and provide concrete examples drawn from the literature.

RESULTS

Standardization offers many rewards: its common articulations and assumptions promote patient safety, foster continuous quality improvement, and enable the spread of best practices. Standardization may also contribute to greater fairness, equity, reliability and validity in high stakes processes, and can provide stakeholders, including the public, with tangible reassurance and a sense of the stable and timeless. At the same time, contextual variation in medical education can afford myriad learning opportunities, and it can improve alignment between training and local workforce needs. The inevitable diversity of contexts for learning and practice renders any absolute standardization of programs, experiences, or outcomes an impossibility.

CONCLUSIONS

The authors propose a number of ways to examine the interplay of contextual diversity and standardization and suggest three ways to move beyond an either/or stance. In reconciling the laudable goals of standardization and the realities of the innumerable contexts in which we train and deliver care, we are better positioned to design and deliver a medical education system that is globally responsible and locally engaged.

A Narrative Literature Review Considering the Development and Implementation of Longitudinal Integrated Clerkships, Including a Practical Guide for Application

Hailed by supporters as the answer to many challenges facing medical schools and the wider health care system, longitudinal integrated clerkships (LICs) offer a practical and sustainable alternative to more traditional block rotational models. Given this, their popularity as a curricular measure is increasing, although such clerkships remain relatively novel within the United Kingdom. This narrative literature review of international work provides a comprehensive introduction to developing and implementing LICs within medical education. This review generates a practical guide for medical educators with a focus on the development and implementation of LICs within the United Kingdom, on which there is little work. Using illustrated examples and with reference to contemporary literature, it outlines the rationale for considering an LIC within a curriculum, the different types of LIC, barriers and enabling factors to LIC implementation and considers the contemporary application of LIC models within the United Kingdom. The practical guide details key questions educators must consider when developing and implementing an LIC, particularly within the landscape of UK medical education.

The Impact of a National Competency-Based Medical Education Initiative in Family Medicine

PURPOSE

Triple C is the Canadian competency-based medical education (CBME) initiative for family medicine. The authors report on a study exploring the impacts Triple C has had across Canada.

METHOD

A multi-institutional team conducted a realist study to explore the impact of Triple C implementation in different programs across Canada. Data were collected between March and June 2016 from interviews and focus groups with key medical school stakeholders. Data were analyzed using thematic and template analysis techniques.

RESULTS

Data were acquired from 16 of the 17 Canadian medical schools from a combination of program leaders, educators, and trainees. Triple C was implemented in different ways and to different extents depending on context. Newer sites tended to have a more comprehensive implementation than established sites. Urban sites afforded different opportunities to implement Triple C from those afforded by rural sites. Although it was too early to assess its impact on the quality of graduating residents, Triple C was seen as having had a positive impact on identifying and remediating failing learners and on energizing and legitimizing the educational mission in family medicine. Negative impacts included greater costs and tensions in the relationships with other specialties. A principles-based approach to CBME offered flexibility to programs to incorporate variation in their interpretation and implementation of Triple C. Although there was a degree of normalization of practice, it was not standardized across sites or programs.

CONCLUSIONS

Triple C has been successfully implemented across Canada but in differing ways and with different impacts.

Thrown into the world of independent practice: from unexpected uncertainty to new identities

One of the most exciting yet stressful times in a physician's life is transitioning from supervised training into independent practice. The majority of literature devoted to this topic has focused upon a perceived gap between clinical and non-clinical skills and interventions taken to address it. Building upon recent streams of scholarship in identity formation and adaptation to new contexts, this work uses a Heideggerian perspective to frame an autoethnographical exploration of the author's transition into independent paediatric practice. An archive of reflective journal entries and personal communications was assembled from the author's first 3 years of practice in four different contexts and analyzed using Heidegger's linked existentials of understanding, attunement and discourse. Insights from his journey suggest this period is a time of anxiety and vulnerability when one questions one's competence and very identity as a medical professional. At the same time, it illustrates the inseparable link between practitioners and the network of relationships in which they are bound, how these relationships contextually vary and how recognizing and tuning to these differences may allow for a more seamless transition. While this work is the experience of one person, its insights support the ideas that change is a constant in professional practice and competence is contextual. As a result, developing educational content that inculcates contextual flexibility and an increased comfort level with uncertainty may prepare our trainees not just to navigate the unavoidable novelty of transition, but lay the groundwork for professional identities attuned to engage more broadly with change itself.

Examining the readiness of best evidence in medical education guides for integration into educational practice: A meta-synthesis

BACKGROUND

To support evidence-informed education, health professions education (HPE) stakeholders encourage the creation and use of knowledge syntheses or reviews. However, it is unclear if these knowledge syntheses are ready for translation into educational practice. Without understanding the readiness, defined by three criteria-quality, accessibility and relevance-we risk translating weak evidence into practice and/or providing information that is not useful to educators.

METHODS

A librarian searched Web of Science for knowledge syntheses, specifically Best Evidence in Medical Education (BEME) Guides. This meta-synthesis focuses on BEME Guides because of their explicit goal to inform educational practice and policy. Two authors extracted data from all Guides, guided by the 25-item STructured apprOach to the Reporting In healthcare education of Evidence Synthesis (STORIES).

RESULTS

Forty-two Guides published in Medical Teacher between 1999 and 2017 were analyzed. No Guide met all STORIES criteria, but all included structured summaries and most described their literature search (n = 39) and study inclusion/exclusion (n = 40) procedures. Eleven Guides reported the presence of theory and/or educational principles, and eight consulted with external subject matter experts. Accessibility to each Guide's full-text and supplemental materials was variable.

DISCUSSION

For a subset of HPE knowledge syntheses, BEME Guides, this meta-synthesis identifies factors that support readiness and indicates potential areas of improvement, such as consistent access to Guides and inclusion of external subject matter experts on the review team. This analysis is useful for understanding the current readiness of HPE knowledge syntheses and informing future reviews to evolve so they can catalyze translation of evidence into educational practice.

Multistation exercises: a combination of problem-based learning and team-based learning instructional design for large-enrollment classes

To introduce active learning session for a large group of 250 students, we combined the strengths of problem-based learning and team-based learning to promote a structured active learning strategy with less faculty involvement. For the implementation of this strategy, a case on anemia was selected based on the module already covered in classes. Structured exercises were preplanned on six different concepts that a student should possess for solving the problem. The large group of 250 students was divided into groups of 41 or 42 each. The groups were facilitated by one faculty member for one structured exercise. At the station, the group of 41 or 42 was further broken down into 6 smaller groups comprising 7 students each. After completion of the exercise, students cycled to the next exercise station facilitated by another faculty member. The case was solved in a plenary session. The effectiveness of the method was assessed by comparing the academic performance of the group with other similar groups from the previous year. The intervention group performed significantly better than the nonintervention group on the related item. Quartile subanalysis found that the effect was present in the performance of average and higher quartile groups, but not in the lower quartile group.

A conceptual framework of game-informed principles for health professions education

Games have been used for training purposes for many years, but their use remains somewhat underdeveloped and under-theorized in health professional education. This paper considers the basis for using serious games (games that have an explicit educational purpose) in health professional education in terms of their underlying concepts and design principles. These principles can be understood as a series of game facets: competition and conflict, chance and luck, experience and performance, simulation and make-believe, tactics and strategies, media, symbols and actions, and complexity and difficulty. Games are distinct and bound in ways that other health professional education activities are not. The differences between games and simulation can be understood in terms of the interconnected concepts of isomorphism (convergence with real-world practice) and anisomorphism (divergence from real-world practice). Gaming facets can extend the instructional design repertoire in health professional education.

Dimensions of integration, continuity and longitudinality in clinical clerkships

CONTEXT

Over the past few decades, longitudinal integrated clerkships (LICs) have been proposed to address many perceived short-coming of traditional block clerkships. This growing interest in LICs has raised broader questions regarding the role of integration, continuity and longitudinality in medical education. A study with complementary theoretical and empirical dimensions was conducted to derive a more precise way of defining these three underlying concepts within the design of medical education curricula.

METHODS

The theoretical dimension involved a thematic review of the literature on integration, continuity and longitudinality in medical education. The empirical dimension surveyed all 17 Canadian medical schools on how they have operationalised integration, continuity and longitudinality in their undergraduate programmes. The two dimensions were iteratively synthesised to explore the meaning and expression of integration, continuity and longitudinality in medical education curriculum design.

RESULTS

Integration, continuity and longitudinality were expressed in many ways and forms, including: integration of clinical disciplines, combined horizontal integration and vertical integration, and programme-level integration. Types of continuity included: continuity of patients, continuity of teaching, continuity of location and peer continuity. Longitudinality focused on connected or repeating episodes of training or on connecting activities, such as encounter logging across educational episodes. Twelve of the 17 schools were running an LIC of some kind, although only one school had a mandatory LIC experience. An ordinal scale of uses of integration, continuity and longitudinality during clerkships was developed, and new definitions of these concepts in the clerkship context were generated.

CONCLUSIONS

Different clerkship designs embodied different forms and levels of integration, continuity and longitudinality. A dichotomous view of LICs and rotation-based clerkships was found not to represent current practices in Canada, which instead tended to fall along a continuum of integration, continuity and longitudinality.

Mapping the dark matter of context: a conceptual scoping review

CONTEXT

Like dark matter, the contexts for medical education are largely invisible to those within them, although context can have profound influences on teaching, learning and practice. For something that is so intrinsic to the field of medical education, the concept of context remains troubling to scholars and those running medical education programmes. This paper reports on a critical and conceptual review of the concept of context within the medical education literature and beyond.

METHODS

A review was undertaken drawing on two sources: concepts of context in the medical education literature, and concepts of context across multiple academic disciplines. This body of material was iteratively, discursively and inductively synthesised.

RESULTS

Few of the articles from the medical education literature described or defined context directly, tending instead to focus on describing specific elements of context, such as clinical disciplines, physical settings and political pressures, that could or did influence learning outcomes. The results were framed in terms of what context 'is', how context works (in terms of context-mechanism-outcome), and how context can be represented using patterns. The authors propose a definition of context in medical education, along with the means to model, contrast and compare different contexts based on recurring patterns.

CONCLUSIONS

Context matters in medical education and it can, despite many challenges, be considered systematically and objectively. The findings from this study both represent a catalyst and challenge medical education researchers to look at context afresh.

Exploring patterns and pattern languages of medical education

CONTEXT

The practices and concepts of medical education are often treated as global constants even though they can take many forms depending on the contexts in which they are realised. This represents challenges in presenting and appraising medical education research, as well as in translating practices and concepts between different contexts. This paper explores the problem and seeks to respond to its challenges.

METHODS

This paper explores the application of architectural theorist Christopher Alexander's work on patterns and pattern languages to medical education. The authors review the underlying concepts of patterns and pattern language, they consider the development of pattern languages in medical education, they suggest possible applications of pattern languages for medical education and they discuss the implications of such use. Examples are drawn from across the field of medical education.

RESULTS

The authors argue that the deliberate and systematic use of patterns and pattern languages in describing medical educational activities, systems and contexts can help us to make sense of the world, and the pattern languages of medical education have the potential to advance understanding and scholarship in medical education, to drive innovation and to enable critical engagement with many of the underlying issues in this field.

Ebola preparedness: a rapid needs assessment of critical care in a tertiary hospital

BACKGROUND

The current outbreak of Ebola has been declared a public health emergency of international concern. We performed a rigorous and rapid needs assessment to identify the desired results, the gaps in current practice, and the barriers and facilitators to the development of solutions in the provision of critical care to patients with suspected or confirmed Ebola.

METHODS

We conducted a qualitative study with an emergent design at a tertiary hospital in Ontario, Canada, recently designated as an Ebola centre, from Oct. 21 to Nov. 7, 2014. Participants included physicians, nurses, respiratory therapists, and staff from infection control, housekeeping, waste management, administration, facilities, and occupational health and safety. Data collection included document analysis, focus groups, interviews and walk-throughs of critical care areas with key stakeholders.

RESULTS

Fifteen themes and 73 desired results were identified, of which 55 had gaps. During the study period, solutions were implemented to fully address 8 gaps and partially address 18 gaps. Themes identified included the following: screening; response team activation; personal protective equipment; postexposure to virus; patient placement, room setup, logging and signage; intrahospital patient movement; interhospital patient movement; critical care management; Ebola-specific diagnosis and treatment; critical care staffing; visitation and contacts; waste management, environmental cleaning and management of linens; postmortem; conflict resolution; and communication.

INTERPRETATION

This investigation identified widespread gaps across numerous themes; as such, we have been able to develop a set of credible and measureable results. All hospitals need to be prepared for contact with a patient with Ebola, and the preparedness plan will need to vary based on local context, resources and site designation.

The educational theory basis of team-based learning

BACKGROUND

Health care providers require the ability to use critical thinking skills and work effectively in a team as a part of an overall set of competencies. Therefore, educational programs should use appropriate methods based in educational theory to effectively graduate learners with these abilities. Team-based learning (TBL) is a method that has been introduced in healthcare education to foster critical thinking skills while students work in high functioning teams.

AIMS

This article will show how TBL follows the principles of constructivist learning theory.

METHOD

The principles of constructivist learning theory are discussed in relation to the teaching method of team-based learning. The effectiveness of TBL in healthcare education is then reviewed.

RESULTS

TBL is learner centered with the teacher acting as an expert facilitator and also provides students with opportunities to expose inconsistencies between their current understandings and new experiences thus stimulating development of new personal mental frameworks built upon previous knowledge. The learning is active using relevant problems and group interaction. Teamwork skills are strengthened by focused reflection on new experiences during the group sessions and on teamwork success by providing feedback to group members.

CONCLUSION

Since these aspects are all essential components of constructivist educational theory, TBL is solidly grounded in the theory and is a promising method to strengthen healthcare education.