1
|
de Graaf J, Bolk M, Dijkstra A, van der Horst M, Hoff RG, Ten Cate O. The Implementation of Entrustable Professional Activities in Postgraduate Medical Education in the Netherlands: Rationale, Process, and Current Status. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2021; 96:S29-S35. [PMID: 34183599 DOI: 10.1097/acm.0000000000004110] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Postgraduate medical education in the Netherlands has adopted competency-based education since the turn of the century. In 2006, the CanMEDS competency framework was introduced. A 2013 government plan to reduce the length and budgets of training programs led the Dutch Association of Medical Specialists (DAMS) to respond with a proposal to create more flexibility and individualization rather than a blunt cut in the length across all training programs. DAMS launched a government-funded, nation-wide, 4-year project (2014-2018) to blueprint the reform of postgraduate medical education in this direction. To achieve competency-based individualization, the fixed duration of postgraduate programs was abandoned, and entrustable professional activities (EPAs) were introduced in all specialty programs. Implementation of this new generation of programs took place in 2017-2019 in all disciplines. The project focused on EPA-based individualization of all programs, while addressing issues of the continuity of patient care in time-variable programs and the legal and regulatory consequences of individualization. About 30 specialty programs were revised at national, regional, local, and individual levels to incorporate EPAs; portfolio systems were adapted, clinical competency committees were installed for all programs, and procedures for summative entrustment decision making were elaborated. This paper reports on the rationale and the process that led to a more time-variable postgraduate education landscape, and, on average, a shortening of training length by 3 months.
Collapse
Affiliation(s)
- Jacqueline de Graaf
- J. de Graaf is professor of Professional Performance in PGME and Director of PGME, Radboudumc Health Academy, Radboud University Medical Center, Nijmegen, The Netherlands; ORCID: https://orcid.org/0000-0001-7662-9284
| | - Marieke Bolk
- M. Bolk is freelance educationalist in postgraduate medical education, currently project manager "Interprofessional education and collaboration" at the Dutch Association of Medical Specialists, Utrecht, The Netherlands
| | - Auk Dijkstra
- A. Dijkstra is freelance educationalist in postgraduate medical education, currently project manager "Integration of current topics and innovation in training" at the Dutch Association of Medical Specialists, Utrecht, The Netherlands
| | - Marieke van der Horst
- M. van der Horst is freelance educationalist in postgraduate medical education, currently project manager "Sustainable development and collaboration" at the Dutch Association of Medical Specialists, Utrecht, The Netherlands
| | - Reinier G Hoff
- R.G. Hoff is professor of education and training in perioperative, intensive, and emergency care and program director, Anesthesiology Residency, Department of Anesthesiology, University Medical Center Utrecht, Utrecht, The Netherlands; ORCID https://orcid.org/0000-0002-7432-7087
| | - Olle Ten Cate
- O. ten Cate is professor of medical education and senior scientist, Center for Research and Development of Education, University Medical Center Utrecht, Utrecht, the Netherlands; ORCID: https://orcid.org/0000-0002-6379-8780
| |
Collapse
|
2
|
Apramian T, Ott M, Roth K, Lingard L, Tipnis R, Cristancho S. Scut to Scholarship: Can Operative Notes be Educationally Useful? JOURNAL OF SURGICAL EDUCATION 2021; 78:168-177. [PMID: 32718727 DOI: 10.1016/j.jsurg.2020.07.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/21/2020] [Revised: 06/04/2020] [Accepted: 07/03/2020] [Indexed: 06/11/2023]
Abstract
OBJECTIVE Efforts to implement competency-based medical education require new sources of workplace-based evidence of growth in learning. We used qualitative analysis of operative notes to explore procedural variation in a simple surgical procedure. DESIGN We used a grounded theory-based mixed methods approach to depict intersurgeon procedural variation. Our grounded theory approach to analysis included follow up interviews with surgeons and residents to probe their understandings of the reasons for variation in the dictated notes and the current and potential utility of operative notes as a reliable source of data for learning and assessment. SETTING Publicly funded tertiary care otolaryngology-head & neck surgery residency program in Ontario, Canada PARTICIPANTS: Using maximum variability sampling, all surgeons performing tonsillectomy in the department (n = 6) contributed operative notes from 65 tonsillectomies, 5 intraoperative observations, and 4 semi-structured interviews. An additional 3 residents from various levels of training contributed semistructured interviews. RESULTS Intersurgeon procedural variations persist even in simple surgical procedures such as tonsillectomy. Operative notes appear to capture procedural variations in a limited way. Surgeons and resident make informal educational use of the clerical work of writing and assessing operative notes, but optimization will be required to shift such hidden work into the formal educational domain. CONCLUSIONS The implementation of competency-based medical education requires surgical educators to both eliminate low-yield tasks for learning and to find new opportunities for multiple low-stakes assessment. Analysis of operative notes may become a high-yield strategy for learning and assessment if residents and surgeons are coached to use operative notes more reliably and efficiently.
Collapse
Affiliation(s)
- Tavis Apramian
- Department of Family Medicine, McMaster University, Hamilton, Ontario, Canada.
| | - Mary Ott
- Faculty of Education, Western University, London, Ontario, Canada; Centre for Education Research & Innovation; Schulich School of Medicine & Dentistry, Western University, London, Ontario, Canada
| | - Kathryn Roth
- Department of Otolaryngology-Head and Neck Surgery, Western University Winnipeg, London, Ontario, Canada
| | - Lorelei Lingard
- Centre for Education Research & Innovation; Schulich School of Medicine & Dentistry, Western University, London, Ontario, Canada
| | - Rajas Tipnis
- Department of Otolaryngology-Head and Neck Surgery, University of Manitoba, Winnipeg, Manitoba, Canada
| | - Sayra Cristancho
- Centre for Education Research & Innovation; Schulich School of Medicine & Dentistry, Western University, London, Ontario, Canada
| |
Collapse
|
3
|
Abstract
Entrustment decision-making has become a topic of interest in workplace-based assessment in the health professions and is germane to the use of entrustable professional activities. Entrustment decisions stem from judgments of a trainee's competence and include the permission to act with a higher level of responsibility or autonomy and a lower level of supervision. Making entrustment decisions differs from regular assessment of trainees, which usually has no consequences beyond marking trainee progress. Studies show that clinicians generally weigh more factors in making an entrustment decision than when merely assessing trainee competence or performance without direct consequences for patient care. To synthesize the varying factors reported in literature, the authors performed a thematic analysis of key qualitative studies that investigated trainee features clinical supervisors find important when making entrustment decisions. Five themes emerged from the 13 publications: Capability (specific knowledge, skills, experience, situational awareness), Integrity (truthful, benevolent, patient-centered), Reliability (conscientious, predictable, accountable, responsible), Humility (recognizes limits, asks for help, receptive to feedback), Agency (proactive toward work, team, safety, personal development). Thoughtful entrustment decisions, made either by individual clinical supervisors or by clinical competency committees, may be enriched by taking into account these five features.
Collapse
Affiliation(s)
- Olle Ten Cate
- Center for Research and Development of Education, University Medical Center Utrecht, Utrecht, The Netherlands
| | - H Carrie Chen
- Georgetown University School of Medicine, Washington, USA
| |
Collapse
|
4
|
van der Aa JE, Aabakke AJM, Ristorp Andersen B, Settnes A, Hornnes P, Teunissen PW, Goverde AJ, Scheele F. From prescription to guidance: a European framework for generic competencies. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2020; 25:173-187. [PMID: 31451981 PMCID: PMC7018687 DOI: 10.1007/s10459-019-09910-8] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/28/2019] [Accepted: 08/09/2019] [Indexed: 06/01/2023]
Abstract
In postgraduate medical education, required competencies are described in detail in existing competency frameworks. This study proposes an alternative strategy for competency-based medical education design, which is supported by change management theories. We demonstrate the value of allowing room for re-invention and creative adaptation of innovations. This new strategy was explored for the development of a new generic competency framework for a harmonised European curriculum in Obstetrics and Gynaecology. The generic competency framework was developed through action research. Data were collected by four European stakeholder groups (patients, nurses, midwives and hospital boards), using a variety of methods. Subsequently, the data were analysed further in consensus discussions with European specialists and trainees in Obstetrics and Gynaecology. These discussions ensured that the framework provides guidance, is specialty-specific, and that implementation in all European countries could be feasible. The presented generic competency framework identifies four domains: 'Patient-centred care', 'Teamwork', 'System-based practice' and 'Personal and professional development'. For each of these four domains, guiding competencies were defined. The new generic competency framework is supported by European specialists and trainees in Obstetrics and Gynaecology, as well as by their European stakeholders. According to change management theories, it seems vital to allow room for re-invention and creative adaptation of the competency framework by medical professionals. Therefore, the generic competency framework offers guidance rather than prescription. The presented strategy for competency framework development offers leads for implementation of competency-based medical education as well as for development of innovations in postgraduate medical education in general.
Collapse
Affiliation(s)
- Jessica E van der Aa
- Department of Research and Education, OLVG Hospital, Amsterdam, The Netherlands.
- Athena Institute, Faculty of Science, VU, Amsterdam, The Netherlands.
| | - Anna J M Aabakke
- Department of Obstetrics and Gynaecology, Herlev University Hospital, Herlev, Denmark
- European Network of Trainees in Obstetrics and Gynaecology (ENTOG), Brussels, Belgium
| | - Betina Ristorp Andersen
- Department of Gynaecology and Obstetrics, North Zealand Hospital, University of Copenhagen, Copenhagen, Denmark
| | - Annette Settnes
- Department of Gynaecology and Obstetrics, North Zealand Hospital, University of Copenhagen, Copenhagen, Denmark
| | - Peter Hornnes
- Department of Gynaecology and Obstetrics, North Zealand Hospital, University of Copenhagen, Copenhagen, Denmark
- European Board and College of Obstetrics and Gynaecology (EBCOG), Brussels, Belgium
| | - Pim W Teunissen
- Department of Obstetrics and Gynaecology, Amsterdam UMC, VU University Medical Centre, Amsterdam, The Netherlands
- School of Health Professions Education (SHE), Faculty of Health Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands
| | - Angelique J Goverde
- European Board and College of Obstetrics and Gynaecology (EBCOG), Brussels, Belgium
- Department of Reproductive Medicine and Gynaecology, University Medical Centre, Utrecht, The Netherlands
| | - Fedde Scheele
- Department of Research and Education, OLVG Hospital, Amsterdam, The Netherlands
- Athena Institute, Faculty of Science, VU, Amsterdam, The Netherlands
- European Board and College of Obstetrics and Gynaecology (EBCOG), Brussels, Belgium
- Department of Obstetrics and Gynaecology, Amsterdam UMC, VU University Medical Centre, Amsterdam, The Netherlands
| |
Collapse
|
5
|
Pearce J. In defence of constructivist, utility-driven psychometrics for the 'post-psychometric era'. MEDICAL EDUCATION 2020; 54:99-102. [PMID: 31867758 DOI: 10.1111/medu.14039] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/11/2023]
Affiliation(s)
- Jacob Pearce
- Australian Council for Educational Research - Assessment and Psychometric Research, Camberwell, Victoria, Australia
| |
Collapse
|
6
|
How Do Thresholds of Principle and Preference Influence Surgeon Assessments of Learner Performance? Ann Surg 2019; 268:385-390. [PMID: 28463897 DOI: 10.1097/sla.0000000000002284] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/21/2023]
Abstract
OBJECTIVE The present study asks whether intraoperative principles are shared among faculty in a single residency program and explores how surgeons' individual thresholds between principles and preferences might influence assessment. BACKGROUND Surgical education continues to face significant challenges in the implementation of intraoperative assessment. Competency-based medical education assumes the possibility of a shared standard of competence, but intersurgeon variation is prevalent and, at times, valued in surgical education. Such procedural variation may pose problems for assessment. METHODS An entire surgical division (n = 11) was recruited to participate in video-guided interviews. Each surgeon assessed intraoperative performance in 8 video clips from a single laparoscopic radical left nephrectomy performed by a senior learner (>PGY5). Interviews were audio recorded, transcribed, and analyzed using the constant comparative method of grounded theory. RESULTS Surgeons' responses revealed 5 shared generic principles: choosing the right plane, knowing what comes next, recognizing normal and abnormal, making safe progress, and handling tools and tissues appropriately. The surgeons, however, disagreed both on whether a particular performance upheld a principle and on how the performance could improve. This variation subsequently shaped their reported assessment of the learner's performance. CONCLUSIONS The findings of the present study provide the first empirical evidence to suggest that surgeons' attitudes toward their own procedural variations may be an important influence on the subjectivity of intraoperative assessment in surgical education. Assessment based on intraoperative entrustment may harness such subjectivity for the purpose of implementing competency-based surgical education.
Collapse
|
7
|
Shorey S, Lau TC, Lau ST, Ang E. Entrustable professional activities in health care education: a scoping review. MEDICAL EDUCATION 2019; 53:766-777. [PMID: 30945329 DOI: 10.1111/medu.13879] [Citation(s) in RCA: 114] [Impact Index Per Article: 22.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/30/2018] [Revised: 11/30/2018] [Accepted: 02/22/2019] [Indexed: 05/13/2023]
Abstract
CONTEXT The shift in medical education from time-based learning to outcome-based learning has drawn much attention to entrustable professional activities (EPAs) as an ideal assessment framework to translate competencies into clinical practice. Given the relative novelty of EPAs, this review aims to highlight research gaps and explore and consolidate available evidence pertaining to the development and implementation of EPAs in health care. METHOD Arksey and O'Malley's scoping review framework was used to present the findings. The authors performed a systematic search of PubMed, Embase, CINAHL, Scopus, MedNar, OpenGrey and ProQuest Dissertation and Theses for English articles published from the inception of each database to May 2018. A manual search of the reference lists of the included studies was conducted and an expert panel was consulted. Two reviewers screened the articles for eligibility using the inclusion criteria. All authors extracted key data and analysed the data descriptively. Thematic analysis was used to categorise the results into themes. RESULTS Eighty articles were included in the review. All articles were published between 2010 and 2018. Three major themes and eight sub-themes were generated: (i) development of EPAs (frameworks for EPA development and implementation, identifying core or specialty-specific EPAs, and EPAs for faculty development), (ii) evaluation of EPAs and EPA entrustment factors (revised curriculum, entrustment decisions for professional activities, and feedback on implemented EPAs and the development process), and (iii) future directions and recommendations for EPAs (implementation of EPAs in undergraduate medical education and specific disciplines, and other criticisms and recommendations for EPAs). CONCLUSIONS Entrustable professional activities are an essential means to translate competencies into observable and measurable clinical practice. However, high-level evidence-based research on the efficacy, development and implementation of EPAs for specific target groups (i.e. undergraduates and staff) and geographical regions (i.e. Asia and Africa) is still lacking, which suggests a direction for future research.
Collapse
Affiliation(s)
- Shefaly Shorey
- Alice Lee Centre for Nursing Studies, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
- National University Health System, Singapore, Singapore
- Yong Loo Lin School of Medicine, Singapore, Singapore
| | | | - Siew Tiang Lau
- Alice Lee Centre for Nursing Studies, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
- National University Health System, Singapore, Singapore
- Yong Loo Lin School of Medicine, Singapore, Singapore
| | - Emily Ang
- Alice Lee Centre for Nursing Studies, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
- National University Health System, Singapore, Singapore
- Yong Loo Lin School of Medicine, Singapore, Singapore
| |
Collapse
|
8
|
Griewatz J, Lammerding-Koeppel M. Intrinsic roles in the crosshair - strategic analysis of multi-site role implementation with an adapted matrix map approach. BMC MEDICAL EDUCATION 2019; 19:237. [PMID: 31248391 PMCID: PMC6598229 DOI: 10.1186/s12909-019-1628-5] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/13/2018] [Accepted: 05/23/2019] [Indexed: 05/26/2023]
Abstract
BACKGROUND The implementation of competency-based intrinsic roles in undergraduate medical education remains a challenge. Faculties in transition need to be provided with generalizable curricular data in order to facilitate orientation on curricular roles' representation and to decide on steps of curriculum development. Explicit and implicit representation of objectives and multi-site agreement can be viewed as status indicators for the adoption of roles. Our aim was to develop a pragmatic cross-locational approach to capture roles' developmental status in an overview and prioritize strategic recommendations. METHODS Based on the mapping data from six German medical faculties, the relationship between explicit and implicit curricular representation of role' objectives (weighting) and extent of programs' consent (agreement) was calculated. Data was visualized in a role-specific Matrix Map to analyse roles' implicit-explicit relation and risk-value potential. The matrix was combined with Roger's stages of innovation diffusion for differentiated interpretation of the developmental role status. RESULTS Entangling multi-site agreement and curricular weighting, the 4-Field-Matrix allows to assess objectives based on their current localization in a quadrant: "Disregard" (lower left) and "Progress" quadrant (upper left) reveal the diffusion period; "Potential" (lower right) and "Emphasis" quadrant (upper right) indicate the adoption period. The role patterns differ in curricular representation, progression and clarity: (1) Scholar: explicit/implicit - scattered across the matrix; most explicit objectives in "Progress". (2) Health Advocate: explicit - primarily in "Emphasis"; only role in which the explicit representation significantly exceeds the implicit. (3) Collaborator: explicit - mainly "Potential"; implicit - "Progress" or "Emphasis". (4) Professional: explicit - primarily "Potential" but also "Emphasis"; implicit - "Progress" and "Emphasis"; appears better adopted but scattered in weighting; high hidden curricula. (5) Manager: explicit and implicit - exclusively in "Potential", without signs of development. Role patterns correspond to evidences from literature. Exemplified with roles, quadrant-specific strategies and measures are suggested. Framework reviewers may gain information for discussion of critical content. CONCLUSION The Matrix Map enables to catch intuitively the status of intrinsic roles' profiles regarding role pattern, implicit-explicit relation and programs agreement. Thus, interpretation and informed discussions are fostered. Further target-oriented analyses and strategic developments can be conducted to enhance transparency and resource-efficiency.
Collapse
Affiliation(s)
- Jan Griewatz
- Eberhard-Karls University of Tuebingen, Competence Centre for University Teaching in Medicine, Baden-Wuerttemberg, Elfriede-Aulhorn-Str. 10, 72076 Tuebingen, Germany
| | - Maria Lammerding-Koeppel
- Eberhard-Karls University of Tuebingen, Competence Centre for University Teaching in Medicine, Baden-Wuerttemberg, Elfriede-Aulhorn-Str. 10, 72076 Tuebingen, Germany
| |
Collapse
|
9
|
Dauphinee WD, Boulet JR, Norcini JJ. Considerations that will determine if competency-based assessment is a sustainable innovation. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2019; 24:413-421. [PMID: 29777463 DOI: 10.1007/s10459-018-9833-2] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/22/2017] [Accepted: 05/11/2018] [Indexed: 06/08/2023]
Abstract
Educational assessment for the health professions has seen a major attempt to introduce competency based frameworks. As high level policy developments, the changes were intended to improve outcomes by supporting learning and skills development. However, we argue that previous experiences with major innovations in assessment offer an important road map for developing and refining assessment innovations, including careful piloting and analyses of their measurement qualities and impacts. Based on the literature, numerous assessment workshops, personal interactions with potential users, and our 40 years of experience in implementing assessment change, we lament the lack of a coordinated approach to clarify and improve measurement qualities and functionality of competency based assessment (CBA). To address this worrisome situation, we offer two roadmaps to guide CBA's further development. Initially, reframe and address CBA as a measurement development opportunity. Secondly, using a roadmap adapted from the management literature on sustainable innovation, the medical assessment community needs to initiate an integrated plan to implement CBA as a sustainable innovation within existing educational programs and self-regulatory enterprises. Further examples of down-stream opportunities to refocus CBA at the implementation level within faculties and within the regulatory framework of the profession are offered. In closing, we challenge the broader assessment community in medicine to step forward and own the challenge and opportunities to reframe CBA as an innovation to improve the quality of the clinical educational experience. The goal is to optimize assessment in health education and ultimately improve the public's health.
Collapse
Affiliation(s)
- W Dale Dauphinee
- Foundation for the Advancement of International Medical Education and Research, 3624 Market Street, Fourth Floor, Philadelphia, PA, 19104, USA.
- McGill University, 1140 Pine Avenue West, Montreal, QC, H3A 1A3, Canada.
- , Saint Andrews, NB, Canada.
| | - John R Boulet
- Foundation for the Advancement of International Medical Education and Research, 3624 Market Street, Fourth Floor, Philadelphia, PA, 19104, USA
| | - John J Norcini
- Foundation for the Advancement of International Medical Education and Research, 3624 Market Street, Fourth Floor, Philadelphia, PA, 19104, USA
| |
Collapse
|
10
|
O'Dowd E, Lydon S, O'Connor P, Madden C, Byrne D. A systematic review of 7 years of research on entrustable professional activities in graduate medical education, 2011-2018. MEDICAL EDUCATION 2019; 53:234-249. [PMID: 30609093 DOI: 10.1111/medu.13792] [Citation(s) in RCA: 101] [Impact Index Per Article: 20.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/31/2018] [Revised: 11/05/2018] [Accepted: 11/20/2018] [Indexed: 05/13/2023]
Abstract
PURPOSE This review aimed to synthesise some of the extant work on the use of entrustable professional activities (EPAs) for postgraduate physicians, to assess the quality of the work and provide direction for future research and practice. METHOD Systematic searches were conducted within five electronic databases (Medline, Scopus, Web of Science, PsycINFO and CINAHL) in September 2018. Reference lists, Google Scholar and Google were also searched. Methodological quality was assessed using the Quality Assessment Tool for Studies with Diverse Designs (QATSDD). RESULTS In total, 49 studies were included, classified as Development of EPAs (n = 37; 76% of total included), Implementation and/or assessment of EPAs (n = 10; 20%), or both (n = 2; 4%). EPAs were described for numerous specialties, including internal medicine (n = 14; 36%), paediatrics (n = 8; 21%) and psychiatry (n = 4; 10%). Of the development studies, 92% utilised more than one method to generate EPAs. The two most commonly used methods were developing initial EPAs in a working group, (n = 27; 69%) and revising through deliberation (n = 21; 54%). Development papers were of variable quality (mean QATSDD score = 20, range 6-41). Implementation and assessment studies utilised methods that included observing trainee performance (n = 6; 50%) and enrolling trainees in competency-based curricula, which included EPAs (n = 4; 33%). The methodological quality of these implementation studies varied (mean QATSDD score = 19.5, range = 6-32). CONCLUSIONS This review highlighted a need for: (i) consideration of best practice guidelines for EPA development; (ii) focus on the methodological quality of research on EPA development and of EPAs, and (iii) further work investigating the implementation of EPAs in the curriculum.
Collapse
Affiliation(s)
- Emily O'Dowd
- Irish Centre for Applied Patient Safety and Simulation, National University of Ireland, Galway, Ireland
- Discipline of General Practice, School of Medicine, National University of Ireland, Galway, Ireland
| | - Sinéad Lydon
- Irish Centre for Applied Patient Safety and Simulation, National University of Ireland, Galway, Ireland
- School of Medicine, National University of Ireland, Galway, Ireland
| | - Paul O'Connor
- Irish Centre for Applied Patient Safety and Simulation, National University of Ireland, Galway, Ireland
- Discipline of General Practice, School of Medicine, National University of Ireland, Galway, Ireland
| | - Caoimhe Madden
- Irish Centre for Applied Patient Safety and Simulation, National University of Ireland, Galway, Ireland
- Discipline of General Practice, School of Medicine, National University of Ireland, Galway, Ireland
| | - Dara Byrne
- Irish Centre for Applied Patient Safety and Simulation, National University of Ireland, Galway, Ireland
- School of Medicine, National University of Ireland, Galway, Ireland
| |
Collapse
|
11
|
Lefebvre C, Hiestand B, Glass C, Masneri D, Hosmer K, Hunt M, Hartman N. Examining the Effects of Narrative Commentary on Evaluators’ Summative Assessments of Resident Performance. Eval Health Prof 2018; 43:159-161. [DOI: 10.1177/0163278718820415] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Abstract
Anchor-based, end-of-shift ratings are commonly used to conduct performance assessments of resident physicians. These performance evaluations often include narrative assessments, such as solicited or “free-text” commentary. Although narrative commentary can help to create a more detailed and specific assessment of performance, there are limited data describing the effects of narrative commentary on the global assessment process. This single-group, observational study examined the effect of narrative comments on global performance assessments. A subgroup of the clinical competency committee, blinded to resident identity, assigned a single, consensus-based performance score (1–6) to each resident based solely on end-of-shift milestone scores. De-identified narrative comments from end-of-shift evaluations were then included and the process was repeated. We compared milestone-only scores to milestone plus narrative commentary scores using a nonparametric sign test. During the study period, 953 end-of-shift evaluations were submitted on 41 residents. Of these, 535 evaluations included free-text narrative comments. In 17 of the 41 observations, performance scores changed after the addition of narrative comments. In two cases, scores decreased with the addition of free-text commentary. In 15 cases, scores increased. The frequency of net positive change was significant ( p = .0023). The addition of narrative commentary to anchor-based ratings significantly influenced the global performance assessment of Emergency Medicine residents by a committee of educators. Descriptive commentary collected at the end of shift may inform more meaningful appraisal of a resident’s progress in a milestone-based paradigm. The authors recommend clinical training programs collect unstructured narrative impressions of residents’ performance from supervising faculty.
Collapse
Affiliation(s)
- Cedric Lefebvre
- Department of Emergency Medicine, Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - Brian Hiestand
- Department of Emergency Medicine, Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - Casey Glass
- Department of Emergency Medicine, Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - David Masneri
- Department of Emergency Medicine, Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - Kathleen Hosmer
- Department of Emergency Medicine, Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - Meagan Hunt
- Department of Emergency Medicine, Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - Nicholas Hartman
- Department of Emergency Medicine, Wake Forest School of Medicine, Winston-Salem, NC, USA
| |
Collapse
|
12
|
Duijn CCMA, Ten Cate O, Kremer WDJ, Bok HGJ. The Development of Entrustable Professional Activities for Competency-Based Veterinary Education in Farm Animal Health. JOURNAL OF VETERINARY MEDICAL EDUCATION 2018; 46:218-224. [PMID: 30565977 DOI: 10.3138/jvme.0617-073r] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Entrustable professional activities (EPAs) are professional tasks that can be entrusted to a student under a given level of supervision once he or she has demonstrated competence in these tasks. The EPA construct was conceived to increase transparency in objectives for clinical workplace learning and to help ensure patient safety and the quality of care. A first step in implementing EPAs in a veterinary curriculum is to identify the core EPAs of the profession. The aim of this study was to develop EPAs for farm animal health. An initial set of 36 EPAs for farm animal health was prepared by a team of six veterinarians and curriculum developers and used in a modified Delphi study. In this iterative process, the EPAs were evaluated until higher than 80% agreement was reached. Of 83 veterinarians who participated, 39 (47%) completed the Delphi procedure. After two rounds, the panel reached consensus. A small expert group further refined and reorganized the EPAs for educational purposes into seven core EPAs for farm animal health and 29 sub-EPAs. This study is an important step in optimizing competency-based training in veterinary medicine. Future steps are to implement EPAs in the curriculum and train supervisors to assess students' ability to perform EPAs with increasing levels of independence.
Collapse
|
13
|
van Loon KA. Formal Versus Informal Judgments: Faculty Experiences With Entrustment in Graduate Medical Education. J Grad Med Educ 2018; 10:537-542. [PMID: 30386479 PMCID: PMC6194887 DOI: 10.4300/jgme-d-18-00120.1] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/06/2018] [Revised: 05/26/2018] [Accepted: 06/01/2018] [Indexed: 12/27/2022] Open
Abstract
BACKGROUND Entrustment of residents has been formalized in many competency-based graduate medical education programs, but its relationship with informal decisions to entrust residents with clinical tasks is unclear. In addition, the effects of formal entrustment on training practice are still unknown. OBJECTIVE Our objective was to learn from faculty members in training programs with extensive experience in formal entrustment how formal entrustment relates to informal entrustment decisions. METHODS A questionnaire was e-mailed to all Dutch obstetrics and gynecology program directors to gather information on how faculty entrusts residents with clinical independence. We also interviewed faculty members to explore the relationship between formal entrustment and informal entrustment. Interviews were analyzed with conventional content analysis. RESULTS Of 92 programs, 54 program directors completed the questionnaire (59% response rate). Results showed that formal entrustment was seen as valuable for generating formative feedback and giving insight into residents' progress in technical competencies. Interviewed faculty members (n = 12) used both formal and informal entrustment to determine the level of resident independence. Faculty reported they tended to favor informal entrustment because it can be reconsidered. In contrast, formal entrustment was reported to feel like a fixed state. CONCLUSIONS In a graduate medical education program where formal entrustment has been used for more than a decade, faculty used a combination of formal and informal entrustment. Informal entrustment is key in deciding if a resident can work independently. Faculty members reported being unsure how to optimally use formal entrustment in practice next to their informal decisions.
Collapse
|
14
|
Parson L, Childs B, Elzie P. Using Competency-Based Curriculum Design to Create a Health Professions Education Certificate Program the Meets the Needs of Students, Administrators, Faculty, and Patients. HEALTH PROFESSIONS EDUCATION 2018. [DOI: 10.1016/j.hpe.2018.03.008] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/17/2022] Open
|
15
|
Linsenmeyer M, Wimsatt L, Speicher M, Powers J, Miller S, Katsaros E. Assessment Considerations for Core Entrustable Professional Activities for Entering Residency. J Osteopath Med 2018; 118:243-251. [DOI: 10.7556/jaoa.2018.049] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
Abstract
Abstract
Context
In the process of analyzing entrustable professional activities (EPAs) for use in medical education, ten Cate and others identified challenges, including the need for valid and reliable EPA assessment strategies.
Objective
To provide osteopathic medical schools with a database of assessment tools compiled from the literature to assist them with the development and implementation of robust, evidence-based assessment methods.
Methods
MEDLINE, ERIC, PubMed, and other relevant databases were searched using MeSH keywords for articles outlining robust, evidence-based assessment tools that could be used in designing assessments for EPAs 1 through 6.
Results
A total of 55 publications were included in content analysis and reporting. All but 2 of the assessment articles were conducted in an undergraduate or graduate medical education setting. The majority of the 55 articles related to assessment of competencies affiliated with EPA 2 (16 articles) and EPA 4 (15 articles). Four articles focused on EPA 3.
Conclusion
Osteopathic medical schools can use this database of assessment tools to support the development of EPA-specific assessment plans that match the unique context and needs of their institution.
Collapse
|
16
|
Teunissen PW, Kogan JR, Ten Cate O, Gruppen LD, Lingard LA. Learning in Practice: A Valuation of Context in Time-Variable Medical Training. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2018; 93:S22-S26. [PMID: 29485483 DOI: 10.1097/acm.0000000000002070] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
The logical consequence of implementing competency-based education is moving to time-variable training. Competency-based, time-variable training (CBTVT) requires an understanding of how learners interact with their learning context and how that leads to competence. In this article, the authors discuss this relationship. They first explain that the time required to achieve competence in clinical practice depends on the availability of clinical experiences that are conducive to ongoing competence development. This requires both curricular flexibility in light of the differences in individual learners' development and a balance between longitudinal placements and transitions to different environments.Along with the deliberate use of the opportunities that learning environments offer, there is value for learners in spending ample time-in-context. For instance, guided independence is possible when trainees do not progress immediately after meeting curricular learning objectives. Next, the potential implications of CBTVT can be illustrated by two learning perspectives-Sfard's acquisition and participation metaphors-which leads to the assertion that competence is both an individual characteristic and a quality that emerges from a purposeful social interaction between individuals and their context. This theory recognizes that the deliberate use of context could be used to approach learning as acquiring collective competence.Based on this relationship between learner, context, and competence, the authors propose an approach to CBTVT that recognizes that all learners will have to meet a number of standard preset learning targets in their workplace, while still having room for further context-specific competence development and personal growth within strategically organized learning environments.
Collapse
Affiliation(s)
- Pim W Teunissen
- P.W. Teunissen is professor, School of Health Professions Education, Maastricht University, Maastricht, the Netherlands, and maternal fetal medicine specialist, VU University Medical Center, Amsterdam, the Netherlands. J.R. Kogan is professor of medicine, Department of Medicine, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania. O. ten Cate is professor of medical education, Center for Research and Development of Education, University Medical Center Utrecht, Utrecht, the Netherlands. L.D. Gruppen is professor, Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor, Michigan. L.A. Lingard is professor, Department of Medicine, and director, Centre for Education Research & Innovation, Schulich School of Medicine & Dentistry, Western University, London, Ontario, Canada
| | | | | | | | | |
Collapse
|