1
|
Frank JR, Karpinski J, Sherbino J, Snell LS, Atkinson A, Oswald A, Hall AK, Cooke L, Dojeiji S, Richardson D, Cheung WJ, Cavalcanti RB, Dalseg TR, Thoma B, Flynn L, Gofton W, Dudek N, Bhanji F, Wong BMF, Razack S, Anderson R, Dubois D, Boucher A, Gomes MM, Taber S, Gorman LJ, Fulford J, Naik V, Harris KA, St. Croix R, van Melle E. Competence By Design: a transformational national model of time-variable competency-based postgraduate medical education. PERSPECTIVES ON MEDICAL EDUCATION 2024; 13:201-223. [PMID: 38525203 PMCID: PMC10959143 DOI: 10.5334/pme.1096] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/04/2023] [Accepted: 02/16/2024] [Indexed: 03/26/2024]
Abstract
Postgraduate medical education is an essential societal enterprise that prepares highly skilled physicians for the health workforce. In recent years, PGME systems have been criticized worldwide for problems with variable graduate abilities, concerns about patient safety, and issues with teaching and assessment methods. In response, competency based medical education approaches, with an emphasis on graduate outcomes, have been proposed as the direction for 21st century health profession education. However, there are few published models of large-scale implementation of these approaches. We describe the rationale and design for a national, time-variable competency-based multi-specialty system for postgraduate medical education called Competence by Design. Fourteen innovations were bundled to create this new system, using the Van Melle Core Components of competency based medical education as the basis for the transformation. The successful execution of this transformational training system shows competency based medical education can be implemented at scale. The lessons learned in the early implementation of Competence by Design can inform competency based medical education innovation efforts across professions worldwide.
Collapse
Affiliation(s)
- Jason R. Frank
- Centre for Innovation in Medical Education and Professor, Department of Emergency Medicine, Faculty of Medicine, University of Ottawa, ON, Canada
| | - Jolanta Karpinski
- Department of Medicine, University of Ottawa, Ottawa, ON, Canada
- Competency Based Medical Education, University of Ottawa, Ottawa, ON, Canada
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | | | - Linda S. Snell
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Medicine and Health Sciences Education, McGill University, Montreal, QC, Canada
| | - Adelle Atkinson
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Paediatrics, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
| | - Anna Oswald
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Medicine, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, AB, Canada
- Competency Based Medical Education, University of Alberta, Edmonton, AB, Canada
| | - Andrew K. Hall
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Emergency Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Lara Cooke
- Division of Neurology, Department of Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Susan Dojeiji
- Physical Medicine and Rehabilitation, University of Ottawa, Ottawa, ON, Canada
| | - Denyse Richardson
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Physical Medicine and Rehabilitation, Queen’s University, Kingston, ON, Canada
| | - Warren J. Cheung
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Medicine, University of Toronto, Toronto, ON, Canada
| | - Rodrigo B. Cavalcanti
- Department of Medicine, University of Toronto, Toronto, ON, Canada
- HoPingKong Centre, University Health Network, Toronto, ON, Canada
| | - Timothy R. Dalseg
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Division of Emergency Medicine, University of Toronto, Toronto, ON, Canada
| | - Brent Thoma
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Emergency Medicine, University of Saskatchewan, Saskatoon, SK, Canada
| | - Leslie Flynn
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Departments of Psychiatry and Family Medicine, and Co-Director Master of Health Sciences Education, Queen’s University, Kingston, ON, Canada
| | - Wade Gofton
- Department of Surgery (Division of Orthopedic Surgery), The Ottawa Hospital and University of Ottawa, Ottawa, ON, Canada
| | - Nancy Dudek
- Department of Medicine (Division of Physical Medicine & Rehabilitation) and The Ottawa Hospital, University of Ottawa, Ottawa, Ontario, Canada
| | - Farhan Bhanji
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Faculty of Medicine and Health Sciences, McGill University, Montreal, QC, Canada
| | - Brian M.-F. Wong
- Centre for Quality Improvement and Patient Safety, University of Toronto, Toronto, Canada
| | - Saleem Razack
- Centre for Health Education Scholarship, University of British Columbia and BC Children’s Hospital, Vancouver, BC, Canada
| | - Robert Anderson
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Northern Ontario School of Medicine University, Sudbury, ON, Canada
| | - Daniel Dubois
- Department of Anesthesiology and Pain Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Andrée Boucher
- Department of Medicine (Division of Endocrinology), Universitéde Montréal, Montréal, QC, Canada
| | - Marcio M. Gomes
- Department of Pathology and Laboratory Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Sarah Taber
- Office of Standards and Assessment, Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | - Lisa J. Gorman
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | - Jane Fulford
- Canadian Internet Registration Authority, Canada
| | - Viren Naik
- Department of Anesthesiology and Pain Medicine, University of Ottawa, Ottawa, ON, Canada
- Medical Council of Canada, Ottawa, ON, Canada
| | - Kenneth A. Harris
- Royal College of Physicians and Surgeons of Canada, Canada
- Emeritus, Western University, Canada
| | - Rhonda St. Croix
- Learning and Connecting at the Royal College of Physicians and Surgeons of Canada, Canada
| | - Elaine van Melle
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Family Medicine, Queen’s University, Kingston, ON, Canada
| |
Collapse
|
2
|
Oswald A, Dubois D, Snell L, Anderson R, Karpinski J, Hall AK, Frank JR, Cheung WJ. Implementing Competence Committees on a National Scale: Design and Lessons Learned. PERSPECTIVES ON MEDICAL EDUCATION 2024; 13:56-67. [PMID: 38343555 PMCID: PMC10854462 DOI: 10.5334/pme.961] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Accepted: 07/03/2023] [Indexed: 02/15/2024]
Abstract
Competence committees (CCs) are a recent innovation to improve assessment decision-making in health professions education. CCs enable a group of trained, dedicated educators to review a portfolio of observations about a learner's progress toward competence and make systematic assessment decisions. CCs are aligned with competency based medical education (CBME) and programmatic assessment. While there is an emerging literature on CCs, little has been published on their system-wide implementation. National-scale implementation of CCs is complex, owing to the culture change that underlies this shift in assessment paradigm and the logistics and skills needed to enable it. We present the Royal College of Physicians and Surgeons of Canada's experience implementing a national CC model, the challenges the Royal College faced, and some strategies to address them. With large scale CC implementation, managing the tension between standardization and flexibility is a fundamental issue that needs to be anticipated and addressed, with careful consideration of individual program needs, resources, and engagement of invested groups. If implementation is to take place in a wide variety of contexts, an approach that uses multiple engagement and communication strategies to allow for local adaptations is needed. Large-scale implementation of CCs, like any transformative initiative, does not occur at a single point but is an evolutionary process requiring both upfront resources and ongoing support. As such, it is important to consider embedding a plan for program evaluation at the outset. We hope these shared lessons will be of value to other educators who are considering a large-scale CBME CC implementation.
Collapse
Affiliation(s)
- Anna Oswald
- Division of Rheumatology, Department of Medicine, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, AB, Canada
- Competency Based Medical Education, University of Alberta, Edmonton, AB, Canada
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- 8-130 Clinical Sciences building, 11350-83 Avenue, Edmonton, AB, Canada
| | - Daniel Dubois
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Anesthesiology and Pain Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Linda Snell
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Institute of Health Sciences Education and Department of Medicine, McGill University, Montreal, QC, Canada
| | - Robert Anderson
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Northern Ontario School of Medicine University, Sudbury, ON, Canada
| | - Jolanta Karpinski
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Andrew K. Hall
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Dept. of Emergency Medicine, University of Ottawa, Canada
| | - Jason R. Frank
- Centre for Innovation in Medical Education, Faculty of Medicine, University of Ottawa, Canada
| | - Warren J. Cheung
- Dept. of Emergency Medicine, University of Ottawa, Canada
- Royal College of Physicians and Surgeons of Canada, 1053 Carling Avenue, Rm F660, Ottawa, Canada
| |
Collapse
|
3
|
Caretta-Weyer HA, Smirnova A, Barone MA, Frank JR, Hernandez-Boussard T, Levinson D, Lombarts KMJMH, Lomis KD, Martini A, Schumacher DJ, Turner DA, Schuh A. The Next Era of Assessment: Building a Trustworthy Assessment System. PERSPECTIVES ON MEDICAL EDUCATION 2024; 13:12-23. [PMID: 38274558 PMCID: PMC10809864 DOI: 10.5334/pme.1110] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/16/2023] [Accepted: 12/18/2023] [Indexed: 01/27/2024]
Abstract
Assessment in medical education has evolved through a sequence of eras each centering on distinct views and values. These eras include measurement (e.g., knowledge exams, objective structured clinical examinations), then judgments (e.g., workplace-based assessments, entrustable professional activities), and most recently systems or programmatic assessment, where over time multiple types and sources of data are collected and combined by competency committees to ensure individual learners are ready to progress to the next stage in their training. Significantly less attention has been paid to the social context of assessment, which has led to an overall erosion of trust in assessment by a variety of stakeholders including learners and frontline assessors. To meaningfully move forward, the authors assert that the reestablishment of trust should be foundational to the next era of assessment. In our actions and interventions, it is imperative that medical education leaders address and build trust in assessment at a systems level. To that end, the authors first review tenets on the social contextualization of assessment and its linkage to trust and discuss consequences should the current state of low trust continue. The authors then posit that trusting and trustworthy relationships can exist at individual as well as organizational and systems levels. Finally, the authors propose a framework to build trust at multiple levels in a future assessment system; one that invites and supports professional and human growth and has the potential to position assessment as a fundamental component of renegotiating the social contract between medical education and the health of the public.
Collapse
Affiliation(s)
- Holly A. Caretta-Weyer
- Department of Emergency Medicine, Stanford University School of Medicine, Palo Alto, California, USA
| | - Alina Smirnova
- Department of Family Medicine, University of Calgary, Calgary, Alberta, Canada
- Kern Institute for the Transformation of Medical Education, Medical College of Wisconsin, Milwaukee, Wisconsin, USA
| | - Michael A. Barone
- NBME, Philadelphia, Pennsylvania, USA
- Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
| | - Jason R. Frank
- Department of Emergency Medicine, University of Ottawa, Ottawa, Ontario, CA
| | | | - Dana Levinson
- Josiah Macy Jr Foundation, Philadelphia, Pennsylvania, USA
| | - Kiki M. J. M. H. Lombarts
- Department of Medical Psychology, Amsterdam University Medical Centers, University of Amsterdam, NL
- Amsterdam Public Health research institute, Amsterdam, NL
| | - Kimberly D. Lomis
- Undergraduate Medical Education Innovations, American Medical Association, Chicago, Illinois, USA
| | - Abigail Martini
- Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio, USA
| | - Daniel J. Schumacher
- Division of Emergency Medicine, Cincinnati Children’s Hospital Medical Center/University of Cincinnati College of Medicine, Cincinnati, Ohio, USA
| | - David A. Turner
- American Board of Pediatrics, Chapel Hill, North Carolina, USA
| | - Abigail Schuh
- Division of Emergency Medicine, Medical College of Wisconsin, Milwaukee, Wisconsin, USA
| |
Collapse
|
4
|
Tillotson CV, Becetti I, Hwu K, Page L, Krishnan S, Stafford D, Stanley T, Vuguin P, Barker JM. Pediatric Endocrinology Milestones 2.0-guide to their implementation. BMC MEDICAL EDUCATION 2023; 23:985. [PMID: 38124091 PMCID: PMC10734147 DOI: 10.1186/s12909-023-04862-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/10/2023] [Accepted: 11/09/2023] [Indexed: 12/23/2023]
Abstract
The Milestones were initiated by the Accreditation Council for Graduate Medical Education (ACGME) to provide a framework for monitoring a trainee's progression throughout residency/fellowship. The Milestones describe stepwise skill progression through six core domains of clinical competency: Patient Care, Medical Knowledge, Interpersonal and Communication Skills, Practice-based Learning and Improvement, Professionalism, and Systems-based Practice. Since their introduction in 2013, several barriers to implementation have emerged. Thus, the ACGME launched the Milestones 2.0 project to develop updated specialty-specific milestones. The Pediatric Endocrinology Milestones 2.0 project aimed to improve upon Milestones 1.0 by addressing common limitations, providing resources for faculty to easily incorporate milestones into their assessment of trainees, and adding sub-competencies in health disparities, patient safety, and physician well-being.This paper reviews the development of the Pediatric Endocrinology Milestones 2.0 including the major changes from Milestones 1.0, development of the Supplemental Guide, and how Milestones 2.0 can be applied at the program level. Although use of the Milestones are required only for ACGME programs, the tools provided in Milestones 2.0 are applicable to fellowship programs worldwide.
Collapse
Affiliation(s)
- Cara V Tillotson
- Pediatric Endocrinology, New York Presbyterian - Columbia University Medical Center, 622 West 168th street, PH 17 West, Room 307, New York, NY, 10032, USA
| | - Imen Becetti
- Pediatric Endocrinology, Massachusetts General Hospital, Boston, MA, USA
| | - Katherine Hwu
- Pediatric Endocrinology, Baylor College of Medicine, Houston, TX, USA
| | - Laura Page
- Pediatric Endocrinology, Duke University School of Medicine, Durham, NC, USA
| | - Sowmya Krishnan
- Pediatric Endocrinology, The University of Texas Southwesteren Medical Center, Dallas, TX, USA
| | - Dianne Stafford
- Pediatric Endocrinology, Stanford University School of Medicine, Stanford, CA, USA.
| | - Takara Stanley
- Pediatric Endocrinology, Massachusetts General Hospital, Boston, MA, USA
| | - Patricia Vuguin
- Pediatric Endocrinology, New York Presbyterian - Columbia University Medical Center, 622 West 168th street, PH 17 West, Room 307, New York, NY, 10032, USA
| | - Jennifer M Barker
- Pediatric Endocrinology, University of Colorado Anschutz Medical Campus and Children's Hospital Colorado, Aurora, CO, USA
| |
Collapse
|
5
|
Acker A, Leifso K, Crawford L, Braund H, Hawksby E, Hall AK, McEwen L, Dalgarno N, Dagnone JD. Lessons learned and new strategies for success: Evaluating the Implementation of Competency-Based Medical Education in Queen's Pediatrics. Paediatr Child Health 2023; 28:463-467. [PMID: 38638538 PMCID: PMC11022870 DOI: 10.1093/pch/pxad021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Accepted: 04/17/2023] [Indexed: 04/20/2024] Open
Abstract
Objectives In 2017, Queen's University launched Competency-Based Medical Education (CBME) across 29 programs simultaneously. Two years post-implementation, we asked key stakeholders (faculty, residents, and program leaders) within the Pediatrics program for their perspectives on and experiences with CBME so far. Methods Program leadership explicitly described the intended outcomes of implementing CBME. Focus groups and interviews were conducted with all stakeholders to describe the enacted implementation. The intended versus enacted implementations were compared to provide insight into needed adaptations for program improvement. Results Overall, stakeholders saw value in the concept of CBME. Residents felt they received more specific feedback and monthly Competence Committee (CC) meetings and Academic Advisors were helpful. Conversely, all stakeholders noted the increased expectations had led to a feeling of assessment fatigue. Faculty noted that direct observation and not knowing a resident's previous performance information was challenging. Residents wanted to see faculty initiate assessments and improved transparency around progress and promotion decisions. Discussion The results provided insight into how well the intended outcomes had been achieved as well as areas for improvement. Proposed adaptations included a need for increased direct observation and exploration of faculty accessing residents' previous performance information. Education was provided on the performance expectations of residents and how progress and promotion decisions are made. As well, "flex blocks" were created to help residents customize their training experience to meet their learning needs. The results of this study can be used to inform and guide implementation and adaptations in other programs and institutions.
Collapse
Affiliation(s)
- Amy Acker
- Department of Pediatrics, Queen’s University, Kingston, Canada
| | - Kirk Leifso
- Department of Pediatrics, Queen’s University, Kingston, Canada
| | | | - Heather Braund
- Scholarship and Simulation Education, Office of Professional Development and Educational Scholarship, Faculty of Health Sciences, Queen’s University, Kingston, Canada
| | - Emily Hawksby
- Department of Pediatrics, Queen’s University, Kingston, Canada
| | - Andrew K Hall
- Department of Emergency Medicine, University of Ottawa, Ottawa, Canada
- Royal College of Physicians and Surgeons of Canada, Ottawa, Canada
| | - Laura McEwen
- Department of Pediatrics, Queen’s University, Kingston, Canada
- Postgraduate Medical Education Queen’s University, Kingston, Canada
| | - Nancy Dalgarno
- Education Scholarship, Office of Professional Development and Educational Scholarship, Kingston, Canada
- Department of Biomedical and Molecular Sciences, Faculty of Health Sciences, Queen’s University, Kingston, Canada
| | - Jeffrey Damon Dagnone
- Postgraduate Medical Education Queen’s University, Kingston, Canada
- Department of Emergency Medicine, Queen’s University, Kingston, Canada
| |
Collapse
|
6
|
Carney PA, Sebok-Syer SS, Pusic MV, Gillespie CC, Westervelt M, Goldhamer MEJ. Using learning analytics in clinical competency committees: Increasing the impact of competency-based medical education. MEDICAL EDUCATION ONLINE 2023; 28:2178913. [PMID: 36821373 PMCID: PMC9970252 DOI: 10.1080/10872981.2023.2178913] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/18/2022] [Revised: 02/02/2023] [Accepted: 02/07/2023] [Indexed: 06/18/2023]
Abstract
Graduate medical education (GME) and Clinical Competency Committees (CCC) have been evolving to monitor trainee progression using competency-based medical education principles and outcomes, though evidence suggests CCCs fall short of this goal. Challenges include that evaluation data are often incomplete, insufficient, poorly aligned with performance, conflicting or of unknown quality, and CCCs struggle to organize, analyze, visualize, and integrate data elements across sources, collection methods, contexts, and time-periods, which makes advancement decisions difficult. Learning analytics have significant potential to improve competence committee decision making, yet their use is not yet commonplace. Learning analytics (LA) is the interpretation of multiple data sources gathered on trainees to assess academic progress, predict future performance, and identify potential issues to be addressed with feedback and individualized learning plans. What distinguishes LA from other educational approaches is systematic data collection and advanced digital interpretation and visualization to inform educational systems. These data are necessary to: 1) fully understand educational contexts and guide improvements; 2) advance proficiency among stakeholders to make ethical and accurate summative decisions; and 3) clearly communicate methods, findings, and actionable recommendations for a range of educational stakeholders. The ACGME released the third edition CCC Guidebook for Programs in 2020 and the 2021 Milestones 2.0 supplement of the Journal of Graduate Medical Education (JGME Supplement) presented important papers that describe evaluation and implementation features of effective CCCs. Principles of LA underpin national GME outcomes data and training across specialties; however, little guidance currently exists on how GME programs can use LA to improve the CCC process. Here we outline recommendations for implementing learning analytics for supporting decision making on trainee progress in two areas: 1) Data Quality and Decision Making, and 2) Educator Development.
Collapse
Affiliation(s)
- Patricia A. Carney
- Professor of Family Medicine, Oregon Health & Science University, Portland, OR, USA
| | | | | | | | - Marjorie Westervelt
- Director of Assessment, Evaluation and Scholarship, University of California, Davis, CA, USA
| | - Mary Ellen J. Goldhamer
- Medicine, Harvard Medical School, Massachusetts General Hospital, and Mass General Brigham, Boston, MA, USA
| |
Collapse
|
7
|
Bell EA, Rufrano GA, Traylor AM, Ohning BL, Salas E. Enhancing team success in the neonatal intensive care unit: challenges and opportunities for fluid teams. Front Psychol 2023; 14:1284606. [PMID: 38023000 PMCID: PMC10662301 DOI: 10.3389/fpsyg.2023.1284606] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Accepted: 10/20/2023] [Indexed: 12/01/2023] Open
Abstract
Fluid teams, characterized by frequent changes in team membership, are vital in the neonatal intensive care unit (NICU) due to high patient acuity and the need for a wide range of specialized providers. However, many challenges can hinder effective teamwork in this setting. This article reviews the challenges related to fluid teamwork in the NICU and discusses recommendations from team science to address each challenge. Drawing from the current literature, this paper outlines three challenges that can hinder fluid teamwork in the NICU: incorporating patient families, managing hierarchy among team members, and facilitating effective patient handoffs. The review concludes with recommendations for managing NICU teamwork differently using strategies from team science.
Collapse
Affiliation(s)
| | | | | | - Bryan L. Ohning
- School of Medicine Greenville, University of South Carolina, Greenville, SC, United States
| | | |
Collapse
|
8
|
Holmboe ES, Osman NY, Murphy CM, Kogan JR. The Urgency of Now: Rethinking and Improving Assessment Practices in Medical Education Programs. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2023; 98:S37-S49. [PMID: 37071705 DOI: 10.1097/acm.0000000000005251] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Assessment is essential to professional development. Assessment provides the information needed to give feedback, support coaching and the creation of individualized learning plans, inform progress decisions, determine appropriate supervision levels, and, most importantly, help ensure patients and families receive high-quality, safe care in the training environment. While the introduction of competency-based medical education has catalyzed advances in assessment, much work remains to be done. First, becoming a physician (or other health professional) is primarily a developmental process, and assessment programs must be designed using a developmental and growth mindset. Second, medical education programs must have integrated programs of assessment that address the interconnected domains of implicit, explicit and structural bias. Third, improving programs of assessment will require a systems-thinking approach. In this paper, the authors first address these overarching issues as key principles that must be embraced so that training programs may optimize assessment to ensure all learners achieve desired medical education outcomes. The authors then explore specific needs in assessment and provide suggestions to improve assessment practices. This paper is by no means inclusive of all medical education assessment challenges or possible solutions. However, there is a wealth of current assessment research and practice that medical education programs can use to improve educational outcomes and help reduce the harmful effects of bias. The authors' goal is to help improve and guide innovation in assessment by catalyzing further conversations.
Collapse
Affiliation(s)
- Eric S Holmboe
- E.S. Holmboe is chief, Research, Milestones Development and Evaluation, Accreditation Council for Graduate Medical Education, Chicago, Illinois; ORCID: https://orcid.org/0000-0003-0108-6021
| | - Nora Y Osman
- N.Y. Osman is associate professor of medicine, Harvard Medical School, and director of undergraduate medical education, Brigham and Women's Hospital Department of Medicine, Boston, Massachusetts; ORCID: https://orcid.org/0000-0003-3542-1262
| | - Christina M Murphy
- C.M. Murphy is a fourth-year medical student and president, Medical Student Government at Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania; ORCID: https://orcid.org/0000-0003-3966-5264
| | - Jennifer R Kogan
- J.R. Kogan is associate dean, Student Success and Professional Development, and professor of medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania; ORCID: https://orcid.org/0000-0001-8426-9506
| |
Collapse
|
9
|
Hauer KE, Park YS, Bullock JL, Tekian A. "My Assessments Are Biased!" Measurement and Sociocultural Approaches to Achieve Fairness in Assessment in Medical Education. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2023; 98:S16-S27. [PMID: 37094278 DOI: 10.1097/acm.0000000000005245] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/03/2023]
Abstract
Assessing learners is foundational to their training and developmental growth throughout the medical education continuum. However, growing evidence shows the prevalence and impact of harmful bias in assessments in medical education, accelerating the urgency to identify solutions. Assessment bias presents a critical problem for all stages of learning and the broader educational system. Bias poses significant challenges to learners, disrupts the learning environment, and threatens the pathway and transition of learners into health professionals. While the topic of assessment bias has been examined within the context of measurement literature, limited guidance and solutions exist for learners in medical education, particularly in the clinical environment. This article presents an overview of assessment bias, focusing on clinical learners. A definition of bias and its manifestations in assessments are presented. Consequences of assessment bias are discussed within the contexts of validity and fairness and their impact on learners, patients/caregivers, and the broader field of medicine. Messick's unified validity framework is used to contextualize assessment bias; in addition, perspectives from sociocultural contexts are incorporated into the discussion to elaborate the nuanced implications in the clinical training environment. Discussions of these topics are conceptualized within the literature and the interventions used to date. The article concludes with practical recommendations to overcome bias and to develop an ideal assessment system. Recommendations address articulating values to guide assessment, designing assessment to foster learning and outcomes, attending to assessment procedures, promoting continuous quality improvement of assessment, and fostering equitable learning and assessment environments.
Collapse
Affiliation(s)
- Karen E Hauer
- K.E. Hauer is associate dean for competency assessment and professional standards, and professor, Department of Medicine, University of California, San Francisco School of Medicine, San Francisco, California; ORCID: http://orcid.org/0000-0002-8812-4045
| | - Yoon Soo Park
- Y.S. Park is associate professor and associate head, Department of Medical Education, University of Illinois at Chicago College of Medicine, Chicago, Illinois; ORCID: http://orcid.org/0000-0001-8583-4335
| | - Justin L Bullock
- J.L. Bullock is a fellow, Department of Medicine, Division of Nephrology, University of Washington School of Medicine, Seattle, Washington; ORCID: http://orcid.org/0000-0003-4240-9798
| | - Ara Tekian
- A. Tekian is professor and associate dean for international education, Department of Medical Education, University of Illinois at Chicago College of Medicine, Chicago, Illinois; ORCID: http://orcid.org/0000-0002-9252-1588
| |
Collapse
|
10
|
Chang YC, Khwepeya M, Nkambule NS, Chuang RS, Chaou CH. Emergency residents' self-perceived readiness for practice: the association of milestones, entrustable professional activities, and professional identities-a multi-institutional survey. Front Med (Lausanne) 2023; 10:1032516. [PMID: 37250629 PMCID: PMC10213224 DOI: 10.3389/fmed.2023.1032516] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Accepted: 04/17/2023] [Indexed: 05/31/2023] Open
Abstract
Background As a successful innovation, competency-based medical education and its assessment tools continue to be a key strategy in training future doctors and tracking their performance trajectories. Linked to professional identity, evidence suggests that clinical competence is related to thinking, acting and feeling like a physician. Thus, incorporating the values and attitudes of healthcare professions as part of their professional identity in the clinical workplace improves professional performance. Methods Through a cross-sectional study, we examined the association of milestone, entrustable professional activities (EPA) and professional identity using self-reported tools among emergency medicine residents from 12 teaching hospitals across Taiwan. Milestone, EPA and professional identity were assessed using the Emergency Medicine Milestone Scale, Entrustable Professional Activity Scale and Emergency Physician Professional Identity and Value Scale, respectively. Results The results of a Pearson correlation indicated a significant positive correlation between milestone-based core competencies and EPAs (r = 0.40 ~ 0.74, p < 0.01). The professional identity domain of skills acquisition, capabilities and practical wisdom was positively correlated with milestone-based core competencies of patient care, medical knowledge, practice-based learning and improvement, and system-based practice (r = 0.18 ~ 0.21, p ≤ 0.05), and six items of EPA (r = 0.16 ~ 0.22, p < 0.05). Additionally, the professional identity domain of professional recognition and self-esteem was positively correlated with practice-based learning and improvement, and system-based practice milestone competencies (r = 0.16 ~ 0.19, p < 0.05). Conclusion This study demonstrates milestone and EPA assessment tools are highly linked and therefore, can be synergistically used by supervisors and clinical educators to evaluate clinical performance during residency training. Emergency physicians' professional identity is partly influenced by the advancement of skills and a resident's ability to learn, effectively perform tasks and make appropriate medical decisions at the system level in their clinical practice. Further research is warranted to understand the importance of residents' competency in relation to their professional identity development trajectory during clinical training.
Collapse
Affiliation(s)
- Yu-Che Chang
- Chang Gung Medical Education Research Centre, Chang Gung Memorial Hospital, Taoyuan, Taiwan
- Department of Emergency Medicine, Chang Gung Memorial Hospital, Taoyuan, Taiwan
- College of Medicine, Chang Gung University, Taoyuan, Taiwan
| | - Madalitso Khwepeya
- Chang Gung Medical Education Research Centre, Chang Gung Memorial Hospital, Taoyuan, Taiwan
- College of Medicine, Chang Gung University, Taoyuan, Taiwan
| | - Nothando S. Nkambule
- International Graduate Program of Education and Human Development (IGPEHD), National Sun Yat-Sen University, Kaohsiung, Taiwan
| | - Renee S. Chuang
- Health Policy and Leadership Program, School of Public Health, Loma Linda University, Loma Linda, CA, United States
| | - Chung-Hsien Chaou
- Chang Gung Medical Education Research Centre, Chang Gung Memorial Hospital, Taoyuan, Taiwan
- Department of Emergency Medicine, Chang Gung Memorial Hospital, Taoyuan, Taiwan
- College of Medicine, Chang Gung University, Taoyuan, Taiwan
| |
Collapse
|
11
|
Sagalowsky ST, Rudinsky SL, Gray JM, Edgar L. What to know about 2.0: Revisions to the pediatric emergency medicine milestones. AEM EDUCATION AND TRAINING 2023; 7:e10865. [PMID: 37064491 PMCID: PMC10090486 DOI: 10.1002/aet2.10865] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/11/2023] [Accepted: 03/01/2023] [Indexed: 06/19/2023]
Abstract
Introduction The Accreditation Council for Graduate Medical Education (ACGME) adopted educational milestones for trainee assessment in 2013, as a key component of the Next Accreditation System. Two years later, the ACGME, American Board of Pediatrics (ABP), and American Board of Emergency Medicine (ABEM) collaborated to create specialty-specific subcompetencies in pediatric emergency medicine (PEM). Since that time, emerging data have demonstrated the need to revise specialty milestones. Consequently, the ACGME summoned a working group to revise the original PEM milestones. Methods The PEM Milestones 2.0 Working Group convened in April 2021, comprising diverse representation from ACGME staff, PEM and EM attendings, PEM fellowship program directors, PEM fellows, and community members, overseen by the ACGME's vice president for milestones development. The group met virtually six times over 3 months, with concurrent independent and subgroup work, to draft the PEM Milestones 2.0 and supplemental guide. Results The working group's revisions generated concise descriptions of behavioral anchors to define skill acquisition more accurately; attention to the transition from residency to fellowship training; incorporation of harmonized milestones for non-patient care and non-medical knowledge domains; and increased emphasis on diversity, equity, and inclusion. A supplemental guide was also designed to aid programs in designing programmatic assessment related to specialty-specific milestones. Conclusions The PEM Milestones 2.0 provide an updated, specialty-specific framework to guide the development and assessment of PEM fellows and training programs. Future work may focus on faculty and learner development, advancing validity evidence, strengthening content expertise, and integrating milestones with specialty-specific entrustable professional activities.
Collapse
Affiliation(s)
- Selin T. Sagalowsky
- Department of Emergency MedicineNew York University Grossman School of MedicineNew YorkNew YorkUSA
| | - Sherri L. Rudinsky
- Military and Emergency MedicineUniformed Services UniversityBethesdaMarylandUSA
| | - James M. Gray
- Department of PediatricsUniversity of CincinnatiCincinnatiOhioUSA
- Division of Emergency MedicineCincinnati Children's Hospital Medical CenterCincinnatiOhioUSA
| | - Laura Edgar
- Accreditation Council for Graduate Medical EducationChicagoIllinoisUSA
| |
Collapse
|
12
|
Swails JL, Angus S, Barone MA, Bienstock J, Burk-Rafel J, Roett MA, Hauer KE. The Undergraduate to Graduate Medical Education Transition as a Systems Problem: A Root Cause Analysis. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2023; 98:180-187. [PMID: 36538695 DOI: 10.1097/acm.0000000000005065] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/17/2023]
Abstract
The transition from undergraduate medical education (UME) to graduate medical education (GME) constitutes a complex system with important implications for learner progression and patient safety. The transition is currently dysfunctional, requiring students and residency programs to spend significant time, money, and energy on the process. Applications and interviews continue to increase despite stable match rates. Although many in the medical community acknowledge the problems with the UME-GME transition and learners have called for prompt action to address these concerns, the underlying causes are complex and have defied easy fixes. This article describes the work of the Coalition for Physician Accountability's Undergraduate Medical Education to Graduate Medical Education Review Committee (UGRC) to apply a quality improvement approach and systems thinking to explore the underlying causes of dysfunction in the UME-GME transition. The UGRC performed a root cause analysis using the 5 whys and an Ishikawa (or fishbone) diagram to deeply explore problems in the UME-GME transition. The root causes of problems identified include culture, costs and limited resources, bias, systems, lack of standards, and lack of alignment. Using the principles of systems thinking (components, connections, and purpose), the UGRC considered interactions among the root causes and developed recommendations to improve the UME-GME transition. Several of the UGRC's recommendations stemming from this work are explained. Sustained monitoring will be necessary to ensure interventions move the process forward to better serve applicants, programs, and the public good.
Collapse
Affiliation(s)
- Jennifer L Swails
- J.L. Swails is residency program director, codirector of interprofessional education, and associate professor, Department of Medicine, McGovern Medical School, University of Texas Health Science Center, Houston, Texas; ORCID: http://orcid.org/0000-0002-6102-831X
| | - Steven Angus
- S. Angus is designated institutional official, vice-chair for education, and professor, Department of Medicine, University of Connecticut School of Medicine, Farmington, Connecticut
| | - Michael A Barone
- M.A. Barone is vice president of competency-based assessment, NBME, Philadelphia, Pennsylvania, and adjunct associate professor of pediatrics, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | - Jessica Bienstock
- J. Bienstock is professor of gynecology and obstetrics, associate dean for graduate medical education, and designated institutional official, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | - Jesse Burk-Rafel
- J. Burk-Rafel is assistant professor of medicine and assistant director of UME-GME innovation, Institute for Innovations in Medical Education, NYU Grossman School of Medicine, New York, New York
| | - Michelle A Roett
- M.A. Roett is professor and chair, Department of Family Medicine, Georgetown University Medicine Center, and clinical chief of family medicine, MedStar Georgetown University Hospital, Washington, DC
| | - Karen E Hauer
- K.E. Hauer is associate dean for competency assessment and professional standards and professor, Department of Medicine, University of California, San Francisco School of Medicine, San Francisco, California
| |
Collapse
|
13
|
Public Health and General Preventive Medicine Residency Milestones 2.0. Am J Prev Med 2023; 64:765-771. [PMID: 36690545 DOI: 10.1016/j.amepre.2022.11.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/28/2022] [Revised: 10/27/2022] [Accepted: 11/02/2022] [Indexed: 01/23/2023]
|
14
|
Zavodnick J, Doroshow J, Rosenberg S, Banks J, Leiby BE, Mingioni N. Hawks and Doves: Perceptions and Reality of Faculty Evaluations. JOURNAL OF MEDICAL EDUCATION AND CURRICULAR DEVELOPMENT 2023; 10:23821205231197079. [PMID: 37692558 PMCID: PMC10492463 DOI: 10.1177/23821205231197079] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Accepted: 08/08/2023] [Indexed: 09/12/2023]
Abstract
OBJECTIVES Internal medicine clerkship grades are important for residency selection, but inconsistencies between evaluator ratings threaten their ability to accurately represent student performance and perceived fairness. Clerkship grading committees are recommended as best practice, but the mechanisms by which they promote accuracy and fairness are not certain. The ability of a committee to reliably assess and account for grading stringency of individual evaluators has not been previously studied. METHODS This is a retrospective analysis of evaluations completed by faculty considered to be stringent, lenient, or neutral graders by members of a grading committee of a single medical college. Faculty evaluations were assessed for differences in ratings on individual skills and recommendations for final grade between perceived stringency categories. Logistic regression was used to determine if actual assigned ratings varied based on perceived faculty's grading stringency category. RESULTS "Easy graders" consistently had the highest probability of awarding an above-average rating, and "hard graders" consistently had the lowest probability of awarding an above-average rating, though this finding only reached statistical significance only for 2 of 8 questions on the evaluation form (P = .033 and P = .001). Odds ratios of assigning a higher final suggested grade followed the expected pattern (higher for "easy" and "neutral" compared to "hard," higher for "easy" compared to "neutral") but did not reach statistical significance. CONCLUSIONS Perceived differences in faculty grading stringency have basis in reality for clerkship evaluation elements. However, final grades recommended by faculty perceived as "stringent" or "lenient" did not differ. Perceptions of "hawks" and "doves" are not just lore but may not have implications for students' final grades. Continued research to describe the "hawk and dove effect" will be crucial to enable assessment of local grading variation and empower local educational leadership to correct, but not overcorrect, for this effect to maintain fairness in student evaluations.
Collapse
Affiliation(s)
- Jillian Zavodnick
- Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, USA
| | | | - Sarah Rosenberg
- Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, USA
| | - Joshua Banks
- Department of Pharmacology and Experimental Therapeutics, Division of Biostatistics, Thomas Jefferson University, Philadelphia, USA
| | - Benjamin E Leiby
- Department of Pharmacology and Experimental Therapeutics, Division of Biostatistics, Thomas Jefferson University, Philadelphia, USA
| | - Nina Mingioni
- Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, USA
| |
Collapse
|
15
|
Zaccagnini M, Bussières A, Mak S, Boruff J, West A, Thomas A. Scholarly practice in healthcare professions: findings from a scoping review. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2022:10.1007/s10459-022-10180-0. [PMID: 36456756 DOI: 10.1007/s10459-022-10180-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/22/2022] [Accepted: 10/16/2022] [Indexed: 06/17/2023]
Abstract
Scholarly practitioners are broadly defined as healthcare professionals that address critical practice problems using theory, scientific evidence, and practice-based knowledge. Though scholarly practice is included in most competency frameworks, it is unclear what scholarly practice is, how it develops and how it is operationalized in clinical practice. The aim of this review was to determine what is known about scholarly practice in healthcare professionals. We conducted a scoping review and searched MEDLINE, EMBASE, CINAHL from inception to May 2020. We included papers that explored, described, or defined scholarly practice, scholar or scholarly practitioner, and/or related concepts in healthcare professionals. We included a total of 90 papers. Thirty percent of papers contained an explicit definition of scholarly practice. Conceptualizations of scholarly practice were organized into three themes: the interdependent relationship between scholarship and practice; advancing the profession's field; and core to being a healthcare practitioner. Attributes of scholarly practitioners clustered around five themes: commitment to excellence in practice; collaborative nature; presence of virtuous characteristics; effective communication skills; and adaptive change ethos. No single unified definition of scholarly practice exists within the literature. The variability in terms used to describe scholarly practice suggests that it is an overarching concept rather than a definable entity. There are similarities between scholarly practitioners and knowledge brokers regarding attributes and how scholarly practice is operationalized. Individuals engaged in the teaching, research and/or assessment of scholarly practice should make explicit their definitions and expectations for healthcare professionals.
Collapse
Affiliation(s)
- Marco Zaccagnini
- School of Physical and Occupational Therapy, McGill University, 3654 Promenade Sir William Osler, Montréal, QC, H3G 1Y5, Canada
- Centre for Interdisciplinary Research in Rehabilitation of Greater Montréal, Montréal, QC, Canada
| | - André Bussières
- School of Physical and Occupational Therapy, McGill University, 3654 Promenade Sir William Osler, Montréal, QC, H3G 1Y5, Canada
- Centre for Interdisciplinary Research in Rehabilitation of Greater Montréal, Montréal, QC, Canada
- Département Chiropratique, Université du Québec à Trois-Rivières, Trois-Rivières, QC, Canada
| | - Susanne Mak
- School of Physical and Occupational Therapy, McGill University, 3654 Promenade Sir William Osler, Montréal, QC, H3G 1Y5, Canada
- Centre for Interdisciplinary Research in Rehabilitation of Greater Montréal, Montréal, QC, Canada
- Institute of Health Sciences Education, McGill University, Montréal, QC, Canada
| | - Jill Boruff
- Schulich Library of Physical Sciences, Life Sciences, and Engineering, McGill University, Montréal, QC, Canada
| | - Andrew West
- The Canadian Society of Respiratory Therapists, Saint John, NB, Canada
| | - Aliki Thomas
- School of Physical and Occupational Therapy, McGill University, 3654 Promenade Sir William Osler, Montréal, QC, H3G 1Y5, Canada.
- Centre for Interdisciplinary Research in Rehabilitation of Greater Montréal, Montréal, QC, Canada.
- Institute of Health Sciences Education, McGill University, Montréal, QC, Canada.
| |
Collapse
|
16
|
Ekpenyong A, Edgar L, Wilkerson L, Holmboe ES. A multispecialty ethnographic study of clinical competency committees (CCCs). MEDICAL TEACHER 2022; 44:1228-1236. [PMID: 35635737 DOI: 10.1080/0142159x.2022.2072281] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
PURPOSE Clinical competency committees (CCCs) assess residents' performance on their specialty specific milestones, however there is no 'one-size fits all' blueprint for accomplishing this. Thus, CCCs have had to develop their own procedures. The goal of this study was to examine these efforts to assist new programs embarking on this venture and established programs looking to improve their CCC practices and processes. METHODS We purposefully sampled CCCs across multiple specialties and institutions. Data from three sources were triangulated: (1) online demographic survey, (2) ethnographic observations of CCC meetings and (3) post-observation semi-structured interviews with the program director and/or CCC chairperson. Template analysis was used to build the coding structure. RESULTS Sixteen observations were completed with 15 different CCCs at 9 institutions. Three main thematic categories that impact the operations of CCCs emerged: (1) Membership structure and members roles, (2) Roles of the CCC in residency and 3) CCC processes, including trainee presentation to the committee and decision-making. While effective practices were observed, substantial variation existed in all three thematic areas. CONCLUSIONS While CCCs used some known effective practices, substantial variation in structure and processes was notable across CCCs. Future work should explore the impact of this variation on educational outcomes.
Collapse
Affiliation(s)
| | - Laura Edgar
- Accreditation Council for Graduate Medical Education, Chicago, IL, USA
| | - LuAnn Wilkerson
- Dell Medical School, The University of Texas at Austin, Austin, TX, USA
| | - Eric S Holmboe
- Accreditation Council for Graduate Medical Education, Chicago, IL, USA
| |
Collapse
|
17
|
Positive Patient Postoperative Outcomes with Pharmacotherapy: A Narrative Review including Perioperative-Specialty Pharmacist Interviews. J Clin Med 2022; 11:jcm11195628. [PMID: 36233497 PMCID: PMC9572852 DOI: 10.3390/jcm11195628] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2022] [Revised: 09/15/2022] [Accepted: 09/21/2022] [Indexed: 11/19/2022] Open
Abstract
The influence of pharmacotherapy regimens on surgical patient outcomes is increasingly appreciated in the era of enhanced recovery protocols and institutional focus on reducing postoperative complications. Specifics related to medication selection, dosing, frequency of administration, and duration of therapy are evolving to optimize pharmacotherapeutic regimens for many enhanced recovery protocolized elements. This review provides a summary of recent pharmacotherapeutic strategies, including those configured within electronic health record (EHR) applications and functionalities, that are associated with the minimization of the frequency and severity of postoperative complications (POCs), shortened hospital length of stay (LOS), reduced readmission rates, and cost or revenue impacts. Further, it will highlight preventive pharmacotherapy regimens that are correlated with improved patient preparation, especially those related to surgical site infection (SSI), venous thromboembolism (VTE), nausea and vomiting (PONV), postoperative ileus (POI), and emergence delirium (PoD) as well as less commonly encountered POCs such as acute kidney injury (AKI) and atrial fibrillation (AF). The importance of interprofessional collaboration in all periprocedural phases, focusing on medication management through shared responsibilities for drug therapy outcomes, will be emphasized. Finally, examples of collaborative care through shared mental models of drug stewardship and non-medical practice agreements to improve operative throughput, reduce operative stress, and increase patient satisfaction are illustrated.
Collapse
|
18
|
Concordance of Narrative Comments with Supervision Ratings Provided During Entrustable Professional Activity Assessments. J Gen Intern Med 2022; 37:2200-2207. [PMID: 35710663 PMCID: PMC9296736 DOI: 10.1007/s11606-022-07509-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/08/2021] [Accepted: 03/24/2022] [Indexed: 10/18/2022]
Abstract
BACKGROUND Use of EPA-based entrustment-supervision ratings to determine a learner's readiness to assume patient care responsibilities is expanding. OBJECTIVE In this study, we investigate the correlation between narrative comments and supervision ratings assigned during ad hoc assessments of medical students' performance of EPA tasks. DESIGN Data from assessments completed for students enrolled in the clerkship phase over 2 academic years were used to extract a stratified random sample of 100 narrative comments for review by an expert panel. PARTICIPANTS A review panel, comprised of faculty with specific expertise related to their roles within the EPA program, provided a "gold standard" supervision rating using the comments provided by the original assessor. MAIN MEASURES Interrater reliability (IRR) between members of review panel and correlation coefficients (CC) between expert ratings and supervision ratings from original assessors. KEY RESULTS IRR among members of the expert panel ranged from .536 for comments associated with focused history taking to .833 for complete physical exam. CC (Kendall's correlation coefficient W) between panel members' assignment of supervision ratings and the ratings provided by the original assessors for history taking, physical examination, and oral presentation comments were .668, .697, and .735 respectively. The supervision ratings of the expert panel had the highest degree of correlation with ratings provided during assessments done by master assessors, faculty trained to assess students across clinical contexts. Correlation between supervision ratings provided with the narrative comments at the time of observation and supervision ratings assigned by the expert panel differed by clinical discipline, perhaps reflecting the value placed on, and perhaps the comfort level with, assessment of the task in a given specialty. CONCLUSIONS To realize the full educational and catalytic effect of EPA assessments, assessors must apply established performance expectations and provide high-quality narrative comments aligned with the criteria.
Collapse
|
19
|
Reimagining the Clinical Competency Committee to Enhance Education and Prepare for Competency-Based Time-Variable Advancement. J Gen Intern Med 2022; 37:2280-2290. [PMID: 35445932 PMCID: PMC9021365 DOI: 10.1007/s11606-022-07515-3] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/15/2021] [Accepted: 03/25/2022] [Indexed: 12/01/2022]
Abstract
Assessing residents and clinical fellows is a high-stakes activity. Effective assessment is important throughout training so that identified areas of strength and weakness can guide educational planning to optimize outcomes. Assessment has historically been underemphasized although medical education oversight organizations have strengthened requirements in recent years. Growing acceptance of competency-based medical education and its logical extension to competency-based time-variable (CB-TV) graduate medical education (GME) further highlights the importance of implementing effective evidence-based approaches to assessment. The Clinical Competency Committee (CCC) has emerged as a key programmatic structure in graduate medical education. In the context of launching a multi-specialty pilot of CB-TV GME in our health system, we have examined several program's CCC processes and reviewed the relevant literature to propose enhancements to CCCs. We recommend that all CCCs fulfill three core goals, regularly applied to every GME trainee: (1) discern and describe the resident's developmental status to individualize education, (2) determine readiness for unsupervised practice, and (3) foster self-assessment ability. We integrate the literature and observations from GME program CCCs in our institutions to evaluate how current CCC processes support or undermine these goals. Obstacles and key enablers are identified. Finally, we recommend ways to achieve the stated goals, including the following: (1) assess and promote the development of competency in all trainees, not just outliers, through a shared model of assessment and competency-based advancement; (2) strengthen CCC assessment processes to determine trainee readiness for independent practice; and (3) promote trainee reflection and informed self-assessment. The importance of coaching for competency, robust workplace-based assessments, feedback, and co-production of individualized learning plans are emphasized. Individual programs and their CCCs must strengthen assessment tools and frameworks to realize the potential of competency-oriented education.
Collapse
|
20
|
Hauer KE, O'Sullivan PS. Making Sense of Milestones Data-Guiding Residents or Assessing Training Programs? JAMA Netw Open 2021; 4:e2137606. [PMID: 34874410 DOI: 10.1001/jamanetworkopen.2021.37606] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Affiliation(s)
- Karen E Hauer
- School of Medicine, University of California, San Francisco
| | - Patricia S O'Sullivan
- Center for Faculty Educators, School of Medicine, University of California, San Francisco
| |
Collapse
|
21
|
Dhaliwal G, Hauer KE. Excellence in medical training: developing talent-not sorting it. PERSPECTIVES ON MEDICAL EDUCATION 2021; 10:356-361. [PMID: 34415554 PMCID: PMC8377327 DOI: 10.1007/s40037-021-00678-5] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/24/2020] [Revised: 06/22/2021] [Accepted: 06/23/2021] [Indexed: 06/02/2023]
Abstract
Many medical schools have reconsidered or eliminated clerkship grades and honor society memberships. National testing organizations announced plans to eliminate numerical scoring for the United States Medical Licensing Examination Step 1 in favor of pass/fail results. These changes have led some faculty to wonder: "How will we recognize and reward excellence?" Excellence in undergraduate medical education has long been defined by high grades, top test scores, honor society memberships, and publication records. However, this model of learner excellence is misaligned with how students learn or what society values. This accolade-driven view of excellence is perpetuated by assessments that are based on gestalt impressions influenced by similarity between evaluators and students, and assessments that are often restricted to a limited number of traditional skill domains. To achieve a new model of learner excellence that values the trainee's achievement, growth, and responsiveness to feedback across multiple domains, we must envision a new model of teacher excellence. Such teachers would have a growth mindset toward assessing competencies and learning new competencies. Actualizing true learner excellence will require teachers to change from evaluators who conduct assessments of learning to coaches who do assessment for learning. Schools will also need to establish policies and structures that foster a culture that supports this change. In this new paradigm, a teacher's core duty is to develop talent rather than sort it.
Collapse
Affiliation(s)
- Gurpreet Dhaliwal
- Department of Medicine, University of California San Francisco School of Medicine, San Francisco, CA, USA.
- Medical Service, San Francisco VA Medical Center, San Francisco, CA, USA.
| | - Karen E Hauer
- Department of Medicine, University of California San Francisco School of Medicine, San Francisco, CA, USA
| |
Collapse
|
22
|
Andolsek KM, Jones MD, Ibrahim H, Edgar L. Introduction to the Milestones 2.0: Assessment, Implementation, and Clinical Competency Committees Supplement. J Grad Med Educ 2021; 13:1-4. [PMID: 33936527 PMCID: PMC8078078 DOI: 10.4300/jgme-d-21-00298.1] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/06/2022] Open
Affiliation(s)
- Kathryn M. Andolsek
- Kathryn M. Andolsek, MD, MPH, is Professor, Department of Family Medicine and Community Health, Assistant Dean, Premedical Education, and Senior Fellow, Center for Study of Aging and Human Development, Duke University School of Medicine, and Associate Editor, Journal of Graduate Medical Education (JGME)
| | - M. Douglas Jones
- M. Douglas Jones Jr, MD, is Professor of Pediatrics, University of Colorado School of Medicine, and Associate Editor, JGME
| | - Halah Ibrahim
- Halah Ibrahim, MD, MEHP, is Consultant, Department of Medicine, Sheikh Khalifa Medical City, Abu Dhabi, UAE, Adjunct Assistant Professor, Department of Medicine, Johns Hopkins University School of Medicine, and Associate Editor, JGME
| | - Laura Edgar
- Laura Edgar, EdD, CAE, is Vice President, Milestones Development, Accreditation Council for Graduate Medical Education
| |
Collapse
|
23
|
Ekpenyong A, Padmore JS, Hauer KE. The Purpose, Structure, and Process of Clinical Competency Committees: Guidance for Members and Program Directors. J Grad Med Educ 2021; 13:45-50. [PMID: 33936532 PMCID: PMC8078071 DOI: 10.4300/jgme-d-20-00841.1] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/13/2023] Open
Affiliation(s)
- Andem Ekpenyong
- Andem Ekpenyong, MD, MHPE, is Associate Professor, Department of Internal Medicine, Rush University Medical Center
| | - Jamie S. Padmore
- Jamie S. Padmore, DM, is Professor and Senior Associate Dean for Medical Education, Georgetown University Medical Center, and Vice President, Academic Affairs, and Designated Institutional Official, MedStar Health
| | - Karen E. Hauer
- Karen E. Hauer, MD, PhD, is Associate Dean, Competency Assessment and Professional Standards, and Professor of Medicine, University of California, San Francisco
| |
Collapse
|