1
|
Alharbi NS. Evaluating competency-based medical education: a systematized review of current practices. BMC MEDICAL EDUCATION 2024; 24:612. [PMID: 38831271 PMCID: PMC11149276 DOI: 10.1186/s12909-024-05609-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Accepted: 05/27/2024] [Indexed: 06/05/2024]
Abstract
BACKGROUND Few published articles provide a comprehensive overview of the available evidence on the topic of evaluating competency-based medical education (CBME) curricula. The purpose of this review is therefore to synthesize the available evidence on the evaluation practices for competency-based curricula employed in schools and programs for undergraduate and postgraduate health professionals. METHOD This systematized review was conducted following the systematic reviews approach with minor modifications to synthesize the findings of published studies that examined the evaluation of CBME undergraduate and postgraduate programs for health professionals. RESULTS Thirty-eight articles met the inclusion criteria and reported evaluation practices in CBME curricula from various countries and regions worldwide, such as Canada, China, Turkey, and West Africa. 57% of the evaluated programs were at the postgraduate level, and 71% were in the field of medicine. The results revealed variation in reporting evaluation practices, with numerous studies failing to clarify evaluations' objectives, approaches, tools, and standards as well as how evaluations were reported and communicated. It was noted that questionnaires were the primary tool employed for evaluating programs, often combined with interviews or focus groups. Furthermore, the utilized evaluation standards considered the well-known competencies framework, specialized association guidelines, and accreditation criteria. CONCLUSION This review calls attention to the importance of ensuring that reports of evaluation experiences include certain essential elements of evaluation to better inform theory and practice.
Collapse
Affiliation(s)
- Nouf Sulaiman Alharbi
- Department of Medical Education, College of Medicine, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia.
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia.
- Ministry of the National Guard - Health Affairs, Riyadh, Saudi Arabia.
| |
Collapse
|
2
|
Karami M, Hashemi N, Van Merrienboer J. From obese to lean curriculum: exploring students' experiences about developing competencies in medical education. Front Med (Lausanne) 2024; 11:1309548. [PMID: 38841567 PMCID: PMC11150563 DOI: 10.3389/fmed.2024.1309548] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Accepted: 05/10/2024] [Indexed: 06/07/2024] Open
Abstract
Introduction Since the beginning of the 21st century, competency-based education has been proposed as an approach to education in many disciplines including the medical sciences and it has become a dominant approach in many countries. We aimed to explore the lived experiences of general medical students about developing competencies in the academic curriculum. Methods We conducted a phenomenology method to study lived experiences of general medical students through selecting participants via a purposeful sampling strategy. Snowballing and maximum variation samplings were also applied to recruit additional participants. The study was conducted at a Medical School in Iran. Three successive phases of qualitative data analysis, namely, data reduction by coding, data structuring by categorization, and data interpretation by discussion were applied to analyze the interviews. Results The results of the research showed that students' lived experiences fall under 4 main themes with 9 subthemes. The main themes show that (1) the compartmentalized curriculum in basic courses is experienced as the missing parts in a puzzle, (2) the physiopathology curriculum is experienced as swimming on land, (3) the externship is experienced as touring a mysterious land, (4) the internship is experienced as unleashed arrows. Discussion Our findings reveal that despite the changes already made in the curriculum, its compartmentalization is still a main obstacle to achieving competency-based medical education. A strict requirement for leaving the discipline-based curriculum behind is to use an integrated approach, in which basic science courses are connected with clinical cases, and physiopathology courses are connected with externships and internships.
Collapse
Affiliation(s)
- Morteza Karami
- Department of Curriculum Studies and Instruction, Ferdowsi University of Mashhad, Mashhad, Iran
| | | | - Jeroen Van Merrienboer
- Department of Educational Development and Research, Maastricht University, Maastricht, Netherlands
| |
Collapse
|
3
|
Sibicky SL, Daugherty KK, Chen AMH, Rhoney D, Nawarskas J. Enabling Factors for the Implementation of Competency-Based Curricula in Colleges and Schools of Pharmacy. AMERICAN JOURNAL OF PHARMACEUTICAL EDUCATION 2024; 88:100681. [PMID: 38460599 DOI: 10.1016/j.ajpe.2024.100681] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/06/2022] [Revised: 09/22/2023] [Accepted: 03/05/2024] [Indexed: 03/11/2024]
Abstract
OBJECTIVES To review the implementation drivers of competency-based pharmacy education (CBPE) and provide recommendations for enablers. FINDINGS Competency-based education is an emerging model in the health professions, focusing on time-variable competency development and achievement compared with a time-bound, course-based, traditional model. CBPE is an outcomes-based organized framework of competencies enabling pharmacists to meet health care and societal needs. However, challenges need to be recognized and overcome for the successful implementation of CBPE. Competency drivers include defining the competencies and roles of stakeholders, developing transparent learning trajectories and aligned assessments, and establishing lifetime development programs for stakeholders. Organization drivers include developing support systems for stakeholders; facilitating connections between all educational experiences; and having transparent assessment plans, policies, and procedures that align with core CBPE precepts, including the sustainability of time-variability. Leadership drivers include establishing growth mindset and facilitating a culture of connection between workplace and educational environments, program advocacy by institutional leaders, accepting failures as part of the process, shifting the organizational culture away from learner differentiation toward competence, and maintaining sufficient administrative capability to support CBPE. SUMMARY The successful implementation of CBPE involves enabling the competency, organization, and leadership drivers that will lead to program success. More research is needed in the areas of creation, implementation, and assessment of CBPE to determine success in this model. We have reviewed and provided recommendations to enable the drivers of successful implementation of CBPE.
Collapse
Affiliation(s)
- Stephanie L Sibicky
- Northeastern University School of Pharmacy and Pharmaceutical Sciences, Boston, MA, USA.
| | - Kimberly K Daugherty
- Sullivan University College of Pharmacy and Health Sciences, Louisville, KY, USA
| | - Aleda M H Chen
- Cedarville University School of Pharmacy, Cedarville, OH, USA
| | - Denise Rhoney
- University of North Carolina Eshelman School of Pharmacy, Chapel Hill, NC, USA
| | - James Nawarskas
- University of New Mexico College of Pharmacy, University of New Mexico, Albuquerque, NM, USA
| |
Collapse
|
4
|
Shrivastava SR, Bankar NJ, Shrivastava PS, Gajbe U, Singh BR, Tiwade Y, Bandre A. From Likes to Saving Lives: Embracing Social Media's Power to Connect, Learn, and Innovate in Medical Education Delivery. JOURNAL OF PHARMACY AND BIOALLIED SCIENCES 2024; 16:S1043-S1047. [PMID: 38882771 PMCID: PMC11174254 DOI: 10.4103/jpbs.jpbs_1230_23] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2023] [Revised: 12/21/2023] [Accepted: 01/09/2024] [Indexed: 06/18/2024] Open
Abstract
Social media have become an integral part of human life, because the manner in which we communicate, share information, and network with the external world is changing significantly. The purpose of the current review is to explore the role of social media in transforming medical education, identify the merits and the potential concerns, and propose effective strategies to overcome the identified challenges. Social media have significant potential in strengthening the delivery of medical education and meeting the scenarios emerging out of changing dynamics. The most important aspect of social media is that they have global reach and wide accessibility, and promotes active engagement of students. Similar to any other teaching-learning method, even the use of social media has its own limitations-cum-concerns. We must come out with potential solutions to each one of these problems so that overall acceptance and utilization of social media applications can be optimized. In conclusion, social media in medical education have immense potential to revolutionize learning and provide a platform for collaborative learning and networking. The need of the hour is to do systematic planning, train stakeholders, and establish guidelines for promoting effective engagement while using social media, and thereby keeping pace with the rapidly evolving healthcare landscape.
Collapse
Affiliation(s)
- Saurabh RamBihariLal Shrivastava
- Deputy Director (Research and Development), Off Campus, Datta Meghe Institute of Higher Education and Research, Datta Meghe Medical College, Off-campus Centre of Datta Meghe Institute of Higher Education and Research, Hingna Road, Wanadongri, Nagpur, Maharashtra, India
- Department of Community Medicine, Datta Meghe Medical College, Off-campus Centre of Datta Meghe Institute of Higher Education and Research, Hingna Road, Wanadongri, Nagpur, Maharashtra, India
| | - Nandkishor J Bankar
- Department of Microbiology, Jawaharlal Nehru Medical College, Datta Meghe Institute of Higher Education and Research, Sawangi, Wardha, Maharashtra, India
| | | | - Ujwal Gajbe
- Department of Anatomy, Datta Meghe Medical College, Off-campus Centre of Datta Meghe Institute of Higher Education and Research, Hingna Road, Wanadongri, Nagpur, Maharashtra, India
| | - Brij Raj Singh
- Department of Anatomy, Datta Meghe Medical College, Off-campus Centre of Datta Meghe Institute of Higher Education and Research, Hingna Road, Wanadongri, Nagpur, Maharashtra, India
| | - Yugeshwari Tiwade
- Department of Pathology, Jawaharlal Nehru Medical College, Datta Meghe Institute of Higher Education and Research, Sawangi, Wardha, Maharashtra, India
| | - Ankit Bandre
- Department of Microbiology, Datta Meghe Medical College, Off-campus Centre of Datta Meghe Institute of Higher Education and Research, Hingna Road, Wanadongri, Nagpur, Maharashtra, India
| |
Collapse
|
5
|
Frank JR, Hall AK, Oswald A, Dagnone JD, Brand PLP, Reznick R. From Competence by Time to Competence by Design: Lessons From A National Transformation Initiative. PERSPECTIVES ON MEDICAL EDUCATION 2024; 13:224-228. [PMID: 38550713 PMCID: PMC10976982 DOI: 10.5334/pme.1342] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/29/2024] [Accepted: 03/06/2024] [Indexed: 04/02/2024]
Affiliation(s)
- Jason R. Frank
- Centre for Innovation in Medical Education, and Professor, Department of Emergency Medicine, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Andrew K. Hall
- Department of Emergency Medicine, University of Ottawa, Ottawa, ON, Canada
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | - Anna Oswald
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Competency Based Medical Education, and Professor, Division of Rheumatology, Department of Medicine, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, AB, Canada
| | - J. Damon Dagnone
- Department of Emergency Medicine, Queen’s University, Kingston, ON, Canada
- Standards and Accreditation, Royal College of Physicians & Surgeons of Canada, Ottawa, ON, Canada
| | - Paul L. P. Brand
- Clinical Medical Education, University Medical Centre and University of Groningen, the Netherlands
- Medical Education and Faculty Development, Isala Hospital, Zwolle, The Netherlands
| | - Richard Reznick
- Queen’s University, Immediate Past President Royal College of Physicians and Surgeons of Canada, Canada
| |
Collapse
|
6
|
Loi UR, Sorhaindo A, Embo M, Kabra R, Kiarie J, Ganatra B. Description of the methodology for developing and validating the WHO's family planning and comprehensive abortion care competencies for the primary health care workforce. SEXUAL & REPRODUCTIVE HEALTHCARE 2024; 39:100945. [PMID: 38237452 PMCID: PMC10951617 DOI: 10.1016/j.srhc.2023.100945] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2023] [Revised: 12/19/2023] [Accepted: 12/27/2023] [Indexed: 02/12/2024]
Abstract
A qualified health workforce is essential to receiving effective, timely, affordable, equitable and respectful family planning and comprehensive abortion care. However, in many countries, health workers lack the competencies required to deliver quality family planning and comprehensive abortion care services. Competency-based education and learning aims to train and assess competencies. The theory-supported approach focuses on outcomes, emphasizes the learner's ability to perform, promotes learner-centeredness and links the health needs of the population to the competencies required of health workers. In 2011, the World Health Organization published a guidance document, Sexual and reproductive health - Core competencies in primary care, defining the competencies that primary care providers need to safely deliver sexual and reproductive health services at the community level and included family planning and comprehensive abortion care. In this article, we describe the methodology and process undertaken in 2020, by the World Health Organization to produce the family planning and comprehensive abortion care competencies guidance, filling gaps identified in the previous guidance document. The World Health Organization's Family Planning and Comprehensive Abortion Care toolkit for the primary health care workforce was published in 2022 and defines the key competencies for health workers in primary health care providing quality family planning and comprehensive abortion care services, as well as support for developing programmes and curricula for education and lifelong learning. The Toolkit is useful for practitioners, managers/supervisors and employers, educators, regulatory bodies, and policymakers. It is an important advance toward strengthening family planning and comprehensive abortion care services in primary health care.
Collapse
Affiliation(s)
- Ulrika Rehnström Loi
- UNDP-UNFPA-UNICEF-WHO-World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP), Department of Sexual and Reproductive Health and Research, World Health Organization, 20 Avenue Appia, 1211 Geneva, Switzerland.
| | - Annik Sorhaindo
- UNDP-UNFPA-UNICEF-WHO-World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP), Department of Sexual and Reproductive Health and Research, World Health Organization, 20 Avenue Appia, 1211 Geneva, Switzerland
| | - Mieke Embo
- Department of Educational Studies, Faculty of Psychology and Educational Sciences, Ghent University, H. Dunantlaan 2, 9000 Ghent, Belgium
| | - Rita Kabra
- UNDP-UNFPA-UNICEF-WHO-World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP), Department of Sexual and Reproductive Health and Research, World Health Organization, 20 Avenue Appia, 1211 Geneva, Switzerland
| | - James Kiarie
- UNDP-UNFPA-UNICEF-WHO-World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP), Department of Sexual and Reproductive Health and Research, World Health Organization, 20 Avenue Appia, 1211 Geneva, Switzerland
| | - Bela Ganatra
- UNDP-UNFPA-UNICEF-WHO-World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP), Department of Sexual and Reproductive Health and Research, World Health Organization, 20 Avenue Appia, 1211 Geneva, Switzerland
| |
Collapse
|
7
|
de Heer MH, Driessen EW, Teunissen PW, Scheele F. Lessons learned spanning 17 years of experience with three consecutive nationwide competency based medical education training plans. Front Med (Lausanne) 2024; 11:1339857. [PMID: 38455473 PMCID: PMC10917951 DOI: 10.3389/fmed.2024.1339857] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2023] [Accepted: 01/31/2024] [Indexed: 03/09/2024] Open
Abstract
Introduction Curricula for postgraduate medical education have transformed since the introduction of competency based medical education (CBME). Postgraduate training plans offer broader training with different competencies and an outcome-based approach, in addition to the medical technical aspects of training. However, CBME also has its challenges. Over the past years, critical views have been shared on the potential drawbacks of CBME, such as assessment burden and conflicts with practicality in the workplace. Recent studies identified a need for a better understanding of how the evolving concept of CBME has been translated to curriculum design and implemented in the practice of postgraduate training. The aim of this study was to describe the development of CBME translations to curriculum design, based on three consecutive postgraduate training programs spanning 17 years. Method We performed a document analysis of three consecutive Dutch gynecology and obstetrics training plans that were implemented in 2005, 2013, and 2021. We used template analysis to identify changes over time. Results Over time, CBME-based curriculum design changed in several domains. Assessment changed from a model with a focus on summative decision to one with an emphasis on formative, low-stakes assessments aimed at supporting learning. The training plans evolved in parallel to evolving educational insights, e.g., by placing increasing emphasis on personal development. The curricula focused on a competency-based concept by introducing training modules and personalized authorization based on feedback rather than on a set duration of internships. There was increasing freedom in personalized training trajectories in the training plans, together with increasing trust towards the resident. Conclusion The way CBME was translated into training plans has evolved in the course of 17 years of experience with CMBE-based education. The main areas of change were the structure of the training plans, which became increasingly open, the degree to which learning outcomes were mandatory or not, and the way these outcomes were assessed.
Collapse
Affiliation(s)
- Merel H. de Heer
- Amsterdam UMC Location Vrije Universiteit Amsterdam, Research in Education, Amsterdam, Netherlands
| | - Erik W. Driessen
- School of Health Professions Education (SHE), Faculty of Health Medicine and Life Sciences (FHML), Maastricht University, Maastricht, Netherlands
| | - Pim W. Teunissen
- School of Health Professions Education (SHE), Faculty of Health Medicine and Life Sciences (FHML), Maastricht University, Maastricht, Netherlands
- Department of Obstetrics and Gynecology, Maastricht University Medical Center (MUMC+), Maastricht, Netherlands
| | - Fedde Scheele
- Amsterdam UMC Location Vrije Universiteit Amsterdam, Research in Education, Amsterdam, Netherlands
- Athena Institute, Faculty of Science, VU, Amsterdam, Netherlands
| |
Collapse
|
8
|
Hall AK, Oswald A, Frank JR, Dalseg T, Cheung WJ, Cooke L, Gorman L, Brzezina S, Selvaratnam S, Wagner N, Hamstra SJ, Van Melle E. Evaluating Competence by Design as a Large System Change Initiative: Readiness, Fidelity, and Outcomes. PERSPECTIVES ON MEDICAL EDUCATION 2024; 13:95-107. [PMID: 38343556 PMCID: PMC10854467 DOI: 10.5334/pme.962] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Accepted: 08/17/2023] [Indexed: 02/15/2024]
Abstract
Program evaluation is an essential, but often neglected, activity in any transformational educational change. Competence by Design was a large-scale change initiative to implement a competency-based time-variable educational system in Canadian postgraduate medical education. A program evaluation strategy was an integral part of the build and implementation plan for CBD from the beginning, providing insights into implementation progress, challenges, unexpected outcomes, and impact. The Competence by Design program evaluation strategy was built upon a logic model and three pillars of evaluation: readiness to implement, fidelity and integrity of implementation, and outcomes of implementation. The program evaluation strategy harvested from both internally driven studies and those performed by partners and invested others. A dashboard for the program evaluation strategy was created to transparently display a real-time view of Competence by Design implementation and facilitate continuous adaptation and improvement. The findings of the program evaluation for Competence by Design drove changes to all aspects of the Competence by Design implementation, aided engagement of partners, supported change management, and deepened our understanding of the journey required for transformational educational change in a complex national postgraduate medical education system. The program evaluation strategy for Competence by Design provides a framework for program evaluation for any large-scale change in health professions education.
Collapse
Affiliation(s)
- Andrew K. Hall
- Department of Emergency Medicine, University of Ottawa, Ottawa, ON, Canada
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | - Anna Oswald
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Division of Rheumatology, Department of Medicine, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, AB, Canada
| | - Jason R. Frank
- Department of Emergency Medicine, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Tim Dalseg
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Medicine, Division of Emergency Medicine, University of Toronto, Toronto, ON, Canada
| | - Warren J. Cheung
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Emergency Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Lara Cooke
- Neurology, Department of Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Lisa Gorman
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | - Stacey Brzezina
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | | | - Natalie Wagner
- Queen’s Health Sciences Office of Professional Development and Educational Scholarship, Queen’s University, Kingston, ON, Canada
| | - Stanley J. Hamstra
- Department of Surgery, University of Toronto, Toronto, ON, Canada
- Department of Milestones Research and Evaluation, Accreditation Council for Graduate Medical Education, Chicago, IL, USA
- Department of Medical Education, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Elaine Van Melle
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Family Medicine, Queen’s University, Kingston, ON, Canada
| |
Collapse
|
9
|
Fisk D, Clendenning B, St John P, Francois J. Multi-stakeholder validation of entrustable professional activities for a family medicine care of the elderly residency program: A focus group study. GERONTOLOGY & GERIATRICS EDUCATION 2024; 45:12-25. [PMID: 36326195 DOI: 10.1080/02701960.2022.2130913] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
Entrustable Professional Activities (EPAs) have become widely used within Competency-Based Medical Education (CBME) for the training and evaluation of residents. Little is known about the effectiveness of incorporating multiple stakeholder groups in the validation of EPAs. Here, we seek to validate an EPA framework developed for the University of Manitoba Care of the Elderly Enhanced Skills program using online focus groups consisting of five stakeholder groups. Participants were recruited to take part in one of five online focus groups, one for each stakeholder group (physician faculty, residents, non-physician healthcare professionals, administrators/managers, and patients). Each group met one time for 90 minutes over ZOOM®. The themes arising from stakeholder feedback suggest that successful EPAs must neither be too specific nor too expansive in scope, clearly delineate appropriate means of evaluation, and indicate specific clinical settings in which each EPA should be evaluated. Cross-cutting themes included requiring trainees to collaborate with other professionals when it would optimize patient care, and preparing trainees to advocate for their patients' health (Advocacy). The present study demonstrates that multi-stakeholder analysis yields diverse feedback that can help make EPAs more clear, easier to use in evaluation, and more socially accountable.
Collapse
Affiliation(s)
- Derek Fisk
- Max Rady College of Medicine, University of Manitoba, Winnipeg, Manitoba, Canada
| | - Ben Clendenning
- Max Rady College of Medicine, University of Manitoba, Winnipeg, Manitoba, Canada
| | - Philip St John
- Max Rady College of Medicine, Department of Internal Medicine, Section of Geriatric Medicine, Winnipeg, Manitoba, Canada
- University of Manitoba Centre on Aging, Winnipeg, Manitoba, Canada
| | - Jose Francois
- University of Manitoba Centre on Aging, Winnipeg, Manitoba, Canada
- Department of Family Medicine, Max Rady College of Medicine, Winnipeg, Manitoba, Canada
| |
Collapse
|
10
|
Daugherty KK, Chen A, Churchwell MD, Jarrett JB, Kleppinger EL, Meyer S, Nawarskas J, Sibicky SL, Stowe CD, Rhoney DH. Competency-based pharmacy education definition: What components need to be defined to implement it? AMERICAN JOURNAL OF PHARMACEUTICAL EDUCATION 2024; 88:100624. [PMID: 37952584 DOI: 10.1016/j.ajpe.2023.100624] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/19/2022] [Revised: 10/30/2023] [Accepted: 11/06/2023] [Indexed: 11/14/2023]
Abstract
OBJECTIVES This study aimed to define the essential elements in the proposed competency-based pharmacy education (CBPE) definition, provide the key defining components of each essential element on the basis of educational theory and evidence, and define how the essential elements meet the identified needs for CBPE. METHODS best-practice integrative review was conducted as part of the work of the American Association of Colleges of Pharmacy CBPE Task Force to define the essential elements in the CBPE definition and how these elements fit with the need for CBPE. The definition was compared with other published competency-based education definitions across K-12, higher education, medical education, and veterinary education. Task Force members then met to develop a consensus on the core components of the 5 essential elements in the definition. Next, the Task Force evaluated the fit of CBPE by matching the identified needs, discussed in detail elsewhere, across each of the stakeholder perspectives with the core components of the 5 essential elements in the derived definition of CBPE. FINDINGS Upon review of the proposed CBPE definition, the Task Force identified 5 essential elements. These elements include the following: meeting health care and societal needs, outcomes-based curricular model, de-emphasized time, learner-centered culture, and authentic teaching and learning strategies aligned to assessments. SUMMARY This article helps to establish a common language for CBPE by defining the essential elements of the core components of the definition, and provides a starting point for further exploration of CBPE.
Collapse
Affiliation(s)
- Kimberly K Daugherty
- Sullivan University College of Pharmacy and Health Sciences, Louisville, KY, USA
| | - Aleda Chen
- Cedarville University, School of Pharmacy, Cedarville, OH, USA
| | - Marianne D Churchwell
- University of Toledo, College of Pharmacy and Pharmaceutical Sciences, Toledo, OH, USA
| | - Jennie B Jarrett
- University of Illinois Chicago, College of Pharmacy, Department of Pharmacy Practice, Chicago, IL, USA
| | | | | | - James Nawarskas
- University of New Mexico, College of Pharmacy, Albuquerque, NM, USA
| | - Stephanie L Sibicky
- Northeastern University, School of Pharmacy and Pharmaceutical Sciences, Boston, MA, USA
| | - Cindy D Stowe
- University of Arkansas for Medical Sciences, College of Pharmacy, Little Rock, AR, USA
| | - Denise H Rhoney
- University of North Carolina at Chapel Hill, Eshelman School of Pharmacy, Chapel Hill, NC, USA.
| |
Collapse
|
11
|
Moffatt-Bruce SD, Harris K, Rubens FD, Villeneuve PJ, Sundaresan RS. Competency-based training: Canadian cardiothoracic surgery. J Thorac Cardiovasc Surg 2024; 167:407-410. [PMID: 36702679 DOI: 10.1016/j.jtcvs.2023.01.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/03/2022] [Revised: 11/25/2022] [Accepted: 01/05/2023] [Indexed: 01/11/2023]
Affiliation(s)
- Susan D Moffatt-Bruce
- The Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada; Department of Surgery, University of Ottawa, Ottawa, Ontario, Canada.
| | - Ken Harris
- Department of Surgery, University of Western Ontario, London, Ontario, Canada
| | - Fraser D Rubens
- Department of Surgery, University of Ottawa, Ottawa, Ontario, Canada
| | | | | |
Collapse
|
12
|
Varpio L, Sherbino J. Demonstrating causality, bestowing honours, and contributing to the arms race: Threats to the sustainability of HPE research. MEDICAL EDUCATION 2024; 58:157-163. [PMID: 37283076 DOI: 10.1111/medu.15148] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/13/2023] [Revised: 05/18/2023] [Accepted: 05/22/2023] [Indexed: 06/08/2023]
Abstract
As the field of health professions education (HPE) continues to evolve, it is necessary to occasionally pause and reflect on the potential effects and outcomes of our research practices. While future-casting does not guarantee that impending negative consequences will be evaded, the exercise can help us avoid pitfalls. In this paper, we reflect on two terms that have taken hold as powerful idols in HPE research that stand above questioning and apart from critique: patient outcomes and productivity. We argue that these terms, and the ways of thinking they uphold, threaten the sustainability of HPE research-one at the level of the community and one at the level of the scholar. First, we suggest that HPE research's history of endorsing a linear and causal association ethos has driven its quest to connect education to patient outcomes. To ensure the sustainability of HPE scholarship, we must deconstruct and disempower patient outcomes as one of HPE's god-terms, as the pinnacle goal of educational activities. To be sustained, HPE research needs to value all of its contributions equally. A second god-term is productivity; it impairs the sustainability of the careers of individual researchers. Problems of honorary authorship, research output expectations, and comparisons with other fields have constructed a space where only scholars with sufficient privilege can prevail. If productivity persists as a god-term, the field of HPE research could decay into a space where new scholars are silenced-not because they fail to make important contributions, but because access is restricted by existing research metrics. These are two of many god-terms threatening the sustainability of HPE research. By highlighting patient outcomes and productivity and by acknowledging our own participation in propagating them, we hope to encourage others to recognize how our collective choices threaten the sustainability of our field.
Collapse
Affiliation(s)
- Lara Varpio
- Department of Pediatrics, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
| | - Jonathan Sherbino
- Department of Medicine, McMaster University, McMaster Education Research, Innovation and Theory (MERIT) Program, Hamilton, Ontario, Canada
| |
Collapse
|
13
|
Acker A, Leifso K, Crawford L, Braund H, Hawksby E, Hall AK, McEwen L, Dalgarno N, Dagnone JD. Lessons learned and new strategies for success: Evaluating the Implementation of Competency-Based Medical Education in Queen's Pediatrics. Paediatr Child Health 2023; 28:463-467. [PMID: 38638538 PMCID: PMC11022870 DOI: 10.1093/pch/pxad021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Accepted: 04/17/2023] [Indexed: 04/20/2024] Open
Abstract
Objectives In 2017, Queen's University launched Competency-Based Medical Education (CBME) across 29 programs simultaneously. Two years post-implementation, we asked key stakeholders (faculty, residents, and program leaders) within the Pediatrics program for their perspectives on and experiences with CBME so far. Methods Program leadership explicitly described the intended outcomes of implementing CBME. Focus groups and interviews were conducted with all stakeholders to describe the enacted implementation. The intended versus enacted implementations were compared to provide insight into needed adaptations for program improvement. Results Overall, stakeholders saw value in the concept of CBME. Residents felt they received more specific feedback and monthly Competence Committee (CC) meetings and Academic Advisors were helpful. Conversely, all stakeholders noted the increased expectations had led to a feeling of assessment fatigue. Faculty noted that direct observation and not knowing a resident's previous performance information was challenging. Residents wanted to see faculty initiate assessments and improved transparency around progress and promotion decisions. Discussion The results provided insight into how well the intended outcomes had been achieved as well as areas for improvement. Proposed adaptations included a need for increased direct observation and exploration of faculty accessing residents' previous performance information. Education was provided on the performance expectations of residents and how progress and promotion decisions are made. As well, "flex blocks" were created to help residents customize their training experience to meet their learning needs. The results of this study can be used to inform and guide implementation and adaptations in other programs and institutions.
Collapse
Affiliation(s)
- Amy Acker
- Department of Pediatrics, Queen’s University, Kingston, Canada
| | - Kirk Leifso
- Department of Pediatrics, Queen’s University, Kingston, Canada
| | | | - Heather Braund
- Scholarship and Simulation Education, Office of Professional Development and Educational Scholarship, Faculty of Health Sciences, Queen’s University, Kingston, Canada
| | - Emily Hawksby
- Department of Pediatrics, Queen’s University, Kingston, Canada
| | - Andrew K Hall
- Department of Emergency Medicine, University of Ottawa, Ottawa, Canada
- Royal College of Physicians and Surgeons of Canada, Ottawa, Canada
| | - Laura McEwen
- Department of Pediatrics, Queen’s University, Kingston, Canada
- Postgraduate Medical Education Queen’s University, Kingston, Canada
| | - Nancy Dalgarno
- Education Scholarship, Office of Professional Development and Educational Scholarship, Kingston, Canada
- Department of Biomedical and Molecular Sciences, Faculty of Health Sciences, Queen’s University, Kingston, Canada
| | - Jeffrey Damon Dagnone
- Postgraduate Medical Education Queen’s University, Kingston, Canada
- Department of Emergency Medicine, Queen’s University, Kingston, Canada
| |
Collapse
|
14
|
Zwischenberger JB, Hatchett SP, Rahman NA, Chadha P, Sebok-Syer SS, George BC. Implementing Workplace-Based Assessments at Scale: The SIMPL Approach. ANNALS OF SURGERY OPEN 2023; 4:e353. [PMID: 38144481 PMCID: PMC10735067 DOI: 10.1097/as9.0000000000000353] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2023] [Accepted: 09/25/2023] [Indexed: 12/26/2023] Open
Abstract
Over the past decade, medical education has shifted from a time-based approach to a competency-based approach for surgical training. This transition presents many new systemic challenges. The Society for Improving Medical Professional Learning (SIMPL) was created to respond to these challenges through coordinated collaboration across an international network of medical educators. The primary goal of the SIMPL network was to implement a workplace-based assessment and feedback platform. To date, SIMPL has developed, implemented, and sustained a platform that represents the earliest and largest effort to support workplace-based assessment at scale. The SIMPL model for collaborative improvement demonstrates a potential approach to addressing other complex systemic problems in medical education.
Collapse
Affiliation(s)
- Joseph B. Zwischenberger
- From the Department of Surgery, Diagnostic Radiology, Pediatrics, and Biomedical Engineering, College of Medicine, University of Kentucky
| | | | - Naveed A. Rahman
- Department of Surgery, SUNY Upstate Medical University, Syracuse, NY
| | - Priyal Chadha
- College of Medicine, SUNY Upstate Medical University, Syracuse, NY
| | | | - Brian C. George
- Department of Surgery, University of Michigan, College of Medicine, Ann Arbor, MI
| |
Collapse
|
15
|
Carney PA, Sebok-Syer SS, Pusic MV, Gillespie CC, Westervelt M, Goldhamer MEJ. Using learning analytics in clinical competency committees: Increasing the impact of competency-based medical education. MEDICAL EDUCATION ONLINE 2023; 28:2178913. [PMID: 36821373 PMCID: PMC9970252 DOI: 10.1080/10872981.2023.2178913] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/18/2022] [Revised: 02/02/2023] [Accepted: 02/07/2023] [Indexed: 06/18/2023]
Abstract
Graduate medical education (GME) and Clinical Competency Committees (CCC) have been evolving to monitor trainee progression using competency-based medical education principles and outcomes, though evidence suggests CCCs fall short of this goal. Challenges include that evaluation data are often incomplete, insufficient, poorly aligned with performance, conflicting or of unknown quality, and CCCs struggle to organize, analyze, visualize, and integrate data elements across sources, collection methods, contexts, and time-periods, which makes advancement decisions difficult. Learning analytics have significant potential to improve competence committee decision making, yet their use is not yet commonplace. Learning analytics (LA) is the interpretation of multiple data sources gathered on trainees to assess academic progress, predict future performance, and identify potential issues to be addressed with feedback and individualized learning plans. What distinguishes LA from other educational approaches is systematic data collection and advanced digital interpretation and visualization to inform educational systems. These data are necessary to: 1) fully understand educational contexts and guide improvements; 2) advance proficiency among stakeholders to make ethical and accurate summative decisions; and 3) clearly communicate methods, findings, and actionable recommendations for a range of educational stakeholders. The ACGME released the third edition CCC Guidebook for Programs in 2020 and the 2021 Milestones 2.0 supplement of the Journal of Graduate Medical Education (JGME Supplement) presented important papers that describe evaluation and implementation features of effective CCCs. Principles of LA underpin national GME outcomes data and training across specialties; however, little guidance currently exists on how GME programs can use LA to improve the CCC process. Here we outline recommendations for implementing learning analytics for supporting decision making on trainee progress in two areas: 1) Data Quality and Decision Making, and 2) Educator Development.
Collapse
Affiliation(s)
- Patricia A. Carney
- Professor of Family Medicine, Oregon Health & Science University, Portland, OR, USA
| | | | | | | | - Marjorie Westervelt
- Director of Assessment, Evaluation and Scholarship, University of California, Davis, CA, USA
| | - Mary Ellen J. Goldhamer
- Medicine, Harvard Medical School, Massachusetts General Hospital, and Mass General Brigham, Boston, MA, USA
| |
Collapse
|
16
|
Szulewski A, Braund H, Dagnone DJ, McEwen L, Dalgarno N, Schultz KW, Hall AK. The Assessment Burden in Competency-Based Medical Education: How Programs Are Adapting. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2023; 98:1261-1267. [PMID: 37343164 DOI: 10.1097/acm.0000000000005305] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/23/2023]
Abstract
Residents and faculty have described a burden of assessment related to the implementation of competency-based medical education (CBME), which may undermine its benefits. Although this concerning signal has been identified, little has been done to identify adaptations to address this problem. Grounded in an analysis of an early Canadian pan-institutional CBME adopter's experience, this article describes postgraduate programs' adaptations related to the challenges of assessment in CBME. From June 2019-September 2022, 8 residency programs underwent a standardized Rapid Evaluation guided by the Core Components Framework (CCF). Sixty interviews and 18 focus groups were held with invested partners. Transcripts were analyzed abductively using CCF, and ideal implementation was compared with enacted implementation. These findings were then shared back with program leaders, adaptations were subsequently developed, and technical reports were generated for each program. Researchers reviewed the technical reports to identify themes related to the burden of assessment with a subsequent focus on identifying adaptations across programs. Three themes were identified: (1) disparate mental models of assessment processes in CBME, (2) challenges in workplace-based assessment processes, and (3) challenges in performance review and decision making. Theme 1 included entrustment interpretation and lack of shared mindset for performance standards. Adaptations included revising entrustment scales, faculty development, and formalizing resident membership. Theme 2 involved direct observation, timeliness of assessment completion, and feedback quality. Adaptations included alternative assessment strategies beyond entrustable professional activity forms and proactive assessment planning. Theme 3 related to resident data monitoring and competence committee decision making. Adaptations included adding resident representatives to the competence committee and assessment platform enhancements. These adaptations represent responses to the concerning signal of significant burden of assessment within CBME being experienced broadly. The authors hope other programs may learn from their institution's experience and navigate the CBME-related assessment burden their invested partners may be facing.
Collapse
Affiliation(s)
- Adam Szulewski
- A. Szulewski is associate professor, Departments of Emergency Medicine and Psychology, and educational scholarship lead, Postgraduate Medical Education, Queen's University, Kingston, Ontario, Canada; ORCID: https://orcid.org/0000-0002-3076-6221
| | - Heather Braund
- H. Braund is associate director of scholarship and simulation education, Office of Professional Development and Educational Scholarship, and assistant (adjunct) professor, Department of Biomedical and Molecular Sciences and School of Medicine, Queen's University, Kingston, Ontario, Canada; ORCID: https://orcid.org/0000-0002-9749-7193
| | - Damon J Dagnone
- D.J. Dagnone is associate professor, Department of Emergency Medicine, Queen's University, Kingston, Ontario, Canada; ORCID: https://orcid.org/0000-0001-6963-7948
| | - Laura McEwen
- L. McEwen is director of assessment and evaluation of postgraduate medical education and assistant professor, Department of Pediatrics, Postgraduate Medical Education, Queen's University, Kingston, Ontario, Canada; ORCID: https://orcid.org/0000-0003-2457-5311
| | - Nancy Dalgarno
- N. Dalgarno is director of education scholarship, Office of Professional Development and Educational Scholarship, and assistant professor (adjunct), Department of Biomedical and Molecular Sciences and Master of Health Professions Education, Queen's University, Kingston, Ontario, Canada; ORCID: https://orcid.org/0000-0001-7932-9949
| | - Karen W Schultz
- K.W. Schultz is professor, Department of Family Medicine, and associate dean of postgraduate medical education, Queen's University, Kingston, Ontario, Canada; ORCID: https://orcid.org/0000-0003-0208-3981
| | - Andrew K Hall
- A.K. Hall is associate professor and vice chair of education, Department of Emergency Medicine, University of Ottawa, and clinician educator, Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada; ORCID: https://orcid.org/0000-0003-1227-5397
| |
Collapse
|
17
|
Shawahna R, Jaber M, Maqboul I, Khaleel M, Abo-Mokh J, Sabbah H, Assi S, Zuhd J. Teaching approaches and satisfaction of medical students during anesthesia rotations: a multicenter cross-sectional study. BMC MEDICAL EDUCATION 2023; 23:641. [PMID: 37679731 PMCID: PMC10486086 DOI: 10.1186/s12909-023-04603-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/12/2023] [Accepted: 08/21/2023] [Indexed: 09/09/2023]
Abstract
BACKGROUND Anesthesia training is an important component of medical education. This multicenter study was conducted to determine the teaching approaches used during anesthesia training or rotations and to assess the satisfaction of the medical students. METHODS This multicenter study was conducted in a cross-sectional design. The study was conducted among 5th and 6th year medical students who completed their anesthesia rotations in different training centers in Palestine. The data were collected using a questionnaire in adherence to the strengthening the reporting of observational studies in epidemiology statement. RESULTS Questionnaires were returned by 385 medical students. The mean anesthesia rotation length was 12.4 ± 2.1 days. On average, the students witnessed 7.8 ± 4.9 procedures performed under general, regional, and local anesthesia in a week. Of the students, 135 (35.1%) and 126 (32.8%) stated that the educators always or often explained how and why they did procedures or techniques during the anesthesia rotation and assessed their baseline level of knowledge before giving new knowledge or explaining things. On the other hand, stepping back and allowing the trainees to work through, presenting articles or literature relevant to the case, and being open to trying new or different procedures or techniques were less often reported by the medical students. Less than half of the students were satisfied with their competencies gained through the anesthesia rotation. CONCLUSION Educators used active and effective teaching or training approaches less frequently during the anesthesia rotations. The findings of this study also showed that the medical students were not satisfied with their competencies after their anesthesia rotations. More studies are still needed to determine the best ways to improve anesthesia rotations and medical education or training in Palestine.
Collapse
Affiliation(s)
- Ramzi Shawahna
- Department of Physiology, Pharmacology and Toxicology, Faculty of Medicine and Health Sciences, An-Najah National University, New Campus, Building: 19, Office: 1340, P.O. Box 7, Nablus, Palestine.
- Clinical Research Center, An-Najah National University Hospital, 44839, Nablus, Palestine.
| | - Mohammad Jaber
- Department of Medicine, Faculty of Medicine and Health Sciences, An-Najah National University, Nablus, Palestine
- An-Najah National University Hospital, An-Najah National University, Nablus, Palestine
| | - Iyad Maqboul
- Department of Medicine, Faculty of Medicine and Health Sciences, An-Najah National University, Nablus, Palestine.
- An-Najah National University Hospital, An-Najah National University, Nablus, Palestine.
| | - Mansour Khaleel
- Department of Medicine, Faculty of Medicine and Health Sciences, An-Najah National University, Nablus, Palestine
| | - Jenan Abo-Mokh
- Department of Medicine, Faculty of Medicine and Health Sciences, An-Najah National University, Nablus, Palestine
| | - Hisham Sabbah
- Department of Medicine, Faculty of Medicine and Health Sciences, An-Najah National University, Nablus, Palestine
| | - Sudqi Assi
- An-Najah National University Hospital, An-Najah National University, Nablus, Palestine
| | - Jehad Zuhd
- An-Najah National University Hospital, An-Najah National University, Nablus, Palestine
| |
Collapse
|
18
|
Chen JX, Thorne MC, Galaiya D, Campisi P, Gray ST. Competency-based medical education in the United States: What the otolaryngologist needs to know. Laryngoscope Investig Otolaryngol 2023; 8:827-831. [PMID: 37621294 PMCID: PMC10446250 DOI: 10.1002/lio2.1095] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2023] [Revised: 05/29/2023] [Accepted: 06/04/2023] [Indexed: 08/26/2023] Open
Abstract
Competency-based medical education (CBME) is an outcomes-focused approach to educating medical professionals that will be central to future efforts to improve resident training in otolaryngology. The transition to CBME for otolaryngology in the United States will require the development of specialty-specific assessments and benchmarks, the financial and administrative support for implementation, the professional development of faculty and learners, and the cooperation of all major stakeholders in graduate medical education. In this article, we describe the need for evidence-based innovation in surgical training, the history of CBME in the United States, and the progress towards defining "entrustable professional activities" as the building blocks of assessments for CBME. We explore what such a paradigm shift in surgical education could mean for academic otolaryngologists by examining innovative educational practices in other surgical specialties and discussing foreseeable challenges in implementation for the American healthcare system.
Collapse
Affiliation(s)
- Jenny X. Chen
- Department of Otolaryngology—Head and Neck SurgeryJohns Hopkins UniversityBaltimoreMarylandUSA
| | - Marc C. Thorne
- Department of Otolaryngology—Head and Neck SurgeryUniversity of MichiganAnn ArborMichiganUSA
| | - Deepa Galaiya
- Department of Otolaryngology—Head and Neck SurgeryJohns Hopkins UniversityBaltimoreMarylandUSA
| | - Paolo Campisi
- Department of Otolaryngology—Head and Neck SurgeryUniversity of TorontoTorontoOntarioCanada
| | - Stacey T. Gray
- Department of Otolaryngology—Head and Neck SurgeryHarvard Medical SchoolBostonMassachusettsUSA
| |
Collapse
|
19
|
Triola MM, Burk-Rafel J. Precision Medical Education. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2023; 98:775-781. [PMID: 37027222 DOI: 10.1097/acm.0000000000005227] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Medical schools and residency programs are increasingly incorporating personalization of content, pathways, and assessments to align with a competency-based model. Yet, such efforts face challenges involving large amounts of data, sometimes struggling to deliver insights in a timely fashion for trainees, coaches, and programs. In this article, the authors argue that the emerging paradigm of precision medical education (PME) may ameliorate some of these challenges. However, PME lacks a widely accepted definition and a shared model of guiding principles and capacities, limiting widespread adoption. The authors propose defining PME as a systematic approach that integrates longitudinal data and analytics to drive precise educational interventions that address each individual learner's needs and goals in a continuous, timely, and cyclical fashion, ultimately improving meaningful educational, clinical, or system outcomes. Borrowing from precision medicine, they offer an adapted shared framework. In the P4 medical education framework, PME should (1) take a proactive approach to acquiring and using trainee data; (2) generate timely personalized insights through precision analytics (including artificial intelligence and decision-support tools); (3) design precision educational interventions (learning, assessment, coaching, pathways) in a participatory fashion, with trainees at the center as co-producers; and (4) ensure interventions are predictive of meaningful educational, professional, or clinical outcomes. Implementing PME will require new foundational capacities: flexible educational pathways and programs responsive to PME-guided dynamic and competency-based progression; comprehensive longitudinal data on trainees linked to educational and clinical outcomes; shared development of requisite technologies and analytics to effect educational decision-making; and a culture that embraces a precision approach, with research to gather validity evidence for this approach and development efforts targeting new skills needed by learners, coaches, and educational leaders. Anticipating pitfalls in the use of this approach will be important, as will ensuring it deepens, rather than replaces, the interaction of trainees and their coaches.
Collapse
Affiliation(s)
- Marc M Triola
- M.M. Triola is associate dean of educational informatics and director of the Institute for Innovations in Medical Education, NYU Grossman School of Medicine, New York, New York; ORCID: https://orcid.org/0000-0002-6303-3112
| | - Jesse Burk-Rafel
- J. Burk-Rafel is assistant director of precision and translational education, Institute for Innovations in Medical Education, and assistant professor of medicine, Division of Hospital Medicine, NYU Grossman School of Medicine, New York, New York; ORCID: https://orcid.org/0000-0003-3785-2154
| |
Collapse
|
20
|
Affiliation(s)
- Marilyn H Oermann
- Editor-in-Chief, Nurse Educator ; Thelma M. Ingles Professor of Nursing, Duke University School of Nursing, Durham, North Carolina
| |
Collapse
|
21
|
Keeley MG, Bray MJ, Bradley EB, Peterson CM, Waggoner-Fountain LA, Gusic ME. Fidelity to Best Practices in EPA Implementation: Outcomes Supporting Use of the Core Components Framework From the University of Virginia Entrustable Professional Activity Program. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2022; 97:1637-1642. [PMID: 35976718 DOI: 10.1097/acm.0000000000004944] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
PROBLEM The rapid expansion of entrustable professional activity (EPA) assessment programs has led to calls to ensure fidelity in implementation and integrity in meeting the goals of competency-based medical education. Initiated in July 2017, in advance of the articulated core components of EPA implementation, this article describes the structure and outcomes of the University of Virginia (UVA) EPA Program and provides support for the identified essential components. APPROACH The UVA EPA Program includes workplace assessments by residents/fellows, attending faculty, and master assessors (MAs), experienced clinicians who assess students across disciplines and clinical settings. All assessors participate in formal professional development and provide verbal and written comments to support their supervision ratings. The Entrustment Committee, composed of 12 MAs, uses a shared mental model and aggregates all assessor data to make a high-stakes summative entrustment decision about students' readiness to assume the role of an acting intern. OUTCOMES Since 2017, over 2,000 assessors have completed 56,969 EPA assessments for 1,479 students. Ninety-four percent of assessments have been done during the clerkship phase. Residents/fellows have completed a mean of 18 assessments, attending faculty a mean of 27, and MAs a mean of 882. Seventy-four percent of observed encounters involved patients with acute concerns with or without a co-morbid condition. Fifty percent of assessments occurred in inpatient and 32% in ambulatory settings. Eighty-seven percent of assessments contained narrative comments with more than 100 characters. NEXT STEPS Planned next steps will include earlier identification of students who require individualized learning to promote the development of skills related to EPAs, expansion of the remediation program to enable more students to engage in a clinical performance mastery elective, and creation of targeted professional development for assessors to reinforce the tenets of the EPA program.
Collapse
Affiliation(s)
- Meg G Keeley
- M.G. Keeley is senior associate dean for education and professor, Office of Medical Education and Department of Pediatrics, University of Virginia School of Medicine, Charlottesville, Virginia
| | - Megan J Bray
- M.J. Bray is associate dean for curriculum and associate professor, Office of Medical Education and Department of Obstetrics and Gynecology, University of Virginia School of Medicine, Charlottesville, Virginia
| | - Elizabeth B Bradley
- E.B. Bradley is director of evaluation and associate professor, Office of Medical Education and Center for Medical Education Research and Scholarly Innovation, University of Virginia School of Medicine, Charlottesville, Virginia
| | - Christine M Peterson
- C.M. Peterson is assistant dean for student affairs and associate professor, Office of Medical Education, University of Virginia School of Medicine, Charlottesville, Virginia
| | - Linda A Waggoner-Fountain
- L.A. Waggoner-Fountain is professor and associate program director, Department of Pediatrics, University of Virginia School of Medicine, Charlottesville, Virginia
| | - Maryellen E Gusic
- M.E. Gusic is currently senior associate dean for education and professor, Lewis Katz School of Medicine, Temple University, Philadelphia, Pennsylvania. She was formerly senior advisor, educational affairs, Office of Medical Education, University of Virginia School of Medicine, Charlottesville, Virginia
| |
Collapse
|
22
|
Yilmaz Y, Carey R, Chan TM, Bandi V, Wang S, Woods RA, Mondal D, Thoma B. Developing a dashboard for program evaluation in competency-based training programs: a design-based research project. CANADIAN MEDICAL EDUCATION JOURNAL 2022; 13:14-27. [PMID: 36310899 PMCID: PMC9588183 DOI: 10.36834/cmej.73554] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
BACKGROUND Canadian specialist residency training programs are implementing a form of competency-based medical education (CBME) that requires the assessment of entrustable professional activities (EPAs). Dashboards could be used to track the completion of EPAs to support program evaluation. METHODS Using a design-based research process, we identified program evaluation needs related to CBME assessments and designed a dashboard containing elements (data, analytics, and visualizations) meeting these needs. We interviewed leaders from the emergency medicine program and postgraduate medical education office at the University of Saskatchewan. Two investigators thematically analyzed interview transcripts to identify program evaluation needs that were audited by two additional investigators. Identified needs were described using quotes, analytics, and visualizations. RESULTS Between July 1, 2019 and April 6, 2021 we conducted 17 interviews with six participants (two program leaders and four institutional leaders). Four needs emerged as themes: tracking changes in overall assessment metrics, comparing metrics to the assessment plan, evaluating rotation performance, and engagement with the assessment metrics. We addressed these needs by presenting analytics and visualizations within a dashboard. CONCLUSIONS We identified program evaluation needs related to EPA assessments and designed dashboard elements to meet them. This work will inform the development of other CBME assessment dashboards designed to support program evaluation.
Collapse
Affiliation(s)
- Yusuf Yilmaz
- Continuing Professional Development Office, and McMaster program for Education Research, Innovation, and Theory (MERIT), McMaster University, Ontario, Canada
- Department of Medical Education, Ege University, Turkey
| | - Robert Carey
- Department of Emergency Medicine, University of Saskatchewan, Saskatchewan, Canada
| | - Teresa M Chan
- Continuing Professional Development Office, and McMaster program for Education Research, Innovation, and Theory (MERIT), McMaster University, Ontario, Canada
- Division of Emergency Medicine, Department of Medicine at McMaster University
| | - Venkat Bandi
- Department of Computer Science, University of Saskatchewan, Saskatchewan, Canada
| | - Shisong Wang
- Department of Computer Science, University of Saskatchewan, Saskatchewan, Canada
| | - Robert A Woods
- Department of Emergency Medicine, University of Saskatchewan, Saskatchewan, Canada
| | - Debajyoti Mondal
- Department of Computer Science, University of Saskatchewan, Saskatchewan, Canada
| | - Brent Thoma
- Department of Emergency Medicine, University of Saskatchewan, Saskatchewan, Canada
- Royal College of Physicians and Surgeons of Canada, Ontario, Canada
| |
Collapse
|
23
|
Cheung WJ, Wagner N, Frank JR, Oswald A, Van Melle E, Skutovich A, Dalseg TR, Cooke LJ, Hall AK. Implementation of competence committees during the transition to CBME in Canada: A national fidelity-focused evaluation. MEDICAL TEACHER 2022; 44:781-789. [PMID: 35199617 DOI: 10.1080/0142159x.2022.2041191] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
PURPOSE This study evaluated the fidelity of competence committee (CC) implementation in Canadian postgraduate specialist training programs during the transition to competency-based medical education (CBME). METHODS A national survey of CC chairs was distributed to all CBME training programs in November 2019. Survey questions were derived from guiding documents published by the Royal College of Physicians and Surgeons of Canada reflecting intended processes and design. RESULTS Response rate was 39% (113/293) with representation from all eligible disciplines. Committee size ranged from 3 to 20 members, 42% of programs included external members, and 20% included a resident representative. Most programs (72%) reported that a primary review and synthesis of resident assessment data occurs prior to the meeting, with some data reviewed collectively during meetings. When determining entrustable professional activity (EPA) achievement, most programs followed the national specialty guidelines closely with some exceptions (53%). Documented concerns about professionalism, EPA narrative comments, and EPA entrustment scores were most highly weighted when determining resident progress decisions. CONCLUSIONS Heterogeneity in CC implementation likely reflects local adaptations, but may also explain some of the variable challenges faced by programs during the transition to CBME. Our results offer educational leaders important fidelity data that can help inform the larger evaluation and transformation of CBME.
Collapse
Affiliation(s)
- Warren J Cheung
- Department of Emergency Medicine, University of Ottawa, Ottawa, Canada
- Royal College of Physicians and Surgeons of Canada, Ottawa, Canada
| | - Natalie Wagner
- Office of Professional Development & Educational Scholarship and Department of Biomedical & Molecular Sciences, Queen's University, Kingston, Canada
| | - Jason R Frank
- Department of Emergency Medicine, University of Ottawa, Ottawa, Canada
- Royal College of Physicians and Surgeons of Canada, Ottawa, Canada
| | - Anna Oswald
- Royal College of Physicians and Surgeons of Canada, Ottawa, Canada
- Department of Medicine, University of Alberta, Edmonton, Canada
| | - Elaine Van Melle
- Royal College of Physicians and Surgeons of Canada, Ottawa, Canada
- Department of Family Medicine, Queen's University, Kingston, Canada
| | | | - Timothy R Dalseg
- Royal College of Physicians and Surgeons of Canada, Ottawa, Canada
- Department of Medicine, Division of Emergency Medicine, University of Toronto, Toronto, Canada
| | - Lara J Cooke
- Royal College of Physicians and Surgeons of Canada, Ottawa, Canada
- Department of Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, Canada
| | - Andrew K Hall
- Department of Emergency Medicine, University of Ottawa, Ottawa, Canada
- Royal College of Physicians and Surgeons of Canada, Ottawa, Canada
| |
Collapse
|
24
|
Concordance of Narrative Comments with Supervision Ratings Provided During Entrustable Professional Activity Assessments. J Gen Intern Med 2022; 37:2200-2207. [PMID: 35710663 PMCID: PMC9296736 DOI: 10.1007/s11606-022-07509-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/08/2021] [Accepted: 03/24/2022] [Indexed: 10/18/2022]
Abstract
BACKGROUND Use of EPA-based entrustment-supervision ratings to determine a learner's readiness to assume patient care responsibilities is expanding. OBJECTIVE In this study, we investigate the correlation between narrative comments and supervision ratings assigned during ad hoc assessments of medical students' performance of EPA tasks. DESIGN Data from assessments completed for students enrolled in the clerkship phase over 2 academic years were used to extract a stratified random sample of 100 narrative comments for review by an expert panel. PARTICIPANTS A review panel, comprised of faculty with specific expertise related to their roles within the EPA program, provided a "gold standard" supervision rating using the comments provided by the original assessor. MAIN MEASURES Interrater reliability (IRR) between members of review panel and correlation coefficients (CC) between expert ratings and supervision ratings from original assessors. KEY RESULTS IRR among members of the expert panel ranged from .536 for comments associated with focused history taking to .833 for complete physical exam. CC (Kendall's correlation coefficient W) between panel members' assignment of supervision ratings and the ratings provided by the original assessors for history taking, physical examination, and oral presentation comments were .668, .697, and .735 respectively. The supervision ratings of the expert panel had the highest degree of correlation with ratings provided during assessments done by master assessors, faculty trained to assess students across clinical contexts. Correlation between supervision ratings provided with the narrative comments at the time of observation and supervision ratings assigned by the expert panel differed by clinical discipline, perhaps reflecting the value placed on, and perhaps the comfort level with, assessment of the task in a given specialty. CONCLUSIONS To realize the full educational and catalytic effect of EPA assessments, assessors must apply established performance expectations and provide high-quality narrative comments aligned with the criteria.
Collapse
|
25
|
Mennin S. Words matter. CBME is not complex. MEDICAL TEACHER 2022; 44:571. [PMID: 34288791 DOI: 10.1080/0142159x.2021.1954606] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Affiliation(s)
- Stewart Mennin
- School of Medicine, Cell Biology and Physiology, University of New Mexico Albuquerque, NM, USA
| |
Collapse
|
26
|
Nelson MR, Smith AR, Lawrence MG. The continuum of Allergy-Immunology Fellowship Training and continuing certification embraces competency based medical education. Ann Allergy Asthma Immunol 2022; 128:236-237. [PMID: 35216743 DOI: 10.1016/j.anai.2021.12.019] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2021] [Revised: 12/21/2021] [Accepted: 12/27/2021] [Indexed: 10/19/2022]
Affiliation(s)
- Michael R Nelson
- University of Virginia School of Medicine, Charlottesville, VA; American Board of Allergy and Immunology, Philadelphia, PA.
| | - Anna R Smith
- University of Virginia School of Medicine, Charlottesville, VA
| | | |
Collapse
|
27
|
Hall AK, Schumacher DJ, Thoma B, Caretta-Weyer H, Kinnear B, Gruppen L, Cooke LJ, Frank JR, Van Melle E. Outcomes of competency-based medical education: A taxonomy for shared language. MEDICAL TEACHER 2021; 43:788-793. [PMID: 34038673 DOI: 10.1080/0142159x.2021.1925643] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
As the global transformation of postgraduate medical training continues, there are persistent calls for program evaluation efforts to understand the impact and outcomes of competency-based medical education (CBME) implementation. The measurement of a complex educational intervention such as CBME is challenging because of the multifaceted nature of activities and outcomes. What is needed, therefore, is an organizational taxonomy to both conceptualize and categorize multiple outcomes. In this manuscript we propose a taxonomy that builds on preceding works to organize CBME outcomes across three domains: focus (educational, clinical), level (micro, meso, macro), and timeline (training, transition to practice, practice). We also provide examples of how to conceptualize outcomes of educational interventions across medical specialties using this taxonomy. By proposing a shared language for outcomes of CBME, we hope that this taxonomy will help organize ongoing evaluation work and catalyze those seeking to engage in the evaluation effort to help understand the impact and outcomes of CBME.
Collapse
Affiliation(s)
- Andrew K Hall
- Department of Emergency Medicine, Queen's University, Kingston, Canada
- Royal College of Physicians and Surgeons of Canada, Ottawa, Canada
| | - Daniel J Schumacher
- Division of Emergency Medicine, Cincinnati Children's Hospital Medical Center
- Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, USA
| | - Brent Thoma
- Royal College of Physicians and Surgeons of Canada, Ottawa, Canada
- Department of Emergency Medicine, University of Saskatchewan, Saskatoon, Canada
| | - Holly Caretta-Weyer
- Department of Emergency Medicine, Stanford University School of Medicine, Palo Alto, CA, USA
| | - Benjamin Kinnear
- Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, USA
| | - Larry Gruppen
- Department of Learning Health Sciences, University of Michigan, Ann Arbor, MI, USA
| | - Lara J Cooke
- Royal College of Physicians and Surgeons of Canada, Ottawa, Canada
- Division of Neurology, Department of Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, Canada
| | - Jason R Frank
- Royal College of Physicians and Surgeons of Canada, Ottawa, Canada
- Department of Emergency Medicine, University of Ottawa, Ottawa, Canada
| | - Elaine Van Melle
- Royal College of Physicians and Surgeons of Canada, Ottawa, Canada
- Department of Family Medicine, Queen's University, Kingston, Canada
| |
Collapse
|