1
|
Hall AK, Oswald A, Frank JR, Dalseg T, Cheung WJ, Cooke L, Gorman L, Brzezina S, Selvaratnam S, Wagner N, Hamstra SJ, Van Melle E. Evaluating Competence by Design as a Large System Change Initiative: Readiness, Fidelity, and Outcomes. PERSPECTIVES ON MEDICAL EDUCATION 2024; 13:95-107. [PMID: 38343556 PMCID: PMC10854467 DOI: 10.5334/pme.962] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Academic Contribution Register] [Received: 03/08/2023] [Accepted: 08/17/2023] [Indexed: 02/15/2024]
Abstract
Program evaluation is an essential, but often neglected, activity in any transformational educational change. Competence by Design was a large-scale change initiative to implement a competency-based time-variable educational system in Canadian postgraduate medical education. A program evaluation strategy was an integral part of the build and implementation plan for CBD from the beginning, providing insights into implementation progress, challenges, unexpected outcomes, and impact. The Competence by Design program evaluation strategy was built upon a logic model and three pillars of evaluation: readiness to implement, fidelity and integrity of implementation, and outcomes of implementation. The program evaluation strategy harvested from both internally driven studies and those performed by partners and invested others. A dashboard for the program evaluation strategy was created to transparently display a real-time view of Competence by Design implementation and facilitate continuous adaptation and improvement. The findings of the program evaluation for Competence by Design drove changes to all aspects of the Competence by Design implementation, aided engagement of partners, supported change management, and deepened our understanding of the journey required for transformational educational change in a complex national postgraduate medical education system. The program evaluation strategy for Competence by Design provides a framework for program evaluation for any large-scale change in health professions education.
Collapse
Affiliation(s)
- Andrew K. Hall
- Department of Emergency Medicine, University of Ottawa, Ottawa, ON, Canada
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | - Anna Oswald
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Division of Rheumatology, Department of Medicine, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, AB, Canada
| | - Jason R. Frank
- Department of Emergency Medicine, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Tim Dalseg
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Medicine, Division of Emergency Medicine, University of Toronto, Toronto, ON, Canada
| | - Warren J. Cheung
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Emergency Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Lara Cooke
- Neurology, Department of Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Lisa Gorman
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | - Stacey Brzezina
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | | | - Natalie Wagner
- Queen’s Health Sciences Office of Professional Development and Educational Scholarship, Queen’s University, Kingston, ON, Canada
| | - Stanley J. Hamstra
- Department of Surgery, University of Toronto, Toronto, ON, Canada
- Department of Milestones Research and Evaluation, Accreditation Council for Graduate Medical Education, Chicago, IL, USA
- Department of Medical Education, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Elaine Van Melle
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Family Medicine, Queen’s University, Kingston, ON, Canada
| |
Collapse
|
2
|
D’Eon MF, Page R, Reilly M. Education decision making at medical schools: there must be a better way. CANADIAN MEDICAL EDUCATION JOURNAL 2021; 12:1-5. [PMID: 35003425 PMCID: PMC8740264 DOI: 10.36834/cmej.74331] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Academic Contribution Register] [Indexed: 06/14/2023]
Affiliation(s)
- Marcel F D’Eon
- Educational Innovation Institute of the Medical College of Georgia, Augusta University, Georgia, USA
| | - Renee Page
- Educational Innovation Institute of the Medical College of Georgia, Augusta University, Georgia, USA
| | - Mayland Reilly
- Educational Innovation Institute of the Medical College of Georgia, Augusta University, Georgia, USA
| |
Collapse
|