1
|
Brown A, La J, Keri MI, Hillis C, Razack S, Korah N, Karpinski J, Frank JR, Wong B, Goldman J. In EPAs we trust, is quality and safety a must? A cross-specialty analysis of entrustable professional activity guides. MEDICAL TEACHER 2024:1-9. [PMID: 38527417 DOI: 10.1080/0142159x.2024.2323177] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/30/2023] [Accepted: 02/21/2024] [Indexed: 03/27/2024]
Abstract
PURPOSE The inclusion of quality improvement (QI) and patient safety (PS) into CanMEDS reflects an expectation that graduating physicians are competent in these areas upon training completion. To ensure that Canadian postgraduate specialty training achieves this, the translation of QI/PS competencies into training standards as part of the implementation of competency-based medical education requires special attention. METHODS We conducted a cross-specialty, multi-method analysis to examine how QI/PS was incorporated into the EPA Guides across 11 postgraduate specialties in Canada. RESULTS We identify cross-specialty variability in how QI/PS is incorporated, positioned, and emphasized in EPAs and milestones. QI/PS was primarily referenced alongside clinical activities rather than as a sole competency or discrete activity. Patterns were characterized in how QI/PS became incorporated into milestones through repetition and customization. QI/PS was also decoupled, conceptualized, and emphasized differently across specialties. CONCLUSIONS Variability in the inclusion of QI/PS in EPAs and milestones has important implications considering the visibility and influence of EPA Guides in practice. As specialties revisit and revise EPA Guides, there is a need to balance the standardization of foundational QI/PS concepts to foster shared understanding while simultaneously ensuring context-sensitive applications across specialties. Beyond QI/PS, this study illuminates the challenges and opportunities that lie in bridging theoretical frameworks with practical implementation in medical education, prompting broader consideration of how intrinsic roles and emergent areas are effectively incorporated into competency-based medical education.
Collapse
Affiliation(s)
- Allison Brown
- Cumming School of Medicine, University of Calgary, Calgary, Canada
| | - Julie La
- Graduate Program in Health Quality, Queen's University, Kingston, Canada
- Department of Surgery, Queen's University, Kingston, Canada
| | | | - Chris Hillis
- Faculty of Health Sciences, McMaster University, Hamilton, Canada
| | - Saleem Razack
- Faculty of Medicine, University of British Columbia, Kelowna, Canada
| | - Nadine Korah
- Faculty of Medicine, McGill University, Montreal, Canada
| | | | - Jason R Frank
- Faculty of Medicine, University of Ottawa, Ottawa, Canada
| | - Brian Wong
- Centre for Quality Improvement and Patient Safety, Toronto, Canada
- Faculty of Medicine, University of Toronto Temerty, Toronto, Canada
| | - Joanne Goldman
- Centre for Quality Improvement and Patient Safety, Toronto, Canada
- Faculty of Medicine, University of Toronto Temerty, Toronto, Canada
| |
Collapse
|
2
|
Frank JR, Karpinski J, Sherbino J, Snell LS, Atkinson A, Oswald A, Hall AK, Cooke L, Dojeiji S, Richardson D, Cheung WJ, Cavalcanti RB, Dalseg TR, Thoma B, Flynn L, Gofton W, Dudek N, Bhanji F, Wong BMF, Razack S, Anderson R, Dubois D, Boucher A, Gomes MM, Taber S, Gorman LJ, Fulford J, Naik V, Harris KA, St. Croix R, van Melle E. Competence By Design: a transformational national model of time-variable competency-based postgraduate medical education. PERSPECTIVES ON MEDICAL EDUCATION 2024; 13:201-223. [PMID: 38525203 PMCID: PMC10959143 DOI: 10.5334/pme.1096] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/04/2023] [Accepted: 02/16/2024] [Indexed: 03/26/2024]
Abstract
Postgraduate medical education is an essential societal enterprise that prepares highly skilled physicians for the health workforce. In recent years, PGME systems have been criticized worldwide for problems with variable graduate abilities, concerns about patient safety, and issues with teaching and assessment methods. In response, competency based medical education approaches, with an emphasis on graduate outcomes, have been proposed as the direction for 21st century health profession education. However, there are few published models of large-scale implementation of these approaches. We describe the rationale and design for a national, time-variable competency-based multi-specialty system for postgraduate medical education called Competence by Design. Fourteen innovations were bundled to create this new system, using the Van Melle Core Components of competency based medical education as the basis for the transformation. The successful execution of this transformational training system shows competency based medical education can be implemented at scale. The lessons learned in the early implementation of Competence by Design can inform competency based medical education innovation efforts across professions worldwide.
Collapse
Affiliation(s)
- Jason R. Frank
- Centre for Innovation in Medical Education and Professor, Department of Emergency Medicine, Faculty of Medicine, University of Ottawa, ON, Canada
| | - Jolanta Karpinski
- Department of Medicine, University of Ottawa, Ottawa, ON, Canada
- Competency Based Medical Education, University of Ottawa, Ottawa, ON, Canada
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | | | - Linda S. Snell
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Medicine and Health Sciences Education, McGill University, Montreal, QC, Canada
| | - Adelle Atkinson
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Paediatrics, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
| | - Anna Oswald
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Medicine, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, AB, Canada
- Competency Based Medical Education, University of Alberta, Edmonton, AB, Canada
| | - Andrew K. Hall
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Emergency Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Lara Cooke
- Division of Neurology, Department of Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Susan Dojeiji
- Physical Medicine and Rehabilitation, University of Ottawa, Ottawa, ON, Canada
| | - Denyse Richardson
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Physical Medicine and Rehabilitation, Queen’s University, Kingston, ON, Canada
| | - Warren J. Cheung
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Medicine, University of Toronto, Toronto, ON, Canada
| | - Rodrigo B. Cavalcanti
- Department of Medicine, University of Toronto, Toronto, ON, Canada
- HoPingKong Centre, University Health Network, Toronto, ON, Canada
| | - Timothy R. Dalseg
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Division of Emergency Medicine, University of Toronto, Toronto, ON, Canada
| | - Brent Thoma
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Emergency Medicine, University of Saskatchewan, Saskatoon, SK, Canada
| | - Leslie Flynn
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Departments of Psychiatry and Family Medicine, and Co-Director Master of Health Sciences Education, Queen’s University, Kingston, ON, Canada
| | - Wade Gofton
- Department of Surgery (Division of Orthopedic Surgery), The Ottawa Hospital and University of Ottawa, Ottawa, ON, Canada
| | - Nancy Dudek
- Department of Medicine (Division of Physical Medicine & Rehabilitation) and The Ottawa Hospital, University of Ottawa, Ottawa, Ontario, Canada
| | - Farhan Bhanji
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Faculty of Medicine and Health Sciences, McGill University, Montreal, QC, Canada
| | - Brian M.-F. Wong
- Centre for Quality Improvement and Patient Safety, University of Toronto, Toronto, Canada
| | - Saleem Razack
- Centre for Health Education Scholarship, University of British Columbia and BC Children’s Hospital, Vancouver, BC, Canada
| | - Robert Anderson
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Northern Ontario School of Medicine University, Sudbury, ON, Canada
| | - Daniel Dubois
- Department of Anesthesiology and Pain Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Andrée Boucher
- Department of Medicine (Division of Endocrinology), Universitéde Montréal, Montréal, QC, Canada
| | - Marcio M. Gomes
- Department of Pathology and Laboratory Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Sarah Taber
- Office of Standards and Assessment, Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | - Lisa J. Gorman
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | - Jane Fulford
- Canadian Internet Registration Authority, Canada
| | - Viren Naik
- Department of Anesthesiology and Pain Medicine, University of Ottawa, Ottawa, ON, Canada
- Medical Council of Canada, Ottawa, ON, Canada
| | - Kenneth A. Harris
- Royal College of Physicians and Surgeons of Canada, Canada
- Emeritus, Western University, Canada
| | - Rhonda St. Croix
- Learning and Connecting at the Royal College of Physicians and Surgeons of Canada, Canada
| | - Elaine van Melle
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Family Medicine, Queen’s University, Kingston, ON, Canada
| |
Collapse
|
3
|
Dalseg TR, Thoma B, Wycliffe-Jones K, Frank JR, Taber S. Enabling Implementation of Competency Based Medical Education through an Outcomes-Focused Accreditation System. PERSPECTIVES ON MEDICAL EDUCATION 2024; 13:75-84. [PMID: 38343559 PMCID: PMC10854411 DOI: 10.5334/pme.963] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Accepted: 09/08/2023] [Indexed: 02/15/2024]
Abstract
Competency based medical education is being adopted around the world. Accreditation plays a vital role as an enabler in the adoption and implementation of competency based medical education, but little has been published about how the design of an accreditation system facilitates this transformation. The Canadian postgraduate medical education environment has recently transitioned to an outcomes-based accreditation system in parallel with the adoption of competency based medical education. Using the Canadian example, we characterize four features of an accreditation system that can facilitate the implementation of competency based medical education: theoretical underpinning, quality focus, accreditation standards, and accreditation processes. Alignment of the underlying educational theories within the accreditation system and educational paradigm drives change in a consistent and desired direction. An accreditation system that prioritizes quality improvement over quality assurance promotes educational system development and progressive change. Accreditation standards that achieve the difficult balance of being sufficiently detailed yet flexible foster a high fidelity of implementation without stifling innovation. Finally, accreditation processes that recognize the change process, encourage program development, and are not overly punitive all enable the implementation of competency based medical education. We also discuss the ways in which accreditation can simultaneously hinder the implementation of this approach. As education bodies adopt competency based medical education, particular attention should be paid to the role that accreditation plays in successful implementation.
Collapse
Affiliation(s)
- Timothy R. Dalseg
- Department of Medicine, Division of Emergency Medicine, University of Toronto, Toronto, ON, Canada
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Toronto General Hospital, 200 Elizabeth Street, R. Fraser Elliott Building, Ground Floor, Room 480, Toronto, ON M5G 2C4, (416) 833-0121; Canada
| | - Brent Thoma
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Emergency Medicine, University of Saskatchewan, Saskatoon, SK, Canada
| | - Keith Wycliffe-Jones
- Department of Family Medicine, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Jason R. Frank
- Centre for Innovation in Medical Education, University of Ottawa, Ottawa, ON, Canada
| | - Sarah Taber
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| |
Collapse
|
4
|
Karpinski J, Stewart J, Oswald A, Dalseg TR, Atkinson A, Frank JR. Competency-Based Medical Education at Scale: A Road Map for Transforming National Systems of Postgraduate Medical Education. PERSPECTIVES ON MEDICAL EDUCATION 2024; 13:24-32. [PMID: 38371306 PMCID: PMC10870941 DOI: 10.5334/pme.957] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/07/2023] [Accepted: 08/08/2023] [Indexed: 02/20/2024]
Abstract
In the past decade, the Canadian system of postgraduate medical education has been transformed with the implementation of a new approach to competency based medical education called Competence by Design. The Royal College of Physicians and Surgeons of Canada (Royal College) developed an approach to time-variable competency based medical education and adapted that design for medical, surgical, and diagnostic disciplines. New educational standards and entrustable professional activities consistent with this approach were co-created with 67 specialties and subspecialties, and implementation was scaled up across 17 universities and over 1000 postgraduate training programs. Partner engagement, systematic design of workshops to create discipline specific competency-based standards of education, and agile adaptation were all key ingredients for success. This paper describes the strategies applied by the Royal College, lessons learned regarding transformative change in the complex system of postgraduate medical education, and the current status of the Competence by Design initiative. The approach taken and lessons learned by the Royal College may be useful for other educators who are planning a transformation to CBME or any other major educational reform.
Collapse
Affiliation(s)
- Jolanta Karpinski
- Department of Medicine, University of Ottawa, Ottawa, ON, Canada
- Competency Based Medical Education, University of Ottawa, Ottawa, ON, Canada
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | - Jennifer Stewart
- Specialty Standards, Office of Standards and Assessment, Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | - Anna Oswald
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Division of Rheumatology, Department of Medicine, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, AB, Canada
- Competency Based Medical Education, University of Alberta, Edmonton, AB, Canada
| | - Timothy R. Dalseg
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Medicine, Division of Emergency Medicine, University of Toronto, Toronto, ON, Canada
| | - Adelle Atkinson
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Paediatrics, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
| | - Jason R. Frank
- Department of Emergency Medicine, and Director, Centre for Innovation in Medical Education, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada
| |
Collapse
|
5
|
Oswald A, Dubois D, Snell L, Anderson R, Karpinski J, Hall AK, Frank JR, Cheung WJ. Implementing Competence Committees on a National Scale: Design and Lessons Learned. PERSPECTIVES ON MEDICAL EDUCATION 2024; 13:56-67. [PMID: 38343555 PMCID: PMC10854462 DOI: 10.5334/pme.961] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Accepted: 07/03/2023] [Indexed: 02/15/2024]
Abstract
Competence committees (CCs) are a recent innovation to improve assessment decision-making in health professions education. CCs enable a group of trained, dedicated educators to review a portfolio of observations about a learner's progress toward competence and make systematic assessment decisions. CCs are aligned with competency based medical education (CBME) and programmatic assessment. While there is an emerging literature on CCs, little has been published on their system-wide implementation. National-scale implementation of CCs is complex, owing to the culture change that underlies this shift in assessment paradigm and the logistics and skills needed to enable it. We present the Royal College of Physicians and Surgeons of Canada's experience implementing a national CC model, the challenges the Royal College faced, and some strategies to address them. With large scale CC implementation, managing the tension between standardization and flexibility is a fundamental issue that needs to be anticipated and addressed, with careful consideration of individual program needs, resources, and engagement of invested groups. If implementation is to take place in a wide variety of contexts, an approach that uses multiple engagement and communication strategies to allow for local adaptations is needed. Large-scale implementation of CCs, like any transformative initiative, does not occur at a single point but is an evolutionary process requiring both upfront resources and ongoing support. As such, it is important to consider embedding a plan for program evaluation at the outset. We hope these shared lessons will be of value to other educators who are considering a large-scale CBME CC implementation.
Collapse
Affiliation(s)
- Anna Oswald
- Division of Rheumatology, Department of Medicine, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, AB, Canada
- Competency Based Medical Education, University of Alberta, Edmonton, AB, Canada
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- 8-130 Clinical Sciences building, 11350-83 Avenue, Edmonton, AB, Canada
| | - Daniel Dubois
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Anesthesiology and Pain Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Linda Snell
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Institute of Health Sciences Education and Department of Medicine, McGill University, Montreal, QC, Canada
| | - Robert Anderson
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Northern Ontario School of Medicine University, Sudbury, ON, Canada
| | - Jolanta Karpinski
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Andrew K. Hall
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Dept. of Emergency Medicine, University of Ottawa, Canada
| | - Jason R. Frank
- Centre for Innovation in Medical Education, Faculty of Medicine, University of Ottawa, Canada
| | - Warren J. Cheung
- Dept. of Emergency Medicine, University of Ottawa, Canada
- Royal College of Physicians and Surgeons of Canada, 1053 Carling Avenue, Rm F660, Ottawa, Canada
| |
Collapse
|
6
|
Cheung WJ, Bhanji F, Gofton W, Hall AK, Karpinski J, Richardson D, Frank JR, Dudek N. Design and Implementation of a National Program of Assessment Model - Integrating Entrustable Professional Activity Assessments in Canadian Specialist Postgraduate Medical Education. PERSPECTIVES ON MEDICAL EDUCATION 2024; 13:44-55. [PMID: 38343554 PMCID: PMC10854461 DOI: 10.5334/pme.956] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Accepted: 09/04/2023] [Indexed: 02/15/2024]
Abstract
Traditional approaches to assessment in health professions education systems, which have generally focused on the summative function of assessment through the development and episodic use of individual high-stakes examinations, may no longer be appropriate in an era of competency based medical education. Contemporary assessment programs should not only ensure collection of high-quality performance data to support robust decision-making on learners' achievement and competence development but also facilitate the provision of meaningful feedback to learners to support reflective practice and performance improvement. Programmatic assessment is a specific approach to designing assessment systems through the intentional selection and combination of a variety of assessment methods and activities embedded within an educational framework to simultaneously optimize the decision-making and learning function of assessment. It is a core component of competency based medical education and is aligned with the goals of promoting assessment for learning and coaching learners to achieve predefined levels of competence. In Canada, postgraduate specialist medical education has undergone a transformative change to a competency based model centred around entrustable professional activities (EPAs). In this paper, we describe and reflect on the large scale, national implementation of a program of assessment model designed to guide learning and ensure that robust data is collected to support defensible decisions about EPA achievement and progress through training. Reflecting on the design and implications of this assessment system may help others who want to incorporate a competency based approach in their own country.
Collapse
Affiliation(s)
- Warren J. Cheung
- Department of Emergency Medicine, University of Ottawa, Ottawa, ON, CA
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada, 1053 Carling Avenue, Rm F660, Ottawa, ON K1Y 4E9, CA
| | - Farhan Bhanji
- Department of Pediatrics (Critical Care), Faculty of Medicine and Health Sciences, McGill University, Montreal, QC, CA
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, CA
| | - Wade Gofton
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, CA
- Department of Surgery, Division of Orthopaedic Surgery, University of Ottawa, Ottawa, ON, CA
| | - Andrew K. Hall
- Department of Emergency Medicine, University of Ottawa, Ottawa, ON, CA
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, CA
| | - Jolanta Karpinski
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, CA
- Department of Medicine, University of Ottawa, Ottawa, ON, CA
| | - Denyse Richardson
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, CA
- Department of Physical Medicine and Rehabilitation, Queen’s University, Kingston, ON, CA
| | - Jason R. Frank
- Department of Emergency Medicine, Director, Centre for Innovation in Medical Education, Faculty of Medicine, University of Ottawa, Ottawa, ON, CA
| | - Nancy Dudek
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, CA
- Department of Medicine, Division of Physical Medicine and Rehabilitation, University of Ottawa, Ottawa, ON, CA
| |
Collapse
|
7
|
Gilchrist PT, Beaton NSM, Atkin JN, Roberts LJ. The new Diploma of Rural Generalist Anaesthesia: Supporting Australian rural and remote communities. Anaesth Intensive Care 2024; 52:6-15. [PMID: 38006613 DOI: 10.1177/0310057x231196909] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2023]
Abstract
In 2023, a Diploma of Rural Generalist Anaesthesia (DipRGA) was implemented across Australia. Developed collaboratively by the Australian and New Zealand College of Anaesthetists (ANZCA), the Australian College of Rural and Remote Medicine (ACRRM) and the Royal Australian College of General Practitioners (RACGP), the 12-month qualification is completed during or following ACRRM or RACGP Rural Generalist Fellowship training. Focused on the needs of rural and remote communities for elective and emergency surgery, maternity care, resuscitative care for medical illness or injury, and stabilisation for retrieval, the DipRGA supports rural generalist anaesthetists working within collaborative teams in geographically isolated settings. The goal is a graduate who can anaesthetise American Society of Anesthesiologists physical status class 1, 2 and stable 3 patients for elective surgery, provide obstetric anaesthesia and analgesia, anaesthetise paediatric patients and undertake advanced crisis care within their scope of practice. Crucially, they also recognise both limitations of their skills and local resources available when considering whether to provide care, defer, refer or transfer patients. DipRGA curriculum design commenced by adapting the ANZCA specialist training curriculum with consideration of the training approach of both the ACRRM and the RACGP, particularly the rural and remote context. Curriculum content is addressed in seven entrustable professional activities supported by workplace-based assessments and multisource feedback. Trainees are supervised by rural generalist anaesthetists and specialist anaesthetists, and complete flexible learning activities to accommodate geographical dispersion. Standardised summative assessments include an early test of knowledge and an examination, adapted from the ACRRM structured assessment using multiple patient scenarios.
Collapse
Affiliation(s)
- Peter T Gilchrist
- Flinders and Upper North Local Health Network, Whyalla, Australia
- Rural Generalist Program, South Australia Health, Adelaide, Australia
- Royal Australian College of General Practitioners, East Melbourne, Australia
- Rural Clinical School, The University of Adelaide, Adelaide, Australia
| | - Neil St M Beaton
- Australian College of Rural and Remote Medicine, Brisbane, Australia
- James Cook University College of Medicine and Dentistry, Townsville, Australia
| | | | - Lindy J Roberts
- Australian and New Zealand College of Anaesthetists, Melbourne, Australia
| |
Collapse
|
8
|
Moffatt-Bruce SD, Harris K, Rubens FD, Villeneuve PJ, Sundaresan RS. Competency-based training: Canadian cardiothoracic surgery. J Thorac Cardiovasc Surg 2024; 167:407-410. [PMID: 36702679 DOI: 10.1016/j.jtcvs.2023.01.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/03/2022] [Revised: 11/25/2022] [Accepted: 01/05/2023] [Indexed: 01/11/2023]
Affiliation(s)
- Susan D Moffatt-Bruce
- The Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada; Department of Surgery, University of Ottawa, Ottawa, Ontario, Canada.
| | - Ken Harris
- Department of Surgery, University of Western Ontario, London, Ontario, Canada
| | - Fraser D Rubens
- Department of Surgery, University of Ottawa, Ottawa, Ontario, Canada
| | | | | |
Collapse
|
9
|
Monteiro S, McConnell MM. Evaluating the Construct Validity of Competencies: A Retrospective Analysis. MEDICAL SCIENCE EDUCATOR 2023; 33:729-736. [PMID: 37501811 PMCID: PMC10368597 DOI: 10.1007/s40670-023-01794-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 04/28/2023] [Indexed: 07/29/2023]
Abstract
Background A competency-based framework focuses on alignment between professional standards and assessment design. This alignment implies improved measurement validity, yet it has not been established that competence in one context predicts performance in another context. High-stakes competence assessments offer insights into the relationship between assessment design and competencies. Methods/Analyses The internationally educated nurses competency assessment program (IENCAP) was developed at Touchstone Institute in collaboration with the College of Nurses of Ontario (CNO) and includes a 12-station OSCE. Each station evaluated the same 10 competencies. We submitted competency scores to a multi-trait multi-method matrix analysis to evaluate the convergent and discriminant validity of competencies. Results/Observations All correlations were significant and positive; however, we did not find evidence of convergent or discriminant validity. Correlations were higher between different competencies evaluated within the same station (mean correlation = 0.60) compared to identical competencies evaluated across different stations (mean correlation = 0.19). Discussion The results do not provide evidence of construct validity for competencies. While competency-based approaches emphasize various generalized knowledge, skills, and attitudes, these findings indicate that the clinical context is a major determinant of performance. Conclusion The context-dependent nature of competencies requires multiple assessments in varied contexts. Performance on a single competency cannot be determined in a single occasion. Supplementary Information The online version contains supplementary material available at 10.1007/s40670-023-01794-z.
Collapse
Affiliation(s)
- Sandra Monteiro
- Division of Education and Innovation, Department of Medicine, Faculty of Health Sciences Program in Education Research, Innovation, and Theory (MERIT), Centre for Simulation Based Learning (CSBL), McMaster University, David Braley Health Sciences Centre, 100 Main Street, 5Th Floor 5002 A/E, Hamilton, Canada
| | - Meghan M. McConnell
- Department of Innovation in Medical Education (DIME), University of Ottawa, Ottawa, Canada
| |
Collapse
|
10
|
Tan X, Ning N, Hua N, Wiley J, Sun M. Determining the core competencies for international postgraduate nursing students in internship programs: A delphi study. Nurse Educ Pract 2023; 70:103654. [PMID: 37146479 DOI: 10.1016/j.nepr.2023.103654] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2022] [Revised: 03/26/2023] [Accepted: 04/24/2023] [Indexed: 05/07/2023]
Abstract
AIM To determine the core competencies of clinical internships for international postgraduate nursing students undertaking a two-year professional program in China. BACKGROUND Clinical internship is an important part of nursing education and is the cornerstone for the future training of nursing professionals. However, in terms of providing a basis for the training and assessment of international postgraduate nursing students undertaking a two-year professional program in China, their core competencies of clinical internships have not yet been sufficiently determined. METHODS Focus group interviews and a two-round Delphi method were conducted. The preliminary list of core competencies was determined based on a scoping review and focus group interviews. Subsequently, experts provided suggestions for modifications to the core competencies in two rounds of the Delphi survey. The response rate (RR), composite reliability (Cr), coefficient of variation and Kendall coefficient of indices were calculated. RESULTS Twenty experts completed two rounds of Delphi expert consultation and reached consensus on five first-level indices and 13 s-level indices with 27 connotations. The RR values for the two rounds of consultation were 100 %, the Cr values were 0.853 and 0.873 and the Kendall coordination coefficients were 0.134-0.250 (p < 0.05). CONCLUSION The core competencies identified in this research can provide a basis for further training international postgraduate nursing students undertaking a two-year professional program in China through internship programs. This research also provides a reference for effectively evaluating and improving clinical programs.
Collapse
Affiliation(s)
- Xiangmin Tan
- Xiangya School of Nursing, Central South University, Changsha, China
| | - Ni Ning
- Xiangya School of Nursing, Central South University, Changsha, China
| | - Nan Hua
- Xiangya School of Nursing, Central South University, Changsha, China
| | - James Wiley
- School of Nursing, University of California, San Francisco, CA 94118, USA
| | - Mei Sun
- Xiangya School of Nursing, Central South University, Changsha, China.
| |
Collapse
|
11
|
Mackenzie MJ, Hagel C, Lin Y, Hall AK, Grant VJ, Doshi S. The Reliability of the Resuscitation Assessment Tool (RAT) in Assessing Emergency Medicine Resident Competence in Pediatric Resuscitation Scenarios: A Prospective Observational Pilot Study. Cureus 2023; 15:e35869. [PMID: 37033538 PMCID: PMC10079254 DOI: 10.7759/cureus.35869] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/07/2023] [Indexed: 03/09/2023] Open
Abstract
Introduction Emergency medicine (EM) postgraduate medical education in Canada has transitioned from traditional time-based training to competency-based medical education (CBME). In order to promote residents through stages of training, simulated assessments are needed to evaluate residents in high-stakes but low-frequency medical emergencies. There remains a gap in the literature pertaining to the use of evaluative tools in simulation, such as the Resuscitation Assessment Tool (RAT) in the new CBME curriculum design. Methods We completed a pilot study of resident physicians in one Canadian EM training program to evaluate the effectiveness and reliability of a simulation-based RAT for pediatric resuscitation. We recorded 10 EM trainees completing simulated scenarios and had nine EM physicians use the RAT tool to evaluate their performances. Generalizability theory was used to evaluate the reliability of the RAT tool. Results The mean RAT score for the management of pediatric myocarditis, cardiac arrest, and septic shock (appendicitis) across raters was 3.70, 3.73, and 4.50, respectively. The overall generalizability coefficient for testing simulated pediatric performance competency was 0.77 for internal consistency and 0.75 for absolute agreement. The performance of senior participants was superior to that of junior participants in the management of pediatric myocarditis (p = 0.01) but not statistically significant in the management of pediatric septic shock (p=0.77) or cardiac arrest (p =0.61). Conclusion Overall, our findings suggest that with an appropriately chosen simulated scenario, the RAT tool can be used effectively for the simulation of high-stakes and low-frequency scenarios for practice to enhance the new CBME curriculum in emergency medicine training programs.
Collapse
|
12
|
Thoma B, Abbott C, Snell L. The future of the CanMEDS physician competency framework. CANADIAN MEDICAL EDUCATION JOURNAL 2023; 14:1-3. [PMID: 36998483 PMCID: PMC10042786 DOI: 10.36834/cmej.77098] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Affiliation(s)
- Brent Thoma
- Emergency Medicine, University of Saskatchewan, Saskatchewan, Canada
- Royal College of Physicians and Surgeons of Canada, Ontario, Canada
| | - Cynthia Abbott
- Royal College of Physicians and Surgeons of Canada, Ontario, Canada
| | - Linda Snell
- Royal College of Physicians and Surgeons of Canada, Ontario, Canada
- Medicine and Health Sciences Education, McGill University, Quebec, Canada
| |
Collapse
|
13
|
Chan TM, Dowling S, Tastad K, Chin A, Thoma B. Integrating training, practice, and reflection within a new model for Canadian medical licensure: a concept paper prepared for the Medical Council of Canada. CANADIAN MEDICAL EDUCATION JOURNAL 2022; 13:68-81. [PMID: 36091730 PMCID: PMC9441128 DOI: 10.36834/cmej.73717] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
In 2020 the Medical Council of Canada created a task force to make recommendations on the modernization of its practices for granting licensure to medical trainees. This task force solicited papers on this topic from subject matter experts. As outlined within this Concept Paper, our proposal would shift licensure away from the traditional focus on high-stakes summative exams in a way that integrates training, clinical practice, and reflection. Specifically, we propose a model of graduated licensure that would have three stages including: a trainee license for trainees that have demonstrated adequate medical knowledge to begin training as a closely supervised resident, a transition to practice license for trainees that have compiled a reflective educational portfolio demonstrating the clinical competence required to begin independent practice with limitations and support, and a fully independent license for unsupervised practice for attendings that have demonstrated competence through a reflective portfolio of clinical analytics. This proposal was reviewed by a diverse group of 30 trainees, practitioners, and administrators in medical education. Their feedback was analyzed and summarized to provide an overview of the likely reception that this proposal would receive from the medical education community.
Collapse
Affiliation(s)
- Teresa M Chan
- Division of Emergency Medicine, Department of Medicine, Faculty of Health Sciences, McMaster University
- Division of Education & Innovation, Department of Medicine
- Faculty of Health Sciences, McMaster University; McMaster Education Research, Innovation, and Theory (MERIT) program
- Office of Continuing Professional Development; Faculty of Health Sciences, McMaster University
| | - Shawn Dowling
- Department of Emergency Medicine, Cumming School of Medicine, University of Calgary
| | - Kara Tastad
- Royal College Emergency Medicine Training Program, University of Toronto
| | - Alvin Chin
- Division of Emergency Medicine, Department of Medicine, Faculty of Health Sciences, McMaster University
| | - Brent Thoma
- Department of Emergency Medicine, University of Saskatchewan
| |
Collapse
|
14
|
Thoma B, Monteiro S, Pardhan A, Waters H, Chan T. Replacing high-stakes summative examinations with graduated medical licensure in Canada. CMAJ 2022; 194:E168-E170. [PMID: 35131756 PMCID: PMC8900762 DOI: 10.1503/cmaj.211816] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/20/2023] Open
Affiliation(s)
- Brent Thoma
- Department of Emergency Medicine (Thoma), University of Saskatchewan, Saskatoon, Sask.; Royal College of Physicians and Surgeons (Thoma), Ottawa, Ott.; Centre for Simulation-Based Learning (Monteiro); McMaster Education Research, Innovation, and Theory (MERIT) Program (Monteiro, Chan); Division of Education & Innovation, Department of Medicine (Monteiro, Chan); Division of Emergency Medicine (Pardhan), Department of Medicine and Pediatrics, and Fellow of the Royal College of Physicians of Canada Emergency Medicine Program (Pardhan); Department of Family Medicine (Waters); McMaster Office of Postgraduate Medical Education Advisory Board (Waters); Division of Emergency Medicine (Chan), Department of Medicine, Faculty of Health Sciences; Office of Continuing Professional Development (Chan), Faculty of Health Sciences, McMaster University, Hamilton, Ont
| | - Sandra Monteiro
- Department of Emergency Medicine (Thoma), University of Saskatchewan, Saskatoon, Sask.; Royal College of Physicians and Surgeons (Thoma), Ottawa, Ott.; Centre for Simulation-Based Learning (Monteiro); McMaster Education Research, Innovation, and Theory (MERIT) Program (Monteiro, Chan); Division of Education & Innovation, Department of Medicine (Monteiro, Chan); Division of Emergency Medicine (Pardhan), Department of Medicine and Pediatrics, and Fellow of the Royal College of Physicians of Canada Emergency Medicine Program (Pardhan); Department of Family Medicine (Waters); McMaster Office of Postgraduate Medical Education Advisory Board (Waters); Division of Emergency Medicine (Chan), Department of Medicine, Faculty of Health Sciences; Office of Continuing Professional Development (Chan), Faculty of Health Sciences, McMaster University, Hamilton, Ont
| | - Alim Pardhan
- Department of Emergency Medicine (Thoma), University of Saskatchewan, Saskatoon, Sask.; Royal College of Physicians and Surgeons (Thoma), Ottawa, Ott.; Centre for Simulation-Based Learning (Monteiro); McMaster Education Research, Innovation, and Theory (MERIT) Program (Monteiro, Chan); Division of Education & Innovation, Department of Medicine (Monteiro, Chan); Division of Emergency Medicine (Pardhan), Department of Medicine and Pediatrics, and Fellow of the Royal College of Physicians of Canada Emergency Medicine Program (Pardhan); Department of Family Medicine (Waters); McMaster Office of Postgraduate Medical Education Advisory Board (Waters); Division of Emergency Medicine (Chan), Department of Medicine, Faculty of Health Sciences; Office of Continuing Professional Development (Chan), Faculty of Health Sciences, McMaster University, Hamilton, Ont
| | - Heather Waters
- Department of Emergency Medicine (Thoma), University of Saskatchewan, Saskatoon, Sask.; Royal College of Physicians and Surgeons (Thoma), Ottawa, Ott.; Centre for Simulation-Based Learning (Monteiro); McMaster Education Research, Innovation, and Theory (MERIT) Program (Monteiro, Chan); Division of Education & Innovation, Department of Medicine (Monteiro, Chan); Division of Emergency Medicine (Pardhan), Department of Medicine and Pediatrics, and Fellow of the Royal College of Physicians of Canada Emergency Medicine Program (Pardhan); Department of Family Medicine (Waters); McMaster Office of Postgraduate Medical Education Advisory Board (Waters); Division of Emergency Medicine (Chan), Department of Medicine, Faculty of Health Sciences; Office of Continuing Professional Development (Chan), Faculty of Health Sciences, McMaster University, Hamilton, Ont
| | - Teresa Chan
- Department of Emergency Medicine (Thoma), University of Saskatchewan, Saskatoon, Sask.; Royal College of Physicians and Surgeons (Thoma), Ottawa, Ott.; Centre for Simulation-Based Learning (Monteiro); McMaster Education Research, Innovation, and Theory (MERIT) Program (Monteiro, Chan); Division of Education & Innovation, Department of Medicine (Monteiro, Chan); Division of Emergency Medicine (Pardhan), Department of Medicine and Pediatrics, and Fellow of the Royal College of Physicians of Canada Emergency Medicine Program (Pardhan); Department of Family Medicine (Waters); McMaster Office of Postgraduate Medical Education Advisory Board (Waters); Division of Emergency Medicine (Chan), Department of Medicine, Faculty of Health Sciences; Office of Continuing Professional Development (Chan), Faculty of Health Sciences, McMaster University, Hamilton, Ont.
| |
Collapse
|
15
|
Unaka NI, Winn A, Spinks-Franklin A, Poitevien P, Trimm F, Nuncio Lujano BJ, Turner DA. An Entrustable Professional Activity Addressing Racism and Pediatric Health Inequities. Pediatrics 2022; 149:184453. [PMID: 35001103 PMCID: PMC9647957 DOI: 10.1542/peds.2021-054604] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 11/17/2021] [Indexed: 02/03/2023] Open
Abstract
Racism and discrimination are the root of many pediatric health inequities and are well described in the literature. Despite the pervasiveness of pediatric health inequities, we have failed to adequately educate and prepare general pediatricians and pediatric subspecialists to address them. Deficiencies within education across the entire continuum and in our health care systems as a whole contribute to health inequities in unacceptable ways. To address these deficiencies, the field of pediatrics, along with other specialties, has been on a journey toward a more competency-based approach to education and assessment, and the framework created for the future is built on entrustable professional activities (EPAs). Competency-based medical education is one approach to addressing the deficiencies within graduate medical education and across the continuum by allowing educators to focus on the desired equitable patient outcomes and then develop an approach to teaching and assessing the tasks, knowledge, skills, and attitudes needed to achieve the goal of optimal, equitable patient care. To that end, we describe the development and content of a revised EPA entitled: Use of Population Health Strategies and Quality Improvement Methods to Promote Health and Address Racism, Discrimination, and Other Contributors to Inequities Among Pediatric Populations. We also highlight the ways in which this EPA can be used to inform curricula, assessments, professional development, organizational systems, and culture change.
Collapse
Affiliation(s)
- Ndidi I. Unaka
- Division of Hospital Medicine, Cincinnati Children’s Hospital, Cincinnati, Ohio,Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, Ohio,Address correspondence to Ndidi I. Unaka, MD, MEd, Cincinnati Children’s Hospital Medical Center, 3333 Burnet Ave, ML 5018, Cincinnati, OH 45229. E-mail:
| | - Ariel Winn
- Division of General Pediatrics, Department of Pediatrics, Boston Children’s Hospital, Boston, Massachusetts,Department of Pediatrics, Harvard Medical School, Boston, Massachusetts
| | - Adiaha Spinks-Franklin
- Section of Developmental Pediatrics, Department of Pediatrics, Texas Children’s Hospital/Baylor College of Medicine, Houston, Texas
| | - Patricia Poitevien
- Division of Hospitalist Medicine, Department of Pediatrics, Hasbro Children’s Hospital, Warren Alpert Medical School of Brown University, Providence, Rhode Island
| | - Franklin Trimm
- Office of Diversity and Inclusion, Department of Pediatrics, University of South Alabama College of Medicine, Mobile, Alabama
| | | | - David A. Turner
- The American Board of Pediatrics, Chapel Hill, North Carolina
| |
Collapse
|
16
|
Nousiainen MT, Bardana D, Gofton W, Broekhuyse HM, Kraemer W. Creating a National Competency-Based Curriculum for Orthopaedic Surgery Residency: The Canadian Experience. JB JS Open Access 2022; 7:JBJSOA-D-21-00131. [PMID: 35355781 PMCID: PMC8939911 DOI: 10.2106/jbjs.oa.21.00131] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Affiliation(s)
| | - Davide Bardana
- Division of Orthopaedic Surgery, Queen's University at Kingston, Kingston, Canada
| | - Wade Gofton
- Division of Orthopaedic Surgery, University of Ottawa, Ottawa, Canada
| | - Henry M Broekhuyse
- Department of Orthopaedic Surgery, University of British Columbia, Vancouver, Canada
| | - William Kraemer
- Division of Orthopaedic Surgery, University of Toronto, Toronto, Canada
| |
Collapse
|
17
|
Schumacher DJ, Turner DA. Entrustable Professional Activities: Reflecting on Where We Are to Define a Path for the Next Decade. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2021; 96:S1-S5. [PMID: 34183594 DOI: 10.1097/acm.0000000000004097] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Affiliation(s)
- Daniel J Schumacher
- D.J. Schumacher is associate professor of pediatrics, Cincinnati Children's Hospital Medical Center and University of Cincinnati College of Medicine, Cincinnati, Ohio
| | - David A Turner
- D.A. Turner is vice president for competency-based medical education, American Board of Pediatrics, Chapel Hill, North Carolina
| |
Collapse
|