1
|
Wong L, Sacoransky E, Hopman W, Islam O, Chung AD, Kwan BYM. Radiologist preferences for faculty development initiatives to improve resident feedback in the era of competency-based medical education. MEDICAL EDUCATION ONLINE 2024; 29:2357412. [PMID: 38810150 PMCID: PMC11138222 DOI: 10.1080/10872981.2024.2357412] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/10/2024] [Accepted: 05/15/2024] [Indexed: 05/31/2024]
Abstract
INTRODUCTION Since 2022, all Canadian post-graduate medical programs have transitioned to a Competence by Design (CBD) model within a Competency-Based Medical Education (CBME) framework. The CBME model emphasized more frequent, formative assessment of residents to evaluate their progress towards predefined competencies in comparison to traditional medical education models. Faculty members therefore have increased responsibility for providing assessments to residents on a more regular basis, which has associated challenges. Our study explores faculty assessment behaviours within the CBD framework and assesses their openness to opportunities aimed at improving the quality of written feedback. Specifically, we explore faculty's receptiveness to routine metric performance reports that offer comprehensive feedback on their assessment patterns. METHODS Online surveys were distributed to all 28 radiology faculty at Queen's University. Data were collected on demographics, feedback practices, motivations for improving the teacher-learner feedback exchange, and openness to metric performance reports and quality improvement measures. Following descriptive statistics, unpaired t-tests and one-way analysis of variance were conducted to compare groups based on experience and subspecialty. RESULTS The response rate was 89% (25/28 faculty). 56% of faculty were likely to complete evaluations after working with a resident. Regarding the degree to which faculty felt written feedback is important, 62% found it at least moderately important. A majority (67%) believed that performance reports could influence their evaluation approach, with volume of written feedback being the most likely to change. Faculty expressed interest in feedback-focused development opportunities (67%), favouring Grand Rounds and workshops. CONCLUSION Assessment of preceptor perceptions reveals that faculty recognize the importance of offering high-quality written feedback to learners. Faculty openness to quality improvement interventions for curricular reform relies on having sufficient time, knowledge, and skills for effective assessments. This suggests that integrating routine performance metrics into faculty assessments could serve as a catalyst for enhancing future feedback quality.
Collapse
Affiliation(s)
- Laura Wong
- Department of Diagnostic Radiology, Kingston Health Sciences Centre, Kingston, ON, Canada
| | - Ethan Sacoransky
- School of Medicine, Faculty of Health Sciences, Queen’s University, Kingston, ON, Canada
| | - Wilma Hopman
- Kingston General Hospital Research Institute, Kingston Health Sciences Centre, Kingston, ON, Canada
| | - Omar Islam
- Department of Diagnostic Radiology, Kingston Health Sciences Centre, Kingston, ON, Canada
| | - Andrew D. Chung
- Department of Diagnostic Radiology, Kingston Health Sciences Centre, Kingston, ON, Canada
| | - Benjamin Y. M. Kwan
- Department of Diagnostic Radiology, Kingston Health Sciences Centre, Kingston, ON, Canada
| |
Collapse
|
2
|
Braund H, Dagnone JD, Hall AK, Dalgarno N, McEwen L, Schultz KW, Szulewski A. Competency based medical education implementation at the institutional level: A cross-discipline comparative program evaluation. MEDICAL TEACHER 2024:1-8. [PMID: 39101833 DOI: 10.1080/0142159x.2024.2362909] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/06/2024] [Accepted: 05/29/2024] [Indexed: 08/06/2024]
Abstract
INTRODUCTION As an early adopter of competency-based medical education (CBME) our postgraduate institution was uniquely positioned to analyze implementation experience data across programs, while keeping institutional factors constant. We described participants' experiences related to CBME implementation across programs derived from early program evaluation efforts within our setting. METHODS This evaluation focused on eight residency programs at a medium-sized academic institution in Canada. Participants (n = 175) included program leaders, faculty, and residents. The study consisted of 3 phases: (1) describing intended implementation; (2) documenting enacted implementation; and (3) comparing intended with enacted implementation to inform adaptations. Each program's findings were summarized in technical reports which were then analyzed thematically. Cross program data were organized by themes. RESULTS Six themes were identified. All groups emphasized the need for ongoing refinement of CBME resulting from shared tensions such as increased assessment burden. However, there were some disparate CBME-related experiences between programs such as the experience with entrustable professional activities, the interpretation of retrospective entrustment anchors, and quality of feedback. CONCLUSION We detected several cross-program successes and important challenges related to CBME. Our experience can inform other programs engaging in implementation and evaluation of CBME.
Collapse
Affiliation(s)
- Heather Braund
- Department of Biomedical and Molecular Sciences and School of Medicine, Queen's University, Kingston, Ontario, Canada
- Office of Professional Development and Educational Scholarship, Queen's University, Kingston, Ontario, Canada
| | - J Damon Dagnone
- Department of Emergency Medicine, Queen's University, Kingston, Ontario, Canada
| | - Andrew K Hall
- Department of Emergency Medicine, University of Ottawa, Ottawa, Ontario, Canada
- Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada
| | - Nancy Dalgarno
- Office of Professional Development and Educational Scholarship, Queen's University, Kingston, Ontario, Canada
- Department of Biomedical and Molecular Sciences and Master of Health Professions Education, Queen's University, Kingston, Ontario, Canada
| | - Laura McEwen
- Department of Pediatrics, Postgraduate Medical Education, Queen's University, Kingston, Ontario, Canada
| | - Karen W Schultz
- Department of Family Medicine, Queen's University, Kingston, Ontario, Canada
| | - Adam Szulewski
- Departments of Emergency Medicine and Psychology Educational Scholarship Lead, Postgraduate Medical Education, Queen's University, Kingston, Ontario, Canada
| |
Collapse
|
3
|
Kalun P, Braund H, McGuire N, McEwen L, Mann S, Trier J, Schultz K, Curtis R, McGuire A, Pereira I, Dagnone D. Was it all worth it? A graduating resident perspective on CBME. MEDICAL TEACHER 2024:1-9. [PMID: 38742827 DOI: 10.1080/0142159x.2024.2339408] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Accepted: 04/02/2024] [Indexed: 05/16/2024]
Abstract
BACKGROUND Our institution simultaneously transitioned all postgraduate specialty training programs to competency-based medical education (CBME) curricula. We explored experiences of CBME-trained residents graduating from five-year programs to inform the continued evolution of CBME in Canada. METHODS We utilized qualitative description to explore residents' experiences and inform continued CBME improvement. Data were collected from fifteen residents from various specialties through focus groups, interviews, and written responses. The data were analyzed inductively, using conventional content analysis. RESULTS We identified five overarching themes. Three themes provided insight into residents' experiences with CBME, describing discrepancies between the intentions of CBME and how it was enacted, challenges with implementation, and variation in residents' experiences. Two themes - adaptations and recommendations - could inform meaningful refinements for CBME going forward. CONCLUSIONS Residents graduating from CBME training programs offered a balanced perspective, including criticism and recognition of the potential value of CBME when implemented as intended. Their experiences provide a better understanding of residents' needs within CBME curricula, including greater balance and flexibility within programs of assessment and curricula. Many challenges that residents faced with CBME could be alleviated by greater accountability at program, institutional, and national levels. We conclude with actionable recommendations for addressing residents' needs in CBME.
Collapse
Affiliation(s)
- Portia Kalun
- Queen's Health Sciences, Queen's University, Kingston, Canada
| | - Heather Braund
- Queen's Health Sciences, Queen's University, Kingston, Canada
| | - Natalie McGuire
- Queen's Health Sciences, Queen's University, Kingston, Canada
| | - Laura McEwen
- Queen's Health Sciences, Queen's University, Kingston, Canada
| | - Steve Mann
- Queen's Health Sciences, Queen's University, Kingston, Canada
| | - Jessica Trier
- Queen's Health Sciences, Queen's University, Kingston, Canada
- Department of Physical Medicine and Rehabilitation, Queen's University, Kingston, Canada
- Providence Care Hospital, Kingston, Canada
| | - Karen Schultz
- Queen's Health Sciences, Queen's University, Kingston, Canada
| | - Rachel Curtis
- Queen's Health Sciences, Queen's University, Kingston, Canada
- Department of Ophthalmology, Queen's University, Kingston, Canada
| | - Andrew McGuire
- Queen's Health Sciences, Queen's University, Kingston, Canada
| | - Ian Pereira
- Queen's Health Sciences, Queen's University, Kingston, Canada
| | - Damon Dagnone
- Queen's Health Sciences, Queen's University, Kingston, Canada
- Department of Emergency Medicine, Queen's University, Kingston, Canada
| |
Collapse
|
4
|
Miller F, Wood S, Livingston P. The lived experience of Competence by Design: Canadian resident physicians' perspectives. Can J Anaesth 2024; 71:254-263. [PMID: 38133715 DOI: 10.1007/s12630-023-02678-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Revised: 06/12/2023] [Accepted: 07/21/2023] [Indexed: 12/23/2023] Open
Abstract
PURPOSE Canadian specialist residency programs are in the process of transitioning to a hybrid time and competence model, Competence by Design (CBD), developed by the Royal College of Physicians and Surgeons Canada. Although there is extensive literature around competency-based medical education (CBME), few studies have evaluated the experience of residents after CBME implementation. The purpose of this study was to obtain a rich perspective on the lived experience of residents. METHODS We designed a qualitative study with inductive thematic analysis of semistructured interview data. The study population was residents in CBD postgraduate training programs in anesthesiology, internal medicine, or surgery (including all surgical subspecialties) at Dalhousie University (Halifax, NS, Canada). RESULTS Residents identified the following benefits of their programs and CBD: supportive peers and clinical supervisors, a roadmap for residency, formalized feedback opportunities, and program evolution. Resident-identified drawbacks of CBD included: a lack of transparency around CBD, CBD not as advertised, a lack of buy-in, increased administrative burden, difficulties obtaining evidence for entrustable professional activities (EPAs); the onus for CBD on residents, inconsistent feedback, cumbersome technology, and significant psychological burden. Resident-suggested improvements were reducing the number of EPAs, streamlining EPA requirements, increasing transparency and communication with competence committees, providing incentives and continuous education for clinical supervisors, improving on existing electronic interfaces, and developing technology better suited to the needs of CBD. CONCLUSION This study highlights that the significant administrative and psychological burden of CBD detracts from clinical learning and enthusiasm for residency. Future research could explore whether overcoming the identified challenges will improve residents' experiences.
Collapse
Affiliation(s)
- Franziska Miller
- Department of Anesthesia, Pain Management and Perioperative Medicine, Dalhousie University, 1276 South Park Street, Halifax, NS, B3H 2Y9, Canada.
| | - Sarah Wood
- School of Medicine, University of Dundee, Dundee, Scotland, UK
| | - Patricia Livingston
- Department of Anesthesia, Pain Management & Perioperative Medicine, Dalhousie University, Halifax, NS, Canada
| |
Collapse
|
5
|
Szulewski A, Braund H, Dagnone DJ, McEwen L, Dalgarno N, Schultz KW, Hall AK. The Assessment Burden in Competency-Based Medical Education: How Programs Are Adapting. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2023; 98:1261-1267. [PMID: 37343164 DOI: 10.1097/acm.0000000000005305] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/23/2023]
Abstract
Residents and faculty have described a burden of assessment related to the implementation of competency-based medical education (CBME), which may undermine its benefits. Although this concerning signal has been identified, little has been done to identify adaptations to address this problem. Grounded in an analysis of an early Canadian pan-institutional CBME adopter's experience, this article describes postgraduate programs' adaptations related to the challenges of assessment in CBME. From June 2019-September 2022, 8 residency programs underwent a standardized Rapid Evaluation guided by the Core Components Framework (CCF). Sixty interviews and 18 focus groups were held with invested partners. Transcripts were analyzed abductively using CCF, and ideal implementation was compared with enacted implementation. These findings were then shared back with program leaders, adaptations were subsequently developed, and technical reports were generated for each program. Researchers reviewed the technical reports to identify themes related to the burden of assessment with a subsequent focus on identifying adaptations across programs. Three themes were identified: (1) disparate mental models of assessment processes in CBME, (2) challenges in workplace-based assessment processes, and (3) challenges in performance review and decision making. Theme 1 included entrustment interpretation and lack of shared mindset for performance standards. Adaptations included revising entrustment scales, faculty development, and formalizing resident membership. Theme 2 involved direct observation, timeliness of assessment completion, and feedback quality. Adaptations included alternative assessment strategies beyond entrustable professional activity forms and proactive assessment planning. Theme 3 related to resident data monitoring and competence committee decision making. Adaptations included adding resident representatives to the competence committee and assessment platform enhancements. These adaptations represent responses to the concerning signal of significant burden of assessment within CBME being experienced broadly. The authors hope other programs may learn from their institution's experience and navigate the CBME-related assessment burden their invested partners may be facing.
Collapse
Affiliation(s)
- Adam Szulewski
- A. Szulewski is associate professor, Departments of Emergency Medicine and Psychology, and educational scholarship lead, Postgraduate Medical Education, Queen's University, Kingston, Ontario, Canada; ORCID: https://orcid.org/0000-0002-3076-6221
| | - Heather Braund
- H. Braund is associate director of scholarship and simulation education, Office of Professional Development and Educational Scholarship, and assistant (adjunct) professor, Department of Biomedical and Molecular Sciences and School of Medicine, Queen's University, Kingston, Ontario, Canada; ORCID: https://orcid.org/0000-0002-9749-7193
| | - Damon J Dagnone
- D.J. Dagnone is associate professor, Department of Emergency Medicine, Queen's University, Kingston, Ontario, Canada; ORCID: https://orcid.org/0000-0001-6963-7948
| | - Laura McEwen
- L. McEwen is director of assessment and evaluation of postgraduate medical education and assistant professor, Department of Pediatrics, Postgraduate Medical Education, Queen's University, Kingston, Ontario, Canada; ORCID: https://orcid.org/0000-0003-2457-5311
| | - Nancy Dalgarno
- N. Dalgarno is director of education scholarship, Office of Professional Development and Educational Scholarship, and assistant professor (adjunct), Department of Biomedical and Molecular Sciences and Master of Health Professions Education, Queen's University, Kingston, Ontario, Canada; ORCID: https://orcid.org/0000-0001-7932-9949
| | - Karen W Schultz
- K.W. Schultz is professor, Department of Family Medicine, and associate dean of postgraduate medical education, Queen's University, Kingston, Ontario, Canada; ORCID: https://orcid.org/0000-0003-0208-3981
| | - Andrew K Hall
- A.K. Hall is associate professor and vice chair of education, Department of Emergency Medicine, University of Ottawa, and clinician educator, Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada; ORCID: https://orcid.org/0000-0003-1227-5397
| |
Collapse
|
6
|
Kagoma YK, Dobranowski J, Patlas M. Competency-Based Medical Education in Radiology - Planning for the Immediate and Long-Term Future of Radiology Education. Can Assoc Radiol J 2023:8465371231157252. [PMID: 36786330 DOI: 10.1177/08465371231157252] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/15/2023] Open
Affiliation(s)
- Yoan K Kagoma
- Department of Radiology, 3710McMaster University, Hamilton, ON, Canada
| | | | - Michael Patlas
- Department of Radiology, 3710McMaster University, Hamilton, ON, Canada
| |
Collapse
|
7
|
Lin C, Nguyen TK, Sanatani M. Exploring the Inner Lens: Examining Oncology Consultants' Use of Competency and Assessment Frameworks in Clinical Teaching Settings. JOURNAL OF CANCER EDUCATION : THE OFFICIAL JOURNAL OF THE AMERICAN ASSOCIATION FOR CANCER EDUCATION 2022:10.1007/s13187-022-02241-0. [PMID: 36422764 DOI: 10.1007/s13187-022-02241-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 11/15/2022] [Indexed: 06/16/2023]
Abstract
Medical education in oncology has adopted a competency-based medical education (CBME) approach in Canada. Oncologist competencies are arranged within assessment frameworks, but it is unknown whether oncologists routinely use assessment frameworks when determining what to teach oncology trainees. Understanding oncologists' actual approaches to assessment and teaching as a form of enacted CBME could inform and focus faculty development efforts. Using a participatory action research approach, oncology faculty were interviewed regarding their teaching practices and use of assessment frameworks. Faculty suggestions regarding challenges in setting teaching goals and completing assessment within the CBME paradigm were solicited. Furthermore, teaching objectives developed by faculty in-house were assessed regarding the domains of competence targeted. Thirty-one of the 40 faculty members in the Department of Oncology, Western University, agreed to participate. Trainee factors (training level, areas of interest), faculty personal values and experience, the teaching setting, and some components of frameworks (medical expert, communicator, advocate, and professional) were articulated as determinants of teaching goals. Of the 1117 in-house objectives reviewed, 76% focused on the medical role. Faculty held widely disparate views on assessment framework features which would enable teaching and assessment. Oncology faculty members used a multifaceted approach to determine what to teach trainees. This approach contains elements both of the conventional apprenticeship models under which many faculty members themselves were trained and of the CBME paradigm. Future research should focus on exploring ways to resolve the tension between established individual teaching approaches and nationally standardized competency frameworks which are being implemented.
Collapse
Affiliation(s)
- Cindy Lin
- Schulich School of Medicine & Dentistry, Western University, London, ON, Canada
| | - Timothy K Nguyen
- Department of Oncology, Division of Radiation Oncology, Schulich School of Medicine & Dentistry, Western University, London, ON, Canada
| | - Michael Sanatani
- Department of Oncology, Division of Medical Oncology, Schulich School of Medicine & Dentistry, Western University, London, ON, Canada.
| |
Collapse
|