An innovative approach to identifying learning needs for intrinsic CanMEDS roles in continuing professional development

The Ottawa resident observation form for nurses (O-RON): evaluation of an assessment tool's psychometric properties in different specialties

BACKGROUND

Workplace-based assessment (WBA) used in post-graduate medical education relies on physician supervisors' feedback. However, in a training environment where supervisors are unavailable to assess certain aspects of a resident's performance, nurses are well-positioned to do so. The Ottawa Resident Observation Form for Nurses (O-RON) was developed to capture nurses' assessment of trainee performance and results have demonstrated strong evidence for validity in Orthopedic Surgery. However, different clinical settings may impact a tool's performance. This project studied the use of the O-RON in three different specialties at the University of Ottawa.

METHODS

O-RON forms were distributed on Internal Medicine, General Surgery, and Obstetrical wards at the University of Ottawa over nine months. Validity evidence related to quantitative data was collected. Exit interviews with nurse managers were performed and content was thematically analyzed.

RESULTS

179 O-RONs were completed on 30 residents. With four forms per resident, the ORON's reliability was 0.82. Global judgement response and frequency of concerns was correlated (r = 0.627, P < 0.001).

CONCLUSIONS

Consistent with the original study, the findings demonstrated strong evidence for validity. However, the number of forms collected was less than expected. Exit interviews identified factors impacting form completion, which included clinical workloads and interprofessional dynamics.

Competency Framework for Podiatric Medicine Training: A Validation Report Based on an Adapted E-Delphi Across Canada

OBJECTIVES

Competency-based medical education has been introduced into many health professional curricula. Based on CanMEDs, a framework has recently been developed for podiatric medicine education in Canada. This study aimed to validate the framework through a consensus of various podiatric medicine experts across Canada.

METHODS

An adapted Delphi method was used for content validation. Two structured online questionnaires were used to gather expert opinions and agreement on the roles and core competencies described in the framework previously developed. The validation consensus threshold was set at a minimum of 80% agreement. The summary of comments and suggestions was used to reformulate certain items after the research team reached a consensus.

RESULTS

Out of the 51 experts invited from the Canadian Podiatric Medicine Association, 19 completed the first-round questionnaire (mean podiatric experience = 24.5 years; standard deviation 17.6). After the first round, "Podiatric Expert", "Communicator" and "Scholar" roles have been modified. After these modifications, a consensus was obtained at the second round completed by 13 experts. Overall, 95% of the experts agreed that the competency framework was relevant even if some indicators would need to be adapted to suit the requirements of each province and territory.

CONCLUSION

This validated framework supports the excellence and the quality of our podiatric educational program. It also promotes the adoption of a uniform education of podiatrists in Canada and worldwide.

French-speaking Swiss physician's perceptions and perspectives regarding their competencies and training need in leadership and management: a mixed-methods study

INTRODUCTION

Effective leadership and management (L&M) are essential to the success of health care organizations. Young medical leaders often find themselves ill-prepared to take on these new responsibilities, but rarely attend training in L&M skills. The aims of this study were to evaluate physician's self-perceived competencies and training needs for L&M, to identify available regional L&M training, and to highlight opportunities, challenges and threats regarding physicians' training in medical L&M in the French-speaking part of Switzerland.

METHODS

We conducted a mixed methods study in three steps: (1) a survey on perceived L&M competencies and training needs (5 dimensions) to all physicians of a Swiss University Hospital (N = 2247); (2) a mapping of the Swiss French speaking L&M training programs through analysis of hospital websites and interviews; and (3) semi-structured interviews with L&M program coordinators about the programs' strengths and weaknesses as well as the opportunities and challenges to include physicians in such training. We used analysis of variance to compare differences in perceived competences between physicians of different hierarchical status and used Cramer's V to measure the association's degree between physicians' training needs and prior training in L&M and hierarchical status. We analysed semi-structured interviews using thematic analysis.

RESULTS

Five-hundred thirty-two physicians responded (24%). Physicians perceived themselves as rather competent in most leadership dimensions. More experienced physicians reported a higher sense of competence in all dimensions of leadership (e.g. Working with others: F = 15.55, p < .001; Managing services: F = 46.89, p < .001). Three competencies did not vary according to the hierarchical status: emotional intelligence (F = 1.56, p = .20), time management (F = 0.47, p = .70) and communicating (F = 1.97, p = .12). There was a weak to moderate association between the responders' self-perceived needs for training and their hierarchal status for all competencies (Cramer's V ∈ [0.16;0.35]). Physicians expressed a strong desire to seek out training for all competencies, especially for knowing one's leadership style (82%), managing teams (83%), and managing conflict (85%). Although existing local L&M training programs covered most relevant topics, only a forth of responders had attended any type of training. L&M program coordinators identified several facilitators and barriers to physician attendance on institutional (matching reality and training), relational (managing collective intelligence), and individual levels (beliefs and self-perceived identity).

CONCLUSIONS

French-speaking Swiss hospital physicians clearly express training needs for L&M skills although they only rarely attend such training programs. Reasons for non-attendance to such programs should be explored in order to understand physicians' low participation rates in these trainings.

Evaluation of a tool to improve the quality of preceptor written feedback for family medicine residents: training and use of a CanMEDS-MF competency-based criterion guide

Background

Written feedback is essential in resident teaching, but preceptors are not always well equipped to provide relevant feedback. The purpose of this study was to evaluate the effectiveness of multi-episodic training and the use of a criterion-referenced guide for written feedback for family medicine preceptors in a French-language academic hospital.

Method

Twenty-three (23) preceptors participated in the training and used the criterion-referenced guide to guide them during the written evaluation in an evaluation sheet named "Field Notes." The content of these Field Notes was analyzed according to completion, the rate of specific feedback, and the rate of feedback by CanMEDS-MF role before and after the training over a three-month period.

Results

Based on the analysis of the Field Notes (n = 70 pre-test; n = 138 post-test), an increase in the percentage of completion (50% vs. 92%, z = 2.97, p = 0.0030) and specific feedback (59% vs. 92%, z = 2.47, p=0.0137) was noted. There was no significant increase in feedback by CanMEDS-MF role.

Conclusions

The development of multi-episodic training and a criterion-referenced guide, created according to the CanMEDS-MF repository, suggests an improvement in comprehensive and specific written feedback in family medicine education.

Exploring Content Relationships Among Components of a Multisource Feedback Program

INTRODUCTION

A new multisource feedback (MSF) program was specifically designed to support physician quality improvement (QI) around the CanMEDS roles of Collaborator , Communicator , and Professional . Quantitative ratings and qualitative comments are collected from a sample of physician colleagues, co-workers (C), and patients (PT). These data are supplemented with self-ratings and given back to physicians in individualized reports. Each physician reviews the report with a trained feedback facilitator and creates one-to-three action plans for QI. This study explores how the content of the four aforementioned multisource feedback program components supports the elicitation and translation of feedback into a QI plan for change.

METHODS

Data included survey items, rater comments, a portion of facilitator reports, and action plans components for 159 physicians. Word frequency queries were used to identify common words and explore relationships among data sources.

RESULTS

Overlap between high frequency words in surveys and rater comments was substantial. The language used to describe goals in physician action plans was highly related to respondent comments, but less so to survey items. High frequency words in facilitator reports related heavily to action plan content.

DISCUSSION

All components of the program relate to one another indicating that each plays a part in the process. Patterns of overlap suggest unique functions conducted by program components. This demonstration of coherence across components of this program is one piece of evidence that supports the program's validity.

Medical Council of Canada Qualifying Examinations and performance in future practice

The purpose of medical licensing examinations is to protect the public from practitioners who do not have adequate knowledge, skills, and abilities to provide acceptable patient care, and therefore evaluating the validity of these examinations is a matter of accountability. Our objective was to discuss the Medical Council of Canada's Qualifying Examinations (MCCQEs) Part I (QE1) and Part II (QE2) in terms of how well they reflect future performance in practice. We examined the supposition that satisfactory performance on the MCCQEs are important determinants of practice performance and, ultimately, patient outcomes. We examined the literature before the implementation of the QE2 (pre-1992), post QE2 but prior to the implementation of the new Blueprint (1992-2018), and post Blueprint (2018-present). The literature suggests that MCCQE performance is predictive of future physician behaviours, that the relationship between examination performance and outcomes did not attenuate with practice experience, and that associations between examination performance and outcomes made sense clinically. While the evidence suggests the MCC qualifying examinations measure the intended constructs and are predictive of future performance, the validity argument is never complete. As new competency requirements emerge, we will need to develop valid and reliable mechanisms for determining practice readiness in these areas.

Intrinsic or Invisible? An Audit of CanMEDS Roles in Entrustable Professional Activities

PURPOSE

Postgraduate medical education in Canada has quickly transformed to a competency-based model featuring new entrustable professional activities (EPAs) and associated milestones. It remains unclear, however, how these milestones are distributed between the central medical expert role and 6 intrinsic roles of the larger CanMEDS competency framework. A document review was thus conducted to measure how many EPA milestones are classified under each CanMEDS role, focusing on the overall balance between representation of intrinsic roles and that of medical expert.

METHOD

Data were extracted from the EPA guides of 40 Canadian specialties in 2021 to measure the percentage of milestones formally linked to each role. Subsequent analyses explored for differences when milestones were separated by stage of postgraduate training, weighted by an EPA's minimum number of observations, or sorted by surgical and medical specialties.

RESULTS

Approximately half of all EPA milestones (mean = 48.6%; 95% confidence interval [CI] = 45.9, 51.3) were classified under intrinsic roles overall. However, representation of the health advocate role was consistently low (mean = 2.95%; 95% CI = 2.49, 3.41), and some intrinsic roles-mainly leader, scholar, and professional-were more heavily concentrated in the final stage of postgraduate training. These findings held true under all conditions examined.

CONCLUSIONS

The observed distribution of roles in EPA milestones fits with high-level descriptions of CanMEDS in that intrinsic roles are viewed as inextricably linked to medical expertise, implying both are equally important to cultivate through curricula. Yet a fine-grained analysis suggests that a low prevalence or late emphasis of some intrinsic roles may hinder how they are taught or assessed. Future work must explore whether the quantity or timing of milestones shapes the perceived value of each role, and other factors determining the optimal distribution of roles throughout training.

Unobserved Observers: Nurses' Perspectives About Sharing Feedback on the Performance of Resident Physicians

PURPOSE

Postgraduate training programs are incorporating feedback from registered nurses (RNs) to facilitate holistic assessments of resident performance. RNs are a potentially rich source of feedback because they often observe trainees during clinical encounters when physician supervisors are not present. However, RN perspectives about sharing feedback have not been deeply explored. This study investigated RN perspectives about providing feedback and explored the facilitators and barriers influencing their engagement.

METHOD

Constructivist grounded theory methodology was used in interviewing 11 emergency medicine and 8 internal medicine RNs at 2 campuses of a tertiary care academic medical center in Ontario, Canada, between July 2019 and March 2020. Interviews explored RN experiences working with and observing residents in clinical practice. Data collection and analysis were conducted iteratively. Themes were identified using constant comparative analysis.

RESULTS

RNs felt they could observe authentic day-to-day behaviors of residents often unwitnessed by supervising physicians and offer unique feedback related to patient advocacy, communication, leadership, collaboration, and professionalism. Despite a strong desire to contribute to resident education, RNs were apprehensive about sharing feedback and reported barriers related to hierarchy, power differentials, and a fear of overstepping professional boundaries. Although infrequent, a key stimulus that enabled RNs to feel safe in sharing feedback was an invitation from the supervising physician to provide input.

CONCLUSIONS

Perceived hierarchy in academic medicine is a critical barrier to engaging RNs in feedback for residents. Accessing RN feedback on authentic resident behaviors requires dismantling the negative effects of hierarchy and fostering a collaborative interprofessional working environment. A critical step toward this goal may require supervising physicians to model feedback-seeking behavior by inviting RNs to share feedback. Until a workplace culture is established that validates nurses' input and creates safe opportunities for them to contribute to resident education, the voices of nurses will remain unheard.

Beyond the CLAIM: A comprehensive needs assessment strategy for creating an Advanced Medical Education Research Training Program (ARMED-MedEd)

BACKGROUND

The health professions education (HPE) landscape has shifted substantively with increasing professionalization of research and scholarship. Clinician educators seeking to become competitive in this domain often pursue fellowships and master's degrees in HPE, but there are few resources for the continuing professional development (CPD) of those who seek to continue developing their scholarly practice within HPE. Acknowledging the multiple players in this landscape, the authors sought to design a new "beyond beginners" HPE research program using a novel needs assessment planning process.

METHODS

The authors developed and conducted a new three-phase, five-step process that sets forth a programmatic approach to conducting a needs assessment for a CPD course in HPE research. The five steps of the CLAIM method are: Competitive analysis, Literature review with thematic analysis, Ask stakeholders, Internal review by experts, and Mapping of a curriculum. These steps are organized into three phases (Discovery, Convergence, and Synthesis).

RESULTS

Over a 12-month period, the authors completed a comprehensive needs assessment. The CLAIM process revealed that longitudinal digital connection, diverse and in depth exposure to HPE research methods, skills around scholarly publishing, and leadership and management of research would be beneficial to our design.

CONCLUSIONS

The CLAIM method provided scaffolding to help the authors create a robust curriculum that adopts a scholarly approach for developing a HPE research course. This needs assessment methodology may be useful in other CPD contexts.

An Innovative Needs Assessment Approach to Develop Relevant Continuing Professional Development for Psychiatrists

OBJECTIVES

Gaps in psychiatrists' competence can interfere with the delivery of optimal patient care, particularly when these gaps have not been identified. This study aimed to assess the perceived and unperceived continuing professional development needs of psychiatrists practicing in Quebec, Canada.

METHODS

The authors sent an online cross-sectional survey (2018) to members of the 'Association des médecins psychiatres du Québec' and collected data on unperceived needs via the critical incident method (focused on managing challenging clinical cases/situations). Data were analyzed using descriptive statistics, chi-squared tests, thematic analysis, and triangulation of data. Two coders independently analyzed qualitative data.

RESULTS

Of 1150 eligible psychiatrists, 187 (16%) completed the survey. Over half were female (58%), caring for adult patients (60%), and practiced in a university hospital (49%). Top perceived and unperceived learning need areas were neurodevelopmental disorders and psychopharmacology. Three hundred forty-three factors influencing the management of reported challenges were classified as case complexity (53%), patient (22%), environment (19%), and lack of knowledge (4%). Consultation with colleagues (49%) was the most frequently accessed resource for approaching challenging cases. During the previous year and across both self-directed and group learning activities, respondents reported engaging more frequently in in-person than online activities.

CONCLUSIONS

A comprehensive needs assessment integrating perceived and unperceived needs is the cornerstone for planning relevant continuing professional development. The critical incident method is a useful tool to assess psychiatrists' unperceived needs. Critical reflection after solving complex clinical cases might provide an opportunity to optimize psychiatrists' selection of relevant continuing professional development.

Attracting and retaining physicians in less attractive specialties: the role of continuing medical education

BACKGROUND

Less attractive specialties in medicine are struggling to recruit and retain physicians. When properly organized and delivered, continuing medical education (CME) activities that include short courses, coaching in the workplace, and communities of practice might offer a solution to this problem. This position paper discusses how educationalists can create CME activities based on the self-determination theory that increase physicians' intrinsic motivation to work in these specialties.

MAIN CONTENT

The authors propose a set of guidelines for the design of CME activities that offer physicians meaningful training experiences within the limits of the available resources and support. First, to increase physicians' sense of professional relatedness, educationalists must conduct a learner needs assessment, evaluate CME's long-term outcomes in work-based settings, create social learning networks, and involve stakeholders in every step of the CME design and implementation process. Moreover, providing accessible, practical training formats and giving informative performance feedback that authentically connects to learners' working life situation increases physicians' competence and autonomy, so that they can confidently and independently manage the situations in their practice contexts. For each guideline, application methods and instruments are proposed, making use of relevant literature and connecting to the self-determination theory.

CONCLUSIONS

By reducing feelings of professional isolation and reinforcing feelings of competence and autonomy in physicians, CME activities show promise as a strategy to recruit and retain physicians in less attractive specialties.

Analysis of Supervisors' Feedback to Residents on Communicator, Collaborator, and Professional Roles During Case Discussions

BACKGROUND

Literature examining the feedback supervisors give to residents during case discussions in the realms of communication, collaboration, and professional roles (intrinsic roles) focuses on analyses of written feedback and self-reporting.

OBJECTIVES

We quantified how much of the supervisors' verbal feedback time targeted residents' intrinsic roles and how well feedback time was aligned with the role targeted by each case. We analyzed the educational goals of this feedback. We assessed whether feedback content differed depending on whether the residents implied or explicitly expressed a need for particular feedback.

METHODS

This was a mixed-methods study conducted from 2017 to 2019. We created scripted cases for radiology and internal medicine residents to present to supervisors, then analyzed the feedback given both qualitatively and quantitatively. The cases were designed to highlight the CanMEDS intrinsic roles of communicator, collaborator, and professional.

RESULTS

Radiologists (n = 15) spent 22% of case discussions providing feedback on intrinsic roles (48% aligned): 28% when the case targeted the communicator role, 14% for collaborator, and 27% for professional. Internists (n = 15) spent 70% of discussions on intrinsic roles (56% aligned): 66% for communicator, 73% for collaborator, and 72% for professional. Radiologists' goals were to offer advice (66%), reflections (21%), and agreements (7%). Internists offered advice (41%), reflections (40%), and clarifying questions (10%). We saw no consistent effects when residents explicitly requested feedback on an intrinsic role.

CONCLUSIONS

Case discussions represent frequent opportunities for substantial feedback on intrinsic roles, largely aligned with the clinical case. Supervisors predominantly offered monologues of advice and agreements.

Evaluation of a professional development course on research methods for healthcare professionals

Healthcare is constantly evolving and thus requires lifelong learning. Evidence-based learning has been shown to lead to better patient outcomes, yet many healthcare professionals report gaps in their research abilities. We sought to evaluate the efficacy of a professional development program in addressing identified gaps.

"You are in Some Sort of Occupational Adolescence": An Exploratory Study to Understand Knowledge, Skills, and Experiences of CanMEDS Leader Role in Psychiatry

OBJECTIVE

The study explored knowledge, attitude, and practices of Canadian Medical Education Directions for Specialists (CanMEDS) "'Leader" role in faculty psychiatrists in a university setting.

METHODS

This is a qualitative study using a thematic analysis approach. Participants were identified by purposive, convenience, and snowball sampling. Telephone interviews were conducted to explore participants' perceptions of leadership. Qualitative analysis was carried out using a constant comparative analysis approach to identify themes across the interview data. Data was transcribed and coded into themes and categories to form an analysis of physicians' knowledge, attitude, and practices of the CanMEDS "Leader" role.

RESULTS

Twenty-eight faculty psychiatrists participated in the study. The following themes and subthemes emerged from analysis: (a) characteristics of leadership and lack of clarity regarding CanMEDS "Leader" role (subthemes: differences between manager and leader, lack of specificity and practicality, relevance to practice) and (b) the meaning of leadership (subthemes: "positional leadership" and "everyday leadership," physician as team leader, developing leadership skills, getting beyond "occupational adolescence").

CONCLUSION

Participants perceived CanMEDS "Leader" role description as a high-level vision, which needs practice-oriented guidance. Participants conceptualized the dual nature of physician leadership at an individual level and at an organizational level. Leadership training is important both in residency as well as lifelong learning.

How do Lebanese patients perceive the ideal doctor based on the CanMEDS competency framework?

BACKGROUND

During their training, Lebanese medical students develop a high medical expertise but are not focusing on other competencies such as communication, collaboration, erudition, professionalism, leadership and health promotion. There is also insufficient data about patients' preference for these skills. This study describes the different weights patients attribute to these physician's competencies.

METHODS

This is a cross-sectional study based on a questionnaire distributed to 133 Lebanese patients. It included 15 questions assessing how patients prioritize the physician's competencies, with open-ended questions asking them to define "the good doctor". Krippendorff's alpha coefficient was used to analyze the reliability of the competencies' classification.

RESULTS

One hundred twenty five patients completed the questionnaire in this cross-sectional study. Their mean age was 48 ± 16.76 years. When classifying competencies, 73.6% opted for medical expertise as first choice and 48% put communication as second. Based on the Krippendorff's coefficient, we identified a moderate agreement for the seven choices (alpha = 0.44). In open-ended questions, patients defined the good doctor in 325 answers: 64.3% mentioned medical expertise, 34.1% high ethics and 26.2% communication.

CONCLUSIONS

This patient-centered study concurs well with the worldwide practice that puts medical expertise at the center of medical education. However Lebanese patients don't perceive equally other competencies and favor professionalism and communication that should be integrated in priority in students' curricula.

Promoting medical competencies through a didactic tutor qualification programme - a qualitative study based on the CanMEDS Physician Competency Framework

BACKGROUND

In peer-led tutorial courses, qualified medical students ("tutors") provide their peers with opportunities to deepen their theoretical knowledge effectively and to practice clinical skills already in preclinical semesters. At the Medical Faculty of Heidelberg University, a structured medical didactic qualification programme prepares and trains future tutors for their responsibilities. This programme consists of four modules: 1. medical didactics and group leadership, 2. subject-specific training, 3. performance of tutorial courses as well as 4. collegial advice and reflection on the tutors' activities. The aim of this study is to systematically analyse and present the development of role competencies for medical tutors based on the CanMEDS Physician Competency Framework through the didactic qualification programme.

METHODS

We applied a qualitative research approach to detect CanMEDS role competencies acquisition within the tutor qualification programme. The CanMEDS framework describes key competencies, grouped thematically under seven professional roles. Two tutors and three training coordinators independently assigned the individual modules of the tutor qualification programme to the key competencies of the CanMEDS framework. Tutors and training coordinators compared and discussed the allocations within the groups in a consensus finding process. All authors analysed the findings in order to find out the so-called "hidden curriculum". The views of both groups are presented separately.

RESULTS

The training programme promotes the acquisition of competencies in all seven CanMEDS roles. The roles of the scholar and the leader are promoted in all modules. In addition, the first and fourth module focus predominately on the role of the collaborator, the second on the role of the medical expert and communicator, and the fourth on the role of the professional.

CONCLUSIONS

The systematic analysis through assignment of the CanMEDS roles to the individual modules of the tutor qualification programme documents the comprehensive acquisition of competencies, not only with regard to the tutor activity, but generally with regard to the later role of the physician. The reflection on one's own competency acquisition can support the promotion of corresponding competencies in the qualification programme and their transfer into the professional practice later.