Assessment of CanMEDS roles in postgraduate training: the validation of the Compass

Unprofessional behaviour of GP residents leading to a dismissal dispute: characteristics and outcomes of those who appeal

BACKGROUND

Recognition of poor performance in General Practice trainees is important because underperformance compromises patients' health and safety. However, in General Practice, research on persistent underperformance while in training and its ultimate consequences is almost completely lacking. We aim to explore the unprofessional behaviours of residents in General Practice who were dismissed from training and who litigated against dismissal.

METHODS

We performed a structured analysis using open-source data from all General Practice cases before the Conciliation Board of the Royal Dutch Medical Association between 2011 and 2020. Anonymised law cases about residents from all Dutch GP training programmes were analysed in terms of the quantitative and qualitative aspects related to performance.

RESULTS

Between 2011 and 2020, 24 residents who were dismissed from training challenged their programme director's decision. Dismissed residents performed poorly in several competencies, including communication, medical expertise and most prominently, professionalism. Over 90% of dismissed residents failed on professionalism. Most lacked self-awareness and/or failed to profit from feedback. Approximately 80% failed on communication, and about 60% on medical expertise as well. A large majority (more than 80%) of dismissed residents had previously participated in some form of remediation.

CONCLUSIONS

Deficiencies in both professionalism and communication were the most prevalent findings among the dismissed General Practice residents. These two deficiencies overlapped considerably. Dismissed residents who challenged their programme director's decision were considered to lack self-awareness, which requires introspection and the appreciation of feedback from others.

The CanMEDS Competency Framework in laboratory medicine: a phenomenographic study exploring how professional roles are applied outside the clinical environment

Background

The CanMEDS Competency Framework is an internationally recognized model used to outline the proficiencies of a physician. It has predominantly been studied in clinical environments but not all medical specialties take part in direct patient contact. In laboratory medicine, the role of the physician is to promote and enhance patient diagnostics by managing and overseeing the functions of a diagnostic laboratory.

Methods

This phenomenographic study explores the lived experiences of biochemistry, microbiology, and pathology residency program directors to better understand how they utilize the CanMEDS competencies. Eight laboratory medicine program directors from across Canada were individually interviewed using a semi-structured interview, and the data was analysed using inductive thematic analysis.

Results

The findings show that the current framework is disconnected from the unique context of laboratory medicine with some competencies appearing unrelatable using the current standardized definitions and expectations. Nevertheless, participants considered the framework to be an appropriate blueprint of the competencies necessary for their professional environment, but to make it accessible more autonomy is required to adapt the framework to their needs.

Conclusion

Newer renditions of the CanMEDS Competency Framework should better consider the realities of non-clinical disciplines.

Development of individual competencies and team performance in interprofessional ward rounds: results of a study with multimodal observations at the Heidelberg Interprofessional Training Ward

Introduction

Interprofessional training wards (IPTW) aim to improve undergraduates' interprofessional collaborative practice of care. Little is known about the effects of the different team tasks on IPTW as measured by external assessment. In Heidelberg, Germany, four nursing and four medical undergraduates (= one cohort) care for up to six patients undergoing general surgery during a four-week placement. They learn both professionally and interprofessionally, working largely on their own responsibility under the supervision of the medical and nursing learning facilitators. Interprofessional ward rounds are a central component of developing individual competencies and team performance. The aim of this study was to evaluate individual competencies and team performance shown in ward rounds.

Methods

Observations took place in four cohorts of four nursing and four medical undergraduates each. Undergraduates in one cohort were divided into two teams, which rotated in morning and afternoon shifts. Team 1 was on morning shift during the first (t0) and third (t1) weeks of the IPTW placement, and Team 2 was on morning shift during the second (t0) and fourth (t1) weeks. Within each team, a tandem of one nursing and one medical undergraduate cared for a patient room with three patients. Ward round observations took place with each team and tandem at t0 and t1 using the IP-VITA instrument for individual competencies (16 items) and team performance (11 items). Four hypotheses were formulated for statistical testing with linear mixed models and correlations.

Results

A total of 16 nursing and medical undergraduates each were included. There were significant changes in mean values between t0 and t1 in individual competencies (Hypothesis 1). They were statistically significant for all three sum scores: "Roles and Responsibilities", Patient-Centeredness", and "Leadership". In terms of team performance (Hypothesis 2), there was a statistically significant change in mean values in the sum score "Roles and Responsibilities" and positive trends in the sum scores "Patient-Centeredness" and "Decision-Making/Collaborative Clinical Reasoning". Analysis of differences in the development of individual competencies in the groups of nursing and medical undergraduates (Hypothesis 3) showed more significant differences in the mean values of the two groups in t0 than in t1. There were significant correlations between individual competencies and team performance at both t0 and t1 (Hypothesis 4).

Discussion

The study has limitations due to the small sample and some sources of bias related to the external assessment by means of observation. Nevertheless, this study offers insights into interprofessional tasks on the IPTW from an external assessment. Results from quantitative and qualitative analysis of learners self-assessment are confirmed in terms of roles and responsibilities and patient-centeredness. It has been observed that medical undergraduates acquired and applied skills in collaborative clinic reasoning and decision-making, whereas nursing undergraduates acquired leadership skills. Within the study sample, only a small group of tandems remained constant over time. In team performance, the group of constant tandems tended to perform better than the group of random tandems. The aim of IPTW should be to prepare healthcare team members for the challenge of changing teams. Therefore, implications for IPTW implementation could be to develop learning support approaches that allow medical and nursing undergraduates to bring interprofessional competencies to team performance, independent of the tandem partner or team.

Core competencies of healthcare professionals in Oman: Research and evidence-based practice needs attention

AIM

The aim of the study was to examine (1) the perceptions on core competencies of healthcare professionals working at clinical settings in Oman and (2) which demographic characteristics explain the overall core competency.

DESIGN

A cross-sectional design.

METHODS

Healthcare Professional Core Competency Instrument, consisting of 11 sub-scales with 81 items, was distributed to healthcare professionals (n = 1,543; 826 nurses and 717 physicians) who worked at primary, secondary and tertiary healthcare institutions. Descriptive statistics, t-test, ANOVA and linear regression were used for data analysis.

RESULTS

Altogether 1,078 healthcare professionals (628 nurses and 450 physicians) responded representing 70% overall response rate. Healthcare professionals perceived their overall core competence as excellent, safety being the highest, and research and evidence-based practice was the lowest. The multiple linear regression analysis revealed that ethnicity, gender and years of working experience were the characters that explained the overall core competence, where expatriate senior professionals reported higher competency levels compared with counterparts.

How progress evaluations are used in postgraduate education with longitudinal supervisor-trainee relationships: a mixed method study

The combination of measuring performance and giving feedback creates tension between formative and summative purposes of progress evaluations and can be challenging for supervisors. There are conflicting perspectives and evidence on the effects supervisor-trainee relationships have on assessing performance. The aim of this study was to learn how progress evaluations are used in postgraduate education with longitudinal supervisor-trainee relationships. Progress evaluations in a two-year community-pharmacy specialization program were studied with a mixed-method approach. An adapted version of the Canadian Medical Education Directives for Specialists (CanMEDS) framework was used. Validity of the performance evaluation scores of 342 trainees was analyzed using repeated measures ANOVA. Semi-structured interviews were held with fifteen supervisors to investigate their response processes, the utility of the progress evaluations, and the influence of supervisor-trainee relationships. Time and CanMEDS roles affected the three-monthly progress evaluation scores. Interviews revealed that supervisors varied in their response processes. They were more committed to stimulating development than to scoring actual performance. Progress evaluations were utilized to discuss and give feedback on trainee development and to add structure to the learning process. A positive supervisor-trainee relationship was seen as the foundation for feedback and supervisors preferred the roles of educator, mentor, and coach over the role of assessor. We found that progress evaluations are a good method for directing feedback in longitudinal supervisor-trainee relationships. The reliability of scoring performance was low. We recommend progress evaluations to be independent of formal assessments in order to minimize roles-conflicts of supervisors.

General practice residents’ perspectives on their professional identity formation: a qualitative study

Objectives

To move beyond professionalism as a measurable competency, medical educators have highlighted the importance of forming a professional identity, in which learners come to ‘think, act, and feel like physicians’. This socialisation process is known as professional identity formation (PIF). Few empirical studies on PIF in residency have been undertaken. None of these studies focused on PIF during the full length of GP training as well as the interplay of concurrent socialising factors. Understanding the socialisation process involved in the development of a resident’s professional identity and the roles of influencing factors and their change over time could add to a more purposeful approach to PIF. Therefore, we aimed to investigate the process of PIF during the full length of General Practice (GP) training and which factors residents perceive as influential.

Design

A qualitative descriptive study employing focus group interviews.

Setting

Four GP training institutes across the Netherlands.

Participants

Ninety-two GP residents in their final training year participated in 12 focus group interviews.

Results

Study findings indicated that identity formation occurs primarily in the workplace, as residents move from doing to becoming and negotiate perceived norms. A tapestry of interrelated influencing factors—most prominently clinical experiences, clinical supervisors and self-assessments—changed over time and were felt to exert their influence predominantly in the workplace.

Conclusions

This study provides deeper empirical insights into PIF during GP residency. Doing the work of a GP exerted a pivotal influence on residents’ shift from doing as a GP to thinking, acting and feeling like a GP, that is, becoming a GP. Clinical supervisors are of utmost importance as role models and coaches in creating an environment that supports residents’ PIF. Implications for practice include faculty development initiatives to help supervisors be aware of how they can perform their various roles across different PIF stages.

Unprofessional behaviour of GP residents and its remediation: a qualitative study among supervisors and faculty

BACKGROUND

Lapses in professionalism have profound negative effects on patients, health professionals, and society. The connection between unprofessional behaviour during training and later practice requires timely identification and remediation. However, appropriate language to describe unprofessional behaviour and its remediation during residency is lacking. Therefore, this exploratory study aims to investigate which behaviours of GP residents are considered unprofessional according to supervisors and faculty, and how remediation is applied.

METHODS

We conducted eight semi-structured focus group interviews with 55 broadly selected supervisors from four Dutch GP training institutes. In addition, we conducted individual semi-structured interviews with eight designated professionalism faculty members. Interview recordings were transcribed verbatim. Data were coded in two consecutive steps: preliminary inductive coding was followed by secondary deductive coding using the descriptors from the recently developed 'Four I's' model for describing unprofessional behaviours as sensitising concepts.

RESULTS

Despite the differences in participants' professional positions, we identified a shared conceptualisation in pinpointing and assessing unprofessional behaviour. Both groups described multiple unprofessional behaviours, which could be successfully mapped to the descriptors and categories of the Four I's model. Behaviours in the categories 'Involvement' and 'Interaction' were assessed as mild and received informal, pedagogical feedback. Behaviours in the categories 'Introspection' and 'Integrity', were seen as very alarming and received strict remediation. We identified two new groups of behaviours; 'Nervous exhaustion complaints' and 'Nine-to-five mentality', needing to be added to the Four I's model. The diagnostic phase of unprofessional behaviour usually started with the supervisor getting a 'sense of alarm', which was described as either a 'gut feeling', 'a loss of enthusiasm for teaching' or 'fuss surrounding the resident'. This sense of alarm triggered the remediation phase. However, the diagnostic and remediation phases did not appear consecutive or distinct, but rather intertwined.

CONCLUSIONS

The processes of identification and remediation of unprofessional behaviour in residents appeared to be intertwined. Identification of behaviours related to lack of introspection or integrity were perceived as the most important to remediate. The results of this research provide supervisors and faculty with an appropriate language to describe unprofessional behaviours among residents, which can facilitate timely identification and remediation.

A systematic review of healthcare professionals' core competency instruments

While technical and profession-specific competencies are paramount in the delivery of healthcare services, the cross-cutting core competencies of healthcare professionals play an important role in healthcare transformation, innovation, and the integration of roles. This systematic review describes the characteristics and psychometric properties of existing instruments for assessing healthcare professionals' core competencies in clinical settings. It was guided by the JBI methodology and used the COSMIN checklist (Mokkink et al., User manual, 2018, 78, 1) to evaluate the methodological quality of the included studies. A database search (CINAHL, Scopus, and PubMed) and additional manual search were undertaken for peer-reviewed papers with abstracts, published in English between 2008 and 2019. The search identified nine studies that were included in the synthesis demonstrating core competencies in professionalism, ethical and legal issues, research and evidence-based practice, personal and professional development, teamwork and collaboration, leadership and management, and patient-centered care. Few instruments addressed competencies in quality improvement, safety, communication, or health information technology. The findings demonstrate the reviewed tools' validity and reliability and pave the way for a comprehensive evaluation and assessment of core competencies into clinical practice.

Lessons from the implementation of developmental progress assessment: A scoping review

OBJECTIVES

Educators and researchers recently implemented developmental progress assessment (DPA) in the context of competency-based education. To reap its anticipated benefits, much still remains to be understood about its implementation. In this study, we aimed to determine the nature and extent of the current evidence on DPA, in an effort to broaden our understanding of the major goals and intended outcomes of DPA as well as the lessons learned from how it has been executed in, or applied across, educational contexts.

METHODS

We conducted a scoping study based on the methodology of Arksey and O'Malley. Our search strategy yielded 2494 articles. These articles were screened for inclusion and exclusion (90% agreement), and numerical and qualitative data were extracted from 56 articles based on a pre-defined set of charting categories. The thematic analysis of the qualitative data was completed with iterative consultations and discussions until consensus was achieved for the interpretation of the results.

RESULTS

Tools used to document DPA include scales, milestones and portfolios. Performances were observed in clinical or standardised contexts. We identified seven major themes in our qualitative thematic analysis: (a) underlying aims of DPA; (b) sources of information; (c) barriers; (d) contextual factors that can act as barriers or facilitators to the implementation of DPA; (e) facilitators; (f) observed outcomes, and (g) documented validity evidences.

CONCLUSIONS

Developmental progress assessment seems to fill a need in the training of future competent health professionals. However, moving forward with a widespread implementation of DPA, factors such as lack of access to user-friendly technology and time to observe performance may render its operationalisation burdensome in the context of competency-based medical education.

[The "Allgemeinarztbarometer A" - an instrument to assess primary care competencies during medical education and training]

INTRODUCTION

The primary goal of medical education and training is to develop clinical competencies. Competency-based curricula exist for both medical education which - once the current reform is implemented - will be geared more towards primary care, and specialty training for General Practice. The "Allgemeinarztbarometer" is a German-language instrument to assess primary care competencies during specialty training and has so far been mainly used as a self-assessment tool. The aim of this study was (i) to test the application of the "Allgemeinarztbarometer" in the context of medical education as an external assessment tool (in its "Allgemeinarztbarometer A [ABA]" version) and (ii) to evaluate its basic psychometric properties.

METHODS

Physicians involved in teaching were asked to assess medical students after a two-week internship in General Practice during their fifth year of study using the ABA (9-item external assessment version). After data processing and displaying basic distributions, we conducted exploratory factor analysis, followed by confirmatory factor analysis. The detected factor structure was then tested for internal consistency.

RESULTS

A total of 150 completed questionnaires were included in the analysis. One item showed a high number of missing values (response option "Not assessable"). Exploratory factor analysis indicated a two-factor structure ("Patient-oriented skills" and "General Practice decision-making"). Three items were not unambiguously assignable and hence erased. Confirmatory factor analysis validated the identified factor structure. The two factors showed a Cronbach's α of 0.73 and 0.84 (moderate to very good internal consistency).

DISCUSSION

Our study provides a first analysis of the ABA in the context of medical education. The detected two-dimensional structure shows a high goodness-of-fit when giving up three items still used in this survey. The analysis of missing data showed a high number of "I can't say" answers when assessing the development of strategies against burnout.

CONCLUSIONS

The ABA (6-item version) seems to be a suitable tool for the external assessment of General Practice competencies in the context of medical education. Further studies are needed to evaluate both the completeness of the assessment and the external assessability of certain items in the context of medical education.

Alignment of CanMEDS-based Undergraduate and Postgraduate Pharmacy Curricula in The Netherlands

In this article the design of three master programs (MSc in Pharmacy) and two postgraduate specialization programs for community or hospital pharmacist is described. After a preceding BSc in Pharmacy, these programs cover the full pharmacy education capacity for pharmacists in primary and secondary health care in the Netherlands. All programs use the CanMEDS framework, adapted to pharmacy education and specialization, which facilitates the horizontal integration of pharmacists’ professional development with other health care professions in the country. Moreover, it is illustrated that crossing the boundary from formal (university) education to experiential (workplace) education is eased by a gradual change in time spent in these two educational environments and by the use of comparable monitoring, feedback, and authentic assessment instruments. A reflection on the curricula, based on the principles of the Integrative Pedagogy Model and the Self-determination Theory, suggests that the alignment of these educational programs facilitates the development of professional expertise and professional identity of Dutch pharmacists.

How is self-regulated learning documented in e-portfolios of trainees? A content analysis

BACKGROUND

It is assumed that portfolios contribute to self-regulated learning (SRL). Presence of these SRL processes within the documentation kept in portfolios is presupposed in common educational practices, such as the assessment of reflective entries. However, questions can be asked considering the presence of SRL within portfolios. The aim of this study was to gain insight into the documentation of SRL processes within the electronic (e)-portfolio content of medical trainees. SRL consists of numerous processes, for this study the focus was on self-assessment via reflection and feedback, goal-setting and planning, and monitoring, as these are the processes that health professions education research mentions to be supported by portfolios.

METHODS

A database containing 1022 anonymous e-portfolios from General Practitioner trainees was used to provide descriptive statistics of the various available e-portfolio forms. This was followed by a quantitative content analysis of 90 e-portfolios, for which, a codebook was constructed to rate the documentation of the included SRL processes.

RESULTS

The numbers of forms in the e-portfolios varied to a great extent. Content analysis showed a limited documentation of reflective entries, and available entries mainly described events and experiences without explanations and context. Feedback was generally limited to comments on what went well and lacked specificity, context and suggestions for future action. Learning goals and plans were short of specificity, but did contain challenging topics and different goals were compatible with each other. 75% of the e-portfolios showed (limited) signs of monitoring.

CONCLUSIONS

The e-portfolio content showed limited documentation of SRL processes. As documentation of SRL requires time and asks for a high level of introspection and writing skills, one cannot expect documentation of SRL processes to appear in e-portfolio content without efforts.

Development of a Postgraduate Community Pharmacist Specialization Program Using CanMEDS Competencies, and Entrustable Professional Activities

Objectives. To develop and implement a postgraduate, workplace-based curriculum for community pharmacy specialists in the Netherlands, conduct a thorough evaluation of the program, and revise any deficiencies found. Methods. The experiences of the Dutch Advisory Board for Postgraduate Curriculum Development for Medical Specialists were used as a guideline for the development of a competency-based postgraduate education program for community pharmacists. To ensure that community pharmacists achieved competence in 10 task areas and seven roles defined by the Canadian Medical Education Directions for Specialists (CanMEDS), a two-year workplace-based curriculum was built. A development path along four milestones was constructed using 40 entrustable professional activities (EPAs). The assessment program consisted of 155 workplace-based assessments, with the supervisor serving as the main assessor. Also, 360-degree feedback and 22 days of classroom courses were included in the curriculum. In 2014, the curriculum was evaluated by two focus groups and a review committee. Results. Eighty-two first-year trainees enrolled in the community pharmacy specialist program in 2012. That number increased to 130 trainees by 2016 (a 59% increase). In 2015, based on feedback from pharmacy supervisors, trainees, and other stakeholders, 22.5% of the EPAs were changed and the number of workplace-based assessments was reduced by 48.5%. Conclusion. Using design approaches from the medical field in the development of postgraduate workplace-based pharmacy education programs proved to be feasible and successful. How to address the concerns and challenges encountered in developing and maintaining competency-based postgraduate pharmacy education programs merits further research.

A practical approach to programmatic assessment design

Assessment of complex tasks integrating several competencies calls for a programmatic design approach. As single instruments do not provide the information required to reach a robust judgment of integral performance, 73 guidelines for programmatic assessment design were developed. When simultaneously applying these interrelated guidelines, it is challenging to keep a clear overview of all assessment activities. The goal of this study was to provide practical support for applying a programmatic approach to assessment design, not bound to any specific educational paradigm. The guidelines were first applied in a postgraduate medical training setting, and a process analysis was conducted. This resulted in the identification of four steps for programmatic assessment design: evaluation, contextualisation, prioritisation and justification. Firstly, the (re)design process starts with sufficiently detailing the assessment environment and formulating the principal purpose. Key stakeholders with sufficient (assessment) expertise need to be involved in the analysis of strengths and weaknesses and identification of developmental needs. Central governance is essential to balance efforts and stakes with the principal purpose and decide on prioritisation of design decisions and selection of relevant guidelines. Finally, justification of assessment design decisions, quality assurance and external accountability close the loop, to ensure sound underpinning and continuous improvement of the assessment programme.

How Entrustment Is Informed by Holistic Judgments Across Time in a Family Medicine Residency Program: An Ethnographic Nonparticipant Observational Study

PURPOSE

Entrustment has mainly been conceptualized as delegating discrete professional tasks. Because residents provide most of their patient care independently, not all resident performance is visible to supervisors; the entrustment process involves more than granting discrete tasks. This study explored how supervisors made entrustment decisions based on residents' performance in a long-term family medicine training program.

METHOD

A qualitative nonparticipant observational study was conducted in 2014-2015 at competency-based family medicine residency programs in the Netherlands. Seven supervisor-resident pairs participated. During two days, one researcher observed first-year residents' patient encounters, debriefing sessions, and supervisor-resident educational meetings and interviewed them separately afterwards. Data were collected and analyzed using iterative, phenomenological inductive research methodology.

RESULTS

The entrustment process developed over three phases. Supervisors based their initial entrustment on prior knowledge about the resident. In the ensuing two weeks, entrustment decisions regarding independent patient care were derived from residents' observed general competencies necessary for a range of health problems (clinical reasoning, decision making, relating to patients); medical knowledge and skills; and supervisors' intuition. Supervisors provided supervision during and after encounters. Once residents performed independently, supervisors kept reevaluating their decisions, informed by residents' overall growth in competencies rather than by adhering to a predefined set of tasks.

CONCLUSIONS

Supervisors in family medicine residency training took a holistic approach to trust, based on general competencies, knowledge, skills, and intuition. Entrustment started before training and developed over time. Building trust is a mutual process between supervisor and resident, requiring a good working relationship.

Assessing medical professionalism: A systematic review of instruments and their measurement properties

BACKGROUND

Over the last three decades, various instruments were developed and employed to assess medical professionalism, but their measurement properties have yet to be fully evaluated. This study aimed to systematically evaluate these instruments' measurement properties and the methodological quality of their related studies within a universally acceptable standardized framework and then provide corresponding recommendations.

METHODS

A systematic search of the electronic databases PubMed, Web of Science, and PsycINFO was conducted to collect studies published from 1990-2015. After screening titles, abstracts, and full texts for eligibility, the articles included in this study were classified according to their respective instrument's usage. A two-phase assessment was conducted: 1) methodological quality was assessed by following the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) checklist; and 2) the quality of measurement properties was assessed according to Terwee's criteria. Results were integrated using best-evidence synthesis to look for recommendable instruments.

RESULTS

After screening 2,959 records, 74 instruments from 80 existing studies were included. The overall methodological quality of these studies was unsatisfactory, with reasons including but not limited to unknown missing data, inadequate sample sizes, and vague hypotheses. Content validity, cross-cultural validity, and criterion validity were either unreported or negative ratings in most studies. Based on best-evidence synthesis, three instruments were recommended: Hisar's instrument for nursing students, Nurse Practitioners' Roles and Competencies Scale, and Perceived Faculty Competency Inventory.

CONCLUSION

Although instruments measuring medical professionalism are diverse, only a limited number of studies were methodologically sound. Future studies should give priority to systematically improving the performance of existing instruments and to longitudinal studies.

GP supervisors' experience in supporting self-regulated learning: a balancing act

Self-regulated learning is essential for professional development and lifelong learning. As self-regulated learning has many inaccuracies, the need to support self-regulated learning has been recommended. Supervisors can provide such support. In a prior study trainees reported on the variation in received supervisor support. This study aims at exploring supervisors' perspectives. The aim is to explore how supervisors experience self-regulated learning of postgraduate general practitioners (GP) trainees and their role in this, and what helps and hinders them in supervising. In a qualitative study using a phenomenological approach, we interviewed 20 supervisors of first- and third-year postgraduate GP trainees. Supervisors recognised trainee activity in self-regulated learning and adapted their coaching style to trainee needs, occasionally causing conflicting emotions. Supervisors' beliefs regarding their role, trainees' role and the usefulness of educational interventions influenced their support. Supervisors experienced a relation between patient safety, self-regulated learning and trainee capability to learn. Supervisor training was helpful to exchange experience and obtain advice. Supervisors found colleagues helpful in sharing supervision tasks or in calibrating judgments of trainees. Busy practice occasionally hindered the supervisory process. In conclusion, supervisors adapt their coaching to trainees' self-regulated learning, sometimes causing conflicting emotions. Patient safety and entrustment are key aspects of the supervisory process. Supervisors' beliefs about their role and trainees' role influence their support. Supervisor training is important to increase awareness of these beliefs and the influence on their behaviour, and to improve the use of educational instruments. The results align with findings from other (medical) education, thereby illustrating its relevance.

Designing faculty development to support the evaluation of resident competency in the intrinsic CanMEDS roles: practical outcomes of an assessment of program director needs

BACKGROUND

The Royal College of Physicians and Surgeons of Canada and the College of Family Physicians of Canada mandate that faculty members demonstrate they are evaluating residents on all CanMEDS (Canadian Medical Education Directions for Specialists) roles as part of the accreditation process. Postgraduate Medical Education at the University of Ottawa initiated a 5-year project to develop and implement a comprehensive system to assess the full spectrum of CanMEDS roles. This paper presents the findings from a needs assessment with Program Directors, in order to determine how postgraduate medical faculty can be motivated and supported to evaluate residents on the intrinsic CanMEDS roles.

METHODS

Semi-structured individual interviews were conducted with 60 Postgraduate Program Directors in the Faculty of Medicine. Transcribed interviews were analyzed using qualitative analysis. Once the researchers were satisfied the identified themes reflected the views of the participants, the data was assigned to categories to provide rich, detailed, and comprehensive information that would indicate what faculty need in order to effectively evaluate their residents on the intrinsic roles.

RESULTS

Findings indicated faculty members need faculty development and shared point of care resources to support them with how to not only evaluate, but also teach, the intrinsic roles. Program Directors expressed the need to collaborate and share resources across departments and national specialty programs. Based on our findings, we designed and delivered workshops with companion eBooks to teach and evaluate residents at the point of care (Developing the Professional, Health Advocate and Scholar).

CONCLUSIONS

Identifying stakeholder needs is essential for designing effective faculty development. By sharing resources, faculties can prevent 'reinventing the wheel' and collaborate to meet the Colleges' accreditation requirements more efficiently.

Assessing family medicine trainees--what can we learn from the European neighbours?

Background: Although demands on family physicians (FP) are to a large extent similar in the European Union, uniform assessment standards for family medicine (FM) specialty training and assessment do not exist. Aim of this pilot study was to elicit and compare the different modalities and assessment methods of FM specialty training in five European countries.

Methods: A semi structured survey was undertaken based on a convenient sample in five European countries (Denmark, Germany, Poland, the Netherlands and the United Kingdom). The respondents were asked to respond to ten items about aspects of FM specialty training and assessment methods in their respective countries. If available, this data was completed with information from official websites of the countries involved.

Results: FM specialty training is performed heterogeneously in the surveyed countries. Training time periods range from three to five years, in some countries requiring a foundation program of up to two years. Most countries perform longitudinal assessment during FM specialty training using a combination of competence-based approach with additional formative and summative assessment. There is some evidence on the assessments methods used, however the assessment method used and costs of assessment differs remarkably between the participating countries.

Conclusions: Longitudinal and competence-based assessment is the presently preferred approach for FM specialty training. Countries which use less multifaceted methods for assessment could learn from best practice. Potential changes have significant cost implications.

A competency based selection procedure for Dutch postgraduate GP training: a pilot study on validity and reliability

Abstract Background: Historically, semi-structured interviews (SSI) have been the core of the Dutch selection for postgraduate general practice (GP) training. This paper describes a pilot study on a newly designed competency-based selection procedure that assesses whether candidates have the competencies that are required to complete GP training.

OBJECTIVES

The objective was to explore reliability and validity aspects of the instruments developed.

METHODS

The new selection procedure comprising the National GP Knowledge Test (LHK), a situational judgement tests (SJT), a patterned behaviour descriptive interview (PBDI) and a simulated encounter (SIM) was piloted alongside the current procedure. Forty-seven candidates volunteered in both procedures. Admission decision was based on the results of the current procedure.

RESULTS

Study participants did hardly differ from the other candidates. The mean scores of the candidates on the LHK and SJT were 21.9 % (SD 8.7) and 83.8% (SD 3.1), respectively. The mean self-reported competency scores (PBDI) were higher than the observed competencies (SIM): 3.7(SD 0.5) and 2.9(SD 0.6), respectively. Content-related competencies showed low correlations with one another when measured with different instruments, whereas more diverse competencies measured by a single instrument showed strong to moderate correlations. Moreover, a moderate correlation between LHK and SJT was found. The internal consistencies (intraclass correlation, ICC) of LHK and SJT were poor while the ICC of PBDI and SIM showed acceptable levels of reliability.

CONCLUSION

Findings on content validity and reliability of these new instruments are promising to realize a competency based procedure. Further development of the instruments and research on predictive validity should be pursued.

[Professionalism in general practice in Germany - a qualitative approximation]

BACKGROUND

Pre- and postgraduate education is meant to be competency-based. Over the last two decades various competency frameworks have been published. One competency is professionalism, a definition of which has not yet been developed but is being discussed in the literature. The aim of this qualitative study is an approximation to professionalism among German general practitioners and general practitioner trainees.

METHODS

A qualitative study was conducted by interviewing seven pairs of GPs and their trainees. All interviews were recorded and transcribed. The analysis was performed according to Mayring supported by the software Atlas.ti.

RESULTS

Four categories of professionalism emerged: responsibility towards patients, responsibility towards other professionals, responsibility towards the society and responsibility towards oneself. Professionalism was perceived as important for general practice in Germany. In addition, barriers of professional behaviour have been identified.

CONCLUSION

The perception of professionalism among German GPs and GP trainees is in accordance with the frameworks of professionalism found in the literature. These results underline the need for conceptualising professionalism among general practice trainees in Germany.