Are some of the challenging aspects of the CanMEDS roles valid outside Canada?

Comparability of outcome frameworks in medical education: Implications for framework development

BACKGROUND

Given the increasing mobility of medical students and practitioners, there is a growing need for harmonization of medical education and qualifications. Although several initiatives have sought to compare national outcome frameworks, this task has proven a challenge. Drawing on an analysis of existing outcome frameworks, we identify factors that hinder comparability and suggest ways of facilitating comparability during framework development and revisions.

METHODS

We searched MedLine, EmBase and the Internet for outcome frameworks in medical education published by national or governmental organizations. We analyzed these frameworks for differences and similarities that influence comparability.

RESULTS

Of 1816 search results, 13 outcome frameworks met our inclusion criteria. These frameworks differ in five core features: history and origins, formal structure, medical education system, target audience and key terms. Many frameworks reference other frameworks without acknowledging these differences. Importantly, the level of detail of the outcomes specified differs both within and between frameworks.

CONCLUSION

The differences identified explain some of the challenges involved in comparing outcome frameworks and medical qualifications. We propose a two-level model distinguishing between "core" competencies and culture-specific "secondary" competencies. This approach could strike a balance between local specifics and cross-national comparability of outcome frameworks and medical education.

The 'missing person' in roles-based competency models: a historical, cross-national, contrastive case study

CONTEXT

The use of roles such as medical expert, advocate or communicator to define competencies is currently popular in health professions education. CanMEDS is one framework that has been subject to great uptake across multiple countries and professions. The examination of the historical and cultural choices of names for roles generates insight into the nature and construction of roles. One role that has appeared in and disappeared from roles-based frameworks is that of the 'person'.

METHODS

In order to examine the implications of explicitly including or excluding the role of the 'physician as person' in a competency framework, we conducted a contrastive analysis of the development of frameworks in Canada and the Netherlands. We drew upon critical social science theoretical understandings of the power of language in our analysis.

RESULTS

In Canada, the 'person' role was a late addition to the precursory work that informed CanMEDS, and was then excluded from the final set of CanMEDS role names. In the Netherlands, a 'reflector' role was added in some Dutch schools and programmes when CanMEDS was adopted. This was done in order to explicitly emphasise the importance of the 'person' of the trainee.

CONCLUSIONS

In analysing choices of names for roles, we have the opportunity to see how cultural and historical contexts affect conceptions of the roles of doctors. The taking up and discarding of the 'person' role in Canada and the Netherlands suggest that as medical educators we may need to further consider the ways in which we wish the trainee as a person to be made visible in the curriculum and in assessment tools.

The CanMEDS framework: relevant but not quite the whole story

BACKGROUND

Despite acknowledgement that the Canadian Medical Educational Directives for Specialists (CanMEDS) framework covers the relevant competencies of physicians, many educators and medical professionals struggle to translate the CanMEDS roles into comprehensive training programmes for specific specialties.

AIM

To gain insight into the applicability of the CanMEDS framework to guide the design of educational programmes for specific specialties by exploring stakeholders' perceptions of specialty specific competencies and examining differences between those competencies and the CanMEDS framework.

METHODS

This case study is a sequel to a study among ObsGyn specialists. It explores the perspectives of patients, midwives, nurses, general practitioners, and hospital boards on gynaecological competencies and compares these with the CanMEDS framework.

RESULTS

Clinical expertise, reflective practice, collaboration, a holistic view, and involvement in practice management were perceived to be important competencies for gynaecological practice. Although all the competencies were covered by the CanMEDS framework, there were some mismatches between stakeholders' perceptions of the importance of some competencies and their position in the framework.

CONCLUSION

The CanMEDS framework appears to offer relevant building blocks for specialty specific postgraduate training, which should be combined with the results of an exploration of specialty specific competencies to arrive at a postgraduate curriculum that is in alignment with professional practice.

Surgeons' attitude toward a competency-based training and assessment program: results of a multicenter survey

BACKGROUND

Currently, most surgical training programs are focused on the development and evaluation of professional competencies. Also in the Netherlands, competency-based training and assessment programs were introduced to restructure postgraduate medical training. The current surgical residency program is based on the Canadian Medical Education Directives for Specialists (CanMEDS) competencies and uses assessment tools to evaluate residents' competence progression. In this study, we examined the attitude of surgical residents and attending surgeons toward a competency-based training and assessment program used to restructure general surgical training in the Netherlands in 2009.

METHODS

In 2011, all residents (n = 51) and attending surgeons (n = 108) in 1 training region, consisting of 7 hospitals, were surveyed. Participants were asked to rate the importance of the CanMEDS competencies and the suitability of the adopted assessment tools. Items were rated on a 5-point Likert scale and considered relevant when at least 80% of the respondents rated an item with a score of 4 or 5 (indicating a positive attitude). Reliability was evaluated by calculating the Cronbach's α, and the Mann-Whitney test was applied to assess differences between groups.

RESULTS

The response rate was 88% (n = 140). The CanMEDS framework demonstrated good reliability (Cronbach's α = 0.87). However, the importance of the competencies 'Manager' (78%) and 'Health Advocate' (70%) was undervalued. The assessment tools failed to achieve an acceptable reliability (Cronbach's α = 0.55), and individual tools were predominantly considered unsuitable for assessment. Exceptions were the tools 'in-training evaluation report' (91%) and 'objective structured assessment of technical skill' (82%). No significant differences were found between the residents and the attending surgeons.

CONCLUSION

This study has demonstrated that, 2 years after the reform of the general surgical residency program, residents and attending surgeons in a large training region in the Netherlands do not acknowledge the importance of all CanMEDS competencies and consider the assessment tools generally unsuitable for competence evaluation.

In the eye of the beholder: student perspectives on professional roles in practice

CONTEXT

Learning about professional roles in clinical settings is confounded by the gap between espoused theory and the professional practice of the workplace. Workplace learning is grounded in that which is afforded to learners and individuals' engagement with those affordances. The meaning students make of the real-world performance of professional roles and how this relates to formal professionalism frameworks remain unclear. Construal of experience is individual. Professional roles are enacted in the eye of the beholder. In their reflections, student subjectivities, intentionalities and engagement with workplace affordances are revealed. Our research question was: How do students' perspectives of professional roles in practice, revealed through written reflections, relate to the formal professionalism curriculum?

METHODS

Year 3 students (n = 108) wrote reflections during hospital and community placements. Thematic content analysis was performed. A priori categories based on the CanMEDS Physician Roles Framework were used to map content.

RESULTS

A total of 107 students consented to the use of their reflections (n = 315). The CanMEDS roles of Communicator, Professional and Scholar predominated. Students were seen applying prior knowledge to new situations and reflecting on them. For some, the confirmation of previous learning was the outcome; for others, the mismatch between practice and the formal curriculum led to the questioning of both. The roles of Manager, Collaborator and Health Advocate were less frequently reflected upon. Differences between the affordances of hospital and community placements were seen. Means to address findings are discussed with reference to Billett's duality of workplace learning.

CONCLUSIONS

Reflective narratives reveal how students construe professional roles in practice. Mapping the content of reflections to a competency framework confirmed the mismatch between the formal and enacted curricula. Billett's duality of workplace learning provides a useful lens through which to identify means to address this, through the structural aspects of access and guidance, and through the promotion of individual engagement and reflection.

How Dutch medical specialists perceive the competencies and training needs of medical residents in healthcare management

BACKGROUND

The Dutch postgraduate medical training has been revised to focus on seven competencies. The role as manager is one of these competencies. Recent studies show that this competency receives little attention during the residency training. In an earlier study, we discovered that residents perceived their competency as managers to be moderate.

AIM

In this study, we investigated how medical specialists perceived the managerial competencies of medical residents and their need for management education.

METHODS

In September 2010, a 46-item questionnaire was designed which examined medical specialists' perceptions of the competency and needs of residents in the field of medical management. Two hundred ninety-eight specialists were invited via email to participate.

RESULTS

Hundred twenty-nine specialists (43.3%) responded to our survey. They rated the residents' competencies in contract negotiating skills, knowledge of the healthcare system, and specialist department poorly. They felt that residents were competent in updating their medical knowledge. Ninety-four percent reported a need for training in management among residents. Preferred topics were time management and healthcare organization. The preferred training method was a workshop given during residency by an extramural expert.

CONCLUSION

Dutch medical specialists perceive the management competencies of residents in some areas to be inadequate. They feel that training in medical management during residency is necessary.

Competencies of specialised wound care nurses: a European Delphi study

Health care professionals responsible for patients with complex wounds need a particular level of expertise and education to ensure optimum wound care. However, uniform education for those working as wound care nurses is lacking. We aimed to reach consensus among experts from six European countries as to the competencies for specialised wound care nurses that meet international professional expectations and educational systems. Wound care experts including doctors, wound care nurses, lecturers, managers and head nurses were invited to contribute to an e-Delphi study. They completed online questionnaires based on the Canadian Medical Education Directives for Specialists framework. Suggested competencies were rated on a 9-point Likert scale. Consensus was defined as an agreement of at least 75% for each competence. Response rates ranged from 62% (round 1) to 86% (rounds 2 and 3). The experts reached consensus on 77 (80%) competences. Most competencies chosen belonged to the domain 'scholar' (n = 19), whereas few addressed those associated with being a 'health advocate' (n = 7). Competencies related to professional knowledge and expertise, ethical integrity and patient commitment were considered most important. This consensus on core competencies for specialised wound care nurses may help achieve a more uniform definition and education for specialised wound care nurses.

A Delphi study to construct a CanMEDS competence based inventory applicable for workplace assessment

BACKGROUND

During workplace based learning students develop professional competences and an appropriate performance. To gain insight in the learning process and to evaluate competences and performance, assessment tools are essential and need to be of good quality. We aimed to construct a competence inventory applicable as an instrument to measure the content validity of workplace based assessment tools, such as portfolio.

METHODS

A Delphi study was carried out based on the CanMEDS Roles Framework. In three rounds, experts (N = 25-30) were invited to score the key competences per CanMEDS role on relevance (6-point Likert-scale), and to comment on the content and formulation bearing in mind its use in workplace based assessment. A descriptive analysis of relevances and comments was performed.

RESULTS

Although all competences were scored as relevant, many comments pointed at a lack of concrete, transparent and applicable descriptions of the key competences for the purpose of assessment. Therefore, the CanMEDS roles were reformulated in this Delphi procedure as concrete learning outcomes, observable and suitable for workplace based assessment.

CONCLUSIONS

A competence based inventory, ready for validating workplace based assessment tools, was constructed using a Delphi procedure and based on a clarification and concretisation of the CanMEDS roles.

Determining the weighting and relative importance of CanMEDS roles and competencies

Background

The CanMEDS roles and competencies are being used as the framework to support the development of the Manitoba Practice Assessment Program (MPAP) designed to assess the competence of physicians practicing with a conditional license. Establishing the link between clinical practice and assessment of performance is critical in the development of the MPAP. A first step in establishing this link is to identify activities performed in actual clinical practice as well as the importance of those activities.

Methods

A descriptive survey design was used to answer the research questions: (1) How do physicians rate the complexity, frequency, and criticality of CanMEDS roles? (2) What is the distribution of perceived importance scores for the CanMEDS roles? Two online surveys, one specific to family practice physicians, and one specific to specialists, were emailed to a sample of Canadian physicians.

Results

Overall perceived importance scores were calculated for each of the CanMEDS roles. It appears that each role is considered to be at least moderately important. The Medical Expert role was ranked as the most important, followed by the roles of Communicator, Professional, Collaborator, Scholar, Manager, and Health Advocate. There were no significant differences in overall CanMEDS perceived importance scores between family practice physicians and specialists (N = 88).

Conclusions

Given that each of the CanMEDS roles is considered at least moderately important, a variety of assessment tools are needed to evaluate competencies across the entire spectrum of roles. The results underscore the importance of incorporating a multifaceted approach when developing a practice assessment program.

Negotiating authority: a comparative study of reform in medical training regimes

Recently the medical profession has faced increased outside pressure to reform postgraduate medical training programs to better equip young doctors for changing health care needs and public expectations. In this article we explore the impact of reform on professional self-governance by conducting a comparative historical-institutional analysis of postgraduate medical training reform in the United Kingdom and the Netherlands. In both countries the medical training regime has shifted from professional self-regulation to coregulation. Yet there are notable differences in each country that cannot be explained solely by diverging institutional contexts. They also result from the strategic actions by the actors involved. Based on an assessment of the recent literature on institutional transformation, this article shows how strategic actions set negotiating authority processes into motion, producing new and sometimes surprising institutional arrangements that can have profound effects on the distribution and allocation of authority in the medical training regime. It stresses the need to study the interactions among political context, the properties of institutions, and negotiating authority processes, as they are crucially important to understanding institutional transformation.

Facilitators and barriers to a nationwide implementation of competency-based postgraduate medical curricula: a qualitative study

BACKGROUND

Postgraduate medical education (PGME) curricula are being redesigned across the western world.

AIM

This study examined the implementation process (what works where and why) of new competency-based PGME curricula and relevant factors influencing this process.

METHODS

In a nationwide project (2006-2010) in the Netherlands, competency-based PGME curricula were implemented for residents in Pediatrics and Obstetrics & Gynecology. The authors conducted 25 semi-structured interviews and used a multi-level theoretical framework to guide coding.

RESULTS

The implementation process proved to be highly dynamic, non-linear, and influenced by many factors. These could be divided into attributes of the innovations/adopters, the implementation process, and the organization. The context determined the speed, quality, and direction of the process and how a factor affected the process.

CONCLUSIONS

We identified specific features of PGME innovation: the challenge of implementing other competencies than that of the medical expert; the importance of regional implementation strategies and educational support; the balance between training and patient care; and the need for regional inter-organizational networks of hospitals. The authors recommend: design the curriculum with the needs of the users in mind; facilitate knowledge sharing; organize educational support; translate the national curriculum to the local workplace; and promote regional inter-organizational networks between hospitals.

Validation of core competencies during residency training in anaesthesiology

BACKGROUND AND GOAL

Curriculum development for residency training is increasingly challenging in times of financial restrictions and time limitations. Several countries have adopted the CanMEDS framework for medical education as a model into their curricula of specialty training. The purpose of the present study was to validate the competency goals, as derived from CanMEDS, of the Department of Anaesthesiology and Intensive Care Medicine of the Berlin Charité University Medical Centre, by conducting a staff survey. These goals for the qualification of specialists stipulate demonstrable competencies in seven areas: expert medical action, efficient collaboration in a team, communications with patients and family, management and organisation, lifelong learning, professional behaviour, and advocacy of good health. We had previously developed a catalogue of curriculum items based on these seven core competencies. In order to evaluate the validity of this catalogue, we surveyed anaesthetists at our department in regard to their perception of the importance of each of these items. In addition to the descriptive acquisition of data, it was intended to assess the results of the survey to ascertain whether there were differences in the evaluation of these objectives by specialists and registrars.

METHODS

The questionnaire with the seven adapted CanMEDS Roles included items describing each of their underlying competencies. Each anaesthetist (registrars and specialists) working at our institution in May of 2007 was asked to participate in the survey. Individual perception of relevance was rated for each item on a scale similar to the Likert system, ranging from 1 (highly relevant) to 5 (not at all relevant), from which ratings means were calculated. For determination of reliability, we calculated Cronbach's alpha. To assess differences between subgroups, we performed analysis of variance.

RESULTS

All seven roles were rated as relevant. Three of the seven competency goals (expert medical action, efficient collaboration in a team, and communication with patients and family) achieved especially high ratings. Only a few items differed significantly in their average rating between specialists and registrars.

CONCLUSIONS

We succeeded in validating the relevance of the adapted seven CanMEDS competencies for residency training within our institution. So far, many countries have adopted the Canadian Model, which indicates the great practicability of this competency-based model in curriculum planning. Roles with higher acceptance should be prioritised in existing curricula. It would be desirable to develop and validate a competency-based curriculum for specialty training in anaesthesiology throughout Germany by conducting a national survey to include specialists as well as registrars in curriculum development.

Development and validation of a competency framework for veterinarians

Changing demands from society and the veterinary profession call for veterinary medical curricula that can deliver veterinarians who are able to integrate specific and generic competencies in their professional practice. This requires educational innovation directed by an integrative veterinary competency framework to guide curriculum development. Given the paucity of relevant information from the veterinary literature, a qualitative multi-method study was conducted to develop and validate such a framework. A competency framework was developed based on the analysis of focus group interviews with 54 recently graduated veterinarians and clients and subsequently validated in a Delphi procedure with a panel of 29 experts, representing the full range and diversity of the veterinary profession. The study resulted in an integrated competency framework for veterinary professionals, which consists of 16 competencies organized in seven domains: veterinary expertise, communication, collaboration, entrepreneurship, health and welfare, scholarship, and personal development. Training veterinarians who are able to use and integrate the seven domains in their professional practice is an important challenge for today's veterinary medical schools. The Veterinary Professional (VetPro) framework provides a sound empirical basis for the ongoing debate about the direction of veterinary education and curriculum development.

Competency-based (CanMEDS) residency training programme in radiology: systematic design procedure, curriculum and success factors

Objective

Based on the CanMEDS framework and the European Training Charter for Clinical Radiology a new radiology curriculum was designed in the Netherlands. Both the development process and the resulting new curriculum are presented in this paper.

Methods

The new curriculum was developed according to four systematic design principles: discursiveness, hierarchical decomposition, systematic variation and satisficing (satisficing is different from satisfying; in this context, satisficing means searching for an acceptable solution instead of searching for an optimal solution).

Results

The new curriculum is organ based with integration of radiological diagnostic techniques, comprises a uniform national common trunk followed by a 2-year subspecialisation, is competency outcome based with appropriate assessment tools and techniques, and is based on regional collaboration among radiology departments.

Discussion

The application of the systematic design principles proved successful in producing a new curriculum approved by all authorities. The principles led to a structured, yet flexible, development process in which creative solutions could be generated and adopters (programme directors, supervisors and residents) were highly involved. Further research is needed to empirically test the components of the new curriculum.

Evaluation of a national process of reforming curricula in postgraduate medical education

CONTEXT

A national reform of the postgraduate medical education in Denmark introduced (1) Outcome-based education, (2) The CanMEDS framework of competence related to seven roles of the doctor, and (3) In-training assessment.

OBJECTIVES

The purpose of the study was to evaluate the process of developing new curricula for 38 specialist training programmes. The research question was: which conditions promote and which conditions impede the process?

METHODS

Evaluation of the process was conducted among 76 contact-persons, who were chairing the curriculum development process within the specialties. Quantitative and qualitative data from a questionnaire survey and telephone interviews were triangulated for data analysis.

RESULTS

The response rate of the questionnaire survey was 83% (63/76). Twenty-six telephone interviews were conducted. Identified promoting factors included positive attitude and motivation in faculty and support from written guidelines and seminars. Identified impeding factors included insufficient pedagogical support, poor introduction to the task, changing and inconsistent information from authorities, replacement of advisors, and stressful deadlines.

CONCLUSIONS

This study identified promoting and impeding factors in a national postgraduate curriculum development process. Surprisingly the study indicates that pedagogical support provided throughout a process in some aspects might not be useful. General suggestions regarding curriculum reform processes are formulated.

What indicates competency in systems based practice? An analysis of perspective consistency among healthcare team members

In many parts of the world the practice of medicine and medical education increasingly focus on providing patient care within context of the larger healthcare system. Our purpose is to solicit perceptions of all professional stakeholders (e.g. nurses) of the system regarding the U.S. ACGME competency Systems Based Practice to uncover the extent to which there is agreement or discrepancy among key system stakeholders. Eighty-eight multidisciplinary personnel (n = 88) from two academic medical centers were invited to participate in one of 14 nominal group process sessions. Participants generated and prioritized resident characteristics that they believed were important for effective System Based Practices. Through content analysis the prioritized attribute statements were coded to identify embedded themes of resident roles and behavior. From the themes, three major resident roles emerged: resident as Self-Manager, Team Collaborator, and Patient Advocate. No one professional group (e.g., nurses, attending physicians, social workers) emphasized all of these roles. Some concepts that are emphasized in the ACGME definition like using cost-benefit analysis were conspicuously absent from the healthcare team generated list. We showed that there are gaps between the key stakeholders prioritizations about the ACGME definition of SBP and, more generally, the behaviors and roles identified by healthcare team stakeholders beyond the U.S. This suggests that within the process of developing a comprehensive working understanding of the Systems Based Practice competency (or other similar competencies, such as in CanMEDS), it is necessary to use multiple stakeholders in the system (perhaps including patients) to more accurately identify key resident roles and observable behaviors.

CanMEDS evaluation in Canadian postgraduate training programmes: tools used and programme director satisfaction

CONTEXT

The Royal College of Physicians and Surgeons of Canada (RCPSC) CanMEDS framework is being incorporated into specialty education worldwide. However, the literature on how to evaluate trainees in the CanMEDS competencies remains sparse.

OBJECTIVES

The goals of this study were to examine the assessment tools used and programme directors' perceptions of how well they evaluate performance of the CanMEDS roles in Canadian postgraduate training programmes.

METHODS

We conducted a web-based survey of programme directors of RCPSC-accredited training programmes. The survey consisted of two questions. Question 1 was designed to establish which assessment tools were used to assess each of the CanMEDS roles. Question 2 was intended to assess programme directors' perceived satisfaction with CanMEDS evaluation in their programmes.

RESULTS

A total of 149 of the eligible 280 programme directors participated in the survey. Programme directors used a variety of assessment tools to evaluate trainees in CanMEDS competencies. Programmes used more tools to evaluate the Medical Expert (mean = 4.03, standard deviation [SD] = 1.59) and Communicator (mean = 2.36, SD = 1.02) roles. Programme directors used the fewest tools for the Collaborator (mean = 1.75, SD = 1.10) and Manager (mean = 1.75, SD = 1.18) roles. More than 92% of the programmes used in-training evaluation reports to evaluate all the CanMEDS roles. Programme directors were satisfied with their evaluation of the Medical Expert role, but less so with assessment of the other CanMEDS competencies.

CONCLUSIONS

This study demonstrates that Canadian postgraduate training programmes use a variety of assessment tools to evaluate the CanMEDS competencies. Programme directors are neutral or concerned about how the CanMEDS roles other than that of Medical Expert are evaluated in their programmes. Further efforts are required to establish best practice in CanMEDS evaluation.