Implementing competency-based medical education: Moving forward

Post competency training in standardized training of resident physicians and integrated postgraduates

We focus on the importance and necessity of post-competency-based training in medical education alongside its current challenges and opportunities. Integrating post competency training into standardized educational frameworks is increasingly recognized as a critical component of preparing residents and postgraduates for real-world clinical practice. Post competency training represents a commitment to excellence in medical education, striving to produce competent, skilled practitioners to meet the challenges of modern healthcare.

Education and Training in Global Neurosurgery: Current State and Path Toward a Uniform Curriculum

Education is a sustainable long-term measure to address the global burden of neurosurgical disease. Neurosurgery residencies in high-income countries are accredited by a regional governing body and incorporate various educational activities. Few opportunities for training may be present in low-income and middle-income countries due to a lack of neurosurgery residency programs, tuition, and health care workforce reductions. Core components of a neurosurgical training curriculum include operative room experience, clinical rounds, managing inpatients, and educational conferences. A gold standard for neurosurgical education is essential for creating comprehensive training experience, though training must be contextually appropriate.

Critical analysis of the subspecialty stroke medicine curriculum: Social and political influences on its design and professionalism

The curriculum comprises all learners' learning experiences that enable them to achieve specific learning outcomes. The Subspeciality Stroke Medicine Curriculum is designed to train doctors in Stroke Medicine as specialists capable of providing holistic healthcare in preventing, treating and rehabilitating stroke through achieving the desired competencies. This article discusses the influence of factors like the development of learning theory, the democratisation of social process, public demand for accountability and transparency, political imperatives, economic factors and professional standards set by professional bodies on curriculum design. The curriculum focuses on an outcome-based educational approach, workplace-based assessment with formative feedback to promote learning, summative evidence for knowledge, skills and attitudes and greater integration to make learning closer to actual practice. This outcome-based, integrated approach is approved by regulating bodies as positively impacting doctors' training and, consequently, the health of individual patients and society.

AACN competency-based essentials revisit: Evidence-based validation of entrustable professional activities in Asia

AIM

This study intended to validate the competency-based approach through an entrustable professional activity in the nursing undergraduate education arena in Taiwan.

BACKGROUND

Entrustable professional activity is a recommended strategy to enhance nursing competencies and skills. It has been widely applied to nursing education in Western countries, especially graduate programs. However, its effects in eastern countries and undergraduate programs remain unclear.

DESIGN

A quasi-experimental comparison design was used.

METHOD

The study is conducted at the Department of Nursing at a university in southern Taiwan. A total of 72 Two-Year Nursing Program students participated in the study. After implementing the designated entrustable professional activity in the Wound Care Nursing course, outcomes were measured using the Competency Inventory of Nursing Students, Learning Satisfaction and Objective Structured Clinical Examinations. These outcomes were then compared with post-test results at the end of the semester. About 90 % of students completed the study with the test group (n=31) receiving extra activity and the comparison group (n=34) receiving usual teaching. Data were analyzed through chi-square, paired t and Student's t-test.

RESULTS

The test group demonstrated significantly higher scores in perceived competency and clinical examinations than the comparison group. However, both groups experienced an increase in learning satisfaction without reaching a significant difference.

CONCLUSION

Results of the study indicate that competency-based pedagogy, e.g., using entrustable professional activities, should be integrated into nursing curricula to meet the new American Association of Colleges of Nursing standards with compelling evidence.

The future of public health doctoral education in Africa: transforming higher education institutions to enhance research and practice

The African Union and the Africa Centers for Disease Control and Prevention issued a Call to Action in 2022 for Africa's New Public Health Order that underscored the need for increased capacity in the public health workforce. Additional domestic and global investments in public health workforce development are central to achieving the aspirations of Agenda 2063 of the African Union, which aims to build and accelerate the implementation of continental frameworks for equitable, people-centred growth and development. Recognising the crucial role of higher education and research, we assessed the capabilities of public health doctoral training in schools and programmes of public health in Africa across three conceptual components: instructional, institutional, and external. Six inter-related and actionable recommendations were derived to advance doctoral training, research, and practice capacity within and between universities. These can be achieved through equitable partnerships between universities, research centres, and national, regional, and global public health institutions.

How well do workplace-based assessments support summative entrustment decisions? A multi-institutional generalisability study

BACKGROUND

Assessment of the Core Entrustable Professional Activities for Entering Residency requires direct observation through workplace-based assessments (WBAs). Single-institution studies have demonstrated mixed findings regarding the reliability of WBAs developed to measure student progression towards entrustment. Factors such as faculty development, rater engagement and scale selection have been suggested to improve reliability. The purpose of this investigation was to conduct a multi-institutional generalisability study to determine the influence of specific factors on reliability of WBAs.

METHODS

The authors analysed WBA data obtained for clerkship-level students across seven institutions from 2018 to 2020. Institutions implemented a variety of strategies including selection of designated assessors, altered scales and different EPAs. Data were aggregated by these factors. Generalisability theory was then used to examine the internal structure validity evidence of the data. An unbalanced cross-classified random-effects model was used to decompose variance components. A phi coefficient of >0.7 was used as threshold for acceptable reliability.

RESULTS

Data from 53 565 WBAs were analysed, and a total of 77 generalisability studies were performed. Most data came from EPAs 1 (n = 17 118, 32%) 2 (n = 10 237, 19.1%), and 6 (n = 6000, 18.5%). Low variance attributed to the learner (<10%) was found for most (59/77, 76%) analyses, resulting in a relatively large number of observations required for reasonable reliability (range = 3 to >560, median = 60). Factors such as DA, scale or EPA were not consistently associated with improved reliability.

CONCLUSION

The results from this study describe relatively low reliability in the WBAs obtained across seven sites. Generalisability for these instruments may be less dependent on factors such as faculty development, rater engagement or scale selection. When used for formative feedback, data from these instruments may be useful. However, such instruments do not consistently provide reasonable reliability to justify their use in high-stakes summative entrustment decisions.

Understanding Emotions Impacted by New Assessment Mandates Implemented in Medical Education: A Survey of Residents and Faculty Across Multiple Specialties

Background Previous research findings show that the overall perception of residents regarding the new entrustable professional activity (EPA) assessment mandates is primarily negative. Hence, this study aims to explore the link between EPA assessment experiences and resident and faculty emotions and expectancy of successfully completing residency training. Methods A standardized questionnaire (Medical Emotions Scale (MES)), which measures 20 unique emotions on a 5-point Likert scale, was used to explore the emotions of residents and faculty members regarding EPA assessments and residents' expectancy of success. Data analysis included descriptive statistics and analysis of variance (ANOVA). Results Ninety-one (N=91) participants (46 faculty members and 45 residents) completed the survey. The results revealed that residents have more negative emotions toward EPA assessments compared to faculty. Additionally, resident and faculty emotions regarding EPA assessments vary across specialty and gender. Conclusions These findings will be crucial in providing the Royal College of Physicians and Surgeons of Canada and medical education programs with concrete evidence and guidance in understanding the perspectives and emotions of residents and faculty towards EPA assessments and residents' beliefs about successfully completing their medical training.

The Need for Competency-Based Education

OBJECTIVES

While pharmacy education updates learning as new information arises, changes to learning experiences can trail behind current practices and technology. There have been multiple calls for radical changes in how health professions education is delivered to ensure patients are receiving high-quality care. Competency-based education has been one way discussed in the literature for how to handle this need to develop students who have a willingness to learn and can problem-solve. The goal of this review is to examine whether competency-based education is needed to drive the profession of pharmacy forward.

FINDINGS

To address, we collaboratively identified stakeholder perspectives to evaluate the need. The following stakeholders achieved consensus among the committee members: patients/society, learners, workplace/profession, and academic institutions.

SUMMARY

Based on those perspectives, needs, and gaps to address those needs were identified and are presented in this review.

From obese to lean curriculum: exploring students' experiences about developing competencies in medical education

Introduction

Since the beginning of the 21st century, competency-based education has been proposed as an approach to education in many disciplines including the medical sciences and it has become a dominant approach in many countries. We aimed to explore the lived experiences of general medical students about developing competencies in the academic curriculum.

Methods

We conducted a phenomenology method to study lived experiences of general medical students through selecting participants via a purposeful sampling strategy. Snowballing and maximum variation samplings were also applied to recruit additional participants. The study was conducted at a Medical School in Iran. Three successive phases of qualitative data analysis, namely, data reduction by coding, data structuring by categorization, and data interpretation by discussion were applied to analyze the interviews.

Results

The results of the research showed that students' lived experiences fall under 4 main themes with 9 subthemes. The main themes show that (1) the compartmentalized curriculum in basic courses is experienced as the missing parts in a puzzle, (2) the physiopathology curriculum is experienced as swimming on land, (3) the externship is experienced as touring a mysterious land, (4) the internship is experienced as unleashed arrows.

Discussion

Our findings reveal that despite the changes already made in the curriculum, its compartmentalization is still a main obstacle to achieving competency-based medical education. A strict requirement for leaving the discipline-based curriculum behind is to use an integrated approach, in which basic science courses are connected with clinical cases, and physiopathology courses are connected with externships and internships.

Co-designing Entrustable Professional Activities in General Practitioner's training: a participatory research study

BACKGROUND

In medical education, Entrustable Professional Activities (EPAs) have been gaining momentum for the last decade. Such novel educational interventions necessitate accommodating competing needs, those of curriculum designers, and those of users in practice, in order to be successfully implemented.

METHODS

We employed a participatory research design, engaging diverse stakeholders in designing an EPA framework. This iterative approach allowed for continuous refinement, shaping a comprehensive blueprint comprising 60 EPAs. Our approach involved two iterative cycles. In the first cycle, we utilized a modified-Delphi methodology with clinical competence committee (CCC) members, asking them whether each EPA should be included. In the second cycle, we used semi-structured interviews with General Practitioner (GP) trainers and trainees to explore their perceptions about the framework and refine it accordingly.

RESULTS

During the first cycle, 14 CCC members agreed that all the 60 EPAs should be included in the framework. Regarding the formulation of each EPAs, 20 comments were given and 16 adaptations were made to enhance clarity. In the second cycle, the semi-structured interviews with trainers and trainees echoed the same findings, emphasizing the need of the EPA framework for improving workplace-based assessment, and its relevance to real-world clinical scenarios. However, trainees and trainers expressed concerns regarding implementation challenges, such as the large number of EPAs to be assessed, and perception of EPAs as potentially high-stakes.

CONCLUSION

Accommodating competing stakeholders' needs during the design process can significantly enhance the EPA implementation. Recognizing users as experts in their own experiences empowers them, enabling a priori identification of implementation barriers and potential pitfalls. By embracing a collaborative approach, wherein diverse stakeholders contribute their unique viewpoints, we can only create effective educational interventions to complex assessment challenges.

Enabling Factors for the Implementation of Competency-Based Curricula in Colleges and Schools of Pharmacy

OBJECTIVES

To review the implementation drivers of competency-based pharmacy education (CBPE) and provide recommendations for enablers.

FINDINGS

Competency-based education is an emerging model in the health professions, focusing on time-variable competency development and achievement compared with a time-bound, course-based, traditional model. CBPE is an outcomes-based organized framework of competencies enabling pharmacists to meet health care and societal needs. However, challenges need to be recognized and overcome for the successful implementation of CBPE. Competency drivers include defining the competencies and roles of stakeholders, developing transparent learning trajectories and aligned assessments, and establishing lifetime development programs for stakeholders. Organization drivers include developing support systems for stakeholders; facilitating connections between all educational experiences; and having transparent assessment plans, policies, and procedures that align with core CBPE precepts, including the sustainability of time-variability. Leadership drivers include establishing growth mindset and facilitating a culture of connection between workplace and educational environments, program advocacy by institutional leaders, accepting failures as part of the process, shifting the organizational culture away from learner differentiation toward competence, and maintaining sufficient administrative capability to support CBPE.

SUMMARY

The successful implementation of CBPE involves enabling the competency, organization, and leadership drivers that will lead to program success. More research is needed in the areas of creation, implementation, and assessment of CBPE to determine success in this model. We have reviewed and provided recommendations to enable the drivers of successful implementation of CBPE.

Analysis of ten-year teaching evaluation of oral microbiology lab curriculum

BACKGROUND

Based on the updated teaching philosophy of oral microbiology, Wuhan University School of Stomatology initiated a reform in the teaching of oral microbiology in 2009. As part of this reform, an oral microbiology laboratory course was introduced to cultivate students' fundamental skills, professional competence, comprehensive abilities, and innovation capabilities through experimental design. This paper provides thorough examination of the teaching experiment findings from 2013 to 2022, a ten-year timeframe, building on earlier data.

METHODS

The curriculum targets fourth-year undergraduate students in a five-year program and adopts a cooperative learning approach. The experimental teaching mainly involves four parts: plaque collection and processing, isolation and cultivation of dental plaque bacteria, staining and biochemical identification of dental plaque bacteria. This article presents a comprehensive analysis of the student experiment results from 2013 to 2022. Statistical analysis was conducted using the chi-square test to assess whether there were any differences in students' experimental grades between different years. A significance level of P < 0.05 was considered statistically significant. Additionally, we evaluated the impact of teaching methods and educational systems on improving students' practical skills and overall innovative abilities.

RESULTS

The performance of 664 undergraduate students showed improvement in the oral microbiology laboratory course, with a noticeable decrease in the proportion of "C" grades in Experiments 2, 3, and 4 compared to Experiment 1. These results indicate that the laboratory course enhanced students' academic achievements, aiding their understanding and mastery of course content, and received positive feedback from the students.

CONCLUSION

This lab curriculum, through systematic laboratory teaching and practical experience, contributes to the enhancement of students' professional skills and research abilities. It fosters students' interest in scientific research and improves the quality of dental education.

Designing the Future of Medical Education: The EPA Framework as a Catalyst to Inform Family Planning and Reproductive Health Fellowship Training Program in Ethiopia Medical Education: An Exploratory Sequential Mixed Method Study

Introduction

Entrustable Professional Activities (EPAs) are tasks or responsibilities within a specific field that can be given to a learner once they are competent to perform them independently. EPAs are being used in various specialty programs and serving as valuable tool to inform educational program. However, due to disparities in professional practice between different contexts, the automatic transfer of a set of core EPAs is not feasible. Hence, our study aims to develop an EPA framework to inform the Family Planning and Reproductive Health Fellowship Program in the local context of Ethiopia.

Methods

We employed an exploratory mixed-method design, which involved the collection of qualitative data using the Nominal Group Technique and quantitative data through a nationwide survey in all residency training institutions across the country. Qualitative data analysis involved several steps, including compiling a list of tasks, removing duplicate tasks, reviewing EPAs using criteria and an equal rubric tool. For quantitative data analysis, descriptive statistics, validity index analysis, and intra-class correlation coefficients, were used.

Results

Seven senior panelists were able to propose a total of 57 EPAs, with 17 remaining after qualitative data analysis. The panelist evaluated the relevance of each EPA in the second phase. As a result, 17 EPAs received a content validity index of >0.83, indicating satisfactory relevance. In the national survey, experts reached a high level of final agreement regarding the relevance and representativeness of all 17 EPAs (ICC = 0.815, 95% CI [0.0.756,0.865], p.0001).

Conclusion

The final set of 17 end-of-training EPAs is valid, acceptable and representative of the discipline, and they can be used as a framework to inform Family planning and Reproductive Health Fellowship Program in Ethiopian medical education once these core EPA statements are described in sufficient detail. This can contribute to raise the quality of training and hence the quality of patient care.

Simulation-based medical education in Canadian anesthesiology academic institutions: a national survey

PURPOSE

Simulation-based medical education (SBME) is provided by all anesthesiology residency programs in Canada. The purpose of this study was to characterize SBME in Canadian anesthesiology residency training programs.

METHODS

We administered a 21-question survey to the simulation director/coordinator for all 17 Canadian academic departments of anesthesiology from October 2019 to January 2020. The survey consisted of questions pertaining to the characteristics of the simulation centres, their faculty, learners, curriculum, and assessment processes.

RESULTS

All 17 residency training programs participated in the survey and reported large variability in the number and formal training of simulation faculty and in content delivery. Five programs (29%) did not provide faculty recognition for curriculum design and running simulation sessions. Most programs offered one to four simulation sessions per academic year for each year of residency. All programs offered mannequin-based and part-task trainers for teaching technical and nontechnical skills. Fourteen programs (82%) offered interprofessional and interdisciplinary simulation sessions, and ten programs (59%) did not include in situ simulation training. Commonly reported barriers to faculty involvement were lack of protected time (12 programs, 71%), lack of financial compensation (ten programs, 59%), and lack of appreciation for SBME (seven programs, 41%).

CONCLUSION

Large variability exists in the delivery of SBME in Canadian anesthesiology residency simulation programs, in part because of differences in financial/human resources and educational content. Future studies should explore whether training and patient outcomes differ between SBME programs and, if so, whether additional standardization is warranted.

Communities of practice to facilitate change in health professions education: A realist synthesis

BACKGROUND

Communities of practice could contribute to transformations in health professions education to meet complex and emerging challenges. However, little is known about the underlying mechanisms of communities of practice in this setting, and how context influences outcomes.

OBJECTIVE

To understand when, why and how communities of practice with health professions education faculty work to facilitate higher education change.

DESIGN

A realist synthesis according to the RAMESES standards and steps described by Pawson and colleagues.

REVIEW METHODS

Early scoping of the literature informed the development of an initial program theory to describe underlying assumptions about how communities of practice in higher education, implemented with health professions education faculty, were likely to work. The theory was tested and further refined through a realist synthesis. A systematic search for evidence using search terms 'faculty', 'communities of practice' and 'higher education' and related terms was supplemented with citation tracking and hand searching of significant authors and journals. Following study appraisal, data were extracted and synthesised from 21 manuscripts describing 16 communities of practice. The realist synthesis focused on identifying patterns in context-mechanism-outcome interactions, and the alignment with substantive theory.

RESULTS

From the included manuscripts, ten context-mechanism-outcome configurations were identified that describe a range of individual, interpersonal and institutional outcomes of communities of practice with health professions education faculty and context-mechanism interactions that contribute to achieving these outcomes.

CONCLUSIONS

This study expands theoretical understandings of how and why communities of practice work. There is value in communities of practice in the higher education sector, primarily in the field of health professions education. Communities of practice implemented in the context of complex change with participants who have a desire to participate can facilitate change in health professions education, including institutional level changes, through reflection, experiential learning and creating a shared agenda for change. Findings from this study can be used by policy and decision-makers within health education to best apply communities of practice to achieve meaningful outcomes.

Description of the methodology for developing and validating the WHO's family planning and comprehensive abortion care competencies for the primary health care workforce

A qualified health workforce is essential to receiving effective, timely, affordable, equitable and respectful family planning and comprehensive abortion care. However, in many countries, health workers lack the competencies required to deliver quality family planning and comprehensive abortion care services. Competency-based education and learning aims to train and assess competencies. The theory-supported approach focuses on outcomes, emphasizes the learner's ability to perform, promotes learner-centeredness and links the health needs of the population to the competencies required of health workers. In 2011, the World Health Organization published a guidance document, Sexual and reproductive health - Core competencies in primary care, defining the competencies that primary care providers need to safely deliver sexual and reproductive health services at the community level and included family planning and comprehensive abortion care. In this article, we describe the methodology and process undertaken in 2020, by the World Health Organization to produce the family planning and comprehensive abortion care competencies guidance, filling gaps identified in the previous guidance document. The World Health Organization's Family Planning and Comprehensive Abortion Care toolkit for the primary health care workforce was published in 2022 and defines the key competencies for health workers in primary health care providing quality family planning and comprehensive abortion care services, as well as support for developing programmes and curricula for education and lifelong learning. The Toolkit is useful for practitioners, managers/supervisors and employers, educators, regulatory bodies, and policymakers. It is an important advance toward strengthening family planning and comprehensive abortion care services in primary health care.

Nurses retrospective view on nursing education: A repeated cross-sectional study over three decades

Background

As the healthcare landscape undergoes transformative shifts due to factors like ageing demographics, technological innovations, rapid global dissemination of infectious diseases, and imperatives for accessible, cost-effective care, a pressing need emerges for the contemporisation of nursing education. Notably, there is a paucity of research delving into nurses' introspective evaluations of their educational experiences after their immersion in professional settings.

Objective

This study aimed to examine nurses' evaluations of their educational background over 30 years and identify relationships between their assessments and their demographic.

Design

The study embraced a recurrent cross-sectional survey methodology, encompassing three distinct quantitative cross-sectional evaluations conducted in the years 1999, 2009, and 2021.

Context/participants

The cohort for this inquiry consisted of nurses stationed in Estonian general hospitals and inpatient departments of developmental plan institutions, each with at least one year of professional experience. Cumulatively, 832 nurses were engaged across three sequential evaluations: Study I (n = 463), Study II (n = 198), and Study III (n = 171).

Methods

Data procurement was executed via a structured survey, with subsequent analytical procedures encompassing descriptive and correlational methodologies.

Results

A discernible augmentation in the educational calibre of nurses was observed with each successive evaluation. This escalation concomitated enhanced positive assessments in areas like evidence-informed education, skill development, and autonomous operational capabilities. Yet, a critical appraisal persisted concerning their competencies in navigating complex patient interactions and addressing socio-religious dilemmas.

Conclusions

The merit of this investigation lies in its illumination of nursing education's evolution, as perceived retrospectively by nurses who have operationalized their academic learnings in real-world scenarios. Their vantage point, inherently informed by practice, uniquely positions them to earmark avenues of refinement. This exploration paves the way for enrichments in nursing education, spotlighting the imperative of equipping nurses to adeptly manage intricate situations.

Evaluating Competence by Design as a Large System Change Initiative: Readiness, Fidelity, and Outcomes

Program evaluation is an essential, but often neglected, activity in any transformational educational change. Competence by Design was a large-scale change initiative to implement a competency-based time-variable educational system in Canadian postgraduate medical education. A program evaluation strategy was an integral part of the build and implementation plan for CBD from the beginning, providing insights into implementation progress, challenges, unexpected outcomes, and impact. The Competence by Design program evaluation strategy was built upon a logic model and three pillars of evaluation: readiness to implement, fidelity and integrity of implementation, and outcomes of implementation. The program evaluation strategy harvested from both internally driven studies and those performed by partners and invested others. A dashboard for the program evaluation strategy was created to transparently display a real-time view of Competence by Design implementation and facilitate continuous adaptation and improvement. The findings of the program evaluation for Competence by Design drove changes to all aspects of the Competence by Design implementation, aided engagement of partners, supported change management, and deepened our understanding of the journey required for transformational educational change in a complex national postgraduate medical education system. The program evaluation strategy for Competence by Design provides a framework for program evaluation for any large-scale change in health professions education.

Examining Competency-Based Education Through the Lens of Implementation Science: A Scoping Review

OBJECTIVES

This study aimed to identify evidence for the implementation and assessment of competency-based education (CBE) in health professions curricula using an implementation science framework.

FINDINGS

Using the PRISMA framework, a systematic review of the literature applying a prespecified and piloted search strategy from 2017 to the present in PubMed and CINAHL was performed. References identified from the search strategy were imported into Covidence for title and abstract screening and full-text review by 2 researchers. A third researcher resolved discrepancies. Data were extracted and synthesized to identify key elements from the article related to implementation science, with a quality appraisal. A total of 25 studies out of 304 initially identified records were included. The studies covered a broad range of health professions and countries. Key findings were limited use of implementation science elements, including variability in CBE implementation, limited fidelity assessment, and partial examination of the process continuum. Programs with a more robust implementation approach have a team-based strategy to lead, implement, and support CBE. Motivation and training of faculty are also key components of successful CBE implementation.

SUMMARY

Competency-based education is implemented differently across institutions, with variation among programs in their choice of elements of implementation science used. Further research is needed to examine CBE from an implementation science perspective and address remaining questions.

Case Curve: A Novel Web-Based Platform and Mobile Phone Application to Evaluate Surgical Competence in Graduate Medical Education

BACKGROUND AND OBJECTIVES

Competency-based medical education is well established, yet methods to evaluate and document acquisition of surgical skill remain underdeveloped. We describe a novel web-based application for competency-based surgical education at a single neurosurgical department over a 3-year period.

METHODS

We used a web-based application to track procedural and cognitive skills acquisition for neurosurgical residents. This process included self-assessment, resident peer evaluations, evaluations from supervising attending physicians, and blinded video reviews. Direct observation by faculty and video recordings were used to evaluate surgical skill. Cases were subdivided into component skills, which were evaluated using a 5-point scale. The learning curve for each skill was continuously updated and compared with expectations. Progress was reviewed at a monthly surgical skills conference that involved discussion and analysis of recorded surgical performances.

RESULTS

During an escalating 3-year pilot from 2019 to 2022, a total of 1078 cases in 39 categories were accrued by 17 resident physicians with 10 neurosurgical faculty who participated as evaluators. A total of 16 251 skill performances in 110 categories were evaluated. The most evaluated skills were those that were common to several types of procedures, such as positioning, hemostasis, and wound closure. The concordance between attending evaluations and resident self-evaluations was 76%, with residents underestimating their performance in 17% of evaluations and overestimating in 7%.

CONCLUSION

We developed a method for evaluating and tracking surgical resident skill performance with an application that provides timely and actionable feedback. The data collected from this system could allow more accurate assessments of surgical skills and deeper insights into factors influencing surgical skill acquisition.

Evaluating Feedback Comments in Entrustable Professional Activities: A Cross-Sectional Study

INTRODUCTION

Competency-based medical education (CBME) has transformed postgraduate medical training, prioritizing competency acquisition over traditional time-based curricula. Integral to CBME are Entrustable Professional Activities (EPAs), that aim to provide high-quality feedback for trainee development. Despite its importance, the quality of feedback within EPAs remains underexplored.

METHODS

We employed a cross-sectional study to explore feedback quality within EPAs, and to examine factors influencing length of written comments and their relationship to quality. We collected and analyzed 1163 written feedback comments using the Quality of Assessment for Learning (QuAL) score. The QuAL aims to evaluate written feedback from low-stakes workplace assessments, based on 3 quality criteria (evidence, suggestion, connection). Afterwards, we performed correlation and regression analyses to examine factors influencing feedback length and quality.

RESULTS

EPAs facilitated high-quality written feedback, with a significant proportion of comments meeting quality criteria. Task-oriented and actionable feedback was prevalent, enhancing value of low-stakes workplace assessments. From the statistical analyses, the type of assessment tool significantly influenced feedback length and quality, implicating that direct and video observations can yield superior feedback in comparison to case-based discussions. However, no correlation between entrustment scores and feedback quality was found, suggesting potential discrepancies between the feedback and the score on the entrustability scale.

CONCLUSION

This study indicates the role of the EPAs to foster high-quality feedback within CBME. It also highlights the multifaceted feedback dynamics, suggesting the influence of factors such as feedback length and assessment tool on feedback quality. Future research should further explore contextual factors for enhancing medical education practices.

Competency-based pharmacy education definition: What components need to be defined to implement it?

OBJECTIVES

This study aimed to define the essential elements in the proposed competency-based pharmacy education (CBPE) definition, provide the key defining components of each essential element on the basis of educational theory and evidence, and define how the essential elements meet the identified needs for CBPE.

METHODS

best-practice integrative review was conducted as part of the work of the American Association of Colleges of Pharmacy CBPE Task Force to define the essential elements in the CBPE definition and how these elements fit with the need for CBPE. The definition was compared with other published competency-based education definitions across K-12, higher education, medical education, and veterinary education. Task Force members then met to develop a consensus on the core components of the 5 essential elements in the definition. Next, the Task Force evaluated the fit of CBPE by matching the identified needs, discussed in detail elsewhere, across each of the stakeholder perspectives with the core components of the 5 essential elements in the derived definition of CBPE.

FINDINGS

Upon review of the proposed CBPE definition, the Task Force identified 5 essential elements. These elements include the following: meeting health care and societal needs, outcomes-based curricular model, de-emphasized time, learner-centered culture, and authentic teaching and learning strategies aligned to assessments.

SUMMARY

This article helps to establish a common language for CBPE by defining the essential elements of the core components of the definition, and provides a starting point for further exploration of CBPE.

Entrustable professional activities of graduate accredited General Medical Sonographers in Australia - Industry perceptions

INTRODUCTION

Linking individual competencies to entrustable professional tasks provides a holistic view of Sonography graduate work readiness. The Australian Sonographers Accreditation Registry (ASAR) publishes a set of entrustable professional activities (EPAs) as part of its Standards for Accreditation of Sonography Courses. EPAs are distinct ultrasound examinations grouped within six critical practice units. This study reports on industry perspectives of current EPAs and their classification for graduates completing general sonography courses in Australia. The article also examines the value of EPAs and links their function to the assessment of graduate competency.

METHODS

An online survey tool elicited stakeholder feedback on graduate EPAs across six critical practice units and the potential for including a new Paediatric unit. From an original sample size of 655, 309 responded to questions about general sonography courses.

RESULTS

A majority (55.3%) recommended no changes to the existing EPA list, and 44.7% recommended amending the list. From respondents that recommended changes (138/309), all current EPAs received >80% agreement to be retained; in addition, nine new examinations received >70% agreement for inclusion at the graduate level. Whilst 42.7% (132/309) supported the current ASAR model requiring competency in five out of six critical practice units, 45.6% (141/309) recommended increasing it to all six. There was limited support, 11.7% (36/309), to reduce this number. Responding to the potential to add a new Paediatric specific critical practice unit, 61.8% (181/293) recommended its inclusion.

CONCLUSIONS

The findings demonstrate that the current list of EPAs aligns with industry expectations. In contrast, there are divergent views on the modelling and grouping of critical practice units. The article's critical analysis of the results and implications provides stakeholders with a practical approach to clinical teaching and EPA assessment, and helps to inform any review of accreditation standards.

Unintended consequences of technology in competency-based education: a qualitative study of lessons learned in an OtoHNS program

BACKGROUND

Formative feedback and entrustment ratings on assessments of entrustable professional activities (EPAs) are intended to support learner self-regulation and inform entrustment decisions in competency-based medical education. Technology platforms have been developed to facilitate these goals, but little is known about their effects on these new assessment practices. This study investigates how users interacted with an e-portfolio in an OtoHNS surgery program transitioning to a Canadian approach to competency-based assessment, Competence by Design.

METHODS

We employed a sociomaterial perspective on technology and grounded theory methods of iterative data collection and analysis to study this OtoHNS program's use of an e-portfolio for assessment purposes. All residents (n = 14) and competency committee members (n = 7) participated in the study; data included feedback in resident portfolios, observation of use of the e-portfolio in a competency committee meeting, and a focus group with residents to explore how they used the e-portfolio and visualize interfaces that would better meet their needs.

RESULTS

Use of the e-portfolio to document, access, and interpret assessment data was problematic for both residents and faculty, but the residents faced more challenges. While faculty were slowed in making entrustment decisions, formative assessments were not actionable for residents. Workarounds to these barriers resulted in a "numbers game" residents played to acquire EPAs. Themes prioritized needs for searchable, contextual, visual, and mobile aspects of technology design to support use of assessment data for resident learning.

CONCLUSION

Best practices of technology design begin by understanding user needs. Insights from this study support recommendations for improved technology design centred on learner needs to provide OtoHNS residents a more formative experience of competency-based training.

Knowledge and Perceptions of Competency-Based Medical Education in Diagnostic Radiology Post-Graduate Medical Education: Identifying Priorities and Developing a Framework for Professional Development Activities

Introduction: We evaluated knowledge and perceptions of an established Competency-Based Medical Education (CBME) model developed by the Royal College of Physicians and Surgeons of Canada, Competence by Design (CBD), and identified evidence-informed priorities for professional development activities (PDAs). Materials and Methods: Teaching faculty and residents at a single, large diagnostic radiology post-graduate medical education (PGME) program were eligible to participate in this cross-sectional, survey-based study. Knowledge of CBD was evaluated through multiple choice questions (MCQs), which assessed participants' understanding of major principles and terms associated with CBD. Participants' perceptions of the anticipated impact of CBD on resident education and patient care were evaluated and priorities for PDAs were identified, which informed a framework for CBD PDAs. Results: Fifty faculty and residents participated. The faculty and resident response rates were 11.6% (n = 29/249) and 55.3% (n = 21/38), respectively. The mean ± standard deviation overall score on MCQs was 39.0% ± 20.4%. The majority of participants perceived the impact of CBD on resident education to be equivocal and to not impact patient care. Knowledge of CBD was not statistically significantly associated with participants' perceptions of the impact of CBD on either resident education or patient care (P > .05). Delivery of high-quality feedback was the greatest priority identified for PDAs. Discussion: Our results and proposed CBD PDAs framework may help to guide diagnostic radiology PGME programs in designing evidence-informed PDAs, which may meaningfully contribute to the successful implementation of CBD in diagnostic radiology PGME. As diagnostic radiology PGME programs throughout the world increasingly implement CBME models, evidence-informed PDAs will become of increasing importance.

Defining new roles and competencies for administrative staff and faculty in the age of competency-based medical education

PURPOSE

These authors sought to define the new roles and competencies required of administrative staff and faculty in the age of CBME.

METHOD

A modified Delphi process was used to define the new CBME roles and competencies needed by faculty and administrative staff. We invited international experts in CBME (volunteers from the ICBME Collaborative email list), as well as faculty members and trainees identified via social media to help us determine the new competencies required of faculty and administrative staff in the CBME era.

RESULTS

Thirteen new roles were identified. The faculty-specific roles were: National Leader/Facilitator in CBME; Institutional/University lead for CBME; Assessment Process & Systems Designer; Local CBME Leads; CBME-specific Faculty Developers or Trainers; Competence Committee Chair; Competence Committee Faculty Member; Faculty Academic Coach/Advisor or Support Person; Frontline Assessor; Frontline Coach. The staff-specific roles were: Information Technology Lead; CBME Analytics/Data Support; Competence Committee Administrative Assistant.

CONCLUSIONS

The authors present a new set of faculty and staff roles that are relevant to the CBME context. While some of these new roles may be incorporated into existing roles, it may be prudent to examine how best to ensure that all of them are supported within all CBME contexts in some manner.

Competency-Based Medical Education (CBME): an Overview and Relevance to the Education of Future Surgical Oncologists

In the next two decades, the global cancer burden is expected to rise by 47%, and the demand for global cancer surgery will increase by 52%. At present, only 25% of the estimated 80% of patients needing surgical intervention have access to timely surgical care. The shortage of a trained workforce of surgical oncologists is one of the main barriers to providing the optimal surgical intervention needed for cancer patients. Some of the contributing factors to the shortage of trained surgical oncologists are variations in the current global educational platforms, long training programs, and physician burnout. Therefore, the availability of a credible training framework and a sustainable certification pipeline for future surgical oncologists is critical to meet the global demand for an adequate healthcare workforce. The current surgical oncology educational program is a time-based construct that trains surgeons to function seamlessly in the multidisciplinary care of cancer patients. However, there is a lack of flexibility in the training framework and timeline despite differences in trainees' abilities. Developing a globally acceptable standard curriculum for surgical oncology training based on the competency-based medical education (CBME) framework and tailoring it to local needs can increase the surgical oncology workforce ready to tackle the rising cancer burden. However, successful implementation of the global CBME-based surgical oncology training curriculum requires an innovative approach to ensure that this framework produces a competent surgical oncologist that meets the local needs.

A Practical Review of Mastery Learning

Objective. To review mastery learning and provide recommendations for implementation in a competency-based curriculum.Findings. Mastery learning, introduced in the 1960s, was developed to ensure all students reach a desired level of mastery or competency. In this model, students acquire knowledge, skills, or attitudes and then complete formative assessments on that learning. If they achieve the desired level, they can proceed to enrichment activities. Students who do not meet the desired level of mastery proceed through corrective activities and retesting. Evidence suggests students within a mastery learning model perform better academically than those in nonmastery learning models with moderate effect sizes. Mastery learning may result in better performance due to several theoretical reasons, including aspects of motivation, testing, and feedback.Conclusion. We make several recommendations on how to modernize mastery learning for apply it to the pharmacy education, including the recommendation to use more cumulative testing and assessment of baseline knowledge and skills. In addition, models of successive relearning may be applied to this model.

A survey of resident physicians' perceptions of competency-based education in standardized resident training in China: a preliminary study

BACKGROUND

Understanding resident physicians' perceptions of competency-based medical education(CBME) may help improve approaches for implementing such education in standardized resident training (SRT). We conducted surveys of residents in China to identify their perceptions of CBME and determine the degree to which such education impacts their career plans.

METHODS

Questionnaire contained a total of 24 questions, which were answered using multiple choice or yes/no, was distributed to residents who were undergoing SRT, regardless of specialty, at 7 accredited training bases located across six provinces of China. The survey aimed to investigate residents' reasons for participating in SRT, perceptions of CBME, interest in receiving CBME-associated courses, and attitudes towards CBME.

RESULTS

Overall, 441 residents completed the questionnaire.17.7% (78/441) responded "no clear objective" before the participated in SRT. Only 3.9% (17/441) fully understood the objectives, training contents, and assessment system of the current "competency-based" standardized training program for residents in China. Residents ranked clinical skills and patient care, interpersonal communication, and professionalism, as the three most important competencies. Most were interested in the CBME residency programs. 90.7% felt that implementing CBME could help them clarify their professional direction and improve their career planning.

CONCLUSION

Residents had positive perceptions of the incorporation of CBME into SRT. Administrators, educational leaders, and clinical faculty should seek to further publicize and increase the popularity of CBME.

Teaching Practical Skills in Anesthesia, Intensive Care, Emergency and Pain Medicine—What Is Really Relevant for Medical Students? Results of a German National Survey of Nearly 3000 Anesthesiologists

As a part of a major reform of the medical curriculum in Germany, the national catalogue of learning objectives is being revised with the focus shifting from theory-based learning to teaching practical skills. Therefore, we conducted an online survey to answer the question, which practical skills are essential in anesthesia. Participants were asked to rate the relevance of several skills, that medical students should be able to perform at the time of graduation. A total of 2898 questionnaires could be evaluated. The highest ratings were made for “bringing a patient into lateral recumbent position” and “diagnosing a cardiac arrest”. All learning objectives regarding regional anesthesia were rated as irrelevant. Furthermore, learning objectives like “performing a bronchoscopy” or “performing a rapid sequence induction” had low ratings. In the subgroup analysis, physicians with advanced training and those who were working at university hospitals rated most skills with higher relevance compared to others. Our survey provides a good prioritization of practical skills for the development of new curricula and assessment frameworks. The results can also help to establish our discipline as a cross-sectional subject in competency-based medical education, thus further increasing the attractiveness for medical students.

Investigating the Relationship between Multiple Mini-Interview Communication Skills Outcomes and First-Year Communication Skills Performance and Reflections in Students at the Ontario Veterinary College

An important outcome for veterinary education is ensuring that graduates can provide an appropriate level of care to patients and clients by demonstrating core competencies such as communication skills. In addition, accreditation requirements dictate the need to assess learning outcomes and may drive the motivation to incorporate relevant and appropriate methods of entry assessments for incoming students. Predicting the success of Doctor of Veterinary Medicine (DVM) students based on entry assessment performance has been scantly investigated and can be challenging. Specifically, no research presently exists on predicting DVM students' first-year performance in relation to communication skills at the time of program entry. Objectives of this exploratory study were to investigate (a) the relationship between communication skills outcomes from multiple mini-interview (MMI) data and first-year academic performance related to communication and (b) the relationship between communication skills outcomes from MMI data and self-reported first-year communication reflections. A retrospective single-class study was conducted. Data were analyzed using descriptive statistics, correlation statistics, regression models, and paired t-tests to identify relationships among variables. Paired t-tests showed that students felt more prepared to meet second-year expectations over first-year expectations. Spearman's correlation revealed an association between MMI communication scores and one pre-year 1 survey question related to professionalism. Noo relationships were observed between MMI communication scores and marks from a self-reflection assignment in a communications course, or grades from a clinical medicine course that included clinical communication. The merit for further exploration of the relationship between communication competencies and student performance is discussed.

Competency assessment of residents of Intensive Care Medicine through a simulation-based objective structured clinical evaluation (OSCE). A multicenter observational study

OBJECTIVES

The current official model of training in Intensive Care Medicine (ICM) in Spain is based on exposure to experiences through clinical rotations. The main objective was to determine the level of competency (I novice to V independent practitioner) achieved by the residents at the end of the 3rd year of training (R3) in ICM through a simulation-based OSCE. Secondary objectives were: (1) To identify gaps in performance, and (2) To investigate the reliability and feasibility of conducting simulation-based assessment at multiple sites.

DESIGN

Observational multicenter study.

SETTING

Thirteen Spanish ICU Departments.

PARTICIPANTS

Thirty six R3.

INTERVENTION

The participants performed on five, 15-min, high-fidelity crisis scenarios in four simulation centers. The performances were video recorded for later scoring by trained raters.

MAIN VARIABLES OF INTEREST

Via a Delphi technique, an independent panel of expert intensivists identified critical essential performance elements (CEPE) for each scenario to define the levels of competency.

RESULTS

A total of 176 performances were analyzed. The internal consistency of the check-lists were adequate (KR-20 range 0.64-0.79). Inter-rater reliability was strong [median Intraclass Correlation Coefficient across scenarios: 0.89 (0.65-0.97)]. Competency levels achieved by R3 were: Level I (18.8%), II (35.2%), III (42.6%), IV/V (3.4%). Overall, a great heterogeneity in performance was observed.

CONCLUSION

The expected level of competency after one year in the ICU was achieved only in half of the performances. A more evidence-based educational approach is needed. Multiple center simulation-based assessment showed feasibility and reliability as an evaluation method of competency.

TRIAL REGISTRATION

COBALIDATION. NCT04278976. (https://register.

CLINICALTRIALS

gov).

Moving toward co-production: five ways to get a grip on collaborative implementation of Movement Behaviour curricula in undergraduate medical education

Several "calls to action" have imposed upon medical schools to include physical activity content in their overextended curricula. These efforts have often neither considered medical education stakeholders' views nor the full complexity of medical education, such as competency-based learning and educational inflation. With this external pressure for change, few medical schools have implemented physical activity curricula. Moreover, Canada's new 24-Hour Movement Guidelines focus on the continuum of movement behaviours (physical activity, sedentary behaviour, and sleep). Thus, a more integrated process to overcome the "black ice" of targeting all movement behaviours, medical education stakeholder engagement, and the overextended curriculum is needed. We argue for co-production in curriculum change and offer five strategies to integrate movement behaviour curricula that acknowledge the complexity of the medical education context, helping to overcome our "black ice." Our objectives were to investigate 24-Hour Movement Guideline content in the medical curriculum and develop an integrated process for competency-based curriculum renewal. Stakeholders were equal collaborators in a two-phased environmental scan of 24-Hour Movement Guideline content in the Queen's University School of Medicine. Findings and a working curriculum map highlight how new, competency-based content may be embedded in an effort to guide more relevant and feasible curriculum changes.

Re-thinking "I"dentity in medical education: genealogy and the possibilities of being and becoming

Professional identity formation has emerged as a key topic for medical education research, with contributions from perspectives of psychological development and socialization opening up needed conversations in the field. Yet mainstream training practices may have the unintended effects of educating for a physician typology that may be too narrow to account for the complexity of learners' personal identities. Alternative approaches, such as Foucauldian genealogy, offer ways to empirically investigate how the legitimate contours of being and becoming have come to be as they are, how they shape professional identities, and to which degree their borders may be made more inclusive. Drawing upon an example of the contemporary practice of competency-based medical education in the Canadian context, this paper considers how genealogy's methodological tools of critical distancing, the dispositif, and problematization may help reveal how educational practices shape the identities of physicians-in-training in ways both intended and unintended. From this perspective it becomes apparent that any attempt to explore professional identity is incomplete without also considering that a trainee's evolving sense of self is inexorably bound up with forces of knowledge, power, and ethics that shape them into becoming certain kinds of physician subjects rather than others. In mapping this terrain, a genealogical approach determines how we reached the now in which we find ourselves and how we might transform it, such that we may shift the possibilities afforded to health professionals to establish professional identities aligned with their personal identities in ways that maximize inclusivity and minimize marginalization.

Training to Support ePortfolio Users During Clinical Placements: a Scoping Review

ePortfolios are frequently used to support students' competency development, and teachers' and clinical mentors' supervision during clinical placements. User training is considered a critical success factor for the implementation of these ePortfolios. However, there is ambiguity about the design and outcomes of ePortfolio user training. A scoping review was conducted to consolidate evidence from studies describing the design of ePortfolio user training initiatives and their outcomes. The search yielded 1180 articles of which 16 were included in this review. Based on the results, an individual, ongoing training approach which grounds in a fitting theoretical framework is recommended.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s40670-022-01583-0.

Core competencies for a biomedical laboratory scientist - a Delphi study

BACKGROUND

After completing university education, biomedical laboratory scientists work in clinical laboratories, in biomedical research laboratories, in biotech, and in pharmaceutical companies. Laboratory diagnostics have undergone rapid development over the recent years, with the pace showing no signs of abatement. This rapid development challenges the competence of the staff and will most certainly influence the education of future staff. This study aimed to examine what was considered the necessary competencies needed to pursue a career as a biomedical laboratory scientist.

METHODS

A modified Delphi technique was used, with the panel of experts expressing their views in a series of three questionnaire. Consensus was defined as the point which 75 % or more of the panel participants agreed that a particular competency was necessary.

RESULTS

The study highlights the perceived importance of mostly generic competencies that relate to quality, quality assurance, and accuracy, as well as different aspects of safety, respect, trustworthiness (towards patients/clients and colleagues), and communication skills. The results also stress the significance of self-awareness and professionality.

CONCLUSIONS

We identified important competencies for biomedical laboratory scientists. Together with complementary information from other sources, i.e., guidelines, laws, and scientific publications, the competencies identified can be used as learning outcomes in a competency-based education to provide students with all the competencies needed to work as professional biomedical laboratory scientists.

Monitoring the integrity and usability of policy evaluation tools within an evolving sociocultural context: A demonstration of reflexivity using the CFPC Family Medicine Longitudinal Survey

RATIONALE, AIMS, AND OBJECTIVES

Over the last decade, policy changes have prompted Canadian medical education to emphasize a transformation to competency-based education, and subsequent development of evaluation tools. The pandemic provides a unique opportunity to emphasize the value of reflexive monitoring, a cyclical and iterative process of appraisal and adaptation, since tools are influenced by social and cultural factors relevant at the time of their development.

METHODS

Deductive content analysis of documents and resources about the advancement of primary care. Reflexive monitoring of the Family Medicine Longitudinal Survey (FMLS), an evaluation tool for physician training.

RESULTS

The FMLS tool does not explore all training experiences that are currently relevant; including, incorporating technology, infection control and safety, public health services referrals, patient preferences for care modality, and trauma-informed culturally safe care.

CONCLUSION

The results illustrate that reflection promotes the validity and usefulness of the data collected to inform policy performance and other initiatives.

Crossing the Virtual Chasm: Practical Considerations for Rethinking Curriculum, Competency, and Culture in the Virtual Care Era

Virtual care, introduced previously as a replacement for in-person visits, is now being integrated into clinical care delivery models to complement in-person visits. The COVID-19 pandemic sped up this process. The rapid uptake of virtual care at the start of the pandemic prevented educators from taking deliberate steps to design the foundational elements of the related learning environment, including workflow, competencies, and assessment methods. Educators must now pursue an informed and purposeful approach to design a curriculum and implement virtual care in the learning environment. Engaging learners in virtual care offers opportunities for novel ways to teach and assess their performance and to effectively integrate technology such that it is accessible and equitable. It also offers opportunities for learners to demonstrate professionalism in a virtual environment, to obtain a patient's history incorporating interpersonal and communication skills, to interact with multiple parties during a patient encounter (patient, caregiver, translator, telepresenter, faculty member), to enhance physical examination techniques via videoconferencing, and ideally to optimize demonstrations of empathy through "webside manner." Feedback and assessment, important features of training in any setting, must be timely, specific, and actionable in the new virtual care environment. Recognizing the importance of integrating virtual care into education, leaders from across the United States convened on September 10, 2020, for a symposium titled, "Crossing the Virtual Chasm: Rethinking Curriculum, Competency, and Culture in the Virtual Care Era." In this article, the authors share recommendations that came out of this symposium for the implementation of educational tools in the evolving virtual care environment. They present core competencies, assessment tools, precepting workflows, and technology to optimize the delivery of high-quality virtual care that is safe, timely, effective, efficient, equitable, and patient-centered.

The Senior Medical Resident's New Role in Assessment in Internal Medicine

PURPOSE

With the introduction of competency-based medical education, senior residents have taken on a new, formalized role of completing assessments of their junior colleagues. However, no prior studies have explored the role of near-peer assessment within the context of entrustable professional activities (EPAs) and competency-based medical education. This study explored internal medicine residents' perceptions of near-peer feedback and assessment in the context of EPAs.

METHOD

Semistructured interviews were conducted from September 2019 to March 2020 with 16 internal medicine residents (8 first-year residents and 8 second- and third-year residents) at the University of Toronto, Toronto, Ontario, Canada. Interviews were conducted and coded iteratively within a constructivist grounded theory approach until sufficiency was reached.

RESULTS

Senior residents noted a tension in their dual roles of coach and assessor when completing EPAs. Senior residents managed the relationship with junior residents to not upset the learner and potentially harm the team dynamic, leading to the documentation of often inflated EPA ratings. Junior residents found senior residents to be credible providers of feedback; however, they were reticent to find senior residents credible as assessors.

CONCLUSIONS

Although EPAs have formalized moments of feedback, senior residents struggled to include constructive feedback comments, all while knowing the assessment decisions may inform the overall summative decision of their peers. As a result, EPA ratings were often inflated. The utility of having senior residents serve as assessors needs to be reexamined because there is concern that this new role has taken away the benefits of having a senior resident act solely as a coach.

Core procedural skills competencies and the maintenance of procedural skills for medical students: a Delphi study

BACKGROUND

It is well recognised that medical students need to acquire certain procedural skills during their medical training, however, agreement on the level and acquisition of competency to be achieved in these skills is under debate. Further, the maintenance of competency of procedural skills across medical curricula is often not considered. The purpose of this study was to identify core procedural skills competencies for Australian medical students and to establish the importance of the maintenance of such skills.

METHODS

A three-round, online Delphi method was used to identify consensus on competencies of procedural skills for graduating medical students in Australia. In Round 1, an initial structured questionnaire was developed using content identified from the literature. Respondents were thirty-six experts representing medical education and multidisciplinary clinicians involved with medical students undertaking procedural skills, invited to rate their agreement on the inclusion of teaching 74 procedural skills and 11 suggested additional procedures. In Round 2, experts re-appraised the importance of 85 skills and rated the importance of maintenance of competency (i.e., Not at all important to Extremely important). In Round 3, experts rated the level of maintenance of competence (i.e., Observer, Novice, Competent, Proficient) in 46 procedures achieving consensus.

RESULTS

Consensus, defined as > 80% agreement, was established with 46 procedural skills across ten categories: cardiovascular, diagnostic/measurement, gastrointestinal, injections/intravenous, ophthalmic/ENT, respiratory, surgical, trauma, women's health and urogenital procedures. The procedural skills that established consensus with the highest level of agreement included cardiopulmonary resuscitation, airway management, asepsis and surgical scrub, gown and gloving. The importance for medical students to demonstrate maintenance of competency in all procedural skills was assessed on the 6-point Likert scale with a mean of 5.03.

CONCLUSIONS

The findings from the Delphi study provide critical information about procedural skills for the Clinical Practice domain of Australian medical curricula. The inclusion of experts from medical faculty and clinicians enabled opportunities to capture a range of experience independent of medical speciality. These findings demonstrate the importance of maintenance of competency of procedural skills and provides the groundwork for further investigations into monitoring medical students' skills prior to graduation.

Assessing changes in the quality of quantitative health educations research: a perspective from communities of practice

BACKGROUND

As a community of practice (CoP), medical education depends on its research literature to communicate new knowledge, examine alternative perspectives, and share methodological innovations. As a key route of communication, the medical education CoP must be concerned about the rigor and validity of its research literature, but prior studies have suggested the need to improve medical education research quality. Of concern in the present study is the question of how responsive the medical education research literature is to changes in the CoP. We examine the nature and extent of changes in the quality of medical education research over a decade, using a widely cited study of research quality in the medical education research literature as a benchmark to compare more recent quality indicators.

METHODS

A bibliometric analysis was conducted to examine the methodologic quality of quantitative medical education research studies published in 13 selected journals from September 2013 to December 2014. Quality scores were calculated for 482 medical education studies using a 10-item Medical Education Research Study Quality Instrument (MERSQI) that has demonstrated strong validity evidence. These data were compared with data from the original study for the same journals in the period September 2002 to December 2003. Eleven investigators representing 6 academic medical centers reviewed and scored the research studies that met inclusion and exclusion criteria. Primary outcome measures include MERSQI quality indicators for 6 domains: study design, sampling, type of data, validity, data analysis, and outcomes.

RESULTS

There were statistically significant improvements in four sub-domain measures: study design, type of data, validity and outcomes. There were no changes in sampling quality or the appropriateness of data analysis methods. There was a small but significant increase in the use of patient outcomes in these studies.

CONCLUSIONS

Overall, we judge this as equivocal evidence for the responsiveness of the research literature to changes in the medical education CoP. This study identified areas of strength as well as opportunities for continued development of medical education research.

Rheumatology Milestones 2.0: A Roadmap for Competency-Based Medical Training of Rheumatology Fellows in the 21st Century

OBJECTIVE

Since 2014, rheumatology fellows have been assessed not only based on their ability to provide patient care and possession of medical knowledge but also on their skill in serving as patient advocates, navigators of health systems, and members of a health care team. Such assessments have been carried out through the use of competency-based milestones from the Accreditation Council of Graduate Medical Education (ACGME). However, a needs assessment has demonstrated interest in more context validity and subspecialty relevance since the development of the ACGME internal medicine (IM) subspecialty reporting milestones. The ACGME thus created a milestones working group, and the present study was undertaken to develop Rheumatology Milestones 2.0 as well as a supplemental guide to assist with implementation.

METHODS

The working group, consisting of 7 rheumatology program directors, 2 division directors, a community practice rheumatologist, a rheumatology fellow in training, and a public member who is a rheumatology patient, was overseen by the ACGME vice president for milestones development and met through three 12-hour, in-person meetings to compose the rheumatology specialty milestones and supplemental guide within the ACGME Milestones 2.0 project.

RESULTS

Informed by the needs assessment data and stakeholders, the working group revised and adapted the ACGME IM subspecialty reporting milestones to create a rheumatology-specific set of milestones and a supplemental guide for their implementation.

CONCLUSION

The Rheumatology Milestones 2.0 provides a specialty-specific, competency-based evaluation tool that can be used by program directors, clinical competency committees, and others to assess the competencies of rheumatology fellows during training and help measure readiness for independent practice.

¿Se deben confiar actividades profesionales en cirugía mínimamente invasiva al médico general? Una propuesta basada en el aprendizaje experiencial

Introducción. El entrenamiento en cirugía mínimamente invasiva y la definición de las actividades profesionales específicas, socialmente responsables y seguras para el paciente, son un desafío y un deber en la educación médica. Nuestro objetivo es argumentar a favor de esta intervención y describir las especificaciones, alcances y limitaciones de las actividades profesionales confiables a este nivel, así como su lugar en un modelo de aprendizaje experiencial en cirugía mínimamente invasiva para toda la vida. Métodos. Se evalúa el contexto de la actividad quirúrgica asistencial de los médicos generales, con relación a su participación en el equipo quirúrgico y la necesidad de incorporación en su perfil profesional de unas competencias propias de la cirugía mínimamente invasiva, acorde con su nivel y funciones. Mediante una postura académica y reflexiva, se identifican vacíos y oportunidades de avanzar en el tema. Resultados. El perfil profesional de un médico general debe contemplar la cirugía mínimamente invasiva como una competencia dentro de su formación, de manera análoga a las propias de la cirugía tradicional abierta. Para ello, se requiere afrontar coherentemente el cambio de teoría y educación quirúrgica, el cambio en la demanda de servicios quirúrgicos y procedimientos, así como fortalecer el rol activo del médico general en el equipo quirúrgico. Conclusiones. Es necesario modificar el paradigma educativo en cirugía, desde la formación médica en el pregrado. Se plantea un modelo de aprendizaje experiencial de cirugía mínimamente invasiva, y se establecen las competencias fundamentadas en actividades profesionales confiables, necesarias para el perfil profesional de un médico general del siglo XXI.

Video-Assisted Peer Teaching for Surgical Skills Training - Innovative Potential for the Medical Curriculum and Beyond: A Randomized Controlled Trial

OBJECTIVE

Hygienic healthcare standards are essential for avoiding hospital infections. However, medical students and staff lack training in this field, which may be due to high personnel resources of present educational approaches. Thus, there is an urgent need for a novel and efficient approach. Aim of the study is to compare a newly developed video-assisted peer feedback (VAPF) method for teaching wound dressings to the traditional teaching method with qualified instructor feedback (QIF) with respect to essential learning outcomes.

DESIGN, SETTING AND PARTICIPANTS

In this randomized controlled noninferiority trial, 251 medical undergraduates were randomly assigned to one of two interventions (QIF n = 127; VAPF n = 124). In QIF, participants received feedback from a qualified instructor. In VAPF, participants video-recorded each other while performing a wound dressing and gave each other feedback assisted by a standardized checklist. Outcome measures were participants' score in an objective structured practical examination (OSPE) and a written exam after the course.

RESULTS

Noninferiority of VAPF (n = 123) compared to QIF (n = 127) was confirmed for both OSPE (QIF: 8.83 ± 1.30; VAPF: 8.88 ± 1.04; mean difference -0.04, 95% CI -0.34 to 0.25) and written exam (QIF: 8.99 ± 1.06; VAPF: 9.14 ± 1.05; mean difference -0.15, 95% CI -0.41 to 0.12).

CONCLUSIONS

VAPF is a cost-efficient and viable alternative to QIF commonly used in medical education. It provides comparable training outcomes to the traditional training method with lower personnel investment. VAPF is a promising educational method for improving essential clinical competencies.

Competency-Based Medical Education: Considering Its Past, Present, and a Post-COVID-19 Era

Advancement toward competency-based medical education (CBME) has been hindered by inertia and a myriad of implementation challenges, including those associated with assessment of competency, accreditation/regulation, and logistical considerations. The COVID-19 pandemic disrupted medical education at every level. Time-in-training sometimes was shortened or significantly altered and there were reductions in the number and variety of clinical exposures. These and other unanticipated changes to existing models highlighted the need to advance the core principles of CBME. This manuscript describes the impact of COVID-19 on the ongoing transition to CBME, including the effects on training, curricular, and assessment processes for medical school and graduate medical education programs. The authors outline consequences of the COVID-19 disruption on learner training and assessment of competency, such as conversion to virtual learning modalities in medical school, redeployment of residents within health systems, and early graduation of trainees based on achievement of competency. Finally, the authors reflect on what the COVID-19 pandemic taught them about realization of CBME as the medical education community looks forward to a postpandemic future.

The ASK-SEAT: a competency-based assessment scale for students majoring in clinical medicine

BACKGROUND

To validate a competency-based assessment scale for students majoring in clinical medicine, ASK-SEAT. Students' competency growth across grade years was also examined for trends and gaps.

METHODS

Questionnaires were distributed online from May through August in 2018 to Year-2 to Year-6 students who majored in clinical medicine at the Shantou University Medical College (China). Cronbach alpha values were calculated for reliability of the scale, and exploratory factor analysis employed for structural validity. Predictive validity was explored by correlating Year-4 students' self-assessed competency ratings with their licensing examination scores (based on Kendall's tau-b values). All students' competency development over time was examined using the Mann-Whitney U test.

RESULTS

A total of 760 questionnaires meeting the inclusion criteria were analyzed. The overall Cronbach's alpha value was 0.964, and the item-total correlations were all greater than 0.520. The overall KMO measure was 0.966 and the KMO measure for each item was greater than 0.930 (P < 0.001). The eigenvalues of the top 3 components extracted were all greater than 1, explaining 55.351, 7.382, and 5.316% of data variance respectively, and 68.048% cumulatively. These components were aligned with the competency dimensions of skills (S), knowledge (K), and attitude (A). Significant and positive correlations (0.135 < Kendall's tau-b < 0.276, p < 0.05) were found between Year-4 students' self-rated competency levels and their scores for the licensing examination. Steady competency growth was associated with almost all indicators, with the most pronounced growth in the domain of skills. A lack of steady growth was seen in the indicators of "applying the English language" and "conducting scientific research & innovating".

CONCLUSIONS

The ASK-SEAT, a competency-based assessment scale developed to measure medical students' competency development shows good reliability and structural validity. For predictive validity, weak-to-moderate correlations are found between Year-4 students' self-assessment and their performance at the national licensing examination (Year-4 students start their clinical clerkship during the 2nd semester of their 4th year of study). Year-2 to Year-6 students demonstrate steady improvement in the great majority of clinical competency indicators, except in the indicators of "applying the English language" and "conducting scientific research & innovating".

What Behaviors Define a Good Physician? Assessing and Communicating About Noncognitive Skills

Once medical students attain a certain level of medical knowledge, success in residency often depends on noncognitive attributes, such as conscientiousness, empathy, and grit. These traits are significantly more difficult to assess than cognitive performance, creating a potential gap in measurement. Despite its promise, competency-based medical education (CBME) has yet to bridge this gap, partly due to a lack of well-defined noncognitive observable behaviors that assessors and educators can use in formative and summative assessment. As a result, typical undergraduate to graduate medical education handovers stress standardized test scores, and program directors trust little of the remaining information they receive, sometimes turning to third-party companies to better describe potential residency candidates. The authors have created a list of noncognitive attributes, with associated definitions and noncognitive skills-called observable practice activities (OPAs)-written for learners across the continuum to help educators collect assessment data that can be turned into valuable information. OPAs are discrete work-based assessment elements collected over time and mapped to larger structures, such as milestones, entrustable professional activities, or competencies, to create learning trajectories for formative and summative decisions. Medical schools and graduate medical education programs could adapt these OPAs or determine ways to create new ones specific to their own contexts. Once OPAs are created, programs will have to find effective ways to assess them, interpret the data, determine consequence validity, and communicate information to learners and institutions. The authors discuss the need for culture change surrounding assessment-even for the adoption of behavior-based tools such as OPAs-including grounding the work in a growth mindset and the broad underpinnings of CBME. Ultimately, improving assessment of noncognitive capacity should benefit learners, schools, programs, and most importantly, patients.

Perspectives of the Key Stakeholders of the Alignment and Integration of the SaudiMEDs Framework into the Saudi Medical Licensure Examination: A Qualitative Study

PURPOSE

The purpose of the Saudi Medical Education Directives Framework (SaudiMEDs) is to assure the essential level of competencies for medical graduates, which should be reflected in the Saudi Medical Licensure Examination (SMLE). This study explored the opinions of key stakeholders of the alignment and integration of the SMLE's blueprint and contents with the SaudiMEDs competency framework's themes and domains.

PARTICIPANTS AND METHODS

This was a qualitative case study, using a purposive sampling technique. Ten participants participated in the study representing the main stakeholders. The participants were four deans, an assistant dean, two residents, and three interns of various colleges of medicine (COM). In-depth interviews were conducted through a semi-structured format, either online or in-person. The interviews were recorded, transcribed verbatim and analyzed according to the general guidelines of qualitative content analysis.

RESULTS

Four major themes emerged from the data, including the current alignment practices of the COM, competencies enhanced through preparing according to the SMLE, the SaudiMEDs representation in the SMLE, and finally a roadmap to achieve optimum alignment between the SaudiMEDs and the SMLE.

CONCLUSION

The participants were knowledgeable about the SaudiMEDs and perceived the SMLE blueprint and contents to be partially aligned with the themes and domains of the SaudiMEDs competency framework. The responses suggested that additional effort is required to improve the current alignment to assess the competencies of COM graduates appropriately.