Impact of a competency-based curriculum on medical student advancement: a ten-year analysis

Interventions for undergraduate and postgraduate medical learners with academic difficulties: A BEME systematic review update: BEME Guide No. 85

BACKGROUND

Clinical teachers often struggle to record trainee underperformance due to lacking evidence-based remediation options.

OBJECTIVES

To provide updated evidence-based recommendations for addressing academic difficulties among undergraduate and postgraduate medical learners.

METHODS

A systematic review searched databases including MEDLINE, CINAHL, EMBASE, ERIC, Education Source, and PsycINFO (2016-2021), replicating the original Best Evidence Medical Education 56 review strategy. Original research/innovation reports describing intervention(s) for medical learners with academic difficulties were included. Data extraction used Michie's Behaviour Change Techniques (BCT) Taxonomy and program evaluation models from Stufflebeam and Kirkpatrick. Quality appraised used the Mixed Methods Appraisal Tool (MMAT). Authors synthesized extracted evidence by adapting GRADE approach to formulate recommendations.

RESULTS

Eighteen articles met the inclusion criteria, primarily addressing knowledge (66.7%), skills (66.7%), attitudinal problems (50%) and learner's personal challenges (27.8%). Feedback and monitoring was the most frequently employed BCT. Study quality varied (MMAT 0-100%). We identified nineteen interventions (UG: n = 9, PG: n = 12), introducing twelve new thematic content. Newly thematic content addressed contemporary learning challenges such as academic procrastination, and use of technology-enhanced learning resources. Combined with previous interventions, the review offers a total dataset of 121 interventions.

CONCLUSION

This review offers additional evidence-based interventions for learners with academic difficulties, supporting teaching, learning, faculty development, and research efforts.

Assisting physicians who exhibit disruptive behavior: Understanding the costs, contributors, and corrections

OBJECTIVE

The aim of this article is to discuss the etiology, prevalence, impact, and management of disruptive behavior in physicians. These various aspects will be examined at both the individual and system level, to provide appropriate perspective and detail effective approaches to address these behaviors.

METHOD

Clinical experience and review of the authors' and other researchers' findings provide consensus on numerous key aspects of physician disruptive behavior.

RESULTS

Physicians demonstrating disruptive behavior are often distressed. The behavior should be understood as arising from biopsychosocial contributors, knowledge gaps, insight, and systems factors. These contributors are inclusive and may interact with each other.

CONCLUSIONS

A comprehensive approach is required which can include assessment/reassessment tools, individualized programming (therapy, coaching, instruction), deliberate practice, medical follow-up, and system intervention. Complications include the diversity of disruptive behaviors, the many contributory factors therein, disagreement about methodology/measurement, and the role of the system.

Assessing professional identity formation (PIF) amongst medical students in Oncology and Palliative Medicine postings: a SEBA guided scoping review

BACKGROUND

Introduction to a multi-professional team who are working and caring for the dying, and facing complex moral and ethical dilemmas during Oncology and Palliative Medicine postings influence a medical student's professional identity formation (PIF). However, limited appreciation of PIF, inadequate assessments and insufficient support jeopardise this opportunity to shape how medical students think, feel and act as future physicians. To address this gap, a systematic scoping review (SSR) of PIF assessment methods is proposed.

METHODS

A Systematic Evidence-based Approach (SEBA) guided SSR of assessments of PIF in medical schools published between 1^st January 2000 and 31^st December 2021 in PubMed, Embase, ERIC and Scopus databases was carried out. Included articles were concurrently content and thematically analysed using SEBA's Split Approach and the themes and categories identified were combined using SEBA's Jigsaw Perspective. The review hinged on the following questions: "what is known about the assessment of professional identity formation amongst medical students?", "what are the theories and principles guiding the assessment of professional identity formation amongst medical students?", "what factors influence PIF in medical students?", "what are the tools used to assess PIF in medical students?", and "what considerations impact the implementation of PIF assessment tools amongst medical students?".

RESULTS

Two thousand four hundred thirty six abstracts were reviewed, 602 full-text articles were evaluated, and 88 articles were included. The 3 domains identified were 1) theories, 2) assessment, and 3) implementation in assessing PIF. Differing attention to the different aspects of the PIF process impairs evaluations, jeopardise timely and appropriate support of medical students and hinder effective implementation of PIF assessments.

CONCLUSION

The Krishna-Pisupati model combines current theories and concepts of PIF to provide a more holistic perspective of the PIF process. Under the aegis of this model, Palliative Care and Oncology postings are envisaged as Communities of Practice influencing self-concepts of personhood and identity and shaping how medical students see their roles and responsibilities as future physicians. These insights allow the forwarding of nine recommendations to improve assessments of PIF and shape the design of a PIF-specific tool that can direct timely and personalized support of medical students.

Emotion in remediation: A scoping review of the medical education literature

OBJECTIVES

Remediation can be crucial and high stakes for medical learners, and experts agree it is often not optimally conducted. Research from other fields indicates that explicit incorporation of emotion improves education because of emotion's documented impacts on learning. Because this could present an important opportunity for improving remediation, we aimed to investigate how the literature on remediation interventions in medical education discusses emotion.

METHODS

The authors used Arksey and O'Malley's framework to conduct a scoping literature review of records describing remediation interventions in medical education, using PubMed, CINAHL Complete, ERIC, Web of Science and APA PsycInfo databases, including all English-language publications through 1 May 2020 meeting search criteria. They included publications discussing remediation interventions either empirically or theoretically, pertaining to physicians or physician trainees of any level. Two independent reviewers used a standardised data extraction form to report descriptive information; they reviewed included records for the presence of mentions of emotion, described the mentions and analysed results thematically.

RESULTS

Of 1644 records, 199 met inclusion criteria and were reviewed in full. Of those, 112 (56%) mentioned emotion in some way; others focused solely on cognitive aspects of remediation. The mentions of emotion fell into three themes based on when the emotion was cited as present: during regular coursework or practice, upon referral for remediation and during remediation. One-quarter of records (50) indicated potential intentional incorporation of emotion into remediation programme design, but they were non-specific as to how emotions related to the learning process itself.

CONCLUSION

Even though emotion is omnipresent in remediation, medical educators frequently do not factor emotion into the design of remediation approaches and rarely explicitly utilise emotion to improve the learning process. Applications from other fields may help medical educators leverage emotion to improve learning in remediation, including strategies to frame and design remediation.

Objective Structured Assessment of technical skill (OSATS) in the Surgical Skills and Technology Elective Program (SSTEP): Comparison of peer and expert raters

BACKGROUND

A major challenge with Competency Based Medical Education (CBME) is that of increased assessment burden on faculty. To reduce this burden, the accuracy and reliability of peer-assessment for surgical skills requires further exploration.

METHODS

Forty-two second year medical students were video recorded while performing a simple interrupted suture and an instrument tie. Four novice raters underwent a short training session on the use of the Objective Structured Assessment of Technical Skills (OSATS) checklists. Videos of the suturing task were then independently assessed by the four novice raters and two expert raters on two occasions. Agreement between novice and expert rater scores was calculated using the intraclass correlation coefficient (ICC).

RESULTS

For both simple interrupted suturing (ICC = 0.78, CI = 0.66-0.86, p < 0.001) and instrument ties (ICC = 0.87, CI = 0.80-0.92, p < 0.001), there was good agreement between novice and expert raters.

CONCLUSIONS

Novice raters can be taught to use the OSATS checklists to assess peers on simple suturing and instrument tying tasks.

Remediating professionalism lapses in medical students and doctors: A systematic review

CONTEXT

A remediation intervention aims to facilitate the improvement of an individual whose competence in a particular skill has dropped below the level expected. Little is known regarding the effectiveness of remediation, especially in the area of professionalism. This review sought to identify and assess the effectiveness of interventions to remediate professionalism lapses in medical students and doctors.

METHODS

Databases Embase, MEDLINE, Education Resources Information Center and the British Education Index were searched in September 2017 and October 2018. Studies reporting interventions to remediate professionalism lapses in medical students and doctors were included. A standardised data extraction form incorporating a previously described behaviour change technique taxonomy was utilised. A narrative synthesis approach was adopted. Quality was assessed using the Critical Appraisal Skills Programme checklist.

RESULTS

A total of 19 studies on remediation interventions reported in 23 articles were identified. Of these, 13 were case studies, five were cohort studies and one was a qualitative study; 37% targeted doctors, 26% medical students, 16% residents and 21% involved mixed populations. Most interventions were multifaceted and addressed professionalism issues concomitantly with clinical skills, but some focused on specific areas (eg sexual boundaries and disruptive behaviours). Most used three or more behaviour change techniques. The included studies were predominantly of low quality as 13 of the 19 were case studies. It was difficult to assess the effectiveness of the interventions as the majority of studies did not carry out any evaluation.

CONCLUSIONS

The review identifies a paucity of evidence to guide best practice in the remediation of professionalism lapses in medical students and doctors. The literature tentatively suggests that remediating lapses in professionalism, as part of a wider programme of remediation, can facilitate participants to graduate from a programme of study, and pass medical licensing and mock oral board examinations. However, it is not clear from this literature whether these interventions are successful in remediating lapses in professionalism specifically. Further research is required to improve the design and evaluation of interventions to remediate professionalism lapses.

Developing an Entrustment Process: Insights from the AAMC CoreEPA Pilot

One of the main goals of the CoreEPA pilot has been to determine the feasibility of developing a process to make summative entrustment decisions regarding entrustable professional activities (EPAs). Five years into the pilot, we report results of a research study we conducted to explore approaches to the entrustment process undertaken by our ten participating schools. We sought to identify the choices that participating schools made regarding the entrustment process and why these decisions were made. We are sharing these results, highlighting ongoing challenges that were identified with the intent of helping other medical schools that are moving toward EPA-based assessment. We conducted semi-structured interviews with representatives of all 10 medical schools in the CoreEPA pilot to understand their choices in designing the entrustment process. Additional information was obtained through follow-up communication to ensure completeness and accuracy of the findings. Several common themes are described. Our results indicate that, while approaches to the entrustment process vary considerably, all schools demonstrated consistent adherence to the guiding principles of the pilot. Several common barriers to the entrustment process emerged, and there was a consensus that more experience is needed with the process before consequential entrustment decisions can be made. The CoreEPA pilot schools continue to address challenges identified in implementing entrustment processes and making entrustment decisions for our students graduating in the Class of 2020.

Interventions for undergraduate and postgraduate medical learners with academic difficulties: A BEME systematic review: BEME Guide No. 56

Background: Clinical teachers often struggle to report unsatisfactory trainee performance, partly because of a lack of evidence-based remediation options. Objectives: To identify interventions for undergraduate (UG) and postgraduate (PG) medical learners experiencing academic difficulties, link them to a theory-based framework and provide literature-based recommendations around their use. Methods: This systematic review searched MEDLINE, CINAHL, EMBASE, ERIC, Education Source and PsycINFO (1990-2016) combining these concepts: medical education, professional competence/difficulty and educational support. Original research/innovation reports describing intervention(s) for UG/PG medical learners with academic difficulties were included. Data extraction employed Michie's Behavior Change Techniques (BCT) Taxonomy and program evaluation models from Stufflebeam and Kirkpatrick. Quality appraisal used the Mixed Methods Appraisal Tool (MMAT). The authors synthesized extracted evidence by adapting the GRADE approach to formulate recommendations. Results: Sixty-eight articles met the inclusion criteria, most commonly addressing knowledge (66.2%), skills (53.9%) and attitudinal problems (26.2%), or learner personal issues (41.5%). The most common BCTs were Shaping knowledge, Feedback/monitoring, and Repetition/substitution. Quality appraisal was variable (MMAT 0-100%). A thematic content analysis identified 109 interventions (UG: n = 84, PG: n = 58), providing 24 strong, 48 moderate, 26 weak and 11 very weak recommendations. Conclusion: This review provides a repertoire of literature-based interventions for teaching/learning, faculty development, and research purposes.

Criterion-Based Assessment in a Norm-Based World: How Can We Move Past Grades?

In the United States, the medical education community has begun a shift from the Flexnerian time-based model to a competency-based medical education model. The graduate medical education (GME) community is substantially farther along in this transition than is the undergraduate medical education (UME) community.GME has largely adopted the use of competencies and their attendant milestones and increasingly is employing the framework of entrustable professional activities (EPAs) to assess trainee competence. The UME community faces several challenges to successfully navigating a similar transition. First is the reliance on norm-based reference standards in the UME-GME transition, comparing students' performance versus their peers' with grades, United States Medical Licensing Examination Step 1 and Step 2 score interpretation, and the structured Medical School Performance Evaluation, or dean's letter. Second is the reliance on proxy assessments rather than direct observation of learners. Third is the emphasis on summative rather than formative assessments.Educators have overcome a major barrier to change by establishing UME outcomes assessment criteria with the advent and general acceptance of the physician competency reference set and the Core EPAs for Entering Residency in UME. Now is the time for the hard work of developing assessments steeped in direct observation that can be accepted by learners and faculty across the educational continuum and can be shown to predict clinical performance in a much more meaningful way than the current measures of grades and examinations. The acceptance of such assessments will facilitate the UME transition toward competency-based medical education.

The Association Between Peer and Self-Assessments and Professionalism Lapses Among Medical Students

Peer and self-assessments are widely used to assess professionalism during medical school as part of a multisource feedback model. The purpose of this study was to examine the associations between peer and self-assessments and professionalism lapses at a large medical school. A retrospective case-control study design was used to compare peer and self-assessment scores from Years 1 to 3 of medical school for students who had been cited for professionalism lapses during medical school (case group; n = 78) with those of a randomly selected control group ( n = 230). Students in the case group had significantly lower peer assessment scores than students in the control group during all 3 years. Year 3 peer assessment scores showed the greatest difference (cases = 7.81 ± 0.65, controls = 8.22 ± 0.34, p < .01). Students with lower peer assessment scores were also significantly more likely to have been cited for a professionalism lapse (odds ratio = 6.25, 95% CI [3.13, 11.11], p < .01). This study reinforces the value of peer assessments of professionalism, which may be useful to help identify students who may be at risk for professionalism lapses during medical school.

Roadmap for creating an accelerated three-year medical education program

Medical education is undergoing significant transformation. Many medical schools are moving away from the concept of seat time to competency-based education and introducing flexibility in the curriculum that allows individualization. In response to rising student debt and the anticipated physician shortage, 35% of US medical schools are considering the development of accelerated pathways. The roadmap described in this paper is grounded in the experiences of the Consortium of Accelerated Medical Pathway Programs (CAMPP) members in the development, implementation, and evaluation of one type of accelerated pathway: the three-year MD program. Strategies include developing a mission that guides curricular development - meeting regulatory requirements, attaining institutional buy-in and resources necessary to support the programs, including student assessment and mentoring - and program evaluation. Accelerated programs offer opportunities to innovate and integrate a mission benefitting students and the public.

ABBREVIATIONS

CAMPP: Consortium of accelerated medical pathway programs; GME: Graduate medical education; LCME: Liaison committee on medical education; NRMP: National residency matching program; UME: Undergraduate medical education.

Is Reflective Ability Associated With Professionalism Lapses During Medical School?

PURPOSE

Recently, many have argued that learning to reflect on one's experiences is a critical component of professional identity formation and of professionalism. However, little empirical evidence exists to support this claim. This study explored the association between reflective ability and professionalism lapses among medical students.

METHOD

The authors conducted a retrospective case-control study of all students who matriculated at Indiana University School of Medicine from 2001 to 2009. The case group (n = 70) included those students who had been cited for a professionalism lapse during medical school; the students in the control group (n = 230) were randomly selected from the students who had not been cited for a professionalism lapse. Students' professionalism journal entries were scored using a validated rubric to assess reflective ability. Mean reflection scores were compared across groups using t tests, and logistic regression analysis was used to assess the relationship between reflective ability and professionalism lapses.

RESULTS

Reflection scores for students in the case group (2.46 ± 1.05) were significantly lower than those for students in the control group (2.82 ± 0.83) (P = .01). A lower reflection score was associated with an increased likelihood that the student had been cited for a professionalism lapse (odds ratio = 1.56; P < .01).

CONCLUSIONS

This study revealed a significant relationship between reflective ability and professionalism, although further study is needed to draw any conclusions regarding causation. These findings provide quantitative evidence to support current anecdotal claims about the relationship between reflection and professionalism.

Implementation of competency-based medical education: are we addressing the concerns and challenges?

CONTEXT

Competency-based medical education (CBME) has emerged as a core strategy to educate and assess the next generation of physicians. Advantages of CBME include: a focus on outcomes and learner achievement; requirements for multifaceted assessment that embraces formative and summative approaches; support of a flexible, time-independent trajectory through the curriculum; and increased accountability to stakeholders with a shared set of expectations and a common language for education, assessment and regulation.

OBJECTIVES

Despite the advantages of CBME, numerous concerns and challenges to the implementation of CBME frameworks have been described, including: increased administrative requirements; the need for faculty development; the lack of models for flexible curricula, and inconsistencies in terms and definitions. Additionally, there are concerns about reductionist approaches to assessment in CBME, lack of good assessments for some competencies, and whether CBME frameworks include domains of current importance. This study will outline these issues and discuss the responses of the medical education community.

METHODS

The concerns and challenges expressed are primarily categorised as: (i) those related to practical, administrative and logistical challenges in implementing CBME frameworks, and (ii) those with more conceptual or theoretical bases. The responses of the education community to these issues are then summarised.

CONCLUSIONS

The education community has begun to address the challenges involved in implementing CBME. Models and guidance exist to inform implementation strategies across the continuum of education, and focus on the more efficient use of resources and technology, and the use of milestones and entrustable professional activities-based frameworks. Inconsistencies in CBME definitions and frameworks remain a significant obstacle. Evolution in assessment approaches from in vitro task-based methods to in vivo integrated approaches is responsive to many of the theoretical and conceptual concerns about CBME, but much work remains to be done to bring rigour and quality to work-based assessment.

Time to Loosen the Apron Strings: Cohort-based Evaluation of a Learner-driven Remediation Model at One Medical School

BACKGROUND

Remediation in the era of competency-based assessment demands a model that empowers students to improve performance.

AIM

To examine a remediation model where students, rather than faculty, develop remedial plans to improve performance.

SETTING/PARTICIPANTS

Private medical school, 177 medical students.

PROGRAM DESCRIPTION

A promotion committee uses student-generated portfolios and faculty referrals to identify struggling students, and has them develop formal remediation plans with personal reflections, improvement strategies, and performance evidence. Students submit reports to document progress until formally released from remediation by the promotion committee.

PROGRAM EVALUATION

Participants included 177 students from six classes (2009-2014). Twenty-six were placed in remediation, with more referrals occurring during Years 1 or 2 (n = 20, 76 %). Unprofessional behavior represented the most common reason for referral in Years 3-5. Remedial students did not differ from classmates (n = 151) on baseline characteristics (Age, Gender, US citizenship, MCAT) or willingness to recommend their medical school to future students (p < 0.05). Two remedial students did not graduate and three did not pass USLME licensure exams on first attempt. Most remedial students (92 %) generated appropriate plans to address performance deficits.

DISCUSSION

Students can successfully design remedial interventions. This learner-driven remediation model promotes greater autonomy and reinforces self-regulated learning.

National undergraduate medical core curriculum in Turkey: evaluation of residents

BACKGROUND

There is very little information available on self-perceived competence levels of junior medical doctors with regard to definitions by the National Core Curriculum (NCC) for Undergraduate Medical Education.

AIMS

This study aims to determine the perceived level of competence of residents during undergraduate medical education within the context of the NCC.

STUDY DESIGN

Descriptive study.

METHODS

The survey was conducted between February 2010 and December 2011; the study population comprised 450 residents. Of this group, 318 (71%) participated in the study. Self-assessment questionnaires on competencies were distributed and residents were asked to assess their own competence in different domains by scoring them on a scale of 1 to 10.

RESULTS

Nearly half of the residents reported insufficient experience of putting clinical skills into practice when they graduated. In the theoretical part of NCC, the lowest competency score was reported for health-care administration, while the determination of level of chlorine in water, delivering babies, and conducting forensic examinations had the lowest perceived levels of competency in the clinical skills domain.

CONCLUSION

Residents reported low levels of perceived competency in skills they rarely performed outside the university hospital. They were much more confident in skills they performed during their medical education.