A remedial intervention linked to a formative assessment is effective in terms of improving student performance in subsequent degree examinations

Exploring challenges in learning and study skills among first-year medical students: a case study

BACKGROUND

Despite being high-achieving students, many medical students face academic challenges, particularly during their first year of study. Research indicates that self-regulated learning, involving metacognitive processes and adaptive strategies, can positively influence academic achievement. This study aimed to assess the early learning and study skills of first-year medical students in an international medical school with the goal of developing a learner-centered educational intervention to promote self-regulated learning.

METHODS

We conducted a retrospective analysis of the Learning and Study Skills Inventory (LASSI) questionnaire that was administered annually each August to first-year medical students from 2019 to 2022. The distribution of students across different percentile ranges for each selected variable was determined for each year and all years collectively. Students were counted within distinct percentile brackets (50th and below, between 51st and 75th, and above 75th ) for each variable.

RESULTS

A total of 147 students completed the LASSI questionnaire over the 4-year time period. Using academic resources was the greatest concern, with 67% of students in the 50th or below percentile, followed by selecting the main idea (56%), motivation (51%), and concentration (50%). Attitude scored highest across all cohorts, scoring between 21.55 ± 0.73 and 26.49 ± 0.34. In comparing mean scores of LASSI variables across all cohorts, attitude, motivation, test-taking strategies, time management, and the use of academic resources differed significantly (p < 0.05).

CONCLUSION

LASSI data can provide an early picture of students' support needs. We posit that early identification of student learning and study skills and areas of struggle can inform personalized educational interventions and programs to support first-year medical students.

Interventions for undergraduate and postgraduate medical learners with academic difficulties: A BEME systematic review update: BEME Guide No. 85

BACKGROUND

Clinical teachers often struggle to record trainee underperformance due to lacking evidence-based remediation options.

OBJECTIVES

To provide updated evidence-based recommendations for addressing academic difficulties among undergraduate and postgraduate medical learners.

METHODS

A systematic review searched databases including MEDLINE, CINAHL, EMBASE, ERIC, Education Source, and PsycINFO (2016-2021), replicating the original Best Evidence Medical Education 56 review strategy. Original research/innovation reports describing intervention(s) for medical learners with academic difficulties were included. Data extraction used Michie's Behaviour Change Techniques (BCT) Taxonomy and program evaluation models from Stufflebeam and Kirkpatrick. Quality appraised used the Mixed Methods Appraisal Tool (MMAT). Authors synthesized extracted evidence by adapting GRADE approach to formulate recommendations.

RESULTS

Eighteen articles met the inclusion criteria, primarily addressing knowledge (66.7%), skills (66.7%), attitudinal problems (50%) and learner's personal challenges (27.8%). Feedback and monitoring was the most frequently employed BCT. Study quality varied (MMAT 0-100%). We identified nineteen interventions (UG: n = 9, PG: n = 12), introducing twelve new thematic content. Newly thematic content addressed contemporary learning challenges such as academic procrastination, and use of technology-enhanced learning resources. Combined with previous interventions, the review offers a total dataset of 121 interventions.

CONCLUSION

This review offers additional evidence-based interventions for learners with academic difficulties, supporting teaching, learning, faculty development, and research efforts.

Exam prediction and the general Practice Registrar Competency Assessment Grid (GPR-CAG)

BACKGROUND

In GP training, identifying early predictors of poor summative examination performance can be challenging. We aimed to establish whether external clinical teaching visit (ECTV) performance, measured using a validated instrument (GP Registrar Competency Assessment Grid, GPR-CAG) is predictive of Royal Australian College of General Practitioners (RACGP) Fellowship examination performance.

METHODS

A retrospective cohort study including GP registrars in New South Wales/Australian Capital Territory with ECTV data recorded during their first training term (GPT1), between 2014 and 2018, who attempted at least one Fellowship examination. Independent variables of interest included the four GPR-CAG factors assessed in GPT1 ('patient-centredness/caring', 'formulating hypotheses/management plans', 'professional responsibilities', 'physical examination skills'). Outcomes of interest included individual scores of the three summative examinations (Applied Knowledge Test (AKT); Key Feature Problem (KFP); and the Objective Structured Clinical Examination (OSCE)) and overall Pass/Fail status. Univariable and multivariable regression analyses were performed.

RESULTS

Univariably, there were statistically significant associations (p < 0.01) between all four GPR-CAG factors and all four summative examination outcomes, except for 'formulating hypotheses/management plans' and OSCE score (p = 0.07). On multivariable analysis, each factor was significantly associated (p < 0.05) with at least one exam outcome, and 'physical examination skills' was significantly associated (p < 0.05) with all four exam outcomes.

DISCUSSION

ECTV performance, via GPR-CAG scores, is predictive of RACGP Fellowship exam performance. The univariable findings highlight the pragmatic utility of ECTVs in flagging registrars who are at-risk of poor exam performance, facilitating early intervention. The multivariable associations of GPR-CAG scores and examination performance suggest that these scores provide predictive ability beyond that of other known predictors.

Addressing academic challenges: A quasi-experimental study on the effect of remedial exam strategy for nursing students with low academic performance

Background

Universities are responsible for providing education and support to all students, including those who may be underachieving. To fulfill this responsibility, universities need effective strategies to help these students graduate successfully.

Objective

This study aimed to evaluate the effectiveness of a midterm remedial exam strategy in improving the academic performance of nursing students.

Methods

A quasi-experimental design was employed to conduct this research at a private college in Dammam, Saudi Arabia's eastern region. The study included 306 convenient bachelor nursing students who had failed their midterm exam. Of these, 103 students voluntarily participated in the remedial midterm exam (intervention group) to enhance their midterm scores, while 203 students did not take the remedial exam (control group). Data collection took place in the spring of 2022 by reviewing course files and student records, encompassing formative and summative evaluations across 14 courses. Data analysis involved using SPSS (2023) and Jamovi (version 2.3, 2022) software to conduct paired t-tests, Mann-Whitney, and correlational tests.

Results

The remedial group demonstrated significant differences between their midterm and remedial exam scores. However, only 15.5% of students in the remedial group passed the exam, despite 68% showing improvement. Additionally, there were significant differences between the intervention and control groups in midterm and overall course scores. The control group outperformed the remedial group in the midterm, final exams, and overall course scores (p <0.05).

Conclusion

Remedial exams can be beneficial for students who are close to passing but may not effectively support those who perform poorly. To address this, implementing a comprehensive remedial program or providing ongoing remedial activities throughout the course can offer better support for strongly underperforming students. Additionally, educational institutions can enhance students' motivation and performance by providing study skills classes and time-management workshops, equipping them with valuable tools for academic success.

Student Use and Perceptions of Embedded Formative Assessments in a Basic Science Veterinary Program

This work describes the implementation of online timed closed-book formative assessments across several modules of a first-year undergraduate veterinary program. This process does not require significant time investment since it can be implemented into existing programs of study. Students were surveyed on how they used these formative assessments for learning and overall, were overwhelmingly positive about the opportunity to practice and receive feedback on their performance. Quantitative statistics on preferences as well as qualitative thematic analysis of open free-text questions reveal clear preferences in how they choose to engage with the assessments for learning, as well as how they prefer assessments to be administered. Students were positive about the online nature of the exams and prefer formative assessments to be distributed across the teaching semesters without any time restrictions, allowing them to be completed as and when they choose. Immediate feedback in the form of model answers is the students' preference, although some value signposting to relevant resources for further research. Furthermore, students report that they want more questions and tests to complement their learning, and overwhelming rely on guided and structured activities for learning and revision which will need to be balanced with opportunities to develop critical thinking and independent learning skills when studying in a professional course, given students are not likely to default into such behavior. This work models a process many curriculum designers have, and continue to undergo in higher education as online, hybrid, and blended approaches to teaching have received renewed interest.

Associations between admissions factors and the need for remediation

This study examines the way in which student characteristics and pre-admissions measures are statistically associated with the likelihood a student will require remediation for academic and professionalism offenses. We anchor our inquiry within Irby and Hamstra's (2016) conceptual framework of constructs of professionalism. Data from five graduating cohorts (2014-2018) from McMaster University (Hamilton, Canada) (N = 1,021) were retroactively collected and analyzed using traditional and multinominal logistic regression analyses. The relationship among student characteristics, pre-admissions variables, and referral for potential remediation both by occurrence (yes/no) as well as type (academic/professional/no referral) were examined separately. Findings indicate that gender (OR = 0.519, 95% CI 0.326-0.827, p < 0.01) and undergraduate grade point average (GPA) (OR = 0.245, 95% CI 0.070-0.855, p < 0.05) were significantly associated with instances of referral for potential professionalism and academic remediation, respectively. Women were less likely than men to require remediation for professionalism (OR = 0.332, 95% CI 0.174-0.602, p < 0.001). Undergraduate GPAs (OR = 0.826, 95% CI 0.021-0.539, p < 0.01) were significantly associated with remediation for academic reasons. Lower undergraduate GPAs were associated with a higher likelihood of remediation. These findings point to the admissions variables that are associated with instances that prompt referral for potential remediation. Where associations are not significant, we consider the application of different conceptualizations of professionalism across periods of admissions and training. We encourage those involved in applicant selection and student remediation to emphasize the importance of the interactions that occur between personal and contextual factors to influence learner behaviour and professional identity formation.

Emotion in remediation: A scoping review of the medical education literature

OBJECTIVES

Remediation can be crucial and high stakes for medical learners, and experts agree it is often not optimally conducted. Research from other fields indicates that explicit incorporation of emotion improves education because of emotion's documented impacts on learning. Because this could present an important opportunity for improving remediation, we aimed to investigate how the literature on remediation interventions in medical education discusses emotion.

METHODS

The authors used Arksey and O'Malley's framework to conduct a scoping literature review of records describing remediation interventions in medical education, using PubMed, CINAHL Complete, ERIC, Web of Science and APA PsycInfo databases, including all English-language publications through 1 May 2020 meeting search criteria. They included publications discussing remediation interventions either empirically or theoretically, pertaining to physicians or physician trainees of any level. Two independent reviewers used a standardised data extraction form to report descriptive information; they reviewed included records for the presence of mentions of emotion, described the mentions and analysed results thematically.

RESULTS

Of 1644 records, 199 met inclusion criteria and were reviewed in full. Of those, 112 (56%) mentioned emotion in some way; others focused solely on cognitive aspects of remediation. The mentions of emotion fell into three themes based on when the emotion was cited as present: during regular coursework or practice, upon referral for remediation and during remediation. One-quarter of records (50) indicated potential intentional incorporation of emotion into remediation programme design, but they were non-specific as to how emotions related to the learning process itself.

CONCLUSION

Even though emotion is omnipresent in remediation, medical educators frequently do not factor emotion into the design of remediation approaches and rarely explicitly utilise emotion to improve the learning process. Applications from other fields may help medical educators leverage emotion to improve learning in remediation, including strategies to frame and design remediation.

Predicting failure before it happens: A 5-year, 1042 participant prospective study

PURPOSE OF THE ARTICLE

Students who fail assessments are at risk of negative consequences, including emotional distress and cessation of studies. Identifying students at risk of failure before they experience difficulties may considerably improve their outcomes.

METHODS

Using a prospective design, we collected simple measures of engagement (formative assessment scores, compliance with routine administrative tasks, and attendance) over the first 6 weeks of Year 1. These measures were combined to form an engagement score which was used to predict a summative examination sat 14 weeks after the start of medical school. The project was repeated for five cohorts, giving a total sample size of 1042.

RESULTS

Simple linear regression showed engagement predicted performance (R²_adj = 0.03, F(1,1040) = 90.09, p < 0.001) with a small effect size. More than half of failing students had an engagement score in the lowest two deciles.

CONCLUSIONS

At-risk medical students can be identified with some accuracy immediately after starting medical school using routinely collected, easily analysed data, allowing for tailored interventions to support students. The toolkit provided here can reproduce the predictive model in any equivalent educational context. Medical educationalists must evaluate how the advantages of early detection are balanced against the potential invasiveness of using student data.

Redefining scholarship for health professions education: AMEE Guide No. 142

Scholarship is an integral aspect of academia. It shapes the practice of individuals and the field and is often used to inform career progression decisions and policies. This makes it high stakes. Yet it is complex, and ambiguous. Definitions vary and the term does not necessarily translate well across contexts. In this AMEE guide, our aim is to establish a contemporary definition of scholarship which is appropriate to health professions education (HPE). Our specific objectives are to provide guidance to support colleagues in their career development as professional educators and to challenge biases and assumptions about scholarship which may still exist in educational systems and structures. Ultimately, we hope that this work will advance the stature/standing of scholarship in the field. We provide a general definition of scholarship and how this relates to the scholarship of teaching (and learning) (SoT[L]) and scholarly teaching. Drawing on Boyer's seminal work, we describe different types of scholarship and reflect on how these apply to HPE, before moving on to describe different types of engagement with scholarship in HPE, including scope of contribution and influence. Using cases and examples, we illustrate differences in scholarly engagement across stages of a career, contexts, and ways of engaging. We provide guidance on how to assess 'quality' of scholarship. We offer practical advice for health professions' educators seeking academic advancement. We advocate that institutional leaders consider their systems and structures, so that these align with faculty work patterns, and judge teaching and professional practice appropriately. We conclude by offering a new definition of scholarship in HPE.

Prediction of success at UK Specialty Board Examinations using the mandatory postgraduate UK surgical examination

Background

While performance in other mandatory examinations taken at the beginning of a doctor's career are predictive of final training outcomes, the influence early postgraduate surgical examinations might have on success at Specialty Board Exams in the UK is currently unknown. The aim was to investigate whether performance at the mandatory Membership of the Royal College of Surgeons (MRCS) examination, and other variables, are predictive of success at the Fellowship of the Royal College of Surgeons (FRCS) examination, thus potentially identifying those who may benefit from early academic intervention.

Methods

Pearson correlation coefficients examined the linear relationship between both examinations and logistic regression analysis identified potential independent predictors of FRCS success. All UK medical graduates who attempted either section of FRCS (Sections 1 and 2) between 2012 and 2018 were included.

Results

First attempt pass rates for Sections 1 and 2 FRCS were 87.4 per cent (n = 854) and 91.8 per cent (n = 797) respectively. In logistic regression analysis, sex (male: odds ratio (OR) 2.32, 95 per cent c.i 1.43 to 3.76), age (less than 29 years at graduation: OR 3.22, 1.88 to 5.51), Part B MRCS attempts (1 attempt: OR 1.77, 1.08 to 3.00), Part A score (OR 1.14, 1.09 to 1.89) and Part B score (OR 1.06, 1.03 to 1.09) were independent predictors of Section 1 FRCS success. Predictors of Section 2 FRCS success were age (less than 29 years at graduation: OR 3.55, 2.00 to 6.39), Part A score (OR 1.06, 1.02 to 1.11) and Section 1 FRCS score (OR 1.13, 1.07 to 1.18).

Conclusion

Part A and B MRCS performance were independent predictors of FRCS success, providing further evidence to support the predictive validity of this mandatory postgraduate exam. However, future research must explore the reasons between the attainment gaps observed for different groups of doctors.

Interventions for undergraduate and postgraduate medical learners with academic difficulties: A BEME systematic review: BEME Guide No. 56

Background: Clinical teachers often struggle to report unsatisfactory trainee performance, partly because of a lack of evidence-based remediation options. Objectives: To identify interventions for undergraduate (UG) and postgraduate (PG) medical learners experiencing academic difficulties, link them to a theory-based framework and provide literature-based recommendations around their use. Methods: This systematic review searched MEDLINE, CINAHL, EMBASE, ERIC, Education Source and PsycINFO (1990-2016) combining these concepts: medical education, professional competence/difficulty and educational support. Original research/innovation reports describing intervention(s) for UG/PG medical learners with academic difficulties were included. Data extraction employed Michie's Behavior Change Techniques (BCT) Taxonomy and program evaluation models from Stufflebeam and Kirkpatrick. Quality appraisal used the Mixed Methods Appraisal Tool (MMAT). The authors synthesized extracted evidence by adapting the GRADE approach to formulate recommendations. Results: Sixty-eight articles met the inclusion criteria, most commonly addressing knowledge (66.2%), skills (53.9%) and attitudinal problems (26.2%), or learner personal issues (41.5%). The most common BCTs were Shaping knowledge, Feedback/monitoring, and Repetition/substitution. Quality appraisal was variable (MMAT 0-100%). A thematic content analysis identified 109 interventions (UG: n = 84, PG: n = 58), providing 24 strong, 48 moderate, 26 weak and 11 very weak recommendations. Conclusion: This review provides a repertoire of literature-based interventions for teaching/learning, faculty development, and research purposes.

Deliberate Practice as an Effective Remediation Strategy for Underperforming Medical Students Focused on Clinical Skills: a Prospective Longitudinal Study

BACKGROUND

It is critical to develop remedial education for underperforming medical students, but little is known about how to create an effective remediation program. Deliberate practice (DP) is a structured and reflective activity that is designed to optimize performance. Here we applied the concept of DP to create remedial education to improve the clinical practices of medical students. We also analyzed the effectiveness of the remediation program.

METHODS

Based on the expert performance approach of DP, we designed a 4-week remedial program for clinical performance that included feedback and reflection. There were 74 student participants in this program from 2014 to 2017. Their clinical performance was re-evaluated after completion, and changes in their clinical performance scores were analyzed.

RESULTS

Students who completed the remediation program showed significant improvements in clinical performance scores (P < 0.001). Most students found the program to be instructive and helpful for improving their clinical performance. They reported that role play with peers was the most helpful for improving their skills.

CONCLUSION

The DP-based remediation program improved the clinical performance of failing medical students. This remediation program should continue to be offered to underperforming students to ensure that medical school graduates are competent.

How does preclinical laboratory training impact physical examination skills during the first clinical year? A retrospective analysis of routinely collected objective structured clinical examination scores among the first two matriculating classes of a reformed curriculum in one Polish medical school

OBJECTIVE

As a result of a curriculum reform launched in 2012 at our institution, preclinical training was shortened to 2 years instead of the traditional 3 years, creating additional incentives to optimise teaching methods. In accordance with the new curriculum, a semester-long preclinical module of clinical skills (CS) laboratory training takes place in the second year of study, while an introductory clinical course (ie, brief introductory clerkships) is scheduled for the Fall semester of the third year. Objective structured clinical examinations (OSCEs) are carried out at the conclusion of both the preclinical module and the introductory clinical course. Our aim was to compare the scores at physical examination stations between the first and second matriculating classes of a newly reformed curriculum on preclinical second-year OSCEs and early clinical third-year OSCEs.

DESIGN

Analysis of routinely collected data.

SETTING

One Polish medical school.

PARTICIPANTS

Complete OSCE records for 462 second-year students and 445 third-year students.

OUTCOME MEASURES

OSCE scores by matriculation year.

RESULTS

In comparison to the first class of the newly reformed curriculum, significantly higher (ie, better) OSCE scores were observed for those students who matriculated in 2013, a year after implementing the reformed curriculum. This finding was consistent for both second-year and third-year cohorts. Additionally, the magnitude of the improvement in median third-year OSCE scores was proportional to the corresponding advancement in preceding second-year preclinical OSCE scores for each of two different sets of physical examination tasks. In contrast, no significant difference was noted between the academic years in the ability to interpret laboratory data or ECG - tasks which had not been included in the second-year preclinical training.

CONCLUSION

Our results suggest the importance of preclinical training in a CS laboratory to improve students' competence in physical examination at the completion of introductory clinical clerkships during the first clinical year.

A BEME (Best Evidence in Medical Education) review of the use of workplace-based assessment in identifying and remediating underperformance among postgraduate medical trainees: BEME Guide No. 43

INTRODUCTION

The extent to which workplace-based assessment (WBA) can be used as a facilitator of change among trainee doctors has not been established; this is particularly important in the case of underperforming trainees. The aim of this review is to examine the use of WBA in identifying and remediating performance among this cohort.

METHODS

Following publication of a review protocol a comprehensive search of eight databases took place to identify relevant articles published prior to November 2015. All screening, data extraction and analysis procedures were performed in duplicate or with quality checks and necessary consensus methods throughout. Given the study-level heterogeneity, a descriptive synthesis approach informed the study analysis.

RESULTS

Twenty studies met the inclusion criteria. The use of WBA within the context of remediation is not supported within the existing literature. The identification of underperformance is not supported by the use of stand-alone, single-assessor WBA events although specific areas of underperformance may be identified. Multisource feedback (MSF) tools may facilitate identification of underperformance.

CONCLUSION

The extent to which WBA can be used to detect and manage underperformance in postgraduate trainees is unclear although evidence to date suggests that multirater assessments (i.e. MSF) may be of more use than single-rater judgments (e.g. mini-clinical evaluation exercise).

The effect of peer-group size on the delivery of feedback in basic life support refresher training: a cluster randomized controlled trial

BACKGROUND

Students are largely providing feedback to one another when instructor facilitates peer feedback rather than teaching in group training. The number of students in a group affect the learning of students in the group training. We aimed to investigate whether a larger group size increases students' test scores on a post-training test with peer feedback facilitated by instructor after video-guided basic life support (BLS) refresher training. Students' one-rescuer adult BLS skills were assessed by a 2-min checklist-based test 1 year after the initial training.

METHODS

A cluster randomized controlled trial was conducted to evaluate the effect of student number in a group on BLS refresher training. Participants included 115 final-year medical students undergoing their emergency medicine clerkship. The median number of students was 8 in the large groups and 4 in the standard group. The primary outcome was to examine group differences in post-training test scores after video-guided BLS training. Secondary outcomes included the feedback time, number of feedback topics, and results of end-of-training evaluation questionnaires.

RESULTS

Scores on the post-training test increased over three consecutive tests with instructor-led peer feedback, but not differ between large and standard groups. The feedback time was longer and number of feedback topics generated by students were higher in standard groups compared to large groups on the first and second tests. The end-of-training questionnaire revealed that the students in large groups preferred the smaller group size compared to their actual group size.

CONCLUSIONS

In this BLS refresher training, the instructor-led group feedback increased the test score after tutorial video-guided BLS learning, irrespective of the group size. A smaller group size allowed more participations in peer feedback.

Evaluation of outcomes of a formative objective structured clinical examination for second-year UK medical students

OBJECTIVE

To explore how formative OSCEs influence student performance and perception when undertaking summative OSCEs.

METHODS

We introduced formative OSCEs for second-year medical students at a large London medical school. Examination data from both formative and subsequent summative OSCEs were analysed to determine the effect on summative OSCE performance. We gathered student perceptions using an anonymous online survey tool. The data was investigated using a standard scale of 1 to 5 and qualitative analysis of free text.

RESULTS

Overall, 46.6% and 85.0% of students passed the formative and summative OSCEs respectively. Formative OSCEs did not improve overall pass rates in summative OSCEs. Inclusion of an individual formative station was associated with improved performance in that station in summative OSCEs, with one exception. Formative OSCEs had a positive predictive value of 92.5% for passing the summative OSCE but limited negative predictive value. Students who passed fewer than two out of three formative OSCE stations were significantly more likely to fail the summative OSCE (78.2% vs 89.7%, p <0.001). Students felt formative OSCEs were good exam preparation and suggested logistical changes.

CONCLUSIONS

Formative OSCEs were associated with improved performance in subsequent summative OSCEs only for identical stations. They did not improve overall pass rates in summative OSCEs, and did not predict performance well. Students viewed the formative OSCE as a positive and useful activity. However, to maximise its benefit as a tool for learning, students need better communication about the role and purpose of formative OSCEs.

Effect of online formative assessment on summative performance in integrated musculoskeletal system module

BACKGROUND

The impact of web-based formative assessment practices on performance of undergraduate medical students in summative assessments is not widely studied. This study was conducted among third-year undergraduate medical students of a designated university in Malaysia to compare the effect, on performance in summative assessment, of repeated computer-based formative assessment with automated feedback with that of single paper-based formative assessment with face-to face feedback.

METHODS

This quasi-randomized trial was conducted among two groups of undergraduate medical students who were selected by stratified random technique from a cohort undertaking the Musculoskeletal module. The control group C (n = 102) was subjected to a paper-based formative MCQ test. The experimental group E (n = 65) was provided three online formative MCQ tests with automated feedback. The summative MCQ test scores for both these groups were collected after the completion of the module.

RESULTS

In this study, no significant difference was observed between the mean summative scores of the two groups. However, Band 1 students from group E with higher entry qualification showed higher mean score in the summative assessment. A trivial, but significant and positive correlation (r(2) = +0.328) was observed between the online formative test scores and summative assessment scores of group E. The proportionate increase of performance in group E was found to be almost double than group C.

CONCLUSION

The use of computer based formative test with automated feedback improved the performance of the students with better academic background in the summative assessment. Computer-based formative test can be explored as an optional addition to the curriculum of pre-clinical integrated medical program to improve the performance of the students with higher academic ability.

The effect of English proficiency and ethnicity on academic performance and progress

Non-local ethnicity or nationality and lower English proficiency have been linked with poor performance in health professional education. This study sought to compare the relative contributions of ethnicity and English proficiency, and to do so in a context where students had not been selected via interviews or some other proxy for language proficiency. Ethnicity, citizenship, and demographic data, as well as academic performance and progress were obtained for three successive cohorts (entering in 2007, 2008 and 2009) into a three-year pharmacy programme, following a common first-year. Complete data was available for all 297 students entering via this path. By the end of the programme, controlling for grades at entry, students from some non-local ethnic backgrounds (p < .001) underperformed local students, as did males (p < .001) and those who had been identified as having low English proficiency in year 1 (p < .01). Males (p < .01) and one non-local ethnic group (p < .01) were also more likely to have to repeat a year of study. Ethnicity was a stronger predictor than English proficiency or acculturation, but English proficiency was still independently and additively predictive. This suggests that targeted support strategies for students with lower English language proficiency are still important, but that cultural differences should not be underestimated.

Twelve tips for providing effective student support in undergraduate medical education

Medical students often require high levels of specialised institutional and personal support to facilitate success. Contributory factors may include personality type, course pressures and financial hardship. Drawing from research literature and the authors' experience, 12 tips are listed under five subheadings: policy and systems; people and resources; students; delivering support; limits of support. The 12 tips provide guidance to organisations and individual providers that encourages implementation of good practice and helps them better visualise their role within the system. By following the tips, medical schools can make more effective provisions for the expected, diverse and sometimes specialist needs of their students. Schools must take a proactive, anticipatory approach to provide appropriately for their entire student body. This ensures that students receive the best quality support, are more likely to succeed and are adequately prepared for their medical careers.

Clinical-performance remediation program for dyscompetent medical students

PURPOSE

Medical schools endeavor to ensure that students are competent with regard to clinical skills. Skills remediation is implemented in cases of poor clinical performance examination (CPX) grades, although little is known about the effectiveness of such techniques. In this study, we examined the effectiveness of a remediation program that was designed to improve the clinical performance of medical students.

METHODS

A 6-week remediation program, administered jointly by Seoul National University College of Medicine's Departments of Internal Medicine (IM) and Family Medicine (FM), was initiated. The program was divided into 2 parts: 3 weeks each of IM classes that were run by specialists in various fields and FM classes that were conducted by a chief resident. Twenty-three students were required to undergo remediation after posting poor scores on 2 sessions of a CPX. On completion of the remediation program, the students' clinical performance was re-evaluated, and the changes in clinical performance scores were analyzed.

RESULTS

After the remediation program, the students' total scores and scores on history taking, physical examination, physician's manner, and physician-patient interaction improved significantly. However, patient education did not improve. Most students found the remediation program to be instructive and helpful in preparation for the CPX. They were more satisfied with the chief resident's serial tutoring than with specialists' tutoring sessions.

CONCLUSION

The remediation program improves clinical performance. Continued development and implementation of this program will help failing students be competent physicians.

The remediation challenge: theoretical and methodological insights from a systematic review

OBJECTIVES

Remediation is usually offered to medical students and doctors in training who underperform on written or clinical examinations. However, there is uncertainty and conflicting evidence about the effectiveness of remediation. The aim of this systematic review was to synthesise the available evidence to clarify how and why remediation interventions may have worked in order to progress knowledge on this topic.

METHODS

The MEDLINE, EMBASE, CINAHL (Cumulative Index to Nursing and Allied Health Literature), ERIC (Educational Resources Information Centre), Web of Science and Scopus databases were searched for papers published from 1984 to April 2012, using the search terms 'remedial teaching', 'education', 'medical', 'undergraduate'/or 'clinical clerkship'/or 'internship and residency', 'at risk' and 'struggling'. Only studies that included an intervention, then provided retest data, and reported at least one outcome measure of satisfaction, knowledge, skills or effects on patients were eligible for inclusion. Studies of practising doctors were excluded. Data were abstracted independently in duplicate for all items. Coding differences were resolved through discussion.

RESULTS

Thirty-one of 2113 studies met the review criteria. Most studies were published after 2000 (n=24, of which 12 were published from 2009 onwards), targeted medical students (n=22) and were designed to improve performance on an immediately subsequent examination (n=22). Control or comparison groups, conceptual frameworks, adequate sample sizes and long-term follow-up measures were rare. In studies that included long-term follow-up, improvements were not sustained. Intervention designs tended to be highly complex, but their design or reporting did not enable the identification of the active components of the remedial process.

CONCLUSIONS

Most remediation interventions in medical education focus on improving performance to pass a re-sit of an examination or assessment and provide no insight into what types of extra support work, or how much extra teaching is critical, in terms of developing learning. More recent studies are generally of better quality. Rigorous approaches to developing and evaluating remediation interventions are required.

Profiling strugglers in a graduate-entry medicine course at Nottingham: a retrospective case study

BACKGROUND

10-15% of students struggle at some point in their medicine course. Risk factors include weaker academic qualifications, male gender, mental illness, UK ethnic minority status, and poor study skills. Recent research on an undergraduate medicine course provided a toolkit to aid early identification of students likely to struggle, who can be targeted by established support and study interventions. The present study sought to extend this work by investigating the number and characteristics of strugglers on a graduate-entry medicine (GEM) programme.

METHODS

A retrospective study of four GEM entry cohorts (2003-6) was carried out. All students who had demonstrated unsatisfactory progress or left prematurely were included. Any information about academic, administrative, personal, or social difficulties, were extracted from their course progress files into a customised database and examined.

RESULTS

362 students were admitted to the course, and 53 (14.6%) were identified for the study, of whom 15 (4.1%) did not complete the course. Students in the study group differed from the others in having a higher proportion of 2ii first degrees, and scoring less well on GAMSAT, an aptitude test used for admission. Within the study group, it proved possible to categorise students into the same groups previously reported (struggler throughout, pre-clinical struggler, clinical struggler, health-related struggler, borderline struggler) and to identify the majority using a number of flags for early difficulties. These flags included: missed attendance, unsatisfactory attitude or behaviour, health problems, social/family problems, failure to complete immunity status checks, and attendance at academic progress committee.

CONCLUSIONS

Problems encountered in a graduate-entry medicine course were comparable to those reported in a corresponding undergraduate programme. A toolkit of academic and non-academic flags of difficulty can be used for early identification of many who will struggle, and could be used to target appropriate support and interventions.

The effect of a short integrated study skills programme for first-year medical students at risk of failure: a randomised controlled trial

BACKGROUND

There is a need for outcome-based studies on strategies for supporting at-risk medical students that use long-term follow-up and contemporaneous controls.

AIM

To measure the effect of a short integrated study skills programme (SSP) on the study progress of at-risk medical students.

METHODS

First-year students identified as at-risk of academic failure at 7 months after enrolment were invited to participate in the randomised controlled trial. Participants were randomly assigned to the SSP group or to a control group receiving standard academic support. Effects of SSP were measured on the short (passed first exam after intervention), medium (obtained enough credits to proceed to second year) and long term (completed first-year curriculum within 2 years).

RESULTS

SSP participants (n=43) more often passed the first exam after the intervention than controls (n=41; 30% versus 12%; X2(1)=4.06, p<0.005, effect size=0.22), in particular those who had previously passed at least one exam. No medium or long-term effect was found. Participants who had attended four or five SSP sessions outperformed those who had attended fewer sessions on all outcome measures.

CONCLUSION

A short, integrated SSP benefited some, but not all students. Our advice is to focus support efforts on at-risk students who have demonstrated commitment and academic potential.

The role of the teacher in remediating at-risk medical students

BACKGROUND

Previous work identified complex ingredients of a remediation programme for at-risk medical students: mandatory, stable, facilitated small groups promote both cognitive and affective developments, with improved self-regulation, metacognition and reflection resulting in significant performance gains.

AIM

We explore the teachers' role in this intervention, aiming to expand and deepen understanding of remediation methods in medical education.

METHODS

Extensive qualitative data from student surveys and in-depth teacher interviews, along with quantitative student performance data, produced a rich description of remediation processes.

RESULTS

Remediation should support emotional needs and foster cognitive and metacognitive skills for self-regulation and critical thinking. Teachers of remediation need to motivate, critique, challenge and advise their learners, applying teaching and contextual expertise in a constructivist, student-centred environment that fosters curiosity and joy for learning. Teachers of remediation can mediate these processes through embodiment of five core roles: facilitator, nurturing mentor, disciplinarian, diagnostician and modeller of desired skills, attitudes and behaviours.

CONCLUSION

Remediation of struggling medical students can be achieved through a cognitive apprenticeship within a small community of inquiry that motivates and challenges the students. This community needs teachers capable of performing a unique combination of roles that demands high levels of teaching presence and practical wisdom.

Is short-term remediation after OSCE failure sustained? A retrospective analysis of the longitudinal attainment of underperforming students in OSCE assessments

BACKGROUND

Significant improvements in the delivery of criterion-based assessment techniques have improved confidence in standard setting and assessment quality. However, for underperforming students, a lack of evidence about longitudinal performance of this group poses dilemmas to educators when making decisions about the timing and nature of remediation.

AIM

To investigate the longitudinal performance of the UK undergraduate medical degree students, with a particular focus on comparing the poorly performing students (i.e. those with borderline or failing grades) with the main cohort of students.

METHOD

Over a 5-year period, 3200-student objective structured clinical examination (OSCE) assessments from a single medical school were investigated. A poorly performing subgroup of 125 students was identified and their longitudinal performance in the final 3 years of the undergraduate medical degree analysed.

RESULT

The relative performance of this student group declines across serial OSCEs, despite current methods of 'remediation and retest'.

CONCLUSIONS

This analysis demonstrates that typically students in the poorly performing subgroup achieve only short-term success with traditional remediation and retest models, and critically show an absence of longitudinal improvement. There is a clear need for institutions to develop profiling models that can help identify this student group and develop effective, research led models of remediation.