The effect of a final exam on long-term retention

Timing's not everything: Immediate and delayed feedback are equally beneficial for performance in formative multiple-choice testing

INTRODUCTION

Test-enhanced learning (TEL) is an impactful teaching and learning strategy that prioritises active learner engagement through the process of regular testing and reviewing. While it is clear that meaningful feedback optimises the effects of TEL, the ideal timing of this feedback (i.e. immediate or delayed) in a medical education setting is unclear.

METHOD

Forty-one second-year medical students were recruited from the University of Melbourne. Participants were given a multiple-choice question test with a mix of immediate (i.e. post-item) and delayed (i.e. post-item-block) conceptual feedback. Students were then tested on near and far transfer items during an immediate post-test, and at a one-week follow-up.

RESULTS

A logistic mixed effects model was used to predict the probability of successful near and far transfer. As expected, participants in our study tended to score lower on far transfer items than they did on near transfer items. In addition, correct initial response on a parent question predicted subsequent correct responding. Contrary to our hypotheses, the feedback timing effect was non-significant-there was no discernible difference between feedback delivered immediately versus delayed feedback.

DISCUSSION

The findings of this study suggest that the timing of feedback delivery (post-item versus post-item-block) does not influence the efficacy of TEL in this medical education setting. We therefore suggest that educators may consider practical factors when determining appropriate TEL feedback timing in their setting.

Systematic review of distributed practice and retrieval practice in health professions education

To determine the effect of distributed practice (spacing out of study over time) and retrieval practice (recalling information from memory) on academic grades in health professions education and to summarise a range of interventional variables that may affect study outcomes. A systematic search of seven databases in November 2022 which were screened according to predefined inclusion criteria. The Medical Education Research Study Quality Instrument (MERSQI) and Newcastle-Ottawa Scale-Education (NOS-E) were used to critically appraise eligible articles. A summary of interventional variables includes article content type, strategy type, assessment type and delay and statistical significance. Of 1818 records retrieved, 56 were eligible for inclusion and included a total of 63 experiments. Of these studies, 43 demonstrated significant benefits of distributed practice and/or retrieval practice over control and comparison groups. Included studies averaged 12.23 out of 18 on the MERSQI and averaged 4.55 out of 6 on the NOS-E. Study designs were heterogeneous with a variety of interventions, comparison groups and assessment types. Distributed practice and retrieval practice are effective at improving academic grades in health professions education. Future study quality can be improved by validating the assessment instruments, to demonstrate the reliability of outcome measures. Increasing the number of institutions included in future studies may improve the diversity of represented study participants and may enhance study quality. Future studies should consider measuring and reporting time on task which may clarify the effectiveness of distributed practice and retrieval practice. The stakes of the assessments, which may affect student motivation and therefore outcomes, should also be considered.

Pre-clerkship National Board of Medical Examiners Subject Examinations Versus End-of-Semester Final Examinations: How Well Do They Assess Preparedness for the United States Medical Licensing Examination Step 1?

Background The ability to provide performance insights of various United States Medical Licensing Examination (USMLE) Step 1 assessments is of great importance to medical educators. Two custom pre-clerkship assessments used at the Kirk Kerkorian School of Medicine at the University of Nevada, Las Vegas (KSOM) are National Board of Medical Examiners (NBME)-derived end-of-semester final examinations and subject examinations. The authors sought to determine if performance on these custom assessments can provide feedback on a medical student's readiness to undertake the USMLE Step 1 examination. Methodology Deidentified student performance data were provided by institutional databases for the KSOM graduating class of 2023 (N = 60). Pearson correlation analyses were utilized to evaluate the strength of the correlation between USMLE Step 1 performance and NBME subject examinations versus NBME end-of-semester final examinations. Results The results indicated that the NBME end-of-semester final examinations have a statistically higher correlation to the USMLE Step 1 score than the majority of the individual NBME subject examinations. However, the mean NBME subject examination score (Semester 1: r = 0.53, p < 0.05; Semester 2: r = 0.58, p < 0.05) demonstrated significantly higher correlation to the USMLE Step 1 performance than the NBME end-of-semester final examination score for both Semesters 1 and 2 (Semester 1: r = 0.50, p < 0.05; Semester 2: r = 0.48, p < 0.05). Conclusions These results showed that the mean of the NBME subject examination score was a better metric to assess readiness for the USMLE Step 1 than the NBME end-of-semester final examinations. However, each NBME end-of-semester final examination score showed a better correlation than the majority of the NBME subject examinations.

Exploring (Collaborative) Generation and Exploitation of Multiple Choice Questions: Likes as Quality Proxy Metric

Multiple Choice Questions (MCQs) are an established medium of formal educational contexts. The collaborative generation of MCQs by students follows the perspectives of constructionist and situated learning and is an activity that fosters learning processes. The MCQs generated are—besides the learning processes—further outcomes of collaborative generation processes. Quality MCQs are a valuable resource, so that collaboratively generated quality MCQs might also be exploited in further educational scenarios. However, the quality MCQs first need to be identified from the corpus of all generated MCQs. This article investigates whether Likes distributed by students when answering MCQs are viable as a metric for identifying quality MCQs. Additionally, this study explores whether the process of collaboratively generating MCQs and using the quality MCQs generated in commercial quiz apps is achievable without additional extrinsic motivators. Accordingly, this article describes the results of a two-stage field study. The first stage investigates whether quality MCQs may be identified through collaborative inputs. For this purpose, the Reading Game (RG), a gamified, web-based software aiming at collaborative MCQ generation, is employed as a semester-accompanying learning activity in a bachelor course in Urban Water Management. The reliability of a proxy metric for quality calculated from the ratio of Likes received and appearances in quizzes is compared to the quality estimations of domain experts for selected MCQs. The selection comprised the ten best and the ten worst rated MCQs. Each of the MCQs is rated regarding five dimensions. The results support the assumption that the RG-given quality metric allows identification of well-designed MCQs. In the second stage, MCQs created by RG are provided in a commercial quiz app (QuizUp) in a voluntary educational scenario. Despite the prevailing pressure to learn, neither the motivational effects of RG nor of the app are found in this study to be sufficient for encouraging students to voluntarily use them on a regular basis. Besides confirming that quality MCQs may be generated by collaborative software, it is to be stated that in the collaborative generation of MCQs, Likes may serve as a proxy metric for the quality of the MCQs generated.

The retention benefits of cumulative versus non-cumulative midterms in introductory biology may depend on students' reasoning skills

Assessment has long played an important role as a measurement tool of student mastery over course content. However, testing has also been shown to be an effective learning tool. Cumulative testing, in which all material from the entire learning period is covered, has been assumed to be effective, yet few studies have explicitly tested its effectiveness compared to non-cumulative testing. Studies in psychology and mathematics courses suggest that cumulative final exams increase long-term retention of information, and cumulative testing during the semester can increase cumulative final exam performance and long-term retention. Because frequent testing has also been shown to increase student learning, the purpose of this quasi-experimental study is to investigate the effects of cumulative versus non-cumulative midterms on student learning in a course that uses frequent assessment. In this study, one section of an introductory biology course for non-majors was given seven cumulative midterms, with about half of the questions drawn from previous units and the rest covering the current unit. The other section was given seven non-cumulative midterms that focused on current material while other course characteristics were held constant. Student performance on a common, cumulative final exam and a retention exam five months later were compared. Midterm format had no effect on final exam performance, contradicting the few studies done in psychology and mathematics courses. Thus, there may be no additional benefit of cumulative testing if exams are given frequently. Cumulative midterms appeared to increase retention after five months, but only for students who entered the course with low reasoning skills. Interestingly, students with high reasoning skills appeared to retain more from the course if they were given non-cumulative midterms. Possible explanations and ideas for future research are discussed.

The effects of a teaching intervention on weight bias among kinesiology undergraduate students

OBJECTIVES

Weight bias is present among kinesiology professionals and this may cause a significant negative impact on their clients with obesity. Thus, our objective was to test if learning about uncontrollable cause of obesity and about weight bias would reduce explicit and implicit weight bias among kinesiology undergraduate students compared to the traditional curriculum which is more focused on controllable causes of weight gain.

METHODS

We recruited undergraduates from two classes of the same kinesiology major course taught by the same instructor. In-class teaching activities consisted of 80 min lecture on day 1, video watching session and a group activity on day 3 for both groups. Intervention group (n = 33) learned about uncontrollable causes of obesity and about weight bias and had activities to evoke empathy. Control group (n = 34) learned the traditional curriculum where they learned the role of exercise and diet in weight management. We measured explicit and implicit weight bias using Anti-Fat Attitude Test (AFAT) and Implicit Association Test (IAT), respectively pre-intervention, immediate post intervention and 1 month later.

RESULTS

In mixed model analysis, AFAT Blame scores had significant group by time interaction (p < 0.001). Blame scores significantly reduced with mean differences (standard error (SE)) of -0.35 (0.08) post intervention (p < 0.001) and persisted to be reduced with mean differences (SE) of -0.39 (0.08) even after 4-week follow-up (p < 0.001) only in the intervention group. Odds of having less implicit weight bias was significantly lower at 4-week follow-up than pre-intervention (odds ratio = 0.4; 95% CI: 0.22-0.73) in the control group but no changes were seen in the intervention group.

CONCLUSIONS

"Blame" component of explicit weight bias significantly decreased when students learned about controllable causes of obesity and weight bias, but implicit bias did not reduce. However, implicit weight bias appears to increase when education on obesity is limited to diet and exercise interventions as taught in the traditional curriculum.

Multiple-Choice Questioning Is an Efficient Instructional Methodology That May Be Widely Implemented in Academic Courses to Improve Exam Performance

Distributed multiple-choice questioning during instruction improves exam performance in middle-school and college classes. Instructional multiple-choice questioning is effective whether done in class or online and improves performance on short-answer and free-recall exam questions as well as multiple-choice questions. It may improve performance on exam questions about facts not previously queried.