Challenging medical students with an interim assessment: a positive effect on formal examination score in a randomized controlled study

Conceptualising spaced learning in health professions education: A scoping review

OBJECTIVES

To investigate the definitions and applications of 'spaced learning' and to propose future directions for advancing its study and practice in health professions education.

METHOD

The authors searched five online databases for articles published on spaced learning in health professions education prior to February 2018. Two researchers independently screened articles for eligibility with set inclusion criteria. They extracted and analysed key data using both quantitative and qualitative methods.

RESULTS

Of the 2972 records retrieved, 120 articles were included in the review. More than 90% of these articles were published in the last 10 years. The definition of spaced learning varied widely and was often not theoretically grounded. Spaced learning was applied in distinct contexts, including online learning, simulation training and classroom settings. There was a large variety of spacing formats, ranging from dispersion of information or practice on a single day, to intervals lasting several months. Generally, spaced learning was implemented in practice or testing phases and rarely during teaching.

CONCLUSIONS

Spaced learning is infrequently and poorly defined in the health professions education literature. We propose a comprehensive definition of spaced learning and emphasise that detailed descriptions of spacing formats are needed in future research to facilitate the operationalisation of spaced learning research and practice in health professions education.

Improving Outcomes

OBJECTIVES

To determine the strength of study design and outcomes in literature describing pathology education for medical students.

METHODS

A search was conducted for articles related to pathology education published over 45 years describing an educational intervention. The primary data collected included phase of education, domain of objectives, number of learners and institutions, type of intervention, use of a comparison/control group, randomization, and strength of statistical analysis.

RESULTS

Of 655 articles, 54 (8%) met inclusion criteria. The majority (65%) reported experiences of 100 learners or fewer, and only one was multi-institutional. Only 46% used a comparison/control group. Statistical significance of results was not reported in 39%. None examined outcomes at a point distant from the educational intervention.

CONCLUSIONS

Most studies describing pathology educational interventions are not of strong experimental design. Consumers of educational research should be cognizant of these potential weaknesses in educational studies.

Comparison of a newly established emotional stimulus approach to a classical assessment-driven approach in BLS training: a randomised controlled trial

OBJECTIVE

The study objective was to implement two strategies (short emotional stimulus vs announced practical assessment) in the teaching of resuscitation skills in order to evaluate whether one led to superior outcomes.

SETTING

This study is an educational intervention provided in one German academic university hospital.

PARTICIPANTS

First-yearmedical students (n=271) during the first3 weeks of their studies.

INTERVENTIONS

Participants were randomly assigned to one of two groups following a sequence of random numbers: the emotional stimulus group (EG) and the assessment group (AG). In the EG, the intervention included watching an emotionally stimulating video prior to the Basic Life Support (BLS) course. In the AG, a practical assessment of the BLS algorithm was announced and tested within a 2 min simulated cardiac arrest scenario. After the baseline testing, a standardised BLS course was provided. Evaluation points were defined 1 week and 6 months after.

PRIMARY OUTCOME MEASURES

Compression depth (CD) and compression rate (CR) were recorded as the primary endpoints for BLS quality.

RESULTS

Within the study, 137 participants were allocated to the EG and 134 to the AG. 104 participants from EG and 120 from AG were analysed1 week after the intervention, where they reached comparable chest-compression performance without significant differences (CR P=0.49; CD P=0.28). The chest-compression performance improved significantly for the EG (P<0.01) and the AG (P<0.01) while adhering to the current resuscitation guidelines criteria for CD and CR.

CONCLUSIONS

There was no statistical difference between both groups' practical chest-compression-performance. Nevertheless, the 2 min video sequence used in the EG with its low production effort and costs, compared with the expensive assessment approach, provides broad opportunities for applicability in BLS training.

Cumulative versus end-of-course assessment: effects on self-study time and test performance

CONTEXT

Students tend to postpone preparation for a test until the test is imminent, which raises various risks associated with 'cramming' behaviours, including that for suboptimal learning. Cumulative assessment utilises spaced testing to stimulate students to study more frequently and to prevent procrastination. This randomised controlled study investigated how cumulative assessment affects time spent on self-study and test performance compared with end-of-course assessment.

METHODS

A total of 78 undergraduate medical students in a Year 2 pre-clinical course were randomly assigned to either of two conditions. Students in the cumulative assessment condition were assessed in weeks 4, 8 and 10. Students in the end-of-course assessment condition were assessed in week 10 only. Each week, students reported the number of hours they spent on self-study.

RESULTS

Students in the cumulative assessment condition (n = 25) spent significantly more time on self-study than students in the end-of-course assessment condition (n = 37) in all weeks of the course except weeks 5, 9 and 10. Overall, the cumulative assessment group spent 69 hours more on self-study during the course than did the control group, although the control group spent 7 hours more in studying during the final week of the course than did the cumulative assessment group. Students in the cumulative assessment condition scored slightly higher on questions concerning the content of the last part of the course.

CONCLUSIONS

Cumulative assessment encourages students to distribute their learning activities over a course, which leaves them more opportunity to study the content of the last part of the course prior to the final examination. There was no evidence for a short-term effect of cumulative assessment on overall knowledge gain. We hypothesise that larger positive effects might be found if retention were to be measured in the long term.

Human structure in six and one-half weeks: one approach to providing foundational anatomical competency in an era of compressed medical school anatomy curricula

The University of Oklahoma College of Medicine reduced gross anatomy from a full semester, 130-hour course to a six and one-half week, 105-hour course as part of a new integrated systems-based pre-clinical curriculum. In addition to the reduction in contact hours, content from embryology, histology, and radiology were added into the course. The new curriculum incorporated best practices in the area of regular assessments, feedback, clinical application, multiple teaching modalities, and professionalism. A comparison of the components of the traditional and integrated curriculum, along with end of course evaluations and student performance revealed that the new curriculum was just as effective, if not more effective. This article also provides important lessons learned.

Cumulative assessment: strategic choices to influence students' study effort

BACKGROUND

It has been asserted that assessment can and should be used to drive students' learning. In the current study, we present a cumulative assessment program in which test planning, repeated testing and compensation are combined in order to influence study effort. The program is aimed at helping initially low-scoring students improve their performance during a module, without impairing initially high-scoring students' performance. We used performance as a proxy for study effort and investigated whether the program worked as intended.

METHODS

We analysed students' test scores in two second-year (n = 494 and n = 436) and two third-year modules (n = 383 and n = 345) in which cumulative assessment was applied. We used t-tests to compare the change in test scores of initially low-scoring students with that of initially high-scoring students between the first and second subtest and again between the combined first and second subtest and the third subtest. During the interpretation of the outcomes we took regression to the mean and test difficulty into account.

RESULTS

Between the first and the second subtest in all four modules, the scores of initially low-scoring students increased more than the scores of initially high-scoring students decreased. Between subtests two and three, we found a similar effect in one module, no significant effect in two modules and the opposite effect in another module.

CONCLUSION

The results between the first two subtests suggest that cumulative assessment may positively influence students' study effort. The inconsistent outcomes between subtests two and three may be caused by differences in perceived imminence, impact and workload between the third subtest and the first two. Cumulative assessment may serve as an example of how several evidence-based assessment principles can be integrated into a program for the benefit of student learning.

Immediate detailed feedback to test-enhanced learning: an effective online educational tool

BACKGROUND

Test-enhanced learning has gained popularity because it is an effective way to increase retention of knowledge; provided the student receives the correct answer soon after the test is taken.

AIM

To determine whether detailed feedback provided to test-enhanced learning questions is an effective online educational tool for improving performance on complex biomedical information exams.

METHODS

A series of online multiple choice tests were developed to test knowledge of biomedical information that students were expected to know after each patient-case. Following submission of the student answers, one cohort (n = 52) received answers only while the following year, a second cohort (n = 51) received the answers with detailed feedback explaining why each answer was correct or incorrect.

RESULTS

Students in both groups progressed through the series of online tests with little assessor intervention. Students receiving the answers along with the explanations within their feedback performed significantly better in the final biomedical information exam than those students receiving correct answers only.

CONCLUSIONS

This pilot study found that the detailed feedback to test-enhanced learning questions is an important online learning tool. The increase in student performance in the complex biomedical information exam in this study suggests that detailed feedback should be investigated not only for increasing knowledge, but also be investigated for its effect on retention and application of knowledge.

The overall impact of testing on medical student learning: quantitative estimation of consequential validity

Given medical education's longstanding emphasis on assessment, it seems prudent to evaluate whether our current research and development focus on testing makes sense. Since any intervention within medical education must ultimately be evaluated based upon its impact on student learning, this report seeks to provide a quantitative accounting of the learning gains attained through educational assessments. To approach this question, we estimate achieved learning within a medical school environment that optimally utilizes educational assessments. We compare this estimate to learning that might be expected in a medical school that employs no educational assessments. Effect sizes are used to estimate testing's total impact on learning by summarizing three effects; the direct effect, the indirect effect, and the selection effect. The literature is far from complete, but the available evidence strongly suggests that each of these effects is large and the net cumulative impact on learning in medical education is over two standard deviations. While additional evidence is required, the current literature shows that testing within medical education makes a strong positive contribution to learning.

Explicit feedback to enhance the effect of an interim assessment: a cross-over study on learning effect and gender difference

In a previous study we demonstrated by a prospective controlled design that an interim assessment during an ongoing small group work (SGW) session resulted in a higher score in the course examination. As this reflects the so-called testing effect, which is supposed to be enhanced by feedback, we investigated whether feedback following an interim assessment would have an effect on the score of the course exam, and whether the effect is influenced by the gender of the student. During a General Pathology bachelor course all 386 (bio) medical students took an interim assessment on the topics cell damage (first week) and tumour pathology (fourth week). The intervention consisted of immediate detailed oral feedback on the content of the questions of the interim assessment by the tutor, including the rationale of the correct and incorrect answers. It concerned a prospective randomized study using a cross-over design. Outcome measures were: (1) the difference in the normalized scores (1-10) of the course examination multiple choice questions related to the two topics, (2) effect of gender, and (3) gender-specific scores on formal examination. The effect of feedback was estimated as half the difference in the outcome between the two conditions. Mixed-model analysis was used whereby the SGW group was taken as the study target. The scores of the questions on cell damage amounted to 7.70 (SD 1.59) in the group without and 7.78 (SD 1.39) in the group with feedback, and 6.73 (SD 1.51) and 6.77 (SD 1.60), respectively, for those on tumour pathology. No statistically significant effect of feedback was found: 0.02 on a scale of 1-10 (95 % CI: -0.20; 0.25). There were no significant interactions of feedback with gender. Female students scored 0.43 points higher on the formal examination in comparison with their male colleagues. No additional effect of immediate explicit feedback following an interim assessment during an SGW session in an ongoing bachelor course could be demonstrated in this prospective randomized controlled study. Gender analysis revealed a higher performance of female students on the formal examination, which could not be explained by the effect of feedback in the current study. In this particular learning environment, SGW, explicit feedback may have little added value to the interactive learning that includes implicit feedback.

On doctor-patient relationship and feedback interventions

None