Retrieval Practice Consistently Benefits Student Learning: a Systematic Review of Applied Research in Schools and Classrooms

Nursing students' experiences with test-enhanced learning in teams: A cross-sectional study

BACKGROUND

Many nursing students struggle with the disciplines of biosciences, particularly Anatomy, physiology, and biochemistry, which are introduced in the first year. Nursing students' motivation, prior knowledge, and academic performance matter, but teaching methods may also influence students' learning process. Retrieving knowledge through testing has previously proven to enhance learning to a greater extent than time spent on other classroom activities.

OBJECTIVE

The aim of this study was to explore nursing students' experiences with test-enhanced learning as a way of enhancing learning in Anatomy, physiology, and biochemistry.

DESIGN

The lectures in each topic were followed by testing five days later. The tests were typically multiple-choice tests with short reply-times. The effect was measured in terms of students' self-reported level of satisfaction with test-enhanced learning, and their performance on the final exam in Anatomy, physiology, and biochemistry. The tests were performed in teams to avoid stressful situations that could negatively affect the students' learning process.

RESULTS

A key achievement from introducing test-enhanced learning in the Anatomy, physiology, and biochemistry course was a perceived higher learning outcome and increased engagement and motivation among the students, resulting in resulting in more students achieving the highest grades (A and B). However, the students' academic results from upper secondary school also seemed to matter for their achievements on the final exam.

CONCLUSION

These results indicated that many students benefited from test-enhanced learning, suggesting that test-enhanced learning can be an important teaching strategy in nursing education, particularly for biosciences.

The effect of retrieval practice on vocabulary learning for DHH children

On average, deaf and hard-of-hearing (DHH) children have difficulty developing expressive spoken vocabulary comparable to hearing peers. Yet, there are no evidence-based practices to guide classroom instruction for teachers of the deaf. Retrieval practice-a robust learning strategy-has been shown to improve children's retention of vocabulary, but it has not been investigated with DHH children who use listening and spoken language. The present study examined whether DHH children benefit from using retrieval practice to learn new vocabulary. Sixteen DHH children (in the age range of 5.0-8.11 years) were taught a set of new vocabulary words using retrieval practice or repeated exposure. A recall test was administered two days later. Results showed that DHH children were twice as likely to recall a word taught through retrieval practice than exposure (OR = 2.01, p = .02). Presence of an additional diagnosis and number of practice trials were also significant predicting factors of vocabulary learning.

Testing the Effects of Individual Residents' Retrieval Practice on Standardized Examination Scores

Background

Residents have limited time and much to learn. Retrieval practice-studying using exam-style review questions-is a powerful educational tool for aggregate groups of learners, but it has not been tested for individual graduate medical education (GME) trainees.

Methods

We conducted a retrospective observational study examining how individual residents' self-directed retrieval practice affected their learning. We used linear regression models to compare the number of exam-style multiple-choice practice questions each resident answered per year to their scores on annual In-Training Exams (ITE).

Results

We found that the higher a resident's ITE score was at the start of a year, the fewer practice questions they answered that year for both first- (p = 0.023) and second-year (p = 0.020) trainees. Then, the more questions a resident answered in a year, the more their ITE score increased over that year for both first- and second-year trainees (p = 0.026 and 0.025, respectively). Residents' prior ITE scores also independently predicted their subsequent ITE scores (p = 0.024 in the first and 0.007 in the second year), and the effect of their baseline scores was larger than the effect of the number of practice questions they answered.

Conclusions

Individual residents' prior exam scores significantly predict their future exam scores, and their independent retrieval practice using exam-style review questions also significantly predicts the annual improvement in their scores. These findings add to the growing evidence supporting retrieval practice in GME, and they can inform individualized educational coaching for GME trainees.

The effect of a Schema-based method on correcting persistent errors in mental arithmetic: an experimental study

Introduction

Arithmetic calculation is a fundamental skill for mathematical learning and daily life. However, elementary school students often make errors in practice.

Methods

Grounded in the schema theory and the memory retrieval theory of mental arithmetic, this study employs a controlled experiment to investigate the effect of a schema-based method in correcting persistent errors in mental arithmetic, specifically in the context of simple addition operations. The experimental group utilizes a schema-based method to help participants rectify incorrect answers in memory retrieval, while the control group did not receive this treatment.

Results

The results showed that significant differences emerged between the experimental and control groups in both the post-test performance and the reduction of persistent error count, indicating that the experimental group had rectified incorrect answers in memory; and persistent errors in simple addition were indeed caused by interference from incorrect answers during memory retrieval; and the schema-based method proves to be effective.

Discussion

The findings of this study contribute to enhancing practical mental arithmetic instruction, assisting students in correcting relevant errors, and improving their mental arithmetic abilities. Not only does it offer directive guidance for teaching practices, but it also provides an enlightening reference for promoting innovative teaching methods.

Terracotta: A tool for conducting experimental research on student learning

For researchers seeking to improve education, a common goal is to identify teaching practices that have causal benefits in classroom settings. To test whether an instructional practice exerts a causal influence on an outcome measure, the most straightforward and compelling method is to conduct an experiment. While experimentation is common in laboratory studies of learning, experimentation is increasingly rare in classroom settings, and to date, researchers have argued it is prohibitively expensive and difficult to conduct experiments on education in situ. To address this challenge, we present Terracotta (Tool for Education Research with RAndomized COnTrolled TriAls), an open-source web application that integrates with a learning management system to provide a comprehensive experimental research platform within an online class site. Terracotta automates randomization, informed consent, experimental manipulation of different versions of learning activities, and export of de-identified research data. Here we describe these features, and the results of a live classroom demonstration study using Terracotta, a preregistered replication of McDaniel et al. (Journal of Applied Research in Memory and Cognition, 1(1), 18-26, 2012). Using Terracotta, we experimentally manipulated online review assignments so that consenting students alternated, on a weekly basis, between taking multiple-choice quizzes (retrieval practice) and reading answers to these quizzes (restudy). Students' performance on subsequent exams was significantly improved for items that had been in retrieval practice review assignments. This successful replication demonstrates that Terracotta can be used to experimentally manipulate consequential aspects of students' experiences in education settings.

Learning from errors versus explicit instruction in preparation for a test that counts

BACKGROUND

Although the generation of errors has been thought, traditionally, to impair learning, recent studies indicate that, under particular feedback conditions, the commission of errors may have a beneficial effect.

AIMS

This study investigates the teaching strategies that facilitate learning from errors.

MATERIALS AND METHODS

This 2-year study, involving two cohorts of ~88 students each, contrasted a learning-from-errors (LFE) with an explicit instruction (EI) teaching strategy in a multi-session implementation directed at improving student performance on the high-stakes New York State Algebra 1 Regents examination. In the LFE condition, instead of receiving instruction on 4 sessions, students took mini-tests. Their errors were isolated to become the focus of 4 teacher-guided feedback sessions. In the EI condition, teachers explicitly taught the mathematical material for all 8 sessions.

RESULTS

Teacher time-on in the LFE condition produced a higher rate of learning than did teacher time-on in the EI condition. The learning benefit in the LFE condition was, however, inconsistent across teachers. Second-by-second analyses of classroom activities, directed at isolating learning-relevant differences in teaching style revealed that a highly interactive mode of engaging the students in understanding their errors was more conducive to learning than was teaching directed at getting to the correct solution, either by lecturing about corrections or by interaction focused on corrections.

CONCLUSION

These results indicate that engaging the students interactively to focus on errors, and the reasons for them, facilitates productive failure and learning from errors.

The effects of retrieval versus study on analogical problem solving

We report four experiments, wherein subjects engaged in either problem-solving practice or example study. First, subjects studied an example problem. Subjects in the example study condition then studied two more analogous problems, whereas subjects in the problem-solving practice conditions solved two such problems, each followed by correct-answer feedback. In Experiment 1, subjects returned 1 week later and completed a posttest on an analogous problem; in Experiments 2-4, subjects completed this posttest immediately after the learning phase. Additionally, Experiment 3 consisted of a control condition, wherein subjects solved these same problems, but did not receive feedback. Experiments 3 and 4 also included a mixed study condition, wherein subjects studied two examples and then solved one with feedback during the learning phase. Across four experiments, we found that the training conditions (i.e., problem-solving practice, mixed, and example study) performed equally well on the posttest. Moreover, subjects in the training conditions outperformed control subjects on the posttest, indicating that the null findings were due to the training conditions learning and transferring their knowledge equally well. After the posttest in Experiment 4, subjects were asked to solve repeated problems from the learning phase. Subjects in the problem-solving practice and mixed study conditions performed better on repeated problems than subjects in the example study condition, indicating that they better learned the solution strategies for these problems than subjects in the example study condition. Nevertheless, this benefit was insufficient to produce differential transfer of learning among the training conditions on the posttest.

Retrieval practice benefits for spelling performance in fifth-grade children

Retrieval practice typically benefits learning in children, although little is known about the benefits of retrieval practice for learning spelling. We investigated this issue in three experiments with fifth-grade children from a low-income area of Brazil. In the experiments, children first read a list of words (study), and after a short interval wrote down the studied words after hearing and rereading them (copy) or after only hearing them (retrieval practice). After an interval of 4 days, spelling performance was greater for words from the retrieval practice condition than for words from the copy condition, but only when immediate corrective feedback was provided (Experiment 3). The current findings, therefore, suggest that retrieval practice followed by corrective feedback is an effective strategy to improve spelling performance of fifth-grade children.

Test Experience, Direct Instruction, and Their Combination Promote Accurate Beliefs about the Testing Effect

Practice testing is a highly robust learning strategy that promotes long-term retention, especially in comparison to more passive strategies such as restudying-a finding referred to as the testing effect. However, learners do not always appreciate the memorial benefits of practice testing over restudying, which could limit their use of practice testing during self-regulated learning. The current investigation explored the extent to which learners' metacognitive judgments about the testing effect can be improved via test experience, direct instruction, or a combination of both techniques. Prolific participants underwent two learning cycles. In the first cycle, participants were randomly assigned to either (a) experience a testing effect in their own memory performance (i.e., study unrelated word pairs, practice half the pairs through restudying and half through testing with correct-answer feedback, complete a critical test on the pairs, and receive feedback regarding their performance after using each strategy); (b) imagine they had to learn word pairs and read a passage on the purported benefits of practice testing; or (c) undergo both procedures. In the second cycle, all participants learned a novel set of word pairs. Across both learning cycles, participants estimated memory performance for material learned through testing versus restudying. Both test experience and direct instruction-independently and in combination-led to more accurate memory estimates across learning cycles, but no technique was more effective than the other. In summary, people can learn about the memorial benefits of practice testing when they experience a testing effect on their own memory performance and/or when they receive instruction about its benefits.

Use of practice tests with immediate feedback in an undergraduate molecular biology course

Multiple studies have shown that testing contributes to learning at all educational levels. In this observational classroom study, we report the use of a learning tool developed for a Genetics and Molecular Biology course at the college level. An interactive set of practice exams that included 136 multiple choice questions (MCQ) or matching queries was developed in the open-source Moodle platform. All MCQ questions contained four answer choices and configured for immediate feedback upon answering. Feedback consisted of providing the right answer and a short explanation of the learning objective examined. The interactive material was tested and refined for several semesters. Usefulness of this tool was assessed in two distinct settings: (1) during a face-to-face semester (Fall 2019) by comparing the grades in a final departmental exam between students who used the tool and those who did not, and (2) during an online semester (Fall 2020) by analyzing the grades in the first and last attempts on study sessions and students' performance in monthly exams. We found that when solving practice tests, students obtained a significantly higher scores in the last attempt compared with their first attempt, and that students who used the material performed better than those who did not. In all cases, answering the practice exams was optional, but students made full use of them preferentially during the online semester. This classroom research exemplifies the documented effectiveness of practice tests enhanced with feedback in biological sciences education through an open-source learning platform.

Students Can (Mostly) Recognize Effective Learning, So Why Do They Not Do It?

Cognitive psychology research has emphasized that the strategies that are effective and efficient for fostering long-term retention (e.g., interleaved study, retrieval practice) are often not recognized as effective by students and are infrequently used. In the present studies, we use a mixed-methods approach and challenge the rhetoric that students are entirely unaware of effective learning strategies. We show that whether being asked to describe strategies used by poor-, average-, and high-performing students (Study 1) or being asked to judge vignettes of students using different strategies (Study 2), participants are generally readily able to identify effective strategies: they were able to recognize the efficacy of explanation, pretesting, interpolated retrieval practice, and even some interleaving. Despite their knowledge of these effective strategies, they were still unlikely to report using these strategies themselves. In Studies 2 and 3, we also explore the reasons why students might not use the strategies that they know are effective. Our findings suggest that interventions to improve learners' strategy use might focus less on teaching them about what is effective and more on increasing self-efficacy, reducing the perceived costs, and establishing better habits.

The Testing Effect in the Lecture Hall: Does it Depend on Learner Prerequisites?

The benefits of practice testing for long-term learning are well established in many contexts. However, little is known about learner characteristics that might moderate its effectiveness. The effects of practice tests might depend on individual prerequisites for learning, especially in real-world educational settings. We explored whether the effects of practice testing in a regular university lecture would depend on cognitive (e.g., prior knowledge), motivational (e.g., learning motivation), or emotional (test anxiety) dispositions. We implemented an experimental intervention design in psychology courses for teacher students ( N  =  208). One week before the lecture, focal learner characteristics were assessed. Immediately after the lecture, participants completed an online review session with short-answer questions (practice testing with corrective feedback) or summarizing statements (restudy), alternating within each participant. One week later, retention of learning contents was assessed with a criterial test containing short-answer and multiple-choice questions. A testing effect emerged (ηp²  =  .07), with better retention for the tested compared with the restudied contents. Some learner characteristics affected learning outcomes, but no interactions with testing vs. restudy occurred. These results suggest that the testing effect in the university classroom is a robust phenomenon that benefits learning irrespective of primary individual learning prerequisites.

Attitudes of an international student cohort to the Quizlet study system employed in an advanced clinical health care review course

Computer-based learning applications and mobile technology have transformed many aspects of the educational experience over the last decade, producing software aimed at improving learning efficiency and streamlining the presentation of course materials. One such class of software, purpose-created to take advantage of spaced learning and spaced testing principles, are electronic flashcard applications. We provide a perspective on the novel use of the Quizlet flashcard application in a tertiary educational setting. To reduce cognitive load for international graduate dental students taking a pharmacology review course, we implemented Quizlet, which integrates both spaced learning and self-testing, to improve the student learning experience. This study assessed students' perceptions of the Quizlet flashcard system in a student cohort comprised of two consecutive years' classes (n = 51 students in total). Results indicated broad acceptance of Quizlet based on ease of use of the software and ease of study of the material. Our data provide insight into the use of this common software in a professional healthcare tertiary education setting and further demonstrate the successful application of electronic flashcards for a mixed international student cohort. Further research should include an assessment of the impact of flashcard on long-term knowledge retention in this setting.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s10639-022-11371-3.

Combining Retrieval Practice and Generative Learning in Educational Contexts

Abstract. Engaging learners in practicing the retrieval of learned information fosters the consolidation of learners’ mental representations and hence long-term retention. Retrieval practice research has enriched the instructional design literature by providing a wealth of evidence for these benefits of retrieval-based learning and thus emphasizing the value of means to consolidate knowledge. The present article makes the case that a fruitful next step could be to focus on the interplay between retrieval practice and generative activities. Rather than consolidating mental representations, generative activities should have as their main function the construction of coherent mental representations. Hence, from a theoretical perspective, generative activities and retrieval practice should functionally complement each other; hence, combinations of both activities might be particularly suitable to promote lasting learning. Given the challenge to beneficially combine these activities, we discuss open questions that could substantially advance both the retrieval practice and the generative learning field.

A Meta-Analysis on Mobile-Assisted Language Learning Applications: Benefits and Risks

Mobile language learning applications are a pervasive facet of modern life, however evidence on their effectiveness on L2 learning outcomes is lacking. In the current work, we sought to determine the effect of mobile language learning applications on L2 proficiency between groups who used mobile language learning applications and control groups who learned with traditional methods on L2 achievement. We systematically searched journal articles and grey literature between 2007–2019 and performed a quantitative meta-analysis based on 23 synthesized effect sizes. We also performed risk of bias and quality of evidence assessments on our included papers. We found a moderate-to-strong overall effect (g = 0.88) of learning achievement using mobile language applications compared to control groups who learned with traditional approaches. At the same time, we found high risk of bias and low quality of evidence across all included studies. Our results provide evidence for mobile applications as a beneficial tool for second language learning. However, findings should be treated with caution due to risks of high bias and low quality of evidence. Improvements for future studies are discussed.

Retrieval Practice Is Effective Regardless of Self-Reported Need for Cognition - Behavioral and Brain Imaging Evidence

There is an emerging consensus that retrieval practice is a powerful way to enhance long-term retention and to reduce achievement gaps in school settings. Less is known whether retrieval practice benefits performance in individuals with low intrinsic motivation to spend time and effort on a given task, as measured by self-reported need for cognition (NFC). Here, we examined retrieval practice in relation to individual differences in NFC by combining behavioral and functional magnetic resonance imaging (fMRI) data. Using a within-subject design, upper-secondary school students (N = 274) learned a language-based material (Swahili-Swedish word-pairs), with half of the items by means of retrieval practice with feedback and half by study only. One week later, the students were tested on the word-pairs either in the classroom (n = 204), or in a fMRI scanner (n = 70). In both settings, a retrieval practice effect was observed across different levels of NFC (high or low). Relatedly, comparable fMRI effects were seen in both NFC subgroups. Taken together, our findings provide behavioral and brain-imaging evidence that retrieval practice is effective also for individuals with lower levels of NFC, which is of direct relevance for educational practice.