Distributed Learning: Data, Metacognition, and Educational Implications

Can written prompts help medical residents to accurately monitor their own communication skills and those of others?

In healthcare, effective communication in complex situations such as end of life conversations is critical for delivering high quality care. Whether residents learn from communication training with actors depends on whether they are able to select appropriate information or 'predictive cues' from that learning situation that accurately reflect their or their peers' performance and whether they use those cues for ensuing judgement. This study aimed to explore whether prompts can help medical residents improving use of predictive cues and judgement of communication skills. First and third year Kenyan residents (N = 41) from 8 different specialties were randomly assigned to one of two experimental groups during a mock OSCE assessing advanced communication skills. Residents in the intervention arm received paper predictive cue prompts while residents in the control arm received paper regular prompts for self-judgement. In a pre- and post- test, residents' use of predictive cues and the appropriateness of peer-judgements were evaluated against a pre-rated video of another resident. The intervention improved both the use of predictive cues in self-judgement and peer-judgement. Ensuing accuracy of peer-judgements in the pre- to post-test only partly improved: no effect from the intervention was found on overall appropriateness of judgements. However, when analyzing participants' completeness of judgements over the various themes within the consultation, a reduction in inappropriate judgments scores was seen in the intervention group. In conclusion, predictive cue prompts can help learners to concentrate on relevant cues when evaluating communication skills and partly improve monitoring accuracy. Future research should focus on offering prompts more frequently to evaluate whether this increases the effect on monitoring accuracy in communication skills.

Assessing the Use of Microlearning for Preceptor Development

The objective of this study was to evaluate microlearning as a preceptor development method compared to a traditional method of learning. Twenty-five preceptor participants volunteered to engage in a learning intervention about two preceptor development topics. Participants were randomized 1:1 to either a thirty-minute traditional learning experience or a fifteen-minute microlearning experience; participants then crossed over to the other intervention for comparison. Primary outcomes were satisfaction, changes in knowledge, self-efficacy, and perception of behavior, confidence scale, and self-reported frequency of behavior, respectively. One-way repeated measures ANOVA and Wilcoxon paired t-tests were used to analyze knowledge and self-efficacy, and Wilcoxon paired t-tests were utilized to assess satisfaction and perception of behavior. Most participants preferred microlearning over the traditional method (72% vs. 20%, p = 0.007). Free text satisfaction responses were analyzed using inductive coding and thematic analysis. Participants reported that microlearning was more engaging and efficient. There were no significant differences in knowledge, self-efficacy, or perception of behavior between microlearning and the traditional method. Knowledge and self-efficacy scores for each modality increased compared to the baseline. Microlearning shows promise for educating pharmacy preceptors. Further study is needed to confirm the findings and determine optimal delivery approaches.

The development and preliminary evaluation of a clinician e-learning training platform for a neonatal sepsis risk monitor for use in ICU settings

BACKGROUND

Training clinicians on the use of hospital-based patient monitoring systems (PMS) is vital to mitigate the risk of use errors and of frustration using these devices, especially when used in ICU settings. PMS training is typically delivered through face-to-face training sessions in the hospital. However, it is not always feasible to deliver training in this format to all clinical staff given some constraints (e.g., availability of staff and trainers to attend in-person training sessions and the costs associated with face-to-face training).

OBJECTIVE

The literature indicates that E-learning has the potential to mitigate barriers associated with time restrictions for trainers and trainees and evidence shows it to be more flexible, and convenient for learners in healthcare settings. This study aimed to develop and carry out a preliminary evaluation via a case study of an e-learning training platform designed for a novel neonatal sepsis risk monitor system (Digi-NewB).

METHODS

A multi-modal qualitative research case study approach was used, including the analysis of three qualitative data sources: (i) audio/video recordings of simulation sessions in which participants were asked to operate the system as intended (e.g., update the clinical observations and monitor the sepsis risk), (ii) interviews with the simulation participants and an attending key opinion leader (KOL), who observed all simulation sessions, and (iii) post-simulation survey.

RESULTS

After receiving ethical approval for the study, nine neonatal intensive care unit (NICU) nurses completed the online training and participated in the simulation and follow-up interview sessions. The KOL was also interviewed, and seven out of the nine NICU nurses answered the post-simulation survey. The video/audio analysis of the simulations revealed that participants were able to use and interpret the Digi-NewB interface. Interviews with simulation participants and the KOL, and feedback extracted from the survey, revealed that participants were overall satisfied with the training platform and perceived it as an efficient and effective method to deliver medical device training.

CONCLUSIONS

This study developed an online training platform to train clinicians in the use of a critical care medical device and carried out a preliminary evaluation of the platform via a case study. The e-learning platform was designed to supplement and enhance other training approaches. Further research is required to evaluate the effectiveness of this approach.

How to Promote Lasting Learning in Schools

Abstract. Creating lasting knowledge is an important goal of education. But how much do students retain what they have learned in school beyond the next class assignment? Is school instruction suitable for creating lasting knowledge and skills? And what can teachers do to foster the learning of lasting knowledge? We present a selective overview of research on these questions. The two theoretical strands that deal with lasting learning are meaningful learning and desirable difficulties in learning. We propose combining ideas from these two approaches to develop a comprehensive theoretical account of lasting learning and sketch questions that research should clarify to enable such a theory.

Brief memory reactivations induce learning in the numeric domain

Learning of arithmetic facts such as the multiplication table requires time-consuming, repeated practice. In light of evidence indicating that reactivation of encoded memories can modulate learning and memory processes at the synaptic, system and behavioral levels, we asked whether brief memory reactivations can induce human learning in the numeric domain. Adult participants performed a number-fact retrieval task in which they learned arbitrary numeric facts. Following encoding and a baseline test, 3 passive, brief reactivation sessions of only 40 s each were conducted on separate days. Learning was evaluated in a retest session. Results showed reactivations induced learning, with improved performance at retest relative to baseline test. Furthermore, performance was superior compared to a control group performing test-retest sessions without reactivations, who showed significant memory deterioration. A standard practice group completed active-retrieval sessions on 3 separate days, and showed significant learning gains. Interestingly, while these gains were higher than those of the reactivations group, subjects showing reactivation-induced learning were characterized by superior efficiency relative to standard practice subjects, with higher rate of improvement per practice time. A follow-up long-term retention experiment showed that 30 days following initial practice, weekly brief reactivations reduced forgetting, with participants performing superior to controls undergoing the same initial practice without reactivations. Overall, the results demonstrate that brief passive reactivations induce efficient learning and reduce forgetting within a numerical context. Time-efficient practice in the numeric domain carries implications for enhancement of learning strategies in daily-life settings.

Evidence of the Spacing Effect and Influences on Perceptions of Learning and Science Curricula

The conventional science curricula generally favour educational practices that yield high scores on immediate examination, though it may not accurately predict students’ long-term academic achievement. In view of the pre-exam cramming phenomenon, this article shows the evidence of spacing effect in science education and probes into its theoretical mechanisms, effectiveness in experimental settings, and current applications in science learning. In brief, spacing works by repeatedly presenting the learning material across various temporal intervals. This paper suggests that spacing could significantly result in greater memory strength by alleviating multiple neurocognitive and behavioural properties of learning that are hampered by cramming. Together with the analysis of its relevance in science education, the spacing effect may further provide leverages for promoting long-term conceptual understanding and reflective skill development. However, there are many reasons that students and teachers may not be aware of or fully appreciate its benefits. Finally, this article discusses systemic barriers to why spaced repetition is underutilized in science curricula.

Beyond the Distributed Practice Effect: Is Distributed Learning Also Effective for Learning With Non-repeated Text Materials?

Distributed learning is often recommended as a general learning strategy, but previous research has established its benefits mainly for learning with repeated materials. In two experiments, we investigated distributed learning with complementary text materials. 77 (Experiment 1) and 130 (Experiment 2) seventh graders read two texts, massed vs. distributed, by 1 week (Experiment 1) or 15 min (Experiment 2). Learning outcomes were measured immediately and 1 week later and metacognitive judgments of learning were assessed. In Experiment 1, distributed learning was perceived as more difficult than massed learning. In both experiments, massed learning led to better outcomes immediately after learning but learning outcomes were lower after 1 week. No such decrease occurred for distributed learning, yielding similar outcomes for massed and distributed learning after 1 week. In sum, no benefits of distributed learning vs. massed learning were found, but distributed learning might lower the decrease in learning outcomes over time.

Development of a Novel and Scalable Simulation-Based Teamwork Training Model Using Within-Group Debriefing of Observed Video Simulation

THE CHALLENGE

Effective teamwork and communication skills are essential for safe and reliable health care. These skills require training and practice. Experiential learning is optimal for training adults, and the industry has recognized simulation training as an exemplar of this approach. Yet despite decades of investment, this training is inaccessible and underutilized for most of the more than 12 million health care professionals in the United States.

DESIGNING A SOLUTION

This report describes the design process of an adapted simulation training created to overcome the key barriers to scaling simulation-based teamwork training: access to technology, time away from clinical work, and availability of trained simulation educators. The prototype training is designed for delivery in one-hour segments and relies on observation of video simulation scenarios and within-group debriefing, which are promising variations on traditional simulation training. To our knowledge, these two simulation approaches have not been previously combined. The resulting prototype minimizes the need for an on-site trained simulation educator. This report details the development of a training model, its subsequent modification based on pilot testing, and the evaluation of the resulting redesigned prototype.

PRELIMINARY EVALUATION

Participant evaluations of the redesigned prototype were highly positive, with 92% reporting that they would like to participate in additional, similar training sessions. Positive results were also found in assessment of feasibility, acceptability, psychological safety, and behavioral intention (reported intention to alter behavior).

Prior failures, laboring in vain, and knowing when to give up: Incremental versus entity theories

Against intuition, a set of "desirable difficulties" has been touted as a way in which to improve learning and lengthen retention. This includes, for instance, varying the conditions of learning to allow for more active, effortful, or challenging, contexts. In the current paper, we introduce data that show that, on the contrary, learning to know when to take the easy road may be crucial when it comes to avoiding "laboring in vain." We presented participants with prior problems - either easy or difficult - followed by choices of selecting an easy or a difficult current problem. Our primary goal was to examine the notion that past failures (which are more likely on the difficult prior items) may be a basis for allowing learners to then choose the easy rather than the difficult current problem. In other words, if one has labored in vain already, the easier items may now be more desirable. In addition, we compare the selections that are made between incremental and entity perspectives, given their fundamentally opposing views on effort. Our results showed that, interestingly, incremental theorists, who generally are proponents of effort, were more likely to select the easy problems, but only when they had experienced failure on prior, and similar, difficult tasks. We interpret these data to suggest that those holding an incremental view may be more in tune with their past efforts, resulting in a Metacognition-by-Experience, or ME strategy, and also hint at its generalizability through cross-cultural comparisons.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11409-020-09253-5.

How are SLPs managing services for people with mild aphasia?

At present, there is limited information on the nature and extent of speech-language pathology services in the U.S. specific to people with mild aphasia (PWMA). The goal of the present study was to determine how speech-language pathologists (SLPs) perceive the assessment, treatment, and discharge of PWMA. According to anecdotal reports from SLPs and PWMA, current services may be inadequate and those with more severe aphasia types are being prioritized. To investigate factors potentially limiting SLP involvement and to determine gaps in service, a 26-question survey, consisting of both open-ended and closed-ended questions, was developed and distributed via email and social media to selected groups of SLPs. A total of 133 SLPs responded to the survey. In this brief report, we discuss only the primary findings. Reports of assessment, treatment, and discharge procedures were diverse; however, a number of central themes were observed. The majority of respondents had more than ten years of experience. They reported greater use of informal measures in assessing people with mild aphasia as compared to people with more severe forms of aphasia. Discharge procedures and recommendations were found to involve primarily referrals to the next level of care, recommendations for continued social interaction, and maintenance programs to preserve treatment progress. Responses tended to be appropriate, yet, not necessarily indicative of the actual standard of care. SLPs reported the need for additional assessment tools and resources in order to better serve people with mild aphasia. Perceived constraints included large caseloads, resource limitations, awareness of health professionals, and the prioritization of those with more severe deficits. Information gathered in this survey highlights the gap in services for this population and provides some tangible guidelines for how to proceed. Given that PWMA have the greatest chance of returning to work, we argue that any prioritization bias should lean toward favoring this population.

Promoting Low-Income Preschoolers' Vocabulary Learning From Educational Media: Does Repetition Support Memory for Learned Word Knowledge?

Children from diverse backgrounds are able to learn new words from educational media. However, learning is often partial and fragile, leaving much room for uncovering strategies that can increase the efficacy of educational media in supporting children's vocabulary knowledge. The present study investigated one such strategy—repeated viewing of educational media—in a sample of low-income preschoolers. One hundred thirty one preschoolers were randomly assigned to view an educational media clip teaching three vocabulary words in one of three conditions: (a) once, (b) three times in immediate succession (massed repetition), or (c) three times with views spaced 1 hour apart (spaced repetition). Children completed a target vocabulary assessment both immediately after the final view and 1 week later. Results indicate that certain types of word knowledge were supported by repetition, particularly spaced repetition. Children also effectively retained the vocabulary knowledge they acquired from educational media over a 1-week period in all conditions. This suggests that educational media is a strong platform for teaching low-income preschoolers new words, and that spaced repetition might further support low-income preschoolers' vocabulary learning.

Distributed Learning in the Classroom: Effects of Rereading Schedules Depend on Time of Test

Research with adults in laboratory settings has shown that distributed rereading is a beneficial learning strategy but its effects depend on time of test. When learning outcomes are measured immediately after rereading, distributed rereading yields no benefits or even detrimental effects on learning, but the beneficial effects emerge two days later. In a preregistered experiment, the effects of distributed rereading were investigated in a classroom setting with school students. Seventh-graders (N = 191) reread a text either immediately or after 1 week. Learning outcomes were measured after 4 min or 1 week. Participants in the distributed rereading condition reread the text more slowly, predicted their learning success to be lower, and reported a lower on-task focus. At the shorter retention interval, massed rereading outperformed distributed rereading in terms of learning outcomes. Contrary to students in the massed condition, students in the distributed condition showed no forgetting from the short to the long retention interval. As a result, they performed equally well as the students in the massed condition at the longer retention interval. Our results indicate that distributed rereading makes learning more demanding and difficult and leads to higher effort during rereading. Its effects on learning depend on time of test, but no beneficial effects were found, not even at the delayed test.

Factors Affecting Student Time to Examination Completion

Objective. To investigate factors (prior or current knowledge, metacognitive accuracy, and personality) that might impact the time it takes students to complete an examination. Methods. On the final examination, the time to complete the examination was recorded. Prior to the course, students completed the five-factor personality assessment. During the semester, students completed four cumulative assessments that included prospective judgments of performance to improve their metacognitive accuracy. Measures of metacognitive accuracy were calculated from the difference between the students' prospective judgments of performance and their actual assessment performance for the final examination. Two weeks prior to the final examination, students completed a cumulative assessment, which served as prior knowledge; this was similar in content to the final examination. Results. The time to complete the final examination was significantly negatively correlated with examination score and positively correlated with Agreeableness, and degree of metacognitive bias. However, only current knowledge (β=-.35) and Agreeableness (β=.12) predicted the time to complete the final examination. These two factors explained about 14% of the variability in completion times. Examining the scale for the time to complete the examination, there were some regional differences between the slowest, intermediate and fastest completers. Conclusion. Current knowledge and to a lesser extent, pro-social behavior (agreeableness) influenced examination completion time. Metacognitive accuracy had limited predictability in time to complete the examination.

Avoiding Surgical Skill Decay: A Systematic Review on the Spacing of Training Sessions

OBJECTIVE

Spreading training sessions over time instead of training in just 1 session leads to an improvement of long-term retention for factual knowledge. However, it is not clear whether this would also apply to surgical skills. Thus, we performed a systematic review to find out whether spacing training sessions would also improve long-term retention of surgical skills.

DESIGN

We searched the Medline, PsycINFO, Embase, Eric, and Web of Science online databases. We only included articles that were randomized trials with a sample of medical trainees acquiring surgical motor skills in which the spacing effect was reported. The quality and bias of the articles were assessed using the Cochrane Collaboration's risk of bias assessment tool.

RESULTS

With respect to the spacing effect, 1955 articles were retrieved. After removing duplicates and articles that did not meet the inclusion criteria, 11 articles remained. The overall quality of the experiments was "moderate." Trainees in the spaced condition scored higher in a retention test than students in the massed condition.

CONCLUSIONS

Our systematic review showed evidence that spacing training sessions improves long-term surgical skills retention when compared to massed practice. However, the optimal gap between the re-study sessions is unclear.

Simultaneous utilization of multiple cues in judgments of learning

There is much evidence that metacognitive judgments, such as people's predictions of their future memory performance (judgments of learning, JOLs), are inferences based on cues and heuristics. However, relatively little is known about whether and when people integrate multiple cues in one metacognitive judgment or focus on a single cue without integrating further information. The current set of experiments systematically addressed whether and to what degree people integrate multiple extrinsic and intrinsic cues in JOLs. Experiment 1 varied two cues: number of study presentations (1 vs. 2) and font size (18 point vs. 48 point). Results revealed that people integrated both cues in their JOLs. Experiment 2 demonstrated that the two word characteristics concreteness (abstract vs. concrete) and emotionality (neutral vs. emotional) were integrated in JOLs. Experiment 3 showed that people integrated all four cues in their JOLs when manipulated simultaneously. Finally, Experiment 4 confirmed integration of three cues that varied on a continuum rather than in two easily distinguishable levels. These results demonstrate that people have a remarkable capacity to integrate multiple cues in metacognitive judgments. In addition, our findings render an explanation of cue effects on JOLs in terms of demand characteristics implausible.

Null effects of boot camps and short-format training for PhD students in life sciences

Many PhD programs incorporate boot camps and summer bridge programs to accelerate the development of doctoral students' research skills and acculturation into their respective disciplines. These brief, high-intensity experiences span no more than several weeks and are typically designed to expose graduate students to data analysis techniques, to develop scientific writing skills, and to better embed incoming students into the scholarly community. However, there is no previous study that directly measures the outcomes of PhD students who participate in such programs and compares them to the outcomes of students who did not participate. Likewise, no previous study has used a longitudinal design to assess these outcomes over time. Here we show that participation in such programs is not associated with detectable benefits related to skill development, socialization into the academic community, or scholarly productivity for students in our sample. Analyzing data from 294 PhD students in the life sciences from 53 US institutions, we found no statistically significant differences in outcomes between participants and nonparticipants across 115 variables. These results stand in contrast to prior studies presenting boot camps as effective interventions based on participant satisfaction and perceived value. Many universities and government agencies (e.g., National Institutes of Health and National Science Foundation) invest substantial resources in boot camp and summer bridge activities in the hopes of better supporting scientific workforce development. Our findings do not reveal any measurable benefits to students, indicating that an allocation of limited resources to alternative strategies with stronger empirical foundations warrants consideration.

Evaluating the effect of instruction and practice schedule on the acquisition of ECG interpretation skills

INTRODUCTION

Evidence of the benefit of distributed instruction and interleaved practice comes from studies using simple materials (e. g. word pairs). Furthermore, there is currently no evidence of the combined impact of these strategies in undergraduate medical education. The present study evaluated the impact of varying both instruction and practice schedules for the acquisition of ECG interpretation skills.

METHODS

We conducted a 2 × 2 factorial study with two levels of instruction (massed and distributed) and two levels of practice (interleaved and blocked). A three-module introductory course in ECG interpretation was delivered to 80 first year medical undergraduate students. Students were assigned to one of four Instruction-Practice conditions: Massed-Interleaved, Massed-Blocked, Distributed-Interleaved and Distributed-Blocked. Learning was evaluated by a multiple choice quiz at the end of each module and a final multiple choice quiz at the end of the course.

RESULTS

End of module mean scores showed that distributed instruction was consistently superior to massed instruction (52% vs 42%, p < 0.01). However, there was no effect of practice and no interaction between teaching and practice methods. The delayed final test scores revealed an advantage for blocked over mixed practice (34% vs 24%, p < 0.05) and distributed over massed instruction (34% vs 24%, p < 0.05).

DISCUSSION

These results suggest that these popular strategies may have varying effects with complex learning materials. Further research is required to understand how these strategies affect the learning of simple and very complex skills.

An evidence-based analysis of learning practices: the need for pharmacy students to employ more effective study strategies

INTRODUCTION

Learning is a process of constructing neural connections between what is being learned and what has already been learned. Superficial thought processes associated with memorization produce shallow, short-term learning. Higher-order thought processing (critical thinking) produces deep, long-term learning. Pharmacy students should study in ways that enable them to retain and apply what they learn.

PREDOMINANT LEARNING PRACTICES

Investigators who surveyed the learning practices of pharmacy students have reported that most students resort to cramming in preparation for an upcoming exam. The practice of routinely keeping up with course material through regular study is much less common. Most students highlight or re-read material when studying rather than quizzing themselves, and many multitask or study with distractions such as texting, checking e-mails or using social media.

EVIDENCE-BASED LEARNING PRACTICES

Studies in cognitive psychology and education provide evidence to confirm the efficacy of the following learning practices: plan and manage study time, space out and repeat study, interleave (mix up) topics or methods, incorporate retrieval practice (self-quizzing, deliberative reading, or written paraphrasing), minimize distractions, leverage mistakes, and sleep at least seven hours a night.

CONCLUSION

Pharmacy students need to become proficient, lifelong learners. A superficial, memorization-oriented approach to learning is detrimental to professional growth. Faculty members should guide students to employ more effective evidence-based study strategies, while also exploring how curricular design, course content, academic policy or pedagogy might be predisposing students to pursue suboptimal learning practices. The issue calls for the academy to focus greater attention on how students learn.

An In Vivo Study of Self-Regulated Study Sequencing in Introductory Psychology Courses

Study sequence can have a profound influence on learning. In this study we investigated how students decide to sequence their study in a naturalistic context and whether their choices result in improved learning. In the study reported here, 2061 undergraduate students enrolled in an Introductory Psychology course completed an online homework tutorial on measures of central tendency, a topic relevant to an exam that counted towards their grades. One group of students was enabled to choose their own study sequence during the tutorial (Self-Regulated group), while the other group of students studied the same materials in sequences chosen by other students (Yoked group). Students who chose their sequence of study showed a clear tendency to block their study by concept, and this tendency was positively associated with subsequent exam performance. In the Yoked group, study sequence had no effect on exam performance. These results suggest that despite findings that blocked study is maladaptive when assigned by an experimenter, it may actually be adaptive when chosen by the learner in a naturalistic context.

Developmental changes and the effect of self-generated feedback in metacognitive controlled spacing strategies in 7-year-olds, 10-year-olds, and adults

The current study investigated the development of metacognitive monitoring-based control of spacing choices in children and adults. Moreover, we assessed whether metacognitive learning decisions are influenced by the effects of previous metacognitive decisions. We tested groups of 7-year-olds, 10-year-olds, and adults in a task with two learning blocks in which they needed to monitor their learning through judgments of learning (JoL) and in which they then needed to decide whether to space their study, mass it, or terminate it. Extending previous findings, our study provides the first evidence that already by 7 years of age children can make metacognitive controlled scheduling decisions. The results also revealed that adults had more clearly differentiated strategies related to their JoL. Furthermore, our study provides evidence that participants of all age groups improved their relative monitoring accuracy in the second learning block and adjusted their JoL. However, only adults changed their strategy choices.

Metacognition for spelling in higher education students with dyslexia: is there evidence for the dual burden hypothesis?

We examined whether academic and professional bachelor students with dyslexia are able to compensate for their spelling deficits with metacognitive experience. Previous research suggested that students with dyslexia may suffer from a dual burden. Not only do they perform worse on spelling but in addition they are not as fully aware of their difficulties as their peers without dyslexia. According to some authors, this is the result of a worse feeling of confidence, which can be considered as a form of metacognition (metacognitive experience). We tried to isolate this metacognitive experience by asking 100 students with dyslexia and 100 matched control students to rate their feeling of confidence in a word spelling task and a proofreading task. Next, we used Signal Detection Analysis to disentangle the effects of proficiency and criterion setting. We found that students with dyslexia showed lower proficiencies but not suboptimal response biases. They were as good at deciding when they could be confident or not as their peers without dyslexia. They just had more cases in which their spelling was wrong. We conclude that the feeling of confidence in our students with dyslexia is as good as in their peers without dyslexia. These findings go against the Dual Burden theory (Krüger & Dunning, 1999), which assumes that people with a skills problem suffer twice as a result of insufficiently developed metacognitive competence. As a result, there is no gain to be expected from extra training of this metacognitive experience in higher education students with dyslexia.