Test-enhanced learning in health professions education: A systematic review: BEME Guide No. 48

Test-enhanced learning in Neuroanesthesia for the First Year anesthetic residents: a randomized controlled trial

BACKGROUND

Test enhancing learning (TEL) had shown a significant effect in promoting the learning of many learning contents. However, its effect on the postgraduate medical level was unclear. This study aimed to investigate the effect of TEL in 1st year anesthesiology residents learning neuroanesthesia.

METHOD

The residents were randomized to either group A, which was assigned to do the intervention exam (exam A) for two times during learning in neuroanesthesia, or group B, which studied in the same environment without doing the exam. All participants were assigned to do the assessment exam (exam B) at one month after the end of the rotation. All of the exams were ten multiple choice questions (MCQ). Since the anesthesia residents rotated to neuroanesthesia for two weeks twice during the first year, we conducted the experiments twice, using exams that covered both basic science (BS) and clinical science (CS) topics.

RESULTS

There was no significant difference in mean ± SD of the scores for assessment exams asking about the basic science topic (BS_B) [group A (5.25 ± 2.05) VS group B (4.90 ± 1.80); p = 0.570] and the clinical science topic (CS_B) [group A (6.30 ± 1.26) VS group B (5.95 ± 1.61); p = 0.448].

CONCLUSION

This study showed null findings on the effect of TEL on learning in residents of the first year of anesthesiology. More studies on TEL were required to confirm the effect of TEL and find the appropriate test format that could enhance learning for post-graduate medical trainees.

Effectiveness of spaced education pedagogy in enhancing Nurses' knowledge on emotional resilience-A quasi-experimental trial

INTRODUCTION

Building resilience among nurses has been postulated as one of the key strategies to support nurses and retain them in the profession. This study aimed to evaluate the effectiveness, of spaced education pedagogy in enhancing Nurses' knowledge on emotional resilience. Secondary objectives include evaluation of the usability and acceptability of delivery of the training via a mobile application in one's own mobile device.

DESIGN

A quasi-experimental study with single group pre-test and post-test trial was conducted.

METHODS

Full-time registered nurses working in an acute care hospital were invited to participate from June 2021 to June 2022. The group used the mobile application daily for 1 month. Pre-test measurement includes socio-demographic data and baseline resilience level before the intervention. Post-test measurement includes resilience level, usability and acceptability of mobile-assisted cognitive-behavioral therapy measured upon completion of the training. The mobile application enabled the delivery of resilience educational content in small quantities through a repeating manner, with a concurrent evaluation of learner's understanding.

RESULTS

When compared to their baseline (mean = 24.38, SD = 5.50), participants reported significant increase in the Connor-Davison Resilience Scale score (mean = 26.33, SD = 5.57) (t = -4.40, p < 0.001). Upon 1 month usage of the mobile application, a higher percentage of the participants reported intermediate to high level of resilience (57.4%), as compared to prior usage (54.7%). Respondents reported knowledge of most useful strategies for their daily lives including: (i) managing negative emotions (54.1%); (ii) psychoeducation about mental health and the risks of burnout (44.7%); (iii) achieving work and life balance (43.5%); and (iv) depiction of workplace scenarios to demonstrate what can be and cannot be controlled during times of change (43.5%). Participants reported usability of the mobile application with a mean SUS score 70.5 (SD = 13.0), which was considered "acceptable." Overall, 82.3% of the participants found the mobile application appealing, 64.7% reported they were likely to use the mobile application in the future and 72.9% would recommend it to other nurses.

CONCLUSION

The mobile application provided nurses with the availability and convenience to access resilience building learning content integrated with the spaced education pedagogy.

CLINICAL RELEVANCE

The use of mobile-assisted cognitive behavioral training can aid in increasing nurses' resilience level. Nurses provided acceptable usability ratings and satisfactory acceptance of receiving training via the mobile application, showing promising opportunities in the improvement of overall well-being.

A randomized controlled educational pilot trial of interictal epileptiform discharge identification for neurology residents

OBJECTIVE

To assess the effectiveness of an educational program leveraging technology-enhanced learning and retrieval practice to teach trainees how to correctly identify interictal epileptiform discharges (IEDs).

METHODS

This was a bi-institutional prospective randomized controlled educational trial involving junior neurology residents. The intervention consisted of three video tutorials focused on the six IFCN criteria for IED identification and rating 500 candidate IEDs with instant feedback either on a web browser (intervention 1) or an iOS app (intervention 2). The control group underwent no educational intervention ("inactive control"). All residents completed a survey and a test at the onset and offset of the study. Performance metrics were calculated for each participant.

RESULTS

Twenty-one residents completed the study: control (n = 8); intervention 1 (n = 6); intervention 2 (n = 7). All but two had no prior EEG experience. Intervention 1 residents improved from baseline (mean) in multiple metrics including AUC (.74; .85; p < .05), sensitivity (.53; .75; p < .05), and level of confidence (LOC) in identifying IEDs/committing patients to therapy (1.33; 2.33; p < .05). Intervention 2 residents improved in multiple metrics including AUC (.81; .86; p < .05) and LOC in identifying IEDs (2.00; 3.14; p < .05) and spike-wave discharges (2.00; 3.14; p < .05). Controls had no significant improvements in any measure.

SIGNIFICANCE

This program led to significant subjective and objective improvements in IED identification. Rating candidate IEDs with instant feedback on a web browser (intervention 1) generated greater objective improvement in comparison to rating candidate IEDs on an iOS app (intervention 2). This program can complement trainee education concerning IED identification.

Timing's not everything: Immediate and delayed feedback are equally beneficial for performance in formative multiple-choice testing

INTRODUCTION

Test-enhanced learning (TEL) is an impactful teaching and learning strategy that prioritises active learner engagement through the process of regular testing and reviewing. While it is clear that meaningful feedback optimises the effects of TEL, the ideal timing of this feedback (i.e. immediate or delayed) in a medical education setting is unclear.

METHOD

Forty-one second-year medical students were recruited from the University of Melbourne. Participants were given a multiple-choice question test with a mix of immediate (i.e. post-item) and delayed (i.e. post-item-block) conceptual feedback. Students were then tested on near and far transfer items during an immediate post-test, and at a one-week follow-up.

RESULTS

A logistic mixed effects model was used to predict the probability of successful near and far transfer. As expected, participants in our study tended to score lower on far transfer items than they did on near transfer items. In addition, correct initial response on a parent question predicted subsequent correct responding. Contrary to our hypotheses, the feedback timing effect was non-significant-there was no discernible difference between feedback delivered immediately versus delayed feedback.

DISCUSSION

The findings of this study suggest that the timing of feedback delivery (post-item versus post-item-block) does not influence the efficacy of TEL in this medical education setting. We therefore suggest that educators may consider practical factors when determining appropriate TEL feedback timing in their setting.

Establishing TELLme - an online platform for self-directed, formative assessment of knowledge acquisition across an undergraduate medical curriculum

PURPOSE

The purpose of this study is to report on the design, implementation, use and evaluation of TELLme, an online platform with annotated multiple-choice questions (MCQs) for formative assessment of knowledge acquisition covering an entire undergraduate medical curriculum.

MATERIALS AND METHODS

We used the Educational Design Research (EDR) framework to develop TELLme as an online platform with 24/7 access for students in a co-creation process between educators, faculty and students. EDR cycle 1 focused on prototyping, while EDR cycle 2 focused on upscaling, usage analysis and evaluation. Database entries were analysed for TELLme usage. Online surveys evaluated platform usability and support for student learning.

RESULTS

At the end of EDR cycle 2, TELLme contained 6,713 fully annotated MCQs aligned with the cognitive learning objectives of the curriculum. Up to 600 students per day used TELLme to self-assess their knowledge, answering 3,168,622 MCQs in two semesters. Surveys indicated good usability of TELLme, 75% of students indicated that TELLme helped them to identify knowledge gaps, and 72% agreed that TELLme supported their learning.

CONCLUSIONS

Medical students use TELLme extensively to formatively assess their learning in a self-directed manner across the entire curriculum. TELLme aligns with the existing summative and formative system of assessment of the study programme.

Implementing Formative Assessment in Human Anatomy Practical Sessions: Medical Students' Perception and Effect on Final Exam Performance

Background

Formative assessment with feedback is part of the assessment program in medical education to improve students' learning. Limited research has focused on its application and impact on practical anatomy education.

Methods

This study aimed to examine medical students' perceptions of formative assessment in practical anatomy sessions of body systems-based educational units and explore its influence on final practical exam performance. A descriptive, cross-sectional study was conducted. Data was collected from 173 Year 2 medical students through a survey that addressed their perception of process and importance of formative assessment and feedback. The survey employed a 5-point Likert scale. Two open-ended questions were appended at the end of the survey. Students' performance in Unit 3 (where formative assessment was conducted) was compared to their performance in Unit 2 (where no formative assessment was conducted) and with the performance of the previous academic year's students in Unit 3 (where no formative assessment was conducted). Descriptive statistics were used. The level of statistical significance was set at p-value < 0.05. Responses to open-ended questions (qualitative data) were counted, categorized as themes, and presented as frequencies and percentages.

Results

The survey showed high internal consistency, and its validity was established through exploratory factor analysis. Results showed that the mean mark for the unit with formative assessment and feedback was significantly higher than for the units without formative assessment and feedback. Students showed positive perception of formative assessment and feedback conducted after practical anatomy sessions. They reported useful insights regarding the benefits they gained from formative assessment and feedback as well as constructive suggestions for future improvements.

Conclusion

The study indicates that students positively perceived formative assessment and feedback sessions after practical anatomy sessions. Findings also refer to a positive effect of formative assessment on students' performance in summative practical assessment in anatomy.

Testing the Effects of Individual Residents' Retrieval Practice on Standardized Examination Scores

Background

Residents have limited time and much to learn. Retrieval practice-studying using exam-style review questions-is a powerful educational tool for aggregate groups of learners, but it has not been tested for individual graduate medical education (GME) trainees.

Methods

We conducted a retrospective observational study examining how individual residents' self-directed retrieval practice affected their learning. We used linear regression models to compare the number of exam-style multiple-choice practice questions each resident answered per year to their scores on annual In-Training Exams (ITE).

Results

We found that the higher a resident's ITE score was at the start of a year, the fewer practice questions they answered that year for both first- (p = 0.023) and second-year (p = 0.020) trainees. Then, the more questions a resident answered in a year, the more their ITE score increased over that year for both first- and second-year trainees (p = 0.026 and 0.025, respectively). Residents' prior ITE scores also independently predicted their subsequent ITE scores (p = 0.024 in the first and 0.007 in the second year), and the effect of their baseline scores was larger than the effect of the number of practice questions they answered.

Conclusions

Individual residents' prior exam scores significantly predict their future exam scores, and their independent retrieval practice using exam-style review questions also significantly predicts the annual improvement in their scores. These findings add to the growing evidence supporting retrieval practice in GME, and they can inform individualized educational coaching for GME trainees.

Question banks: credit? Or debit? A qualitative exploration of their use among medical students

BACKGROUND

Online question banks are the most widely used education resource amongst medical students. Despite this there is an absence of literature outlining how and why they are used by students. Drawing on Deci and Ryan's self-determination theory, our study aimed to explore why and how early-stage medical students use question banks in their learning and revision strategies.

METHODS

The study was conducted at Newcastle University Medical School (United Kingdom and Malaysia). Purposive, convenience and snowball sampling of year two students were employed. Ten interviews were conducted. Thematic analysis was undertaken iteratively, enabling exploration of nascent themes. Data collection ceased when no new perspectives were identified.

RESULTS

Students' motivation to use question banks was predominantly driven by extrinsic motivators, with high-stakes exams and fear of failure being central. Their convenience and perceived efficiency promoted autonomy and thus motivation. Rapid feedback cycles and design features consistent with gamification were deterrents to intrinsic motivation. Potentially detrimental patterns of question bank use were evident: cueing, avoidance and memorising. Scepticism regarding veracity of question bank content was absent.

CONCLUSIONS

We call on educators to provide students with guidance about potential pitfalls associated with question banks and to reflect on potential inequity of access to these resources.

Test-Only Learning via Virtual Patients to Improve Surgical Illness Scripts of Preclinical Medical Students as a Preparation for Clinical Clerkships: An Experimental Study

OBJECTIVE

The aim was to determine whether preclinical medical students can develop their illness scripts to a level comparable to that of clerkship students through test-only learning using repeated formative online testlets.

METHODS

In this experimental study, participants were 52 preclinical and 53 clinical medical students. The intervention group consisted of preclinical medical students, and the control group consisted of clinical medical students. The intervention group responded to online testlets containing feedback, an innovative formative assessment method called ContExtended Questions, on general surgery for 8 days by spending no more than 30 minutes each day. The control group completed the general surgery clerkship. The performances were assessed using 20 Key-Feature Question items. The intervention group was assessed twice: immediately after the intervention (the immediate test), and again 1 month later (the delayed test). The control group was assessed once, immediately after the clerkship. All performance tests were identical.

RESULTS

The preclinical students had a significantly higher mean score on the immediate test (83.1 ± 9.6) compared to the clinical students (75.4 ± 8.9), P < .001. The effect size (Cohen's d) was .83. However, the mean score in the delayed test (76.9 ± 13.6) was not significantly different from clinical students' mean score (75.4 ± 8.9), P > .05.

CONCLUSIONS

Test-only learning as a spaced repetition of online formative testlets is effective in preparing preclinical medical students to the clinical clerkship. Through using this approach in preclinical period, they can prepare themselves for the clinical environment to optimize the benefits derived from clerkships.

Curriculum Innovation: A Resident-Created Multiple-Choice Question of the Week to Augment Case-Based Learning

Introduction and Problem Statement

Morning report (MR) has been a foundation of learning in many neurology residency programs. However, fortification of the high-yield learning points during MR cases may be achieved with supplementary educational initiatives to promote effective long-term retention and test-enhanced learning.

Objectives

During the 2020-2021 academic year, chief residents of our neurology training program sought to implement neurology certification board-style multiple-choice questions (MCQs) based on cases presented at MR to enhance case-based learning.

Methods and Curriculum Description

A chief resident was selected weekly to write a MCQ based on an instructive case presented in MR from the prior week. The National Board of Medical Examiners item writing guide and online tutorial were used as guidelines for constructing MCQs. MCQs featured a clinical vignette in the question stem, and images were added to augment select cases. The MCQs were distributed using Qualtrics, which generated a web link and tracked anonymous answers. The Qualtrics link was added to the departmental weekly newsletter and labeled question of the week (QOW). Detailed explanations for each QOW were provided. A feedback survey was sent to the departmental education committee after study completion.

Results and Assessment of Data

Forty MCQs were written by the chief residents, and 1 question was distributed weekly in the departmental newsletter. After week 24, the QOW was restructured to enhance visibility. The mean number of residents who completed the MCQ was 13 (of 29 neurology residents [range 4-29]). The overall median response rate was 38%. When stratified by weeks 1-24 and 25-40 to account for QOW reformatting, the median response rate for weeks 1-24 and weeks 25-40 were 24% and 55%, respectively (p = 0.0013). In a poststudy survey sent to the education committee, 90% of respondents felt that resident-created MCQs were similar to board-style questions and added educational value to resident learning.

Discussion and Lessons Learned

A chief resident QOW initiative was feasible and led to neurology resident academic engagement and enrichment, which supplemented case-based learning through a test-enhanced learning approach. Resident participation was significantly increased with enhanced visibility of QOW in weekly emails compared with hyperlink format.

An Online Program for Primary Care Practitioners to Enhance Confidence in Ability to Care for Patients With or at Risk of Painful Knee Osteoarthritis

OBJECTIVE

Primary care practitioners (PCPs) care for the majority of patients with knee osteoarthritis (KOA). Despite the existing evidence-based guidelines, PCPs often feel unequipped to evaluate and effectively treat patients with KOA. To address this need, we designed and implemented a free internet-based program focusing on the diagnosis and treatment of KOA. We assessed whether the program led to improvements in participants' confidence in their knowledge and skills related to effectively recognizing and caring for patients with or at risk of KOA.

METHODS

We used Caffarella's integrative model to develop a program aligned with the American College of Rheumatology 2019 Guideline for the treatment of KOA. The program incorporated 18 case-based questions to provide retrieval practice and mastery experiences. We assessed changes in participants' confidence in their KOA knowledge and skills after program completion.

RESULTS

Of the first 353 learners completing the program, 53.8% were women, 41.9% had a clinical focus in primary care, and 69.1% were nonphysicians. Overall confidence in KOA knowledge and skills improved after program completion (P < 0.001; effect size = 1.28, 95% confidence interval 1.12-1.45), with largest improvements among participants with lower pre-program confidence. A total of 95.8% of participants indicated they would recommend the program to others.

CONCLUSION

A free online program focusing on the effective care for patients with KOA attracted a wide range of learners, even though it targeted PCPs. Participants overwhelmingly endorsed it as highly relevant and would recommend it to others. Whether improvements in confidence translate into better patient outcomes is an important area for future research.

E-learning modules to improve clinical reasoning and practice: a prospective comparative study

Background

Controversy remains about whether e-learning can improve clinical competences. Our study aimed to compare the effects of e-learning versus traditional education on medical students' reasoning and how they applied their knowledge to clinical competences, assess factors associated with e-learning that might influence exam scores, and evaluate medical students' satisfaction with these two learning methods.

Methods

Prospective study of 299 medical students in two fourth-year pediatric clerkship cohorts (2016-17 and 2017-18) in Switzerland.

Results

We found no evidence of a difference in students' reasoning or how they applied their knowledge to competences in clinical case resolution, whether they had followed e-learning modules or attended traditional lectures. The number of quizzes taken and being female were factors associated with better scores. Even though overall satisfaction with the two learning methods was similar, students claimed that they learned more in e-learning than in traditional lectures and that e-learning explained learning objectives better.

Conclusions

E-learning could be used as a supplement or alternative to traditional face-to-face medical teaching methods without compromising teaching quality. E-learning modules should be better integrated into medical students' curricula but avoid the risk of curriculum overload, especially in case of repeated COVID-like context.

Feedback, fairness, and validity: effects of disclosing and reusing multiple-choice questions in medical schools

BACKGROUND

Disclosure of items used in multiple-choice-question (MCQ) exams may decrease student anxiety and improve transparency, feedback, and test-enhanced learning but potentially compromises the reliability and fairness of exams if items are eventually reused. Evidence regarding whether disclosure and reuse of test items change item psychometrics is scarce and inconclusive.

METHODS

We retrospectively analysed difficulty and discrimination coefficients of 10,148 MCQ items used between fall 2017 and fall 2019 in a large European medical school in which items were disclosed from fall 2017 onwards. We categorised items as 'new'; 'reused, not disclosed'; or 'reused, disclosed'. For reused items, we calculated the difference from their first ever use, that is, when they were new. Differences between categories and terms were analysed with one-way analyses of variance and independent-samples t tests.

RESULTS

The proportion of reused, disclosed items grew from 0% to 48.4%; mean difficulty coefficients increased from 0.70 to 0.76; that is, items became easier, P < .001, η_p² = 0.011. On average, reused, disclosed items were significantly easier (M = 0.83) than reused, not disclosed items (M = 0.71) and entirely new items (M = 0.66), P < .001, η_p² = 0.087. Mean discrimination coefficients increased from 0.21 to 0.23; that is, item became slightly more discriminating, P = .002, η_p² = 0.002.

CONCLUSIONS

Disclosing test items provides the opportunity to enhance feedback and transparency in MCQ exams but potentially at the expense of decreased item reliability. Discrimination was positively affected. Our study may help weigh advantages and disadvantages of using previously disclosed items.

A Scoping Review of Retrieval Practice (Test-Enhanced Learning) in Nursing Education

AIM

The aim of this study was to determine the status of retrieval practice in original research of nursing education.

BACKGROUND

The science of learning is an emerging interdisciplinary field that offers evidence-based strategies to improve learning. One of the most highly effective strategies is retrieval practice, which involves recalling previously learned information from long-term memory prior to additional study.

METHOD

Searching PubMed, CINAHL, Scopus, Psych INFO, and ERIC, an interprofessional team followed a formal scoping review framework and utilized the PRISMA Extension for Scoping Reviews to report the findings.

RESULTS

The review included 25 research studies, with all but one at the prelicensure level. Quantitative designs were most common, and outcomes reflected objective and subjective measures. If present, terms were varied and inconsistent with supporting research. Documentation of many characteristics was lacking.

CONCLUSION

Retrieval practice is an increasingly common strategy being studied at the prelicensure level. Opportunities for improvement include consistent use of standard terminology, documentation of important characteristics, and more attention to subjective outcomes and to graduate and continuing education levels.

The Effectiveness of Spaced Learning, Interleaving, and Retrieval Practice in Radiology Education: A Systematic Review

PURPOSE

Radiology is a highly complex field that requires mastery over an ever-expanding body of knowledge. Spaced learning, interleaving, and retrieval practice are evidence-based learning strategies that enhance long-term retention of information. The aim of this systematic review is to assess the effectiveness of these interventions in the setting of radiology education.

METHODS

The authors searched MEDLINE, Embase, PsycInfo, ERIC, and forward and backward citations for studies published between database inception and February 19, 2023. Eligibility criteria for included studies were randomized and quasi-randomized controlled trials that investigated the impact of spaced, interleaved, or retrieval practice on knowledge retention of medical trainees after education related to medical imaging as assessed by postinterventional examination scores.

RESULTS

Of 1,316 records reviewed, 8 studies met eligibility criteria. Two studies investigated spaced learning, two studies interleaving, and six studies retrieval practice, including two trials that evaluated interventions incorporating both spaced learning and retrieval practice. Five of eight studies reported statistically significant differences between interventional and control groups on either immediate or delayed postinterventional examinations.

CONCLUSIONS

Despite extensive evidence in support of spaced, interleaved, and retrieval practice within the broader literature, few studies have examined the effectiveness of these strategies in radiology education. Additional trials are required to evaluate the usefulness of incorporating these techniques into educational programs related to medical imaging.

An online flipped classroom approach improves the physiology score and subsequent course scores of the top-performing students

An online flipped classroom approach achieved the same teaching effect as online live teaching but had a more positive impact on high-achieving students. The positive impact was not only in Physiology but also in subsequent courses where logical thinking prevailed. However, for low-achieving students, the effect of online live teaching was better.

Evaluating TESLA-G, a gamified, telegram-delivered, quizzing platform for surgical education in medical students: protocol for a pilot randomised controlled trial

INTRODUCTION

Online multiple-choice question (MCQ) quizzes are popular in medical education due to their ease of access and ability for test-enhanced learning. However, a general lack of motivation among students often results in decreasing usage over time. We aim to address this limitation by developing Telegram Education for Surgical Learning and Application Gamified (TESLA-G), an online platform for surgical education that incorporates game elements into conventional MCQ quizzes.

METHODS AND ANALYSIS

This online, pilot randomised control trial will be conducted over 2 weeks. Fifty full-time undergraduate medical students from a medical school in Singapore will be recruited and randomised into an intervention group (TESLA-G) and an active control group (non-gamified quizzing platform) with a 1:1 allocation ratio, stratified by year of study.We will evaluate TESLA-G in the area of endocrine surgery education. Our platform is designed based on Bloom's taxonomy of learning domains: questions are created in blocks of five questions per endocrine surgery topic, with each question corresponding to one level on Bloom's taxonomy. This structure promotes mastery while boosting student engagement and motivation. All questions are created by two board-certified general surgeons and one endocrinologist, and validated by the research team. The feasibility of this pilot study will be determined quantitatively by participant enrolment, participant retention and degree of completion of the quizzes. The acceptability of the intervention will be assessed quantitatively by a postintervention learner satisfaction survey consisting of a system satisfaction questionnaire and a content satisfaction questionnaire. The improvement of surgical knowledge will be assessed by comparing the scores of preintervention and postintervention knowledge tests, which consist of separately created questions on endocrine surgery. Retention of surgical knowledge will be measured using a follow-up knowledge test administered 2 weeks postintervention. Finally, qualitative feedback from participants regarding their experience will be obtained and thematically analysed.

ETHICS AND DISSEMINATION

This research is approved by Singapore Nanyang Technological University (NTU) Institutional Review Boards (Reference Number: IRB-2021-732). All participants will be expected to read and sign a letter of informed consent before they are considered as recruited into the study. This study poses minimal risk to participants. Study results will be published in peer-reviewed open-access journals and presented in conference presentations.

TRIAL REGISTRATION NUMBER

NCT05520671.

Evidence-based teaching in contact lenses education: Teaching and learning strategies

INTRODUCTION

Contact lens (CL) practice is an ever-changing field with clinical knowledge, techniques and equipment continuously evolving. These new developments are backed with clinical trials and research to ensure that practitioners feel confident that there is an evidence base to support these advances. Evidence-based practice is now a crucial part of CL practice, and its importance also filters down to CL education. For example, lectures are one of the most popular tools for an educator but, is standing at the front of a lecture theatre full of students a more effective way of teaching than providing the same material for students to read by themselves? What evidence exists specific to CL education?

METHOD

An expert panel of educators completed a comprehensive literature review of current evidence of teaching methods in CL training, or if not available then what can be learnt from other health care professional training that could be potentially applicable to CL education.

RESULTS

Due to the amount of evidence available in the overall subject area relating to healthcare education, the initial plan of compiling evidence into one narrative review paper was discarded in favour of producing two linked papers. Here, the first paper details definitions of terminology, and also teaching methods. The second paper focuses on assessment and specific clinical training required to attain CL practice competency. In this first paper, no direct evidence of the spreading and benefit of new education strategies evidence such as flipped classrooms, spaced learning, test-enhanced learning, group work, CBL, PBL, TBL, and reflective practice in CL education was found. The only technique that was widely used in the CL field was case reports and the group discussion of them. Nevertheless, the authors found a consensus of opinion from other disciplines that are transferable to CL teaching and could help students meet the intended learning outcomes.

CONCLUSION

There is a small amount of evidence supporting CL education, but most of this seems to be related to the practical element of the training. However, there is a lot of evidence in the field of healthcare education from related disciplines which provides additional but important learning tools that may be effectively implemented in CL education.

Building a student learning-focused assessment and grading system in dental school: One school's experience

PURPOSE/OBJECTIVES

As health professions education moves toward competency-based education, there has been increased focus on the structure of assessment systems that support student competency development and learning. This has been buoyed by a growing body of research supporting assessment for learning processes to promote student growth and learning rather than relying on assessment systems primarily to measure performance. This paper presents the rationale and evidence for moving to an assessment for learning system and the results of a quasi-experimental interrupted time series study using data from 2015 to 2022 to evaluate the impacts of these changes.

METHODS

Columbia University College of Dental Medicine faculty voted to implement assessment for learning system changes beginning in 2017 with the graduating class of 2021. These changes included moving from using a grading system for didactic courses with Honors, Pass, and Fail as available grades to a grading system with only Pass and Fail as available grades, as well as creating synthesis and assessment weeks, weekly problem sets, post-exam review sessions, exam remediation opportunities, and formative progress exams throughout the curriculum. The revised assessment and grading system changes were communicated to residency program directors, and programmatic competency data about student performance across the curriculum were shared with programs in Dean's Letters.

RESULTS

Once assessment system changes were implemented, it was found that student exam failure rates were lower, course exam scores were the same or higher, and performance on board exams improved compared to the national average. Students reported positive perceptions with regard to well-being and learning climate that they associated with the adoption of Pass/Fail grading. Match outcomes, including student satisfaction and program director ratings, have remained consistently positive.

CONCLUSION

As dental educators, our goal is to nurture students to become life-long learners. Adopting a grading structure that is Pass/Fail and an assessment system that fosters learning allows students to shape learning practices that favor long-term retention and application of information, also enhancing the learning environment and student well-being. These system changes may also facilitate the inclusion and support of students whose backgrounds are underrepresented in dentistry.

How Am I Doing in Small Group? Student Perceptions of Feedback in Case-Based Learning Sessions

Introduction

Small group case-based learning (CBL) facilitators are content experts that may provide feedback to students on cognitive reasoning skills and knowledge acquisition. However, student feedback-seeking behavior and response toward faculty feedback in CBL sessions are not known, and it is essential to maximize feedback in this setting where it can be a challenge to observe student performance while groups may have varied emphasis on individual versus team performance. We explored student perceptions of the effectiveness of faculty feedback processes during CBL sessions.

Methods

This qualitative study used semi-structured interviews with ten second year medical students enrolled in the Geisel School of Medicine preclinical neurology course. Investigator triangulation was used with interpretation comparisons that included independent content analysis. The constructed themes were discussed and final theme consensus was reached.

Results

Three major themes arose: (1) students value frequent feedback on their understanding of key clinical case concepts; (2) the CBL learning environment is not conducive to individual feedback; and (3) student feedback-seeking behavior and response are influenced by self-perceived level of preparedness for the sessions and overall comfort with the CBL facilitator and learning environment.

Conclusions

Students value content-based feedback from CBL sessions and need more individualized feedback. The style of the facilitator and overall learning environment can vary widely in the small group setting and has direct impact on feedback opportunities and student feedback-seeking behavior.

Sleep Medicine Evaluation in the Annual Patient Management Problem (PMP) Exam for Otolaryngology Residents

The sleep medicine has not been previously included in the patient management problem (PMP) exam for otolaryngology residents at Tehran University of Medical Sciences. The aim of this study, is to evaluate the results of this first implementation. This was a cross-sectional study. The examinees included all 62 residents from the 1st to 4th years of the Tehran University of Medical Sciences otolaryngology department. The resident scores obtained in this exam were evaluated and compared by residency year. The overall participant score was low for this part of the PMP. There was no score difference between years 1 and 3, but there was a significant positive change in year 4 (p = 0.007). This score pattern was not evident in other parts of the exam. Otolaryngology residents have insufficient knowledge toward the sleep medicine. The reasons for such scoring patterns may be due to the structure of sleep medicine training within the department which may lead to undesirable consequences.

Voluntary participation in flipped classroom application sessions has a negligible effect on assessment outcomes in an accelerated pass-fail course

Increasingly, basic science educators at medical and health science programs are faced with the challenge of delivering fundamental science content using evidence-based pedagogical approaches that build students' fund of knowledge while also supporting their development as self-regulated learners. This has led to an increased use of active learning-based pedagogies such as flipped classroom teaching. However, there are many open questions about the conditions necessary for successful flipped classroom sessions. In particular, the role of student compliance (i.e., participation, engagement, attendance) in mediating performance needs to be evaluated. This is especially important in accelerated curricula where multiple basic science disciplines are integrated together in pass-fail courses, presenting challenges to both students' time and cognitive load. Data on prematriculation performance, in-class participation, weekly quiz performance, and summative assessment performance from three cohorts of medical students (n = 146) at a new medical school were collected and analyzed. We found that historically high-performing students more readily participated in flipped classroom application sessions compared with historically lower-performing students. Correlational analysis of performance on weekly formative quizzes and the summative course exam was not related to in-class participation. However, performance on weekly formative quizzes played the most significant role in students' performance on summative exams. Efforts to understand the benefits of in-class participation beyond short-term assessment performance, such as long-term knowledge retention or development of noncognitive skills, should be undertaken to justify using such time- and human resource-intensive pedagogies.NEW & NOTEWORTHY This study explores the use of flipped classroom teaching in a voluntary and accelerated medical school course. We found that historically high-performing students attend class, whereas historically low-performing students do not attend class as readily. Formative assessment performance appears to be more important than participation in determining the final grade. Correlation of high performance (>90%) with participation may differentiate students who excel in our curriculum from those who simply pass with superficial knowledge.

Enhancing Examination Success: the Cumulative Benefits of Self-Assessment Questions and Virtual Patient Cases

PURPOSE

Research on the learning benefits of the feedback-rich formative assessment environment of virtual patient cases (VPCs) has largely been limited to single institutions and focused on discrete clinical skills or topical knowledge. To augment current understanding, we designed a multi-institutional study to explore the distinct and cumulative effects of VPC formative assessments and optional self-assessment questions (SAQs) on exam performance.

METHOD

In this correlational study, we examined the records of 1,692 students on their family medicine (FM) clerkship at 20 medical schools during the 2014-2015 academic year. Schools utilized an established online curriculum, which included family medicine VPCs, embedded formative assessments, context-rich SAQs corresponding with each VPC, and an associated comprehensive family medicine exam. We used mixed-effects modeling to relate the student VPC composite formative assessment score, SAQ completion, and SAQ performance to students' scores on the FM final examination.

RESULTS

Students scored higher on the final exam when they performed better on the VPC formative assessments, completed associated SAQs, and scored higher on those SAQs. Students' SAQ completion enhanced examination performance above that explained by engagement with the VPC formative assessments alone.

CONCLUSIONS

This large-scale, multi-institutional study furthers the body of research on the effect of formative assessments associated with VPCs on exam performance and demonstrates the added benefit of optional associated SAQs. Findings highlight opportunities for future work on the broader impact of formative assessments for learning, exploring the benefits of integrating VPCs and SAQs, and documenting effects on clinical performance and summative exam scores.

Science of Learning Strategy Series: Article 3, Interleaving

Interleaving is an evidence-based, learning-science strategy that is relevant to the planning and implementation of continuing professional development (CPD). Mixing related but different areas of study forces the brain to reconcile the relationship between the areas while understanding each area well. By doing so, interleaving increases the likelihood of mastery and memory. Research from cognitive psychology and neuroscience provides the rationale for interleaving, and examples of its implementation in health profession education have begun to appear in the literature. If utilized appropriately, some common CPD interventions can leverage interleaving. Through increased understanding, CPD participants can benefit from interleaving by making more-informed educational choices, and CPD planners can benefit in efforts to improve educational activities.

Combining desirable difficulty learning strategies to address the amount-to-learn vs. time-to-learn imbalance in residency training

PURPOSE

Residents have limited time and much to learn. Mounting evidence shows that Desirable Difficulty (DD) learning strategies can ease that imbalance, but few studies have specifically studied combinations of these strategies.

METHODS

We tested two different combinations of DD strategies: a double combination of distributed practice and retrieval practice and a triple combination additionally including interleaved practice. We compared residents' annual In-Training Exam (ITE) scores and graduates' board certification performance between both DD curricula and a historical baseline.

RESULTS

Average ITE scores rose from 149.06 in the historical baseline to 160.04 under the combined DD curricula (p < 0.001). Average ITE scores fell from 162.50 under the double combination to 155.11 under the triple combination (p = 0.03). There were no significant changes in graduates' board performance between any of the curricula.

CONCLUSIONS

These results add to the evidence that DD strategies can enhance residents' learning. The drop in ITE scores under the triple DD combination may suggest that it pushed learners past beneficial desirable difficulty into detrimental overwhelming difficulty. Further research should apply this framework in larger and more diverse settings to clarify how these DD strategies can be optimally used to enhance residents' learning.

The two envelopes method for active learning

PURPOSE

Active learning improves knowledge acquisition and provides medical students with learning habits that become an integral part of their behavior. As an integral element of our institution's transition from a lecture hall teaching culture to active learning, the current project, conducted with fourth year students, aimed to examine the effects of the two envelopes method of teaching on students' knowledge.

METHOD

The class of 120 students was divided into 12 groups of 10 students each. Six experienced senior cardiologists were assigned to teach the 12 groups. When the students arrived at the classroom, they received two envelopes. Students were instructed to open the first envelope and answer a 10-question test in 15 minutes. After completing the test, they returned the tests to the envelope, sealed it, and then opened the second envelope which included the same test and relevant patient information. They then spent the next 30 minutes discussing the test as a group and familiarizing themselves with the patients' case histories and clinical data. After completion of the group discussion, the tutor entered the room for a two-hour discussion of the patients' disease entities including the anatomy, physiology, pathology, clinical presentation, diagnostic measures, and potential therapies.

RESULTS

We compared grades and standard deviations of grades between two classes: one learned in the lecture hall format (2018) and the other learned employing the two-envelopes method (2019). There was a non-statistically significant trend toward better grades with reduced dispersion of grades in the class that learned with the two-envelope method.

CONCLUSIONS

We describe a novel method for active learning that enhances self-learning and peer learning, and we observed better knowledge acquisition and reduced knowledge dispersion that were not statistically significant.

Mobile application adjunct to the WHO basic emergency care course: a mixed methods study

OBJECTIVES

The WHO developed a 5-day basic emergency care (BEC) course using the traditional lecture format. However, adult learning theory suggests that lecture-based courses alone may not promote long-term knowledge retention. We assessed whether a mobile application adjunct (BEC app) can have positive impact on knowledge acquisition and retention compared with the BEC course alone and evaluated perceptions, acceptability and barriers to adoption of such a tool.

DESIGN

Mixed-methods prospective cohort study.

PARTICIPANTS

Adult healthcare workers in six health facilities in Tanzania who enrolled in the BEC course and were divided into the control arm (BEC course) or the intervention arm (BEC course plus BEC app).

MAIN OUTCOME MEASURES

Changes in knowledge assessment scores, self-efficacy and perceptions of BEC app.

RESULTS

92 enrolees, 46 (50%) in each arm, completed the BEC course. 71 (77%) returned for the 4-month follow-up. Mean test scores were not different between the two arms at any time period. Both arms had significantly improved test scores from enrolment (prior to distribution of materials) to day 1 of the BEC course and from day 1 of BEC course to immediately after BEC course completion. The drop-off in mean scores from immediately after BEC course completion to 4 months after course completion was not significant for either arm. No differences were observed between the two arms for any self-efficacy question at any time point. Focus groups revealed five major themes related to BEC app adoption: educational utility, clinical utility, user experience, barriers to access and barriers to use.

CONCLUSION

The BEC app was well received, but no differences in knowledge retention and self-efficacy were observed between the two arms and only a very small number of participants reported using the app. Technologic-based, linguistic-based and content-based barriers likely limited its impact.

The Importance of Sociocultural Factors in Moderating the Applicability of Test-Enhanced Learning to Continuing Professional Development

Many studies have shown that repeated testing promotes learning, an effect known as test-enhanced learning. Although test-enhanced learning has been well-established within health care trainees, there are mixed findings of whether testing facilitates learning within continuing professional development (CPD) contexts. This piece argues that for testing to support learning in CPD, there is need to understand the various social, cultural, and structural factors that influence clinicians' willingness to participate in regular testing activities. The solution to this problem is conceptual in nature, whereby CPD programs must consider how learning cultures shape practicing clinicians' perspectives toward testing.

Knowledge Retention Using "Relay": A Novel Active-Learning Technique

INTRODUCTION

Many residency programs utilize passive didactic lectures despite mixed evidence for knowledge retention. This prospective study investigated the efficacy of "relay," an active-learning technique, as measured by residents' knowledge retention and attitudes compared to traditional format.

METHODS

Faculty presented lectures to four family medicine residency programs on a randomized schedule. Lectures were followed by a 15-minute question-and-answer (Q&A) session or relay session. A relay is a collaborative, question-based game. The primary outcome was knowledge retention at 3 months, comparing Q&A to relay sessions as measured by a multiple-choice assessment. Responses were only included if a given resident completed knowledge assessments for both Q&A and relay sessions, to allow for intraresident adjustments, in addition to program, training year, and lecturer/topic. Secondary outcomes included residents' self-perceived knowledge and engagement as surveyed by an ordinal scale immediately following the learning session.

RESULTS

The primary analysis included 51 responses from 18 unique residents. The adjusted mean knowledge assessment score at 3 months was not statistically different after the relay sessions compared to Q&A (67% vs 60%, respectively; 7% difference, 95% CI: -4 to 18%, P=.20). For the secondary outcomes of learner attitudes (n=143 responses), learners reported greater engagement after the relay sessions compared to Q&A (51% vs 28% "very engaged"; overall P=.003), but self-perceived knowledge was not significantly different (overall P=.05, rounded down).

CONCLUSIONS

The relay technique did not show significant difference in 3-month knowledge retention, nor immediate self-perceived knowledge, despite greater learner self-perceived engagement.

How to Keep Training-After Residency Training

Lifelong learning in medicine is an important skill and ethical obligation, but many residents do not feel prepared to be effective self-directed learners when training ends. The learning sciences offer evidence to guide self-directed learning, but these insights have not been integrated into a practical and actionable plan for residents to improve their clinical knowledge and reasoning. We encourage residents to establish a self-directed learning plan, just as an athlete employs a training plan in the pursuit of excellence. We highlight four evidence-based learning principles (spaced practice, mixed practice, retrieval practice, and feedback) and four training strategies comprising a weekly training plan: case tracking, simulated cases, quizzing, and new evidence integration. We provide tips for residents to implement and refine their approach and discuss how residency programs can foster these routines and habits. By optimizing their scarce self-directed learning time with a training plan, residents may enhance patient care and their career satisfaction through their pursuit of clinical mastery.

Scoping Review of Distributed Practice in Nursing Education

AIM

The aim of this study was to describe the status of distributed practice in research of nursing education.

BACKGROUND

The science of learning has compiled evidence-based strategies that should be integral to nursing education. One long-standing strategy, distributed practice, involves spacing, placing cognitive breaks between study or practice sessions with priority information.

METHOD

Reviewing literature published over 20 years at every level of nursing education, the authors conducted a scoping review to determine the extent to which research of nursing education includes attention to distributed practice.

RESULTS

In the 13 articles meeting criteria, distributed practice was most common in continuing professional development, with an emphasis in learning psychomotor skills. Study authors used a variety of terms and descriptions in referring to the strategy.

CONCLUSION

By way of research, distributed practice appears underutilized in nursing education, especially at prelicensure and graduate levels, and could benefit from clear and consistent use of terminology.

Optimizing preclinical learning with retrieval practice: A call to action

This Personal View is about our experience with preclinical education as medical students. We discuss the problem with current medical education in light of an ever-growing body of medical knowledge and increasing student disengagement with preclinical lectures. We briefly review the concept of retrieval practice as an effective, evidence-based learning strategy that helped us retain knowledge for longer periods and propose that medical educators should adopt this strategy to best prepare medical students to navigate the vastly expanding scope of modern medicine.

Training non-intensivist doctors to work with COVID-19 patients in intensive care units

Background

Due to an expected surge of COVID‐19 patients in need of mechanical ventilation, the intensive care capacity was doubled at Rigshospitalet, Copenhagen, in March 2020. This resulted in an urgent need for doctors with competence in working with critically ill COVID‐19 patients. A training course and a theoretical test for non‐intensivist doctors were developed. The aims of this study were to gather validity evidence for the theoretical test and explore the effects of the course.

Methods

The 1‐day course was comprised of theoretical sessions and hands‐on training in ventilator use, hemodynamic monitoring, vascular access, and use of personal protective equipment. Validity evidence was gathered for the test by comparing answers from novices and experts in intensive care. Doctors who participated in the course completed the test before (pretest), after (posttest), and again within 8 weeks following the course (retention test).

Results

Fifty‐four non‐intensivist doctors from 15 different specialties with a wide range in clinical experience level completed the course. The test consisted of 23 questions and demonstrated a credible pass–fail standard at 16 points. Mean pretest score was 11.9 (SD 3.0), mean posttest score 20.6 (1.8), and mean retention test score 17.4 (2.2). All doctors passed the posttest.

Conclusion

Non‐intensivist doctors, irrespective of experience level, can acquire relevant knowledge for working in the ICU through a focused 1‐day evidence‐based course. This knowledge was largely retained as shown by a multiple‐choice test supported by validity evidence. The test is available in appendix and online.

The Future Comes Early for Medical Educators

Many experts have foretold of a digital transformation in medical education. Yet, until recently, day-to-day practices for frontline clinician-educators, who cherish close physical and intellectual contact between the patient, learner, and teacher, have remained largely unchanged. The COVID-19 pandemic disrupted that model and is forcing teachers to pursue new ways to reach learners. We provide a roadmap for educators to start their transformation from an analog to a digital approach by harnessing existing tools including podcasts, social media, and videoconferencing. Teachers will need to enhance the same pedagogical and interpersonal practices that underpin effective in-person education while they learn new skills as they become curators, creators, and moderators in the digital space. This adaptation is essential, as many of the changes in medical education spurred by COVID-19 will likely far outlast the pandemic.

Twelve tips on guiding preparation for both high-stakes exams and long-term learning

High-stakes exams including admissions, licensing, and maintenance of certification examinations are commonplace in health professions education. Although exam scores and performance can often serve gate-keeping purposes, the broader goal of health professions education is to foster deep, self-directed, meaningful, motivated learning. Establishing strong support structures that emphasize deep learning and understanding rather than exam scores can be beneficial to preparing learners who have the knowledge base to be excellent practitioners. This article offers guidance that can be used by academic support centres, medical educators, learning specialists, and faculty advisors, or even test-takers, to help learners to balance score achievement and knowledge development, while simultaneously cultivating more efficient and motivated studying and increasingly self-regulated learning. This series of tips details considerations for building academic success supports, fostering a growth mindset, planning efficient and effective studying efforts, utilizing test-enhanced learning strategies, exam-taking skills practice, and other support structures that can help strengthen learning experiences overall.

Rethinking Assessment Design: Evidence-Informed Strategies to Boost Educational Impact in the Anatomical Sciences

University assessment is in the midst of transformation. Assessments are no longer designed solely to determine that students can remember and regurgitate lecture content, nor in order to rank students to aid with some future selection process. Instead, assessments are expected to drive, support, and enhance learning and to contribute to student self-assessment and development of skills and attributes for a lifetime of learning. While traditional purposes of certifying achievement and determining readiness to progress remain important, these new expectations for assessment can create tensions in assessment design, selection, and deployment. With the recognition of these tensions, three contemporary approaches to assessment in medical education are described. These approaches include careful consideration of the educational impact of assessment-before, during (test or recall enhanced learning) and after assessments; development of student (and staff) assessment literacy; and planning of cohesive systems of assessment (with a range of assessment tools) designed to assess the various competencies demanded of future graduates. These approaches purposefully straddle the cross purposes of assessment in modern health professions education. The implications of these models are explored within the context of medical education and then linked with contemporary work in the anatomical sciences in order to highlight current synergies and potential future innovations when using evidence-informed strategies to boost the educational impact of assessments.

Science of Learning Strategy Series: Article 2, Retrieval Practice

Retrieval practice is an evidence-based, science of learning strategy that is relevant to the planning and implementation of continuing professional development (CPD). Retrieval practice requires one to examine long-term memory to work with priority information again in working memory. Retrieval practice improves learning in two ways. It improves memory for the information itself (direct benefit), and retrieval practice provides feedback about what needs additional effort (indirect). Both benefits contribute significantly to durable learning. Research from cognitive psychology and neuroscience provides the rationale for retrieval practice, and examples of its implementation in health professions education are increasingly available in the literature. Through appropriate utilization, CPD participants can benefit from retrieval practice by making more-informed educational choices, and CPD planners can benefit in efforts to improve educational activities.

Twelve tips for embedding assessment for and as learning practices in a programmatic assessment system

Programmatic assessment supports the evolution from assessment of learning to fostering assessment for learning and as learning practices. A well-designed programmatic assessment system aligns educational objectives, learning opportunities, and assessments with the goals of supporting student learning, making decisions about student competence and promotion decisions, and supporting curriculum evaluation. We present evidence-based guidance for implementing assessment for and as learning practices in the pre-clinical knowledge assessment system to help students learn, synthesize, master and retain content for the long-term so that they can apply knowledge to patient care. Practical tips are in the domains of culture and motivation of assessment, including how an honour code and competency-based grading system can support an assessment system to develop student self-regulated learning and professional identity, curricular assessment structure, such as how and when to utilize low-stakes and cumulative assessment to drive learning, exam and question structure, including what authentic question and exam types can best facilitate learning, and assessment follow-up and review considerations, such exam retake processes to support learning, and academic success structures. A culture change is likely necessary for administrators, faculty members, and students to embrace assessment as most importantly a learning tool for students and programs.

Consensus statement on the content of clinical reasoning curricula in undergraduate medical education

INTRODUCTION

Effective clinical reasoning is required for safe patient care. Students and postgraduate trainees largely learn the knowledge, skills and behaviours required for effective clinical reasoning implicitly, through experience and apprenticeship. There is a growing consensus that medical schools should teach clinical reasoning in a way that is explicitly integrated into courses throughout each year, adopting a systematic approach consistent with current evidence. However, the clinical reasoning literature is 'fragmented' and can be difficult for medical educators to access. The purpose of this paper is to provide practical recommendations that will be of use to all medical schools.

METHODS

Members of the UK Clinical Reasoning in Medical Education group (CReME) met to discuss what clinical reasoning-specific teaching should be delivered by medical schools (what to teach). A literature review was conducted to identify what teaching strategies are successful in improving clinical reasoning ability among medical students (how to teach). A consensus statement was then produced based on the agreed ideas and the literature review, discussed by members of the consensus statement group, then edited and agreed by the authors.

RESULTS

The group identified 30 consensus ideas that were grouped into five domains: (1) clinical reasoning concepts, (2) history and physical examination, (3) choosing and interpreting diagnostic tests, (4) problem identification and management, and (5) shared decision making. The literature review demonstrated a lack of effectiveness for teaching the general thinking processes involved in clinical reasoning, whereas specific teaching strategies aimed at building knowledge and understanding led to improvements. These strategies are synthesised and described.

CONCLUSION

What is taught, how it is taught, and when it is taught can facilitate clinical reasoning development more effectively through purposeful curriculum design and medical schools should consider implementing a formal clinical reasoning curriculum that is horizontally and vertically integrated throughout the programme.

Science of Learning Strategy Series: Article 1, Distributed Practice

Distributed practice is an evidence-based, learning-science strategy that is relevant to the planning and implementation of continuing professional development (CPD). Spacing-out study or practice over time allows the brain multiple opportunities to process new and complex information in an efficient way, thus increasing the likelihood of mastery and memory. Research from cognitive psychology and neuroscience provide the rationale for distributed practice, and examples of its implementation in health professions education have begun to appear in the literature. If used appropriately or extended creatively, some common CPD interventions can fully leverage distributed practice. Through increased understanding, CPD planners can benefit from distributed practice in efforts to improve educational activities, and CPD participants can benefit by making more informed educational choices.

A Web-Based Virtual Microscopy Platform for Improving Academic Performance in Histology and Pathology Laboratory Courses: A Pilot Study

Virtual microscopy (VM) has been utilized to improve students' learning experience in microscope laboratory sessions, but minimal attention has been given to determining how to use VM more effectively. The study examined the influence of VM on academic performance and teacher and student perceptions and compared laboratory test scores before and after VM incorporation. A total of 662 third-year students studying histology and 651 fourth-year students studying pathology were divided into two groups. The light microscopy (LM) group used a light microscope in 2014 and 2015, while the LM + VM group used the VM platform and a light microscope in 2016 and 2017. Four factors positively predict laboratory scores (R square, 0.323; P < 0.001): (i) the pathology course and test-enhanced learning, (ii) the VM platform and experience, (iii) medical students and lecture scores, and (iv) female students. The LM + VM group exhibited less score variability on laboratory examinations relative to their mean than the LM group. The LM + VM group was also associated with fewer failing grades (F grade; odds ratio, 0.336; P < 0.001) and higher scores (A grade; odds ratio, 2.084; P < 0.001) after controlling for sex, school, course, and lecture grades. The positive effect of the VM platform on laboratory test grades was associated with prior experience using the VM platform and was synergistic with more interim tests. Both teachers and students agreed that the VM platform enhanced laboratory learning. The incorporation of the VM platform in the context of test-enhanced learning may help more students to master microscopic laboratory content.

Summative and Formative Style Anatomy Practical Examinations: Do They Have Impact on Students' Performance and Drive for Learning?

Anatomical knowledge is commonly assessed by practical examinations that are often administered in summative format. The format of anatomy practical examination was changed at the Lee Kong Chian School of Medicine in Singapore from summative (graded; must pass) to formative (ungraded; no pass/fail) in academic year (AY) 2017-2018. Both assessment formats were undertaken online, but the formative mode used a team-based learning activity comprising individual and team assessments. This gave an unique opportunity to investigate: (1) the impact of two different online assessment formats on student performance in practical examination; (2) the impact of new formative practical examination on students' performance in summative examinations; and (3) students' opinions of these two practical examination formats. The class of 2021 perceptions was obtained as they experienced both formats. A retrospective cohort study was also conducted to analyze the Year 2 students' performance in anatomy practical and year-end summative examinations of cohorts AY 2015-2016, AY 2016-2017 (summative format), and AY 2017-2018 (formative format). There were no significant differences in students' performance between two practical examination formats. The cohort who experienced the formative format, performed significantly better in summative examinations (mean ± SD: 82.32 ± 10.22%) compared with the cohort who experienced the summative format (73.77 ± 11.09%) (P < 0.001). Students highlighted positive features of the formative practical examination, including team reinforcement of learning, instant feedback, and enhanced learning. These findings indicate that students continue to study for anatomy practical examination without the need for external drivers. The team-based learning style practical examination enhances students' performance in summative examinations.

Identifying and Supporting Students to Prevent USMLE Step 1 Failures When Testing Follows Clerkships: Insights From 9 Schools

Several schools have moved the United States Medical Licensing Examination Step 1 exam after core clerkships, and others are considering this change. Delaying Step 1 may improve Step 1 performance and lower Step 1 failure rates. Schools considering moving Step 1 are particularly concerned about late identification of struggling students and late Step failures, which can be particularly problematic due to reduced time to remediate and accumulated debt if remediation is ultimately unsuccessful. In the literature published to date, little attention has been given to these students. In this article, authors from 9 medical schools with a postclerkship Step 1 exam share their experiences. The authors describe curricular policies, early warning and identification strategies, and interventions to enhance success for all students and struggling students in particular. Such learners can be identified by understanding challenges that place them "at risk" and by tracking performance outcomes, particularly on other standardized assessments. All learners can benefit from early coaching and advising, mechanisms to ensure early feedback on performance, commercial study tools, learning specialists or resources to enhance learning skills, and wellness programs. Some students may need intensive tutoring, neuropsychological testing and exam accommodations, board preparation courses, deceleration pathways, and options to postpone Step 1. In rare instances, a student may need a compassionate off-ramp from medical school. With the National Board of Medical Examiner's announcement that Step 1 scoring will change to pass/fail as early as January 2022, residency program directors might use failing Step 1 scores to screen out candidates. Institutions altering the timing of Step 1 can benefit from practical guidance by those who have made the change, to both prevent Step 1 failures and minimize adverse effects on those who fail.