Test-enhanced learning of clinical reasoning: a crossover randomised trial

Timing's not everything: Immediate and delayed feedback are equally beneficial for performance in formative multiple-choice testing

INTRODUCTION

Test-enhanced learning (TEL) is an impactful teaching and learning strategy that prioritises active learner engagement through the process of regular testing and reviewing. While it is clear that meaningful feedback optimises the effects of TEL, the ideal timing of this feedback (i.e. immediate or delayed) in a medical education setting is unclear.

METHOD

Forty-one second-year medical students were recruited from the University of Melbourne. Participants were given a multiple-choice question test with a mix of immediate (i.e. post-item) and delayed (i.e. post-item-block) conceptual feedback. Students were then tested on near and far transfer items during an immediate post-test, and at a one-week follow-up.

RESULTS

A logistic mixed effects model was used to predict the probability of successful near and far transfer. As expected, participants in our study tended to score lower on far transfer items than they did on near transfer items. In addition, correct initial response on a parent question predicted subsequent correct responding. Contrary to our hypotheses, the feedback timing effect was non-significant-there was no discernible difference between feedback delivered immediately versus delayed feedback.

DISCUSSION

The findings of this study suggest that the timing of feedback delivery (post-item versus post-item-block) does not influence the efficacy of TEL in this medical education setting. We therefore suggest that educators may consider practical factors when determining appropriate TEL feedback timing in their setting.

Critical thinking pedagogical practices in medical education: a systematic review

Introduction

The development of critical thinking (CT) has been a universal goal in higher education. A systematic review of the literature was conducted to evaluate the effectiveness of currently used pedagogical practices to foster CT/ clinical reasoning (CR)/ clinical judgment (CJ) skills and/or dispositions in undergraduate medical students.

Methods

PubMed, Web of Science and Scopus databases were searched from January 2010 to April 2021 with a predefined Boolean expression.

Results

Of the 3221 articles originally identified, 33 articles were included by using PICOS methodology. From these, 21 (64%) reported CR pedagogical practices and 12 (36%) CT pedagogical practices.

Discussion

Overall, pedagogical practices such as cognitive/visual representation, simulation, literature exposure, test-enhancing and team-based learning, clinical case discussion, error-based learning, game-based learning seem to enhance CT/CR skills and/or dispositions. Further research is required to identify the optimal timing, duration and modality of pedagogical interventions for effectively foster CT/CR in medical education.

Beyond thinking fast and slow: Implications of a transtheoretical model of clinical reasoning and error on teaching, assessment, and research

From dual process to a family of theories known collectively as situativity, both micro and macro theories of cognition inform our current understanding of clinical reasoning (CR) and error. CR is a complex process that occurs in a complex environment, and a nuanced, expansive, integrated model of these theories is necessary to fully understand how CR is performed in the present day and in the future. In this perspective, we present these individual theories along with figures and descriptive cases for purposes of comparison before exploring the implications of a transtheoretical model of these theories for teaching, assessment, and research in CR and error.

Test-Only Learning via Virtual Patients to Improve Surgical Illness Scripts of Preclinical Medical Students as a Preparation for Clinical Clerkships: An Experimental Study

OBJECTIVE

The aim was to determine whether preclinical medical students can develop their illness scripts to a level comparable to that of clerkship students through test-only learning using repeated formative online testlets.

METHODS

In this experimental study, participants were 52 preclinical and 53 clinical medical students. The intervention group consisted of preclinical medical students, and the control group consisted of clinical medical students. The intervention group responded to online testlets containing feedback, an innovative formative assessment method called ContExtended Questions, on general surgery for 8 days by spending no more than 30 minutes each day. The control group completed the general surgery clerkship. The performances were assessed using 20 Key-Feature Question items. The intervention group was assessed twice: immediately after the intervention (the immediate test), and again 1 month later (the delayed test). The control group was assessed once, immediately after the clerkship. All performance tests were identical.

RESULTS

The preclinical students had a significantly higher mean score on the immediate test (83.1 ± 9.6) compared to the clinical students (75.4 ± 8.9), P < .001. The effect size (Cohen's d) was .83. However, the mean score in the delayed test (76.9 ± 13.6) was not significantly different from clinical students' mean score (75.4 ± 8.9), P > .05.

CONCLUSIONS

Test-only learning as a spaced repetition of online formative testlets is effective in preparing preclinical medical students to the clinical clerkship. Through using this approach in preclinical period, they can prepare themselves for the clinical environment to optimize the benefits derived from clerkships.

Using insights from cognitive science for the teaching of clinical skills: AMEE Guide No. 155

Students have to develop a wide variety of clinical skills, from cannulation to advanced life support, prior to entering clinical practice. An important challenge for health professions' educators is the implementation of strategies for effectively supporting students in their acquisition of different types of clinical skills and also to minimize skill decay over time. Cognitive science provides a unified approach that can inform how to maximize clinical skill acquisition and also minimize skill decay. The Guide discusses the nature of expertise and mastery development, the key insights from cognitive science for clinical skill development and skill retention, how these insights can be practically applied and integrated with current approaches used in clinical skills teaching.

Exam prediction and the general Practice Registrar Competency Assessment Grid (GPR-CAG)

BACKGROUND

In GP training, identifying early predictors of poor summative examination performance can be challenging. We aimed to establish whether external clinical teaching visit (ECTV) performance, measured using a validated instrument (GP Registrar Competency Assessment Grid, GPR-CAG) is predictive of Royal Australian College of General Practitioners (RACGP) Fellowship examination performance.

METHODS

A retrospective cohort study including GP registrars in New South Wales/Australian Capital Territory with ECTV data recorded during their first training term (GPT1), between 2014 and 2018, who attempted at least one Fellowship examination. Independent variables of interest included the four GPR-CAG factors assessed in GPT1 ('patient-centredness/caring', 'formulating hypotheses/management plans', 'professional responsibilities', 'physical examination skills'). Outcomes of interest included individual scores of the three summative examinations (Applied Knowledge Test (AKT); Key Feature Problem (KFP); and the Objective Structured Clinical Examination (OSCE)) and overall Pass/Fail status. Univariable and multivariable regression analyses were performed.

RESULTS

Univariably, there were statistically significant associations (p < 0.01) between all four GPR-CAG factors and all four summative examination outcomes, except for 'formulating hypotheses/management plans' and OSCE score (p = 0.07). On multivariable analysis, each factor was significantly associated (p < 0.05) with at least one exam outcome, and 'physical examination skills' was significantly associated (p < 0.05) with all four exam outcomes.

DISCUSSION

ECTV performance, via GPR-CAG scores, is predictive of RACGP Fellowship exam performance. The univariable findings highlight the pragmatic utility of ECTVs in flagging registrars who are at-risk of poor exam performance, facilitating early intervention. The multivariable associations of GPR-CAG scores and examination performance suggest that these scores provide predictive ability beyond that of other known predictors.

A Team-based Learning Approach During Pediatric Clerkship to Promote Clinical Reasoning

BACKGROUND

Clinical reasoning skills are essential to medical practice. The problem lies in assuming junior medical students with limited experience develop clinical reasoning and decision-making skills passively and solely through clinical exposure. To prepare learners for independent practice and care for future patients, explicit teaching and assessment of clinical reasoning in low-stakes, collaborative learning environments are needed.

METHODS

The key-feature question (KFQs) format is an assessment approach that focuses on the reasoning and decision-making aspects of medical problems rather than knowledge recall. This report describes the development, implementation, and evaluation of a team-based learning (TBL) approach using KFQs to foster clinical reasoning in the third-year pediatric clerkship at our institution.

RESULTS

Over the first 2 years of implementation (2017-18, 2018-19), 278 students participated in TBL sessions. Mean individual student scores significantly improved in the group setting for both academic years (P < .001). Individual scores were moderately positively correlated with their total summative Objective Structured Clinical Examination score (r[275] = 0.51; P < .001). The correlation between individual score and their multiple-choice question examination was weaker at 0.29 (P < .001) but remained positive.

CONCLUSIONS

A TBL session using KFQs to both teach and assess clinical reasoning in clerkship students may help educators identify students with gaps in knowledge and/or reasoning. The next steps include developing and implementing individualized coaching opportunities, and expanding this approach within the undergraduate medical curriculum. Outcome measures to assess clinical reasoning in authentic patient encounters require further research and development.

Cognitive perspectives on maintaining physicians' medical expertise: IV. Best practices and open questions in using testing to enhance learning and retention

Although tests and assessments-such as those used to maintain a physician's Board certification-are often viewed merely as tools for decision-making about one's performance level, strong evidence now indicates that the experience of being tested is a powerful learning experience in its own right: The act of retrieving targeted information from memory strengthens the ability to use it again in the future, known as the testing effect. We review meta-analytic evidence for the learning benefits of testing, including in the domain of medicine, and discuss theoretical accounts of its mechanism(s). We also review key moderators-including the timing, frequency, order, and format of testing and the content of feedback-and what they indicate about how to most effectively use testing for learning. We also identify open questions for the optimal use of testing, such as the timing of feedback and the sequencing of complex knowledge domains. Lastly, we consider how to facilitate adoption of this powerful study strategy by physicians and other learners.

Using item response theory to appraise key feature examinations for clinical reasoning

BACKGROUND

Validation of examinations is usually based on classical test theory. In this study, we analysed a key feature examination according to item response theory and compared the results with those of a classical test theory approach.

METHODS

Over the course of five years, 805 fourth-year undergraduate students took a key feature examination on general medicine consisting of 30 items. Analyses were run according to a classical test theory approach as well as using item response theory. Classical test theory analyses are reported as item difficulty, discriminatory power, and Cronbach's alpha while item response theory analyses are presented as item characteristics curves, item information curves and a test information function.

RESULTS

According to classical test theory findings, the examination was labelled as easy. Analyses according to item response theory more specifically indicated that the examination was most suited to identify struggling students. Furthermore, the analysis allowed for adapting the examination to specific ability ranges by removing items, as well as comparing multiple samples with varying ability ranges.

CONCLUSIONS

Item response theory analyses revealed results not yielded by classical test theory. Thus, both approaches should be routinely combined to increase the information yield of examination data.

Enhancing Examination Success: the Cumulative Benefits of Self-Assessment Questions and Virtual Patient Cases

PURPOSE

Research on the learning benefits of the feedback-rich formative assessment environment of virtual patient cases (VPCs) has largely been limited to single institutions and focused on discrete clinical skills or topical knowledge. To augment current understanding, we designed a multi-institutional study to explore the distinct and cumulative effects of VPC formative assessments and optional self-assessment questions (SAQs) on exam performance.

METHOD

In this correlational study, we examined the records of 1,692 students on their family medicine (FM) clerkship at 20 medical schools during the 2014-2015 academic year. Schools utilized an established online curriculum, which included family medicine VPCs, embedded formative assessments, context-rich SAQs corresponding with each VPC, and an associated comprehensive family medicine exam. We used mixed-effects modeling to relate the student VPC composite formative assessment score, SAQ completion, and SAQ performance to students' scores on the FM final examination.

RESULTS

Students scored higher on the final exam when they performed better on the VPC formative assessments, completed associated SAQs, and scored higher on those SAQs. Students' SAQ completion enhanced examination performance above that explained by engagement with the VPC formative assessments alone.

CONCLUSIONS

This large-scale, multi-institutional study furthers the body of research on the effect of formative assessments associated with VPCs on exam performance and demonstrates the added benefit of optional associated SAQs. Findings highlight opportunities for future work on the broader impact of formative assessments for learning, exploring the benefits of integrating VPCs and SAQs, and documenting effects on clinical performance and summative exam scores.

Use of digital teaching resources and predictors of medical student performance during the pandemic: A prospective study

Background

The coronavirus pandemic has led to increased use of digital teaching formats in medical education. A number of studies have assessed student satisfaction with these resources. However, there is a lack of studies investigating changes in student performance following the switch from contact to virtual teaching. Specifically, there are no studies linking student use of digital resources to learning outcome and examining predictors of failure.

Methods

Student performance before (winter term 2019/20: contact teaching) and during (summer term 2020: no contact teaching) the pandemic was compared prospectively in a cohort of 162 medical students enrolled in the clinical phase of a five-year undergraduate curriculum. Use of and performance in various digital resources (case-based teaching in a modified flipped classroom approach; formative key feature examinations of clinical reasoning; daily multiple choice quizzes) was recorded in summer 2020. Student scores in summative examinations were compared to examination scores in the previous term. Associations between student characteristics, resource use and summative examination results were used to identify predictors of performance.

Results

Not all students made complete use of the digital learning resources provided. Timely completion of tasks was associated with superior performance compared to delayed completion. Female students scored significantly fewer points in formative key feature examinations and digital quizzes. Overall, higher rankings within the student cohort (according to summative exams) in winter term 2019/20 as well as male gender predicted summative exam performance in summer 2020. Scores achieved in the first formative key feature examination predicted summative end-of-module exam scores.

Conclusions

The association between timely completion of tasks as well as early performance in a module and summative exams might help to identify students at risk and offering help early on. The unexpected gender difference requires further study to determine whether the shift to a digital-only curriculum disadvantages female students.

Developing Clinical Reasoning in a Physician Assistant Curriculum: The University of Sheffield approach

ABSTRACT

Clinical reasoning remains a key area for development amongst clinical students across the world. Physician assistant (PA) roles (physician associate in the United Kingdom) are rapidly expanding, and there is a need to ensure that this core skill is reflected in course curricula. This article presents how the University of Sheffield has integrated clinical reasoning into the curriculum for its PA course. This includes recognizing the need to consider different approaches to reasoning and how they are taught, the assessment of reasoning across Miller's pyramid, and other considerations that contribute to embedding reasoning within the course. We discuss the implications of our approach and comment on issues that we may need to consider in the future.

Methods to Improve Diagnostic Reasoning in Undergraduate Medical Education in the Clinical Setting: a Systematic Review

BACKGROUND

In recent years, there have been numerous studies exploring different teaching methods for improving diagnostic reasoning in undergraduate medical students. This systematic review examines and summarizes the evidence for the effectiveness of these teaching methods during clinical training.

METHODS

PubMed, Embase, Scopus, and ERIC were searched. The inclusion criteria for the review consist of articles describing (1) methods to enhance diagnostic reasoning, (2) in a clinical setting (3) on medical students. Articles describing original research using qualitative, quantitative, or mixed study designs and published within the last 10 years (1 April 2009-2019) were included. Results were screened and evaluated for eligibility. Relevant data were then extracted from the studies that met the inclusion criteria.

RESULTS

Sixty-seven full-text articles were first identified. Seventeen articles were included in this review. There were 13 randomized controlled studies and four quasi-experimental studies. Of the randomized controlled studies, six discussed structured reflection, four self-explanation, and three prompts for generating differential diagnoses. Of the remaining four studies, two employed the SNAPPS¹ technique for case presentation. Two other studies explored schema-based instruction and using illness scripts. Twelve out of 17 studies reported improvement in clinical reasoning after the intervention. All studies ranked level two on the New World Kirkpatrick model.

DISCUSSION

The authors posit a framework to teach diagnostic reasoning in the clinical setting. The framework targets specific deficiencies in the students' reasoning process. There remains a lack of studies comparing the effectiveness of different methods. More comparative studies with standardized assessment and evaluation of long-term effectiveness of these methods are recommended.

Effectiveness of a serious game addressing guideline adherence: cohort study with 1.5-year follow-up

BACKGROUND

Patients presenting with acute shortness of breath and chest pain should be managed according to guideline recommendations. Serious games can be used to train clinical reasoning. However, only few studies have used outcomes beyond student satisfaction, and most of the published evidence is based on short-term follow-up. This study investigated the effectiveness of a digital simulation of an emergency ward regarding appropriate clinical decision-making.

METHODS

In this prospective trial that ran from summer 2017 to winter 2018/19 at Göttingen Medical University Centre, a total of 178 students enrolled in either the fourth or the fifth year of undergraduate medical education took six 90-min sessions of playing a serious game ('training phase') in which they managed virtual patients presenting with various conditions. Learning outcome was assessed by analysing log-files of in-game activity (including choice of diagnostic methods, differential diagnosis and treatment initiation) with regard to history taking and patient management in three virtual patient cases: Non-ST segment elevation myocardial infarction (NSTEMI), pulmonary embolism (PE) and hypertensive crisis. Fourth-year students were followed up for 1.5 years, and their final performance was compared to the performance of students who had never been exposed to the game but had otherwise taken the same five-year undergraduate course.

RESULTS

During the training phase, overall performance scores increased from 57.6 ± 1.1% to 65.5 ± 1.2% (p < 0.001; effect size 0.656). Performance remained stable over 1.5 years, and the final assessment revealed a strong impact of ever-exposure to the game on management scores (72.6 ± 1.2% vs. 63.5 ± 2.1%, p < 0.001; effect size 0.811). Pre-exposed students were more than twice as likely to correctly diagnose NSTEMI and PE and showed significantly greater adherence to guideline recommendations (e.g., troponin measurement and D-dimer testing in suspected PE).

CONCLUSIONS

The considerable difference observed between previously exposed and unexposed students suggests a long-term effect of using the game although retention of specific virtual patient cases rather than general principles might partially account for this effect. Thus, the game may foster the implementation of guideline recommendations.

Interactive, case-based seminars in the digitized pediatrics block internship from the students' perspective

We report on the pediatric block practice digitized due to the COVID-19 pandemic. Nineteen seminars were recorded, which represent the broad spectrum of pediatrics as comprehensively as possible, and made available on the learning platform ILIAS in a learning sequence. In order to increase attention and learning success formative questions were interspersed into the seminars. The evaluation of the students showed a high acceptance of this course. Especially the formative questions and the high time flexibility were appreciated. A major point of criticism was that not all questions were resolved immediately afterwards. The resolutions were inserted in the meantime. Parts of the digitized block practical course should therefore be used in the pediatric block practical course after the end of the corona restrictions after appropriate revision.

The student is key: A realist review of educational interventions to develop analytical and non-analytical clinical reasoning ability

OBJECTIVES

Clinical reasoning refers to the cognitive processes used by individuals as they formulate a diagnosis or treatment plan. Clinical reasoning is dependent on formal and experiential knowledge. Developing the ability to acquire and recall knowledge effectively for both analytical and non-analytical cognitive processing has patient safety implications. This realist review examines the way educational interventions develop analytical and non-analytical reasoning ability in undergraduate education. A realist review is theory-driven, seeking not only to identify if an intervention works, but also understand the reasons why, for whom, and in what circumstances. The aim of this study is to develop understanding about the way educational interventions develop effective analytical and non-analytical clinical reasoning ability, when they do, for whom and in what circumstances.

METHODS

Literature from a scoping search, combined with expert opinion and researcher experience was synthesised to generate an initial programme theory (IPT). Four databases were searched and articles relevant to the developing theory were selected as appropriate. Factors affecting educational outcomes at the individual student, teacher and wider organisational levels were investigated in order to further refine the IPT.

RESULTS

A total of 28 papers contributed to the overall programme theory. The review predominantly identified evidence of mechanisms for interventions at the individual student level. Key student level factors influencing the effectiveness of interventions included an individual's self-confidence, self-efficacy and pre-existing level of knowledge. These contexts influenced a variety of educational interventions, impacting both positively and negatively on educational outcomes.

CONCLUSIONS

Development of analytical and non-analytical clinical reasoning ability requires activities that enhance knowledge acquisition and recall alongside the accumulation of clinical experience and opportunities to practise reasoning in real or simulated clinical environments. However, factors such as pre-existing knowledge and self-confidence influence their effectiveness, especially amongst individuals with 'low knowledge.' Promoting non-analytical reasoning once novices acquire more clinical knowledge is important for the development of clinical reasoning in undergraduate education.

Implicit expression of uncertainty - suggestion of an empirically derived framework

BACKGROUND

Uncertainty occurs in physicians' daily work in almost every clinical context and is also present in the clinical reasoning process. The way physicians communicate uncertainty in their thinking process during handoffs is crucial for patient safety because uncertainty has diverse effects on individuals involved in patient care. Dealing with uncertainty and expressing uncertainty are important processes in the development of professional identity of undergraduate medical students. Many studies focused on how to deal with uncertainty and whether uncertainty is explicitly expressed. Hardly any research has been done regarding implicit expression of uncertainty. Therefore, we studied the ways in which medical students in the role of beginning residents implicitly express uncertainty during simulated handoffs.

METHODS

Sixty-seven advanced undergraduate medical students participated in a simulated first day of residency including a consultation hour, a patient management phase with interprofessional interaction, and a patient handoff. We transcribed the videographed handoffs verbatim and extracted language with respect to expression of uncertainty using a grounded theory approach. Text sequences expressing patient related information were analyzed and coded with respect to language aspects which implicitly modified plain information with respect to increasing or decreasing uncertainty. Concepts and categories were developed and discussed until saturation of all aspects was reached.

RESULTS

We discovered a framework of implicit expressions of uncertainty regarding diagnostic and treatment-related decisions within four categories: "Statement", "Assessment", "Consideration", and "Implication". Each category was related to either the subcategory "Actions" or "Results" within the diagnostic or therapeutic decisions. Within each category and subcategory, we found a subset of expressions, which implicitly attenuated or strengthened plain information thereby increasing uncertainty or certainty, respectively. Language that implicitly attenuated plain information belonged to the categories questionable, incomplete, alterable, and unreliable while we could ascribe implicit strengtheners to the categories assertive, adequate, focused, and reliable.

CONCLUSIONS

Our suggested framework of implicit expression of uncertainty may help to raise the awareness for expression of uncertainty in the clinical reasoning process and provide support for making uncertainty explicit in the teaching process. This may lead to more transparent communication processes among health care professionals and eventually to improved patient safety.

Musculoskeletal Anatomy Knowledge Retention in the Macquarie University Chiropractic Program: A Cross-Sectional Study

Attrition of anatomy knowledge has been an area of concern in health professions curricula. To ensure safe and effective clinical practice, the study of chiropractic requires a good knowledge of musculoskeletal anatomy. In this study, musculoskeletal limb knowledge retention was investigated among students in the 5-year chiropractic program at Macquarie University, Australia. A test of 20 multiple-choice questions, categorized into low-order (LO) and high-order (HO) cognitive ability according to Bloom's Taxonomy, was developed. Students enrolled in the program were invited to participate with 257 of the 387 choosing to participate, (response rate ranging 56%-72% per year level). No attrition of knowledge across the years was observed, instead, a significant increase in knowledge, measured by total LO and HO scores (P < 0.0005), throughout the program. There were significant increases in both low and high cognitive scores which were not uniform, with high-order scores increasing significantly in the last two year levels. The increase of knowledge, may be explained, at least partially, by the vertical and horizontal integrated curriculum. Retrieval of knowledge, especially in clinically applied formats, may have led to an enhanced ability to apply anatomy knowledge and account for the increased scores in the high-order knowledge seen in the later clinical years. Evaluating anatomy knowledge retention at different cognitive levels seems to provide a better assessment and is worth considering in future anatomy educational research.

Conceptualising spaced learning in health professions education: A scoping review

OBJECTIVES

To investigate the definitions and applications of 'spaced learning' and to propose future directions for advancing its study and practice in health professions education.

METHOD

The authors searched five online databases for articles published on spaced learning in health professions education prior to February 2018. Two researchers independently screened articles for eligibility with set inclusion criteria. They extracted and analysed key data using both quantitative and qualitative methods.

RESULTS

Of the 2972 records retrieved, 120 articles were included in the review. More than 90% of these articles were published in the last 10 years. The definition of spaced learning varied widely and was often not theoretically grounded. Spaced learning was applied in distinct contexts, including online learning, simulation training and classroom settings. There was a large variety of spacing formats, ranging from dispersion of information or practice on a single day, to intervals lasting several months. Generally, spaced learning was implemented in practice or testing phases and rarely during teaching.

CONCLUSIONS

Spaced learning is infrequently and poorly defined in the health professions education literature. We propose a comprehensive definition of spaced learning and emphasise that detailed descriptions of spacing formats are needed in future research to facilitate the operationalisation of spaced learning research and practice in health professions education.

Taking CBL to the Lecture Hall: a Comparison of Outcomes Between Traditional Small Group CBL and a Novel Large Group Team-Based CBL Teaching Method

PURPOSE

Case-based learning (CBL), an important component of medical school curricula, is an effective inquiry-based teaching method associated with high levels of student and teacher satisfaction. However, because traditional CBL requires small groups, its feasibility is limited by faculty and resources. We developed and tested a novel team-based CBL (TB-CBL) method to be implemented in the lecture hall.

METHODS

All second-year students at our institution (n = 121) were randomized to either traditional small group CBL or TB-CBL during the Endocrine block and to the other modality during the Renal block. All students were exposed to both methods. Case content was identical, and sessions were run concurrently. This cross-over, non-inferiority study tested the hypothesis that no difference in knowledge acquisition, clinical reasoning, or student satisfaction would be detected between groups.

RESULTS

Based on student performance on case-relevant exam questions, no difference in knowledge acquisition was seen between groups for either block (p = 0.62 Endocrine, p = 0.38 Renal). There was also no difference in overall final exam performance between groups (p = 0.56 Endocrine, p = 0.26 Renal). Case-relevant script concordance testing revealed no difference in clinical reasoning skills between groups (p = 0.87 Endocrine, p = 0.17 Renal). Satisfaction was higher for the TB-CBL format (p = 0.005). Cost analysis revealed that each small group CBL session costs $2654, while each TB-CBL session costs approximately $221.

CONCLUSIONS

TB-CBL, a novel case-based teaching method, appears to produce similar learner outcomes and higher student satisfaction when compared with small group CBL. TB-CBL may be used to supplement case-based curricula while optimizing resource allocation.

Learners and Luddites in the Twenty-first Century: Bringing Evidence-based Education to Anesthesiology

Anesthesiologists are both teachers and learners and alternate between these roles throughout their careers. However, few anesthesiologists have formal training in the methodologies and theories of education. Many anesthesiology educators often teach as they were taught and may not be taking advantage of current evidence in education to guide and optimize the way they teach and learn. This review describes the most up-to-date evidence in education for teaching knowledge, procedural skills, and professionalism. Methods such as active learning, spaced learning, interleaving, retrieval practice, e-learning, experiential learning, and the use of cognitive aids will be described. We made an effort to illustrate the best available evidence supporting educational practices while recognizing the inherent challenges in medical education research. Similar to implementing evidence in clinical practice in an attempt to improve patient outcomes, implementing an evidence-based approach to anesthesiology education may improve learning outcomes.

Quantifying the medical student learning curve for ECG rhythm strip interpretation using deliberate practice

Objectives: Obtaining competency in medical skills such as interpretation of electrocardiograms (ECGs) requires repeated practice and feedback. Structured repeated practice and feedback for ECGs is likely not provided to most medical students, so skill development is dependent on opportunistic training during clinical rotations. Our aim was to describe: the amount of deliberate practice completed for learning ECG rhythm strip diagnoses in first year medical students, the learning curve for rhythm strip diagnosis, and student experiences with deliberate practice. Methods: First year medical students from two medical schools were provided with online rhythm strip practice cases. Diagnostic accuracy was measured throughout practice, and students were provided feedback for every case they completed. Total cases practiced and time spent practicing were correlated with their performance during practice and on an exam. Results: 314 of 384 (82%) students consented. The mean number of ECGs each student practiced was 59 (range 0-280), representing 18,466 total instances of deliberate practice. We generated mathematical models that accurately correlated both the number of cases practiced and time spent practicing, with diagnostic accuracy on an exam (p<0.001). For example, students would need to spend on average of 112 minutes and complete 34 practice cases to obtain 75% on an ECG rhythm strip exam. Student satisfaction was high using the online cases. Conclusions: We succeeded in delivering deliberate practice for ECG rhythm strip interpretation to a large cohort of students at 2 medical schools. We quantified a learning curve that estimates the number of cases and practice time required to achieve pre-determined levels of diagnostic accuracy. This data can help inform a competency-based approach to curriculum development.

Retrieval practice and judgements of learning enhance transfer of physiology information

It is well-documented that retrieval practice enhances the recall of simple and complex information (Karpicke and Aue in Educ Psychol Rev 27(2):317-326, 2015). Evidence is also accumulating that retrieval practice can enhance other cognitive processes such as the ability to critically evaluate research articles (Dobson et al. in Med Educ 52(5):513-525, 2018) and transfer of learning (Butler in J Exp Psychol Learn Mem Cogn 36(5):1118, 2010). One aim of this investigation was to explore the effects of retrieval practice on transfer of learning with physiology information. A second aim was to compare recall and transfer of physiology information following retrieval practice versus a judgment of learning task (JOL) that is potentially less time consuming for students to use. Participants were randomly assigned to learn three physiology texts using each of the following strategies: (1) studying a text four times (S-S-S-S), (2) studying and then retrieving a text two times (S-R-S-R), and (3) studying a text four times while completing multiple JOL during the second and fourth repetitions (S-S/J-S-S/J). Recall and accuracy on transfer questions were assessed 1 week after the learning phase, and the results were analyzed using repeated measures ANOVAs. The S-R-S-R strategy (21.35 ± 1.08%) produced significantly greater recall than the S-S-S-S strategy (17.35 ± 0.86%), and both the S-R-S-R (44.60 ± 2.55%) and S-S/J-S-S/J (41.79 ± 2.63%) strategies lead to significantly greater transfer than the S-S-S-S strategy (36.07 ± 2.40%). These results provide evidence that retrieval practice enhances recall and transfer of physiology information and that a JOL task can also prove to be beneficial but to a lesser degree.

The Impact of Medical Students' Individual Teaching Format Choice on the Learning Outcome Related to Clinical Reasoning

BACKGROUND

Repeated formative assessments using key feature questions have been shown to enhance clinical reasoning. Key feature questions augmented by videos presenting clinical vignettes may be more effective than text-based questions, especially in a setting where medical students are free to choose the format they would like to work with. This study investigated learning outcomes related to clinical reasoning in students using video- or text-based key feature questions according to their individual preferences.

OBJECTIVE

The aim of this study was to test the hypothesis that repeated exposure to video-based key feature questions enhances clinical reasoning to a greater extent than repeated exposure to text-based key feature questions if students are allowed to choose between those different formats on their own.

METHODS

In this monocentric, prospective, nonrandomized trial, fourth-year medical students attended 12 computer-based case seminars during which they worked on case histories containing key feature questions. Cases were available in a text- and a video-based format. Students chose their preferred presentation format at the beginning of each case seminar. Student performance in key feature questions was assessed in formative entry, exit, and retention exams and was analyzed with regard to preceding exposure to video- or text-based case histories.

RESULTS

Of 102 eligible students, 75 provided written consent and complete data at all study exams (response rate=73.5%). A majority of students (n=52) predominantly chose the text-based format. Compared with these, students preferring the video-based format achieved a nonsignificantly higher score in the exit exam (mean 76.2% [SD 12.6] vs 70.0% [SD 19.0]; P=.15) and a significantly higher score in the retention exam (mean 75.3% [SD 16.6] vs 63.4% [SD 20.3]; P=.02). The effect was independent of the video- or text-based presentation format, which was set as default in the respective exams.

CONCLUSIONS

Despite students' overall preference for text-based case histories, the learning outcome with regard to clinical reasoning was higher in students with higher exposure to video-based items. Time-on-task is one conceivable explanation for these effects as working with video-based items was more time-consuming. The baseline performance levels of students do not account for the results as the preceding summative exam results were comparable across the 2 groups. Given that a substantial number of students chose a presentation format that was less effective, students might need to be briefed about the beneficial effects of using video-based case histories to be able to make informed choices about their study methods.

Generative retrieval results in positive academic emotions and long-term retention of cardiovascular anatomy using transthoracic echocardiography

With increasing medical knowledge, procedural, and diagnostic skills to learn, it is vital for educators to make the limited amount of teaching time available to students effective and efficient. Generative retrieval is an effective and efficient learning tool, improving long-term retention through the practice of retrieval from memory. Forty medical students were randomized to learn normal cardiovascular anatomy using transthoracic echocardiography video clips in a generative retrieval (GR) or standard practice (SP) group. GR participants were required to verbally identify each unlabeled cardiovascular structure after viewing the video. After answering, participants viewed the correctly labeled video. SP participants viewed the same video clips labeled with the correct cardiovascular structure for the same amount of total time without verbally generating an answer. All participants were tested for intermediate (1-wk), late (1-mo), and long-term (6- to 9-mo) retention of cardiovascular anatomy. Additionally, a three-question survey was incorporated to assess perceptions of the learning method. There was no difference in pretest scores. The GR group demonstrated a trend toward improvement in recall at 1 wk [GR = 74.3 (SD 12.3); SP = 65.4 (SD 16.7); P = 0.10] and 1 mo [GR = 69.9 (SD15.6); SP = 64.3 (SD 15.4); P = 0.33]. At the 6- to 9-mo time point, there was a statistically significant difference in scores [GR = 74.3 (SD 9.9); SP = 65.0 (SD 14.1); P = 0.042]. At nearly every time point, learners had a statistically significantly higher perception of effectiveness, enjoyment, and satisfaction with GR. In addition to improved recall, GR is associated with increased perceptions of effectiveness, enjoyment, and satisfaction, which may lead to increased engagement, time spent studying, and improved retention.

Assessment, feedback and the alchemy of learning

CONTEXT

Models of sound assessment practices increasingly emphasise assessment's formative role. As a result, assessment must not only support sound judgements about learner competence, but also generate meaningful feedback to guide learning. Reconciling the tension between assessment's focus on judgement and decision making and feedback's focus on growth and development represents a critical challenge for researchers and educators.

METHODS

We synthesise the literature related to this tension, framed around four trends in education research: (i) shifting perspectives on assessment; (ii) shifting perspectives on feedback; (iii) increasing attention on learners' perceptions of assessment and feedback, and (iv) increasing attention on the influence of culture on assessment and feedback. We describe factors that produce and sustain this tension.

RESULTS

The lines between assessment and feedback frequently blur in medical education. Models of programmatic assessment deliberately use the same data for both purposes: low-stakes individual data points are used formatively, but then are added together to support summative judgements. However, the translation of theory to practice is not straightforward. Efforts to embed meaningful feedback in programmes of learning face a multitude of threats. Learners may perceive assessment with formative intent as summative, restricting their engagement with it as feedback, and thus diminishing its learning value. A learning culture focused on assessment may limit learners' sense of safety to explore, to experiment, and sometimes to fail.

CONCLUSIONS

Successfully blending assessment and feedback demands clarity of purpose, support for learners, and a system and organisational commitment to a culture of improvement rather than a culture of performance.

Pretest Scores Uniquely Predict 1-Year-Delayed Performance in a Simulation-Based Mastery Course for Central Line Insertion

INTRODUCTION

Within simulation-based mastery learning (SBML) courses, there is inconsistent inclusion of learner pretesting, which requires considerable resources and is contrary to popular instructional frameworks. However, it may have several benefits, including its direct benefit as a form of deliberate practice and its facilitation of more learner-specific subsequent deliberate practice. We consider an unexplored potential benefit of pretesting: its ability to predict variable long-term learner performance.

METHODS

Twenty-seven residents completed an SBML course in central line insertion. Residents were tested on simulated central line insertion precourse, immediately postcourse, and after between 64 and 82 weeks. We analyzed pretest scores' prediction of delayed test scores, above and beyond prediction by program year, line insertion experiences in the interim, and immediate posttest scores.

RESULTS

Pretest scores related strongly to delayed test scores (r = 0.59, P = 0.01; disattenuated ρ = 0.75). The number of independent central lines inserted also related to year-delayed test scores (r = 0.44, P = 0.02); other predictors did not discernibly relate. In a regression model jointly predicting delayed test scores, pretest was a significant predictor (β = 0.487, P = 0.011); number of independent insertions was not (β = 0.234, P = 0.198).

CONCLUSIONS

This study suggests that pretests can play a major role in predicting learner variance in learning gains from SBML courses, thus facilitating more targeted refresher training. It also exposes a risk in SBML courses that learners who meet immediate mastery standards may be incorrectly assumed to have equal long-term learning gains.

Teaching Diagnostic Reasoning: A Review of Evidence-Based Interventions

Problem/Background: The ability to accurately diagnose patients based on symptom profiles is a vital yet challenging skill that Nurse Practitioners (NPs) undertake frequently.

PURPOSE

This integrative literature review highlights a variety of evidence based, practical educational strategies that foster the development of diagnostic reasoning.

METHODS

An integrative literature review was conducted in order to identify original research focusing on diagnostic reasoning educational interventions.

RESULTS

Eighteen primary sources met inclusion and exclusion criteria. Results are synthesized in terms of sample and setting, methodological features, interventions, and outcomes. Interventions broadly fit into five educational themes: testing strategies, cognitive biases, simulation programs, course formats, and instructional approaches.

DISCUSSION

Interventions are simple and can be implemented in multiple educational settings. Future research should occur in populations of NP students. Validated, easy-to-use measurement tools as well as more precise diagnostic reasoning concept development should occur.

Training of clinical reasoning with a Serious Game versus small-group problem-based learning: A prospective study

INTRODUCTION

Serious Games are increasingly being used in undergraduate medical education. They are usually intended to enhance learning with a focus on knowledge acquisition and skills development. According to the current literature, few studies have assessed their effectiveness regarding clinical reasoning (CR). The aim of this prospective study was to compare a Serious Game, the virtual Accident & Emergency department 'EMERGE' to small-group problem-based learning (PBL) regarding student learning outcome on clinical reasoning in the short term.

METHODS

A total of 112 final-year medical students self-selected to participate in ten 90-minute sessions of either small-group PBL or playing EMERGE. CR was assessed in a formative examination consisting of six key feature cases and a final 45-minute EMERGE session.

RESULTS

Overall, the EMERGE group (n = 78) scored significantly higher than the PBL group (n = 34) in the key feature examination (62.5 (IQR: 17.7)% vs. 54.2 (IQR: 21.9)%; p = 0.015). There was no significant difference in performance levels between groups regarding those cases which had been discussed in both instructional formats during the training phase. In the final EMERGE session, the EMERGE group achieved significantly better results than the PBL group in all four cases regarding the total score as well as in three of four cases regarding the final diagnosis and the correct therapeutic interventions.

CONCLUSION

EMERGE can be used effectively for CR training in undergraduate medical education. The difference in key feature exam scores was driven by additional exposure to more cases in EMERGE compared to PBL despite identical learning time in both instructional formats. EMERGE is a potential alternative to intensive small-group teaching. Further work is needed to establish how Serious Games enhance CR most effectively.

How can we teach medical students to choose wisely? A randomised controlled cross-over study of video- versus text-based case scenarios

BACKGROUND

The Choosing Wisely campaign highlights the importance of clinical reasoning abilities for competent and reflective physicians. The principles of this campaign should be addressed in undergraduate medical education. Recent research suggests that answering questions on important steps in patient management promotes knowledge retention. It is less clear whether increasing the authenticity of educational material by the inclusion of videos further enhances learning outcome.

METHODS

In a prospective randomised controlled cross-over study, we assessed whether repeated video-based testing is more effective than repeated text-based testing in training students to choose appropriate diagnostic tests, arrive at correct diagnoses and identify advisable therapies. Following an entry exam, fourth-year undergraduate medical students attended 10 weekly computer-based seminars during which they studied patient case histories. Each case contained five key feature questions (items) on the diagnosis and treatment of the presented patient. Students were randomly allocated to read text cases (control condition) or watch videos (intervention), and assignment to either text or video was switched between groups every week. Using a within-subjects design, student performance on video-based and text-based items was assessed 13 weeks (exit exam) and 9 months (retention test) after the first day of term. The primary outcome was the within-subject difference in performance on video-based and text-based items in the exit exam.

RESULTS

Of 125 eligible students, 93 provided data for all three exams (response rate 74.4%). Percent scores were significantly higher for video-based than for text-based items in the exit exam (76.2 ± 19.4% vs. 72.4 ± 19.1%, p = 0.026) but not the retention test (69.2 ± 20.2% vs. 66.4 ± 20.3%, p = 0.108). An additional Bayesian analysis of this retention test suggested that video-based training is marginally more effective than text-based training in the long term (Bayes factor 2.36). Regardless of presentation format, student responses revealed a high prevalence of erroneous beliefs that, if applied to the clinical context, could place patients at risk.

CONCLUSION

Repeated video-based key feature testing produces superior short-term learning outcome compared to text-based testing. Given the high prevalence of misconceptions, efforts to improve clinical reasoning training in medical education are warranted. The Choosing Wisely campaign lends itself to being part of this process.

Retrieval practice enhances the ability to evaluate complex physiology information

OBJECTIVE

Many investigations have shown that retrieval practice enhances the recall of different types of information, including both medical and physiological, but the effects of the strategy on higher-order thinking, such as evaluation, are less clear. The primary aim of this study was to compare how effectively retrieval practice and repeated studying (i.e. reading) strategies facilitated the evaluation of two research articles that advocated dissimilar conclusions. A secondary aim was to determine if that comparison was affected by using those same strategies to first learn important contextual information about the articles.

METHODS

Participants were randomly assigned to learn three texts that provided background information about the research articles either by studying them four consecutive times (Text-S) or by studying and then retrieving them two consecutive times (Text-R). Half of both the Text-S and Text-R groups were then randomly assigned to learn two physiology research articles by studying them four consecutive times (Article-S) and the other half learned them by studying and then retrieving them two consecutive times (Article-R). Participants then completed two assessments: the first tested their ability to critique the research articles and the second tested their recall of the background texts.

RESULTS

On the article critique assessment, the Article-R groups' mean scores of 33.7 ± 4.7% and 35.4 ± 4.5% (Text-R then Article-R group and Text-S then Article-R group, respectively) were both significantly (p < 0.05) higher than the two Article-S mean scores of 19.5 ± 4.4% and 21.7 ± 2.9% (Text-S then Article-S group and Text-R then Article-S group, respectively). There was no difference between the two Article-R groups on the article critique assessment, indicating those scores weren't affected by the different contextual learning strategies.

CONCLUSION

Retrieval practice promoted superior critical evaluation of the research articles, and the results also indicated the strategy enhanced the recall of background information.

Assessing the application of knowledge in clinical problem-solving: The structured professional reasoning exercise

INTRODUCTION

Clinical reasoning is a fundamental and core clinical competence of healthcare professionals. The study aimed to investigate the utility of the Structured Professional Reasoning Exercise (SPRE), a new competence assessment method designed to measure dental students' clinical reasoning in simulated scenarios, covering the clinical areas of Oral Disease, Primary Dental Care and Restorative Dentistry, Child Dental Health and Dental Practice and Clinical Governance.

MATERIALS AND METHODS

A total of 313 year-5 students sat for the assessment. Students spent 45 minutes assimilating the scenarios, before rotating through four pairs of 39 trained examiners who each independently assessed a single scenario over a ten-minute period, using a structured marking sheet. After the assessment, all students and examiners were invited to complete an anonymous perception questionnaire of the exercise. These questionnaires and the examination scores were statistically analysed.

RESULTS AND DISCUSSION

Oral Disease showed the lowest scores; Dental Practice and Governance the highest. The overall Intraclass Correlation Coefficient (ICC) was 0.770, whilst examiner training helped to increase the ICC from 0.716 in 2013 to 0.835 in 2014. Exploratory factor analysis revealed one major factor with an eigenvalue of 2.75 (68.8% of total variance). The Generalizability coefficient was consistent at 0.806. A total of 295 students and 32 examiners completed the perception questionnaire. Students' lowest examination perceptions were an "Unpleasant" and "Unenjoyable" experience, whilst the highest were "Interesting", "Valuable" and "Important". The majority of students and examiners reported the assessment as acceptable, fair and valid.

CONCLUSION

The SPRE offers a reliable, valid and acceptable assessment method, provided it comprises at least four scenarios with two independently marking and trained assessors. 3.

Test-enhanced learning in health professions education: A systematic review: BEME Guide No. 48

BACKGROUND

Cognitive psychology studies demonstrate that subjects who attempt to recall information show better learning, retention, and transfer than subjects who spend the same time studying the same material (test-enhanced learning, TEL). We systematically reviewed TEL interventions in health professions education.

METHODS

We searched 13 databases, 14 medical education journals, and reference lists. Inclusion criteria included controlled studies of TEL that compared TEL to studying the same material or to a different TEL strategy. Two raters screened articles for inclusion, abstracted information, determined quality scores, and calculated the standardized mean difference (SMD) for the learning outcomes.

RESULTS

Inter-rater agreement was excellent for all comparisons. The 19 included studies reported 41 outcomes with data sufficient to determine a SMD. TEL interventions included short answer questions, multiple choice questions, simulation, and standardized patients. Five of six immediate learning outcomes (SMD 0.09-0.44), 21 of 23 retention outcomes (SMD 0.12-2.5), and all seven transfer outcomes (SMD 0.33-1.1) favored TEL over studying.

CONCLUSIONS

TEL demonstrates robust effects across health professions, learners, TEL formats, and learning outcomes. The effectiveness of TEL extends beyond knowledge assessed by examinations to clinical applications. Educators should include TEL in health professions curricula to enhance recall, retention, and transfer.

[Choosing Wisely in medical education]

Choosing Wisely recommendations address situations where physicians will have to make decisions about further diagnostic and therapeutic steps. Undergraduate medical education needs to equip students with the foundations on which clinical reasoning skills can be acquired and fostered throughout their clinical career. Teaching these skills usually involves patients (e.g., bedside teaching, electives, clinical attachments) but it can also be delivered in the format of formalised small-group, case-based learning. Case-based key feature tests have been developed to facilitate the assessment of learning outcomes related to clinical reasoning. Repeated testing with key feature cases yields better medium-term retention than repeatedly studying the same material (without questions). The project 'Choosing Wisely in medical education', which was funded by the German Association for Internal Medicine, involves the creation of key feature cases with reference to the German set of Choosing Wisely recommendations. This article presents the results of the first pilot study using these new cases.

Effectiveness of Test-Enhanced Learning (TEL) in lectures for undergraduate medical students

Objective

To determine the effectiveness of Test-Enhanced learning as a learning tool in lectures for undergraduate medical students.

Method

This quantitative, randomized controlled trial included eighty-four students of 4^th year MBBS from Yusra Medical & Dental College, Islamabad. The duration of study was from March 2016 to August 2016. After obtaining the informed consent; participants were equally assigned to interventional and non-interventional study groups through stratified randomization. Single best answer MCQs of special pathology were used as data collection instrument after validation. A pre- and post-test was taken from both groups, before and after the intervention, respectively and their results were compared using SPSS version 21.

Results

There were 13 male (31%) and 29 female (69%) participants in each study group who showed an equivalent baseline performance on pre-test (p=0.95). Statistically significant difference was found among mean scores of interventional and non-interventional study groups at exit exam (p=0.00). Interventional group also showed a significant improvement in their post-test scores (mean: 17.17±1.59) as compared to pre-test scores (mean: 6.19±1.81).

Conclusions

Test-enhanced learning has significant effect on improving the learning of course content delivered to undergraduate medical students through lectures.

Does Test-enhanced Learning Improve Success Rates of Ultrasound-guided Peripheral Intravenous Insertion? A Randomized Controlled Trial

INTRODUCTION

Teaching procedural skills in medicine is time- and resource-intensive, and it is therefore important to determine which educational strategies are most effective. Test-enhanced learning (TEL) has been demonstrated to be effective in improving learner retention; however, there is little research evaluating the testing effect on the acquisition of procedural skills in medicine. The objective of this study was to examine the impact of TEL on learning ultrasound-guided intravenous (USG IV) insertion on a simulated model.

METHODS

We conducted a prospective randomized controlled trial of medical students at a single tertiary care academic hospital. Participants were randomized to either the TEL group (TEG) or control group (CG). Each group received an USG peripheral IV teaching session that included a didactic portion and hands-on skills training. The training sessions were identical except that the TEG was informed at the outset that there would be an assessment at the end of the session. The TEG then received a formal assessment of the skill during the last 15 minutes of the session, whereas the CG had continued practice time. Subjects in both groups were evaluated 10-14 days later to compare skill performance using a simulation-based assessment tool consisting of a global rating scale (GRS) and checklist items.

RESULTS

Thirty medical students completed the study, 15 in the TEG and 15 in the CG. There were no significant differences between the two groups at baseline based on year of medical training or prior IV or ultrasound experience. The overall procedural success rate was 93.3% (95% CI = 79.0%-100.0%) in the TEG and 80.0% (95% CI = 57.1%-100%) in the CG (p = 0.60). The first-attempt failure rate was 13.3% (95% CI = 0.0%-32.8%) in the TEG and 33.3% (95% CI = 6.3%-60.4%) in the CG (p = 0.39). There were no statistically significant differences between the means of the GRS (TEG = 4.7, CG = 4.2; p = 0.53, r = 0.11) or checklist scores (TEG = 78.6%, CG = 75.3%; p = 0.20, r = 0.24).

CONCLUSIONS

In this study of novice learners, both TEL with structured practice and structured practice alone lead to high success rates performing USG IV insertion on a simulated model. While we noted a trend toward higher procedural success rates and lower first-attempt failure rates in the TEG, these did not meet statistical significance. Further studies with larger sample sizes are required to determine whether the beneficial effects of TEL can be transferred to procedural skills teaching.

Repeated testing improves achievement in a blended learning approach for risk competence training of medical students: results of a randomized controlled trial

BACKGROUND

Adequate estimation and communication of risks is a critical competence of physicians. Due to an evident lack of these competences, effective training addressing risk competence during medical education is needed. Test-enhanced learning has been shown to produce marked effects on achievements. This study aimed to investigate the effect of repeated tests implemented on top of a blended learning program for risk competence.

METHODS

We introduced a blended-learning curriculum for risk estimation and risk communication based on a set of operationalized learning objectives, which was integrated into a mandatory course "Evidence-based Medicine" for third-year students. A randomized controlled trial addressed the effect of repeated testing on achievement as measured by the students' pre- and post-training score (nine multiple-choice items). Basic numeracy and statistical literacy were assessed at baseline. Analysis relied on descriptive statistics (histograms, box plots, scatter plots, and summary of descriptive measures), bootstrapped confidence intervals, analysis of covariance (ANCOVA), and effect sizes (Cohen's d, r) based on adjusted means and standard deviations.

RESULTS

All of the 114 students enrolled in the course consented to take part in the study and were assigned to either the intervention or control group (both: n = 57) by balanced randomization. Five participants dropped out due to non-compliance (control: 4, intervention: 1). Both groups profited considerably from the program in general (Cohen's d for overall pre vs. post scores: 2.61). Repeated testing yielded an additional positive effect: while the covariate (baseline score) exhibits no relation to the post-intervention score, F(1, 106) = 2.88, p > .05, there was a significant effect of the intervention (repeated tests scenario) on learning achievement, F(1106) = 12.72, p < .05, d = .94, r = .42 (95% CI: [.26, .57]). However, in the subgroup of participants with a high initial numeracy score no similar effect could be observed.

CONCLUSION

Dedicated training can improve relevant components of risk competence of medical students. An already promising overall effect of the blended learning approach can be improved significantly by implementing a test-enhanced learning design, namely repeated testing. As students with a high initial numeracy score did not profit equally from repeated testing, target-group specific opt-out may be offered.

Problem-based learning in internal medicine: virtual patients or paper-based problems?

BACKGROUND

Teaching using paper problem-based learning (p-PBL) sessions has left some students fatigued with the learning process. Therefore, attempts have been made to replace p-PBL with digitally enhanced, decision-making PBL in the form of virtual patients (VP). Student enthusiasm for substituting p-PBL with VP has not been quantitatively evaluated on the intended educational effects.

AIM

To determine the educational effects of substituting p-PBL sessions with VP on undergraduate medical students in their internal medicine course.

METHODS

We conducted a randomised controlled study on 34 third-year undergraduate medical students in the academic year 2015-2016. Student performance after an intervention substituting p-PBL sessions with VP was analysed. The educational outcomes were measured with knowledge exams and the Diagnostic Thinking Inventory.

RESULTS

There was no difference in exam performance between groups (P > 0.833) immediately after the intervention, or in long term. Nor was there a significant difference in improvement of diagnostic thinking between groups (P > 0.935 and P > 0.320).

CONCLUSIONS

Our study showed no significant improvement in diagnostic thinking abilities or knowledge exam results with the use of VP. Educators can add VP to sessions to motivate students, but a significant improvement to educational outcome should not be expected.

Kids (learn how to) save lives in the school with the serious game Relive

INTRODUCTION

Relive is a serious game focusing on increasing kids and young adults' awareness on CPR. We evaluated the use of Relive on schoolchildren.

METHODS

A longitudinal, prospective study was carried out in two high schools in Italy over a 8-month period, divided in three phases: baseline, competition, and retention. Improvement in schoolchildren's CPR awareness, in terms of knowledge (MCQ results) and skills (chest compression (CC) rate and depth), was evaluated. Usability of Relive and differences in CC performance according to sex and BMI class were also evaluated.

RESULTS

At baseline, students performed CC with a mean depth of 31mm and a rate of 95 cpm. In the competition phase, students performed CC with a mean depth of 46mm and a rate of 111 cpm. In the retention phase, students performed CC with a mean depth of 47mm and a rate of 131 cpm. Thus, the training session with Relive during the competition phase affected positively both CC depth (p<0.001) and rate (p<0.001). Such an effect persisted up to the retention phase. CC depth was also affected by gender (p<0.01) and BMI class (p<0.01). Indeed, CC depth was significantly greater in male players and in players with higher BMI. Seventy-three percent of students improved their CPR knowledge as represented by an increases in the MCQ score (p<0.001). The participants perceived the Relive to be easy to use with effective feedback.

CONCLUSIONS

Relive is an useful tool to spread CPR knowledge and improve CPR skills in schoolchildren.

Structured Smoking Cessation Training for Medical Students: A Prospective Study

INTRODUCTION

Physician adherence to guideline recommendations regarding the provision of counseling and support for smokers willing to quit is low. A lack of training during undergraduate medical education has been identified as a potential cause. This prospective intervention study evaluated a novel teaching module for medical students.

METHODS

As part of a 6-week cardiovascular course, 125 fourth-year undergraduate medical students received a multimodal and interactive teaching module on smoking cessation, including online learning material, lectures, seminars, and practical skills training. Short- and medium-term effects on knowledge, skills, attitudes, and self-reported practice were measured using written examinations and an objective structured clinical examination at the end of the module and 6 months later. Results were compared to data obtained from a historical control cohort (n = 70) unexposed to the intervention.

RESULTS

At the 6-month follow-up, scores in the knowledge test were significantly higher in the intervention than the control group (61.1% vs. 51.7%; p < .001). A similar pattern was observed in the objective structured clinical examination (71.5% vs. 60.5%; p < .001). More students in the intervention than control group agreed that smoking was a chronic disease (83.1% vs. 68.1%; p = .045). The control group was more likely to report recording smoking status (p = .018), but no group difference was detected regarding the report of advising to quit (p = .154).

CONCLUSIONS

A novel teaching module for undergraduate medical students produced a sustained learning outcome in terms of knowledge, skills, and attitudes but not self-reported practice.

IMPLICATIONS

Studies across the world have identified considerable knowledge gaps and deficits in practical training with regard to smoking cessation counseling in undergraduate medical students. This paper describes a teaching intervention informed by current recommendations for the design of educational activities aimed at enabling medical students to deliver adequate behavior change counseling. The teaching module was tailored to the needs of a specific healthcare system. Given its effectiveness as demonstrated in this prospective study, a rollout of this intervention in medical schools might have the potential to substantially improve medical students' knowledge, skills, and attitudes in relation to smoking cessation counseling.