Repeated testing improves achievement in a blended learning approach for risk competence training of medical students: results of a randomized controlled trial

How to design effective educational videos for teaching evidence-based medicine to undergraduate learners - systematic review with complementing qualitative research to develop a practicable guide

BACKGROUND

eLearning can be an effective tool to achieve learning objectives. It facilitates asynchronous distance learning, increasing flexibility for learners and instructors. In this context, the high educational value of videos provides an invaluable primary component for longitudinal digital curricula, especially for maintaining knowledge on otherwise rarely taught subjects. Although literature concerning eLearning evaluation exists, research comprehensively describing how to design effective educational videos is lacking. In particular, studies on the requirements and design goals of educational videos need to be complemented by qualitative research using grounded theory methodology.

METHODS

Due to the paucity of randomized controlled trials in this area, there is an urgent need to generate recommendations based on a broader fundament than a literature search alone. Thus, the authors have employed grounded theory as a guiding framework, augmented by Mayring's qualitative content analysis and commonly used standards. An adaptive approach was conducted based on a literature search and qualitative semi-structured interviews. Drawing on these results, the authors elaborated a guide for creating effective educational videos.

RESULTS

The authors identified 40 effective or presumedly effective factors fostering the success of video-based eLearning in teaching evidence-based medicine, providing a ready-to-use checklist. The information collected via the interviews supported and enriched much of the advice found in the literature.

DISCUSSION

To the authors' knowledge, this type of comprehensive guide for video-based eLearning needs has not previously been published. The interviews considerably contributed to the results. Due to the grounded theory-based approach, in particular, consensus was achieved without the presence of a formal expert panel. Although the guide was created with a focus on teaching evidence-based medicine, due to the general study selection process and research approach, the recommendations are applicable to a wide range of subjects in medical education where the teaching aim is to impart conceptual knowledge.

Long-term effectiveness and cost-effectiveness of guided internet-based cognitive behavioral therapy for obsessive-compulsive disorder: 24-month follow-up

This study investigated the long-term effectiveness and cost-effectiveness of guided internet-based cognitive behavioral therapy (ICBT) for obsessive-compulsive disorder (OCD). Twenty-five patients with OCD who had undergone guided ICBT in a randomized controlled trial or a single-arm trial were followed up for 6, 12, and 24 months. Missing data were imputed using the mice package in R, and a one-way analysis of variance with repeated measures was performed. The total Yale-Brown Obsessive-Compulsive Scale (Y-BOCS) score significantly decreased from baseline to all endpoints (p < 0.001). OCD remission (the total Y-BOCS score < 14) rates significantly increased from post-treatment (48 %) at the 12 months (80 %) and 24 months (76 %) follow-up (χ2 (1) = 7.11-11.08, p < 0.01) post guided ICBT. Regression analysis predicted the maintenance of remission at 12 and 24 months following post-treatment remission (z = 2.20, p = 0.03). An incremental cost-effectiveness ratio was calculated to assess the cost-effectiveness of guided ICBT. The incremental cost-effectiveness ratio was JPY 999,495, below the JPY 5 million threshold for willingness to pay in Japan. Our findings demonstrate the long-term effectiveness and cost-effectiveness of guided ICBT for OCD. Guided ICBT can mitigate the severity of OCD even after treatment.

Numeracy Education for Health Care Providers: A Scoping Review

INTRODUCTION

Health numeracy (numerical literacy) refers to an individual's ability to use numerical information to make effective health decisions. Numeracy is fundamental in the role of a health care provider, forming the basis of evidence-based medicine and effective patient-provider communication. Despite a high level of education, many health care providers struggle with numeracy. Numeracy is often integrated into training curricula; however, teaching modality, competencies covered, learner satisfaction, and effectiveness of these educational interventions varies.

METHOD

A scoping review was conducted to explore and summarize what is known about numeracy skills education programs for health care providers. A comprehensive literature search was conducted from January 2010 to April 2021 in 10 databases. Controlled vocabulary terms and text words were used. The search was restricted to human studies, adults, and the English language. Articles were included if they were related to numeracy education for health care providers or trainees and provided details regarding methods, evaluation, and results.

RESULTS

The literature search retrieved 31,611 results and 71 met the inclusion criteria. Most interventions were conducted in a university setting, and targeted nursing students, medical students, resident physicians, and pharmacy students. Common numeracy concepts included statistics/biostatistics, medication calculations, evidence-based medicine, research methodology, and epidemiology. A variety of teaching modalities were used, which most often combined active approaches (eg, workshops, laboratories, small-group exercises, and discussion boards) with passive approaches (eg, traditional lectures and didactic teaching). Measured outcomes included knowledge and skills, self-efficacy, attitudes, and engagement.

DISCUSSION

Although efforts have been made to incorporate numeracy into training curricula, greater emphasis should be placed on developing strong numeracy skills in health care providers, particularly given the role numerical information plays in clinical decision making, evidence-based practices, and patient-provider communication.

Caution with competitive gamification in medical education: unexpected results of a randomised cross-over study

BACKGROUND

To intrinsically motivate students in the long term, longitudinal e-learning systems combined with repeated testing and competitive gamification seem promising. The effects of this approach have never been closely examined in the field of evidence-based medicine. The authors investigated if a simple, competitive learning application enhances students' risk competence and intrinsic motivation.

METHODS

Participants were 5.-9. semester medical students (n = 48), recruited in an elective evidence-based medicine subject and randomly distributed to two groups (group 1: n = 23; group 2: n = 25). Both accessed a competitive evidence-based medicine quiz game. Following a cross-over design, each group practiced with one of two thematically different questionnaires A or B, before the allocation switched after one month. To analyse whether there was a measurable learning effect in the practiced topics, a paired t-test was performed with quantitative data from 3 e-tests. Students further reported their experience in evaluation surveys.

RESULTS

Students' improvements in e-test scores after training with the corresponding topics in the learning application can be attributed to chance. Even though the majority enjoyed playing and felt motivated to study, they invested a minimum of time and rejected competition.

CONCLUSION

The authors found no evidence for benefits of the investigated learning programme on students' risk competence or on their internal motivation. The majority disapproved the competitive concept, indicating adverse side effects of the applied gamification element. To intrinsically motivate more students, prospective learning programmes should favour complex and collaborative programmes over simple and competitive ones.

Dependence of learning outcomes in flipped and lecture classrooms on review questions: A randomized controlled trial and observational study

PURPOSE

The effects of class structure (i.e., lecture vs. flipped) and repeated review on: 1) exam scores and 2) student perceptions of learning were compared in a prospective randomized controlled trial (Study 1) and a retrospective cohort study (Study 2).

METHODS

In Study 1, 42 second year students in a Doctor of Physical Therapy program were randomized to either a lecture or flipped section of a neurobiology class. Both sections incorporated repeated review. In Study 2, exam scores were retrospectively compared between two cohorts: a lecture cohort without repeated review (n = 42) and a flipped cohort with repeated review (n = 46). In both studies, outcomes of interest were exam scores and student surveys.

RESULTS

In Study 1, students in the lecture and flipped sections had similar exam averages (lecture = 76.7 ± 17%, flipped = 77.5 ± 17%, p = 0.73). Repeated review significantly improved exam scores by 12.0 percentage points (95% CI: 8.0 to 16.0 percentage points) in the lecture section and 10.8 percentage points (95% CI: 6.9 to 14.8 percentage points) in the flipped section. The flipped section reported higher levels of satisfaction and perceived learning. In Study 2, the flipped cohort had significantly higher exam scores than the lecture cohort (lecture = 70.2 ± 6.9%, flipped = 83.4 ± 7.7%, p < 0.0001). Student satisfaction and perceived learning were similar in both cohorts.

CONCLUSION

Exam scores improve with review questions and quizzes provided in a class, both in a lecture or flipped classroom.

The Importance of Sociocultural Factors in Moderating the Applicability of Test-Enhanced Learning to Continuing Professional Development

Many studies have shown that repeated testing promotes learning, an effect known as test-enhanced learning. Although test-enhanced learning has been well-established within health care trainees, there are mixed findings of whether testing facilitates learning within continuing professional development (CPD) contexts. This piece argues that for testing to support learning in CPD, there is need to understand the various social, cultural, and structural factors that influence clinicians' willingness to participate in regular testing activities. The solution to this problem is conceptual in nature, whereby CPD programs must consider how learning cultures shape practicing clinicians' perspectives toward testing.

Quantifying teaching quality in medical education: The impact of learning gain calculation

BACKGROUND

Student performance is a mirror of teaching quality. The pre-/post-test design allows a pragmatic approach to comparing the effects of interventions. However, the calculation of current knowledge gain scores introduces varying degrees of distortion. Here we present a new metric employing a linear weighting coefficient to reduce skewness on outcome interpretation.

METHODS

We compared and contrasted a number of common scores (raw and relative gain scores) with our new method on two datasets, one simulated and the other empirical from a previous intervention study (n = 180) employing a pre-/post-test design.

RESULTS

The outcomes of the common scores were clearly different, demonstrating a significant dependency on pre-test scores. Only the new metric revealed a linear relationship to the knowledge baseline, was less skewed on the upper or lower extremes, and proved well suited to allow the calculation of negative learning gains. Employing the empirical dataset, the new method also confirmed the interaction effect of teaching formats with specific subgroups of learner characteristics.

CONCLUSION

This work introduces a new weighted metric enabling meaningful comparisons between interventions based on a linear transformation. This method will form the basis to intertwine the calculation of test performance closely with the outcome of learning as an important factor reflecting teaching quality and efficacy. Its regular use can improve the transparency of teaching activities and outcomes, contribute to forming rounded judgements of students' acquisition of knowledge and skills and enable valuable feedforward to develop and enhance curricular concepts.

Evaluating Simulated Consultation Videos in Teaching Patient-Centered General Practice

This article was migrated. The article was marked as recommended. Introduction In the general practice course at Copenhagen University, students are taught patient-centered consultations. The aim of this study is to evaluate the feasibility of a new method for measuring the effect of this teaching, and of adding access to simulated consultation videos to usual teaching. Methods The university assigned 293 final-semester students to three groups: a 'Control Group' with usual curriculum, an 'Access Group' that watched simulated consultation video clips online and a 'Teaching Group' where the video clips were discussed in teaching sessions. The outcome was the change in students' ability to identify patient-centered elements in a test video consultation, measured with a questionnaire before and after the course. Results An overall teaching effect was observed, which was most apparent in communication items such as "making a contract about the topic for the consultation" and "summarizing". Changes in clinical items and general issues were small. Conclusion A tool for measuring the effect of teaching general practice consultation skills combining a test video and a questionnaire is presented. Topics needing to be highlighted in teaching could be identified using the tool.

A Web-Based Virtual Microscopy Platform for Improving Academic Performance in Histology and Pathology Laboratory Courses: A Pilot Study

Virtual microscopy (VM) has been utilized to improve students' learning experience in microscope laboratory sessions, but minimal attention has been given to determining how to use VM more effectively. The study examined the influence of VM on academic performance and teacher and student perceptions and compared laboratory test scores before and after VM incorporation. A total of 662 third-year students studying histology and 651 fourth-year students studying pathology were divided into two groups. The light microscopy (LM) group used a light microscope in 2014 and 2015, while the LM + VM group used the VM platform and a light microscope in 2016 and 2017. Four factors positively predict laboratory scores (R square, 0.323; P < 0.001): (i) the pathology course and test-enhanced learning, (ii) the VM platform and experience, (iii) medical students and lecture scores, and (iv) female students. The LM + VM group exhibited less score variability on laboratory examinations relative to their mean than the LM group. The LM + VM group was also associated with fewer failing grades (F grade; odds ratio, 0.336; P < 0.001) and higher scores (A grade; odds ratio, 2.084; P < 0.001) after controlling for sex, school, course, and lecture grades. The positive effect of the VM platform on laboratory test grades was associated with prior experience using the VM platform and was synergistic with more interim tests. Both teachers and students agreed that the VM platform enhanced laboratory learning. The incorporation of the VM platform in the context of test-enhanced learning may help more students to master microscopic laboratory content.

What and how are students taught about communicating risks to patients? Analysis of a medical curriculum

Background

Communication is a core competence in medical care. Failure of physicians to properly communicate inherent risks of medical interventions has been linked with inadequate training at school. This study analyses a medical curriculum for assessing the content and quality of teaching risk communication to students.

Methods

A checklist based on the national guidelines of core competencies on risk communication required of physicians was developed. Participant observers surveyed all teaching sessions at a medical school during a semester to record the frequency, characteristics and clinical context used by lectures during classes. Data were analyzed using statistical and descriptive methods to determine the prevalence and quality of teaching content.

Results

231 teaching sessions were surveyed. The inter-rater reliability was 81%. Lecturers mentioned topics of risk communication in 61.5% of teaching sessions (83.7% in surgery, 43.3% in internal medicine) but core biostatistics concepts were not discussed in more than 80% of these sessions. Important topics such as patient safety and preventable diseases were underrepresented. Risk communication was mainly taught in large-group, theoretical sessions and rarely with supplementary teaching material (7.4%). Students asked questions in 15.2% of courses, more often in surgery classes than in internal medicine.

Conclusion

Statistical and clinical topics relevant for teaching risk communication to medical students are not only underrepresented but also minimally explained by lecturers. Supplementary material on risk communication is rarely provided to students during classes. High-resource demanding, small-group teaching formats are not necessarily interactive as students ask few questions.

Conceptualising spaced learning in health professions education: A scoping review

OBJECTIVES

To investigate the definitions and applications of 'spaced learning' and to propose future directions for advancing its study and practice in health professions education.

METHOD

The authors searched five online databases for articles published on spaced learning in health professions education prior to February 2018. Two researchers independently screened articles for eligibility with set inclusion criteria. They extracted and analysed key data using both quantitative and qualitative methods.

RESULTS

Of the 2972 records retrieved, 120 articles were included in the review. More than 90% of these articles were published in the last 10 years. The definition of spaced learning varied widely and was often not theoretically grounded. Spaced learning was applied in distinct contexts, including online learning, simulation training and classroom settings. There was a large variety of spacing formats, ranging from dispersion of information or practice on a single day, to intervals lasting several months. Generally, spaced learning was implemented in practice or testing phases and rarely during teaching.

CONCLUSIONS

Spaced learning is infrequently and poorly defined in the health professions education literature. We propose a comprehensive definition of spaced learning and emphasise that detailed descriptions of spacing formats are needed in future research to facilitate the operationalisation of spaced learning research and practice in health professions education.

Health numeracy skills of medical students:cross-sectional and controlled before-and-after study

BACKGROUND

Although numeracy, defined as understanding and handling numbers, is an important skill for the medical profession, it is not clear whether it changes during graduate medical education and whether it can be improved by specific interventions. The objective of this study was to assess objective and subjective numeracy levels at different stages of medical education and explore whether a research methodology/statistics course improves numeracy levels in a longer period.

METHODS

We performed cross-sectional and controlled before-and-after studies. First-year sociology students and first- to sixth-year medical students from the in the cross sectional study and two groups of first-year medical students in a controlled before-and-after study. The intervention was a course on biostatistics and research methodology using blended approach. Numeracy was measured using Subjective Numeracy Scale (Cronbach α = 0.70) and Numeracy Understanding in Medicine instrument (Cronbach α = 0.75).

RESULTS

Whereas first-year medical students did not differ from first-year sociology students in objective numeracy, medicine students had higher results on subjective numeracy. Students from higher years of medical school had generally higher subjective and objective numeracy scores. In the controlled before-and-after study, the intervention group improved more in subjective numeracy (median difference on a 0-8 scale = 0.5, 95% CI 0.3 to 0.7 vs - 0.4, 95% CI - 0.4 to - 0.1, P < 0.001) but not in objective numeracy.

CONCLUSIONS

Although the numeracy levels at the beginning of the medical school are within the range of non-medical population, both objective and subjective numeracy improve during the higher years of medical school. Curriculum during medical school may help in numeracy increase, while research methodology training may help to increase subjective but not objective numeracy skills.