Spaced education improves the feedback that surgical residents give to medical students: a randomized trial

Creation and Implementation of an Online Tool for Feedback on Resident Teaching: A Pilot Study

OBJECTIVE

There are few assessments of the competence and growth of surgical residents as educators. We developed and piloted an observation-based feedback tool (FT) to provide residents direct feedback during a specific teaching session, as perceived by medical students (MS). We hypothesized that residents' performance would improve with frequent, low stakes, observation-based feedback.

SETTING

This prospective study took place at an academic general surgery program.

PARTICIPANTS

Focus groups of MS, surgical residents, and faculty informed FT development. MS completed the FT regarding resident teaching.

DESIGN

The FT utilized 5 slider-bar ratings (0 to 100) about the teaching encounter and a checklist of 16 desirable teaching behaviors. QR codes and weekly email links were distributed for 12 months (6 clerkship blocks) to promote use. Residents were sent their results after each block. A survey after each block assessed motivation for use and gathered feedback on the FT. Descriptive statistics were used for analysis (medians, IQRs). Primary measures of performance were median of the slider-bar scores and the number of teaching behaviors.

RESULTS

The FT was used 111 times; 37 of 46 residents were rated by up to 65 MS. The median rating on the slider-bars was 100 and the median number of desirable teaching behaviors was 12; there were no differences based on gender or PGY level. 10 residents had 5 or more FT observations during the year. Four residents had evaluations completed in 4 or more blocks and 19 residents had evaluations completed in at least 2 blocks. Over time, 13 residents had consistent slider-bar scores, 1 resident had higher scores, and 5 residents had lower scores (defined as a more than 5-point change from initial rating). Frequency of use of the FT decreased over time (38, 32, 9, 21, 7, 5 uses per block). The post-use survey was completed by 24 MS and 19 residents. Most common reasons for usage were interest in improving surgical learning environment, giving positive feedback (MS), and improving teaching skills (residents). Most common reasons for lack of usage from residents were "I did not think I taught enough to ask for feedback," "I forgot it existed," and "I did not know it existed."

CONCLUSIONS

The FT did not lead to any meaningful improvement in resident scores over the course of the year. This may be due to overall high scores, suggesting that the components of the FT may require reevaluation. Additionally, decreased utilization of the instrument over time made it challenging to assess change in performance of specific residents, likely due to lack of awareness of the FT despite frequent reminders. Successful implementation of observation-based teaching assessments may require better integration with residency or clerkship objectives.

Evidence-based teaching in contact lenses education: Teaching and learning strategies

INTRODUCTION

Contact lens (CL) practice is an ever-changing field with clinical knowledge, techniques and equipment continuously evolving. These new developments are backed with clinical trials and research to ensure that practitioners feel confident that there is an evidence base to support these advances. Evidence-based practice is now a crucial part of CL practice, and its importance also filters down to CL education. For example, lectures are one of the most popular tools for an educator but, is standing at the front of a lecture theatre full of students a more effective way of teaching than providing the same material for students to read by themselves? What evidence exists specific to CL education?

METHOD

An expert panel of educators completed a comprehensive literature review of current evidence of teaching methods in CL training, or if not available then what can be learnt from other health care professional training that could be potentially applicable to CL education.

RESULTS

Due to the amount of evidence available in the overall subject area relating to healthcare education, the initial plan of compiling evidence into one narrative review paper was discarded in favour of producing two linked papers. Here, the first paper details definitions of terminology, and also teaching methods. The second paper focuses on assessment and specific clinical training required to attain CL practice competency. In this first paper, no direct evidence of the spreading and benefit of new education strategies evidence such as flipped classrooms, spaced learning, test-enhanced learning, group work, CBL, PBL, TBL, and reflective practice in CL education was found. The only technique that was widely used in the CL field was case reports and the group discussion of them. Nevertheless, the authors found a consensus of opinion from other disciplines that are transferable to CL teaching and could help students meet the intended learning outcomes.

CONCLUSION

There is a small amount of evidence supporting CL education, but most of this seems to be related to the practical element of the training. However, there is a lot of evidence in the field of healthcare education from related disciplines which provides additional but important learning tools that may be effectively implemented in CL education.

Effect of Smartphone App-Based Education on Clinician Prescribing Habits in a Learning Health Care System: A Randomized Cluster Crossover Trial

IMPORTANCE

Effective methods for engaging clinicians in continuing education for learning-based practice improvement remain unknown.

OBJECTIVE

To determine whether a smartphone-based app using spaced education with retrieval practice is an effective method to increase evidence-based practice.

DESIGN, SETTING, AND PARTICIPANTS

A prospective, unblinded, single-center, crossover randomized clinical trial was conducted at a single academic medical center from January 6 to April 24, 2020. Vanderbilt University Medical Center clinicians prescribing intravenous fluids were invited to participate in this study.

INTERVENTIONS

All clinicians received two 4-week education modules: 1 on prescribing intravenous fluids and 1 on prescribing opioid and nonopioid medications (counterbalancing measure), over a 12-week period. The order of delivery was randomized 1:1 such that 1 group received the fluid management module first, followed by the pain management module after a 4-week break, and the other group received the pain management module first, followed by the fluid management module after a 4-week break.

MAIN OUTCOMES AND MEASURES

The primary outcome was evidence-based clinician prescribing behavior concerning intravenous fluids in the inpatient setting and pain medication prescribing on discharge from the hospital.

RESULTS

A total of 354 participants were enrolled and randomized, with 177 in group 1 (fluid then pain management education) and 177 in group 2 (pain management then fluid education). During the overall study period, 16 868 questions were sent to 349 learners, with 11 783 (70.0%) being opened: 10 885 (92.4%) of those opened were answered and 7175 (65.9%) of those answered were answered correctly. The differences between groups changed significantly over time, indicated by the significant interaction between educational intervention and time (P = .002). Briefly, at baseline evidence-concordant IV fluid ordered 7.2% less frequently in group 1 than group 2 (95% CI, -19.2% to 4.9%). This was reversed after training at 4% higher (95% CI, -8.2% to 16.0%) in group 1 than group 2, a more than doubling in the odds of evidence-concordant ordering (OR, 2.56, 95% CI, 0.80-8.21). Postintervention, all gains had been reversed with less frequent ordering in group 1 than group 2 (-9.5%, 95% CI, -21.6% to 2.7%). There was no measurable change in opioid prescribing behaviors at any time point.

CONCLUSIONS AND RELEVANCE

In this randomized clinical trial, use of smartphone app learning modules resulted in statistically significant short-term improvement in some prescribing behaviors. However, this effect was not sustained over the long-term. Additional research is needed to understand how to sustain improvements in care delivery as a result of continuous professional development at the institutional level.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT03771482.

Effect of "Residents as Teachers" Workshop on Learner Perception of Trainee Teaching Skill

BACKGROUND

Surgical residencies use variable structures for formal training in education. We hypothesized that a one-day workshop intervention would improve resident teaching ability measured by self-assessment and learner evaluation.

MATERIALS AND METHODS

Faculty educators delivered a Residents as Teachers (RAT) workshop to general surgery residents on setting expectations, positive learning environment, difficult feedback and the 1-min preceptor model. For three months before and after the workshop, junior residents and medical students evaluated their supervising residents' teaching skill monthly using a Likert scale questionnaire. Pre- and postworkshop surveys were administered to resident participants to assess their knowledge of the material and teaching confidence. Results were analyzed using Wilcoxon rank sum tests. This study was conducted at a tertiary academic center with a large surgical residency program.

RESULTS

Thirty-nine PGY 1-5 residents participated in the Residents as Teachers workshop and were included in the study. Pre- and post- workshop survey results demonstrated significant improvements in participants' knowledge and teaching confidence. On monthly assessments of seniors by junior residents, significant improvements were noted in three domains. Medical student ratings did not reflect significant improvements in resident teaching skill.

CONCLUSIONS

This is the first study using learner evaluation of a comprehensive surgical RAT program. Despite a significant increase in surgery residents' self-assessment following participation in an education workshop, no improvement was seen in resident teaching skill as perceived by medical students.

Fall prevention education for people with multiple sclerosis: a randomized clinical trial

BACKGROUND

Online spaced education (OSE) is a method recognized for promoting long-term knowledge retention, changing behaviors and improving outcomes for students and healthcare professionals. However, there is little evidence about its impacts on patient education.

OBJECTIVES

The aim of this research was to compare knowledge retention using educational brochure and OSE on individuals with multiple sclerosis (MS) and to verify the impact of educational methods on fall outcome.

METHODS

Individuals with MS (n = 230) were randomly assigned to two types of patient education-educational brochure (control) and OSE (intervention). During 12 weeks, the intervention group received multiple-choice tests on fall prevention. Knowledge retention, behavior change and fall incidence were assessed before intervention and after 3 and 6 months. The participants' satisfaction with the education method was also evaluated.

RESULTS

Knowledge retention was similar between groups, and behavior change was observed in both groups. There was a significant reduction in fall rate in the intervention group, from 0.60 to 0.27 at 6 months (P < 0.001). Participants' satisfaction achieved an average of 8.75, with no differences between groups.

CONCLUSION

Individuals demonstrated significant improvement in fall rate outcome in both groups with no significant difference. In regard to test scores and satisfaction, results were similar between groups.

"Residents as Teachers" Workshop Improves Knowledge, Confidence, and Feedback Skills for General Surgery Residents

OBJECTIVE

Surgical residents receive limited formal training in education, yet they are expected to teach medical students and one another. A "Residents as Teachers" curriculum was developed and implemented to improve residents' knowledge of educational strategies, confidence in teaching abilities, and quality of feedback given to learners.

DESIGN

A 6-hour workshop was delivered at an academic general surgery residency program. It included 3 interactive sessions: "Teaching on the wards", "How to give and receive feedback", and "Teaching in the operating room (OR)". Pre- and postsession surveys were administered to evaluate participants' knowledge and confidence regarding teaching skills. Standard statistical analyses were used to compare pre- and postcurriculum scores.

SETTING

General surgery residents at Case Western Reserve University/University Hospitals Cleveland Medical Center Program attended a 6-hour educational seminar entitled "Residents as Teachers". Three attending surgeons with expertise in surgical education administered the curriculum.

PARTICIPANTS

Twenty-four residents completed the course and surveys (5 PGY-5s, 6 PGY-4s, 4 PGY-3s, 5 PGY-2s, and 4 PGY-1s).

RESULTS

On a 5-point Likert scale, residents' self-rating of teaching skills (pre = 1.8, post = 3.04, p < 0.001) and understanding of adult learning (pre = 1.88, post = 4.42, p < 0.001) improved significantly. The greatest gain was in describing effective strategies for teaching in the OR (pre = 1.75, post = 4.38, p < 0.001). Residents reported improved understanding of effective feedback characteristics (pre = 2.5, post = 4.33, p < 0.001), and felt more prepared to provide feedback to learners (pre = 2.73, post = 3.17, p = 0.01). Scores on 5 of 10 knowledge-based questions significantly improved following the intervention. All participants agreed or strongly agreed that the workshop improved the program's surgical education curriculum (mean score 4.42 of 5).

CONCLUSIONS

A "Residents as Teachers" workshop improved resident confidence in teaching and feedback skills, particularly for intraoperative instruction, and improved knowledge in the 3 specific educational domains included in the session.

Conceptualising spaced learning in health professions education: A scoping review

OBJECTIVES

To investigate the definitions and applications of 'spaced learning' and to propose future directions for advancing its study and practice in health professions education.

METHOD

The authors searched five online databases for articles published on spaced learning in health professions education prior to February 2018. Two researchers independently screened articles for eligibility with set inclusion criteria. They extracted and analysed key data using both quantitative and qualitative methods.

RESULTS

Of the 2972 records retrieved, 120 articles were included in the review. More than 90% of these articles were published in the last 10 years. The definition of spaced learning varied widely and was often not theoretically grounded. Spaced learning was applied in distinct contexts, including online learning, simulation training and classroom settings. There was a large variety of spacing formats, ranging from dispersion of information or practice on a single day, to intervals lasting several months. Generally, spaced learning was implemented in practice or testing phases and rarely during teaching.

CONCLUSIONS

Spaced learning is infrequently and poorly defined in the health professions education literature. We propose a comprehensive definition of spaced learning and emphasise that detailed descriptions of spacing formats are needed in future research to facilitate the operationalisation of spaced learning research and practice in health professions education.

Urology in Undergraduate Medical Education

PURPOSE OF REVIEW

Urology is an essential topic in undergraduate medical education (UME). The objective of this article is to review the current state of exposure to urology in medical school, to discuss why it is critical to maintain a urology curriculum, and to review methods in establishing an effective curriculum for all students with limited resources.

RECENT FINDINGS

UME curriculum in urology should be geared toward the widest group of students, namely those entering primary care or internal medicine, where patients with urologic complaints are most likely to first present. Hands-on teaching should focus on skills such as the genitourinary exam and Foley catheter placement, while ancillary modules should be utilized for complex concepts. Medical schools do not sufficiently incorporate didactics in urology as part of their core curriculum. As such, educators in urology must develop curricula that provide fundamental knowledge to all students, especially those pursuing non-urologic specialties who will undoubtedly treat patients with urologic complaints.

Effectiveness of the spaced education pedagogy for clinicians' continuing professional development: a systematic review

CONTEXT

Ensuring clinical practice reflects current evidence is challenging given the rapid proliferation of new knowledge. Changing entrenched clinical behaviours and facilitating the adoption of best practice evidence requires a range of strategies, including affordable, scalable and effective continuing professional development (CPD). Yet, identifying the CPD delivery method most likely to effectively change and improve patient outcomes is difficult given the variability in the evidence for different learning approaches. Although there is moderate level evidence for outreach education, audit and feedback, and face-to-face or online learning, little is known about the capacity of spaced education to change ineffective clinical practice(s). Spaced education harnesses the power of spacing, repetition and testing learning content to increase topic-specific knowledge. Although spaced education is widely used in undergraduate and postgraduate medical programmes, its effectiveness as a CPD delivery method that improves patient outcomes is less certain.

AIM

To determine the effectiveness of the spaced education CPD programmes to change targeted clinical knowledge and practice(s) to improve patient outcomes.

METHOD

A systematic review, appraising the spaced education CPD evidence generated from searching six specialist medical and psychosocial databases. Studies published in English peer-reviewed journals from 1 January, 2000 to 31 August, 2018 were eligible for inclusion. A modified Kirkpatrick four levels of evaluation framework assisted with appraising the effect of spaced education CPD interventions on clinicians and patients.

RESULTS

Of the 2396 studies identified, 17 met the inclusion criteria, involving 2701 practising clinicians from multiple disciplines and specialties. Five randomised controlled trials generated level II evidence, with the remaining 12 studies generating lower levels of evidence. The majority of studies (n = 14) involved the delivery of online spaced education. All studies were evaluated using the modified Kirkpatrick four levels of evaluation framework with: 10 studies demonstrating significant increases in participants' knowledge; seven studies reporting significant changes in clinician behaviour; four studies showing significant increases in clinician confidence; and three studies identifying significant and sustained increases in participants' clinical skills. Only two studies reported positive improvements in patient outcomes.

CONCLUSION

Spaced education via an online platform offers a scalable CPD format that can increase clinical knowledge and change practice. However, further adequately powered randomised controlled trials are required to confirm that spaced education CPD can impact positively on patients' reported outcomes.

Does Performance on Evidence-Based Medicine and Urgent Clinical Scenarios Assessments Deteriorate During the Fourth Year of Medical School? Findings From One Institution

PURPOSE

The fourth year of medical school (M4) should prepare students for residency yet remains generally unstructured, with ill-defined goals. The primary aim of this study was to determine whether there were performance changes in evidence-based medicine (EBM) and urgent clinical scenarios (UCS) assessments before and after M4 year.

METHOD

University of Michigan Medical School graduates who matched into internship at Michigan Medicine completed identical assessments on EBM and UCS at the beginning of M4 year and 13 months later during postgraduate year 1 (PGY1) orientation. Individual scores on these assessments were compared using paired t test analysis. The associations of academic performance, residency specialty classification, and initial performance on knowledge changes were analyzed.

RESULTS

During academic years 2014 and 2015, 76 students matched into a Michigan Medicine internship; 52 completed identical EBM stations and 53 completed UCS stations. Learners' performance on the EBM assessment decreased from M4 to PGY1 (mean 93% [SD = 7%] vs. mean 80% [SD = 13%], P < .01), while performance on UCS remained stable (mean 80% [SD = 9%] vs. mean 82% [SD = 8%], P = .22). High M4 performers experienced a greater rate of decline in knowledge level compared with low M4 performers for EBM (-20% vs. -4%, P = .01). Residency specialty and academic performance did not affect performance.

CONCLUSIONS

This study demonstrated degradation of performance in EBM during the fourth year and adds to the growing literature that highlights the need for curricular reform during this year.

Online Digital Education for Postregistration Training of Medical Doctors: Systematic Review by the Digital Health Education Collaboration

Background

Globally, online and local area network–based (LAN) digital education (ODE) has grown in popularity. Blended learning is used by ODE along with traditional learning. Studies have shown the increasing potential of these technologies in training medical doctors; however, the evidence for its effectiveness and cost-effectiveness is unclear.

Objective

This systematic review evaluated the effectiveness of online and LAN-based ODE in improving practicing medical doctors’ knowledge, skills, attitude, satisfaction (primary outcomes), practice or behavior change, patient outcomes, and cost-effectiveness (secondary outcomes).

Methods

We searched seven electronic databased for randomized controlled trials, cluster-randomized trials, and quasi-randomized trials from January 1990 to March 2017. Two review authors independently extracted data and assessed the risk of bias. We have presented the findings narratively. We mainly compared ODE with self-directed/face-to-face learning and blended learning with self-directed/face-to-face learning.

Results

A total of 93 studies (N=16,895) were included, of which 76 compared ODE (including blended) and self-directed/face-to-face learning. Overall, the effect of ODE (including blended) on postintervention knowledge, skills, attitude, satisfaction, practice or behavior change, and patient outcomes was inconsistent and ranged mostly from no difference between the groups to higher postintervention score in the intervention group (small to large effect size, very low to low quality evidence). Twenty-one studies reported higher knowledge scores (small to large effect size and very low quality) for the intervention, while 20 studies reported no difference in knowledge between the groups. Seven studies reported higher skill score in the intervention (large effect size and low quality), while 13 studies reported no difference in the skill scores between the groups. One study reported a higher attitude score for the intervention (very low quality), while four studies reported no difference in the attitude score between the groups. Four studies reported higher postintervention physician satisfaction with the intervention (large effect size and low quality), while six studies reported no difference in satisfaction between the groups. Eight studies reported higher postintervention practice or behavior change for the ODE group (small to moderate effect size and low quality), while five studies reported no difference in practice or behavior change between the groups. One study reported higher improvement in patient outcome, while three others reported no difference in patient outcome between the groups. None of the included studies reported any unintended/adverse effects or cost-effectiveness of the interventions.

Conclusions

Empiric evidence showed that ODE and blended learning may be equivalent to self-directed/face-to-face learning for training practicing physicians. Few other studies demonstrated that ODE and blended learning may significantly improve learning outcomes compared to self-directed/face-to-face learning. The quality of the evidence in these studies was found to be very low for knowledge. Further high-quality randomized controlled trials are required to confirm these findings.

Utility of intermittent online quizzes as an early warning for residents at risk of failing the pediatric board certification examination

BACKGROUND

Traditionally, quizzes have been applied as a tool for summative assessment, though literature suggests their use as a formative assessment can improve motivation and content retention. With this premise, we implemented a series of intermittent, online quizzes known as the Board Examination Simulation Exercise (BESE). We sought to demonstrate an association between BESE participation and scores and performance on the American Board of Pediatrics (ABP) Certifying Examination (CE).

METHODS

Residents were assigned online quizzes on a single topic at 2 week intervals that consisted of 20 multiple choice questions written by the study authors. This analysis includes graduates of 3 Pediatric and Internal Medicine-Pediatrics residency programs.

RESULTS

Data were available for 329 residents. The overall BESE score weakly correlated with ABP CE score (n = 287; r = 0.39, p < 0.0001). ABP CE pass rates increased from 2009 to 2016 at all programs combined (p = 0.0001). A composite BESE score ≤ 11 had sensitivity of 54% and specificity of 80% for predicting ABP CE failure on the first attempt. There was no difference in ABP CE failure rates or scores by number of completed quizzes.

CONCLUSION

Intermittent online quizzes implemented at three pediatric residency programs were associated with overall increasing ABP CE pass rates. BESE increased program emphasis on board preparation. Residents with lower BESE scores more often failed ABP CE. Though additional data are needed, BESE is a promising tool for pediatric resident learning and board preparation. It may also aid in earlier identification of residents at higher risk of failing the ABP CE and facilitate targeted interventions.

Reducing Redundant Anaerobic Therapy Through Spaced Education and Antimicrobial Stewardship Interventions

BACKGROUND

Decreasing the use of redundant anaerobic therapy is a key target for antimicrobial stewardship. Education techniques that optimize knowledge retention could be an important component of reducing these regimens.

METHODS

We implemented a quality improvement project that incorporated spaced education to reduce the use of redundant anaerobic therapy. The initial interventions (November through December 2015) included education in a hospital-wide newsletter and review of redundant anaerobic regimens by the antimicrobial stewardship program. A spaced education module was then developed with the gastroenterology (GI) service, which had a relatively high rate of redundant anaerobic therapy use. Ten questions with teaching points were delivered to GI physicians at spaced intervals over 2 to 4 weeks (February through March 2016). Knowledge scores were compared at initial and final question presentation using generalized estimating equations. Interrupted time-series analysis was used to compare the rates of redundant-metronidazole-days per 1000 patient-days among patients in the patients admitted to the GI service and those in the non-GI group before and after the intervention.

RESULTS

Of 66 GI physicians, 56 (85%) participated in the spaced education activity. After the intervention, their knowledge scores on all the questions improved, and their mean knowledge score increased from 57% to 86% (P < .001). Nearly all (91%) of the participants were very or generally satisfied with the activity. In the GI group, the rate of redundant-metronidazole-days decreased from 26.2 to 13.0 per 1000 patient-days (relative risk [RR], 0.45 [95% confidence interval (CI), 0.27-0.73]; P = .001). This rate in the non-GI group also decreased from 5.47 to 2.18 per 1000 patient-days (RR, 0.47 [95% CI, 0.36-0.60]; P < .001) after our interventions.

CONCLUSIONS

Spaced education is an effective approach for teaching antimicrobial stewardship topics. Focused provider education was associated with a sustained reduction in the use of redundant anaerobic therapy.

Resident-as-teacher programs in general surgery residency - A review of published curricula

INTRODUCTION

The Liaison Committee on Medical Education (LCME) requires that residents are trained to fulfill their educational duties toward medical students. This study reviews the literature on resident-as-teacher programs (RATPs) aimed at surgical residents.

METHODS

Literature search with MeSH terms internship, residency, general surgery, teaching, education, and curriculum was performed using PubMed, Embase, Web of Science, and ERIC. Curriculum components and how curricula's success was measured were extracted for each study. Quality was scored using the Medical Education Research Study Quality Instrument (MERSQI).

RESULTS

For the seven relevant publications the average MERSQI score was 9.9 (range 6.5-13.5). The RATPs were either lecture based (4/7) or content was distributed electronically (3/7). Change in attitude toward teaching was the most frequently assessed outcome. Highly rated curricular components were individualized feedback and iterative reminders to make teaching part of practice.

CONCLUSIONS

Few published RATPs in general surgery training exist. The literature suggests that pairing lectures with observation and feedback is successful. Distributing the content electronically is a feasible alternative to class-room based teaching in a busy surgical residency.

An Integrated Interactive-Spaced Education Radiology Curriculum for Preclinical Students

Introduction:

The objective of this study is to determine whether a radiology module, together with online spaced education, helps students of an integrated problem-based learning (PBL) curriculum increase their radiology knowledge and long-term retention.

Materials and Methods:

Second-year students at the American University of Antigua College of Medicine participated in small groups of ten students each into two 2 h of radiology laboratories. The study comprised two cohorts: winter and fall 2013 students (control group) and 2014 students (experimental group). Both groups used face-to-face PBL. The students of the experimental group received additional online-spaced education. The skills were assessed for both groups before the beginning of laboratories and 4 weeks and 7 months after laboratories.

Results:

There was no significant difference on pretest between the control and experimental groups. On completion of the radiology laboratories, comparison of test results before and after training showed net improvement for both groups. The corresponding difference for the experimental group was higher compared to the one for the control group (7.83 vs. 6.21, P < 0.001). The difference between the scores on delayed test and pretest showed that the students of both groups demonstrated average knowledge improvement even though their level of performance was slightly below the posttest. The corresponding difference for the experimental group did not differ much from the posttest (P > 0.05), and no significant difference of scores was observed 7 months later for either group. Further, a higher percentage of the students in the experimental group strongly agreed that their learning objectives were met (92% vs. 71%, P > 0.001), and this trend persisted throughout the study.

Conclusion:

Online spaced education combined to a face-to-face PBL enhances not only the student's knowledge of basic radiology along with his/her self-assessment skills but also the long-term retention of radiology material and satisfaction with the integrated interactive system-based module. Future research is needed to see if medical students in need of additional education support may benefit from spaced education in the field of remediation.

Feedback in Medical Education: A Critical Appraisal

OBJECTIVE

The objective was to review and critically appraise the medical education literature pertaining to feedback and highlight influential papers that inform our current understanding of the role of feedback in medical education.

METHODS

A search of the English language literature in querying Education Resources Information Center (ERIC), PsychINFO, PubMed, and Scopus identified 327 feedback-related papers using either quantitative (hypothesis-testing or observational investigations of educational interventions), qualitative methods (exploring important phenomena in emergency medicine [EM] education), or review methods.Two reviewers independently screened each category of publications using previously established exclusion criteria. Six reviewers then independently scored the remaining 54 publications using a qualitative, quantitative, or review paper scoring system. Each scoring system consisted of nine criteria and used parallel scoring metrics that have been previously used in critical appraisals of education research.

RESULTS

Fifty-four feedback papers (25 quantitative studies, 24 qualitative studies, five review papers) met the a priori criteria for inclusion and were reviewed. Eight quantitative studies, nine qualitative studies, and three review papers were ranked highly by the reviewers and are summarized in this article.

CONCLUSIONS

This inaugural Council of Emergency Medicine Residency Directors Academy critical appraisal highlights 20 feedback in medical education papers that describe the current state of the feedback literature. A summary of current factors that influence feedback effectiveness is discussed, along with practical implications for EM educators and the next steps for research.

Residents as teachers: Near peer learning in clinical work settings: AMEE Guide No. 106

This AMEE Guide provides a framework to guide medical educators engaged in the design and implementation of "Resident as Teacher" programs. The suggested approaches are based on established models of program development: the Program Logic model to guide program design, the Dundee three-circle model to inform a systematic approach to planning educational content and the Kirkpatrick pyramid, which forms the backbone of program evaluation. The Guide provides an overview of Resident as Teacher curricula, their benefits and impact, from existing literature supplemented by insights from the authors' own experiences, all of whom are engaged in teaching initiatives at their own institutions. A conceptual description of the Program Logic model is provided, a model that highlights an outcomes-based curricular design. Examples of activities under each step of this model are described, which would allow educational leaders to structure their own program based on the scope, context, institutional needs and resources available. Emphasis is placed on a modular curricular format to not only enhance the teaching skills of residents, but also enable development of future career educators, scholars and leaders. Application of the Dundee three-circle model is illustrated to allow for a flexible curricular design that can cater to varying levels of educational needs and interests. In addition, practical advice is provided on robust assessment of outcomes, both assessment of participants and program evaluation. Finally, the authors highlight the need for congruence between the formal and hidden curriculum through explicit recognition of the value of teaching by institutions, support for development of teaching programs, encouragement of evidence-based approach to education and rewards for all levels of teachers.

Integration of pediatric mental health care: an evidence-based workshop for primary care providers

INTRODUCTION

Pediatric primary care providers (PCPs) are being asked to care for children with mental health (MH) disorders but cite inadequate training as a barrier. An intensive workshop may improve the PCPs' level of knowledge and lead to an increase in quality care for children with MH disorders. We compared pediatric PCPs' knowledge, comfort, and practice in the evaluation and management of pediatric patients with attention deficit-hyperactivity disorder, depression, anxiety, and autism spectrum disorders before and after a 2-day educational workshop.

METHOD

Study participants (n = 30) were recruited from rural areas of Pennsylvania. A pre- and posttest design was used. A 15-question multiple choice knowledge test and a 19-question survey of comfort and practice were administered before and after the workshop.

RESULTS

The mean knowledge test number correct increased from 9.19 before the workshop to 12.23 after the workshop (p < .0001). Survey scores increased from 34.6 before the workshop to 44.14 after the workshop (p < .0001).

DISCUSSION

Intensive workshops may be an effective method of training PCPs on provision of MH care in pediatric primary care practice.

Impact of online education on intern behaviour around joint commission national patient safety goals: a randomised trial

PURPOSE

To compare the effectiveness of two types of online learning methodologies for improving the patient-safety behaviours mandated in the Joint Commission National Patient Safety Goals (NPSG).

METHODS

This randomised controlled trial was conducted in 2010 at Massachusetts General Hospital and Brigham and Women's Hospital (BWH) in Boston USA. Incoming interns were randomised to either receive an online Spaced Education (SE) programme consisting of cases and questions that reinforce over time, or a programme consisting of an online slide show followed by a quiz (SQ). The outcome measures included NPSG-knowledge improvement, NPSG-compliant behaviours in a simulation scenario, self-reported confidence in safety and quality, programme acceptability and programme relevance.

RESULTS

Both online learning programmes improved knowledge retention. On four out of seven survey items measuring satisfaction and self-reported confidence, the proportion of SE interns responding positively was significantly higher (p<0.05) than the fraction of SQ interns. SE interns demonstrated a mean 4.79 (36.6%) NPSG-compliant behaviours (out of 13 total), while SQ interns completed a mean 4.17 (32.0%) (p=0.09). Among those in surgical fields, SE interns demonstrated a mean 5.67 (43.6%) NPSG-compliant behaviours, while SQ interns completed a mean 2.33 (17.9%) (p=0.015). Focus group data indicates that SE was more contextually relevant than SQ, and significantly more engaging.

CONCLUSION

While both online methodologies improved knowledge surrounding the NPSG, SE was more contextually relevant to trainees and was engaging. SE impacted more significantly on both self-reported confidence and the behaviour of surgical residents in a simulated scenario.

Spaced education faculty development may not improve faculty teaching performance ratings in a surgery department

OBJECTIVES

To determine the effectiveness of spaced education as a faculty development tool designed to improve teaching skills in a surgery department.

DESIGN

Faculty members were randomized to receive either weekly spaced education e-mails with content designed to improve teaching skills (group A) or no e-mails (group B). Using qualitative and quantitative surveys, we assessed both medical students' perception of faculty members' teaching effectiveness and faculty members' perception of the usefulness of the spaced education e-mails.

SETTING

Academic medical center.

PARTICIPANTS

Twenty-nine surgery faculty members with teaching responsibility for medical students in their Core Surgery Clerkship.

RESULTS

All 41 medical students who rotated through the Core Surgery Clerkship rated the quality of teaching for each faculty members; 172 online rating surveys were completed. Overall, faculty members received high ratings on the teaching skills included on the surveys. Additionally, no significant differences were found between the perceived skill level of the faculty members who received the weekly e-mails and those who did not. Specifically, 53.8% and 54% (p = 0.47) of the faculty were felt to deliver feedback more than three times per week; 87.1% and 89.9% (p = 0.15) of faculty were felt to deliver useful feedback; 89.2% and 90.8% (p = 0.71) of faculty were perceived to encourage student autonomy; and 78.1% and 81.9% (p = 0.89) of faculty were felt to set clear learning expectations for students. Postprogram comments from faculty revealed they did not find the e-mails useful as a faculty development tool.

CONCLUSIONS

Students perceived high levels of teaching skills among the clinical faculty. Faculty members who received e-mail-based spaced education-based faculty development were not rated to be more effective teachers on the student surveys. Electronically based faculty development does not satisfy faculty expectations.

Surgery course evaluation. Expectations of medical students in surgery rotation? From bench to bedside

The curriculum of the Medical Faculty is a result of a compromise between the need to upgrade and extend the material and the immutability of study duration. In result of reduction of time for acquisition of basic practical skills. THE AIM OF THE STUDY was to evaluate the current curriculum by students and answer the question: What are the students' expectations of teaching surgery? and to compare the opinion in two academic centers in Poland.

MATERIAL AND METHODS

The survey embraced 85 students of the Medical Faculty of IV (25.9%), V (22.35%) and VI (51.75%) year of the Medical University of Gdańsk and VI year students of the Pomeranian University of Szczecin (PUM--34%). Students completed a 19-item questionnaire, send by e-mail. Questions were closed (yes / no or grades 1-5) with the option of opinion adding to each item. The Statistica (version 9) package for calculations was used. Differences with p<0.05 was considered statistically significant. Qualitative data (opinions) were prepared in the form of summary tables, generalized or quoted.

RESULTS

Satisfaction with the education of students amounted to 2,1-2,4 (on a scale 1-5). There is a weak association between gender and choice of surgical specialties. Declaring an interest in surgery does not affect the assessment of classes. Most students believe that the amount of theoretical classes is sufficient, there is lack of practical classes. Among procedures they want to learn, most often were mentioned: bladder catheterization, suturing, wound treatment and putting stomach tube. Additionally, they pay attention to the lack of affordable learning materials.

CONCLUSIONS

Students expect a full "non-corridor" utilization of classes, learn and practice the basic and most frequent activities at the patient. They are dissatisfied with the current training methods, and would be taught in a diverse and active way.

Engaging medical students in the feedback process

BACKGROUND

There are potential advantages to engaging medical students in the feedback process, but efforts to do so have yielded mixed results. The purpose of this study was to evaluate a student-focused feedback instructional session in an experimental setting.

METHODS

Medical students were assigned randomly to either the intervention or control groups and then assigned randomly to receive either feedback or compliments. Tests of knowledge, skills, and attitudes were given before and after the intervention.

RESULTS

There was a significant gain of knowledge and skill in the group that received instruction. Satisfaction was higher after compliments in the control group but higher after feedback in the instructional group. There was no change in the subject's willingness to seek feedback.

CONCLUSIONS

A student-focused component should be carefully included as part of an overall effort to improve feedback in surgical education. The role of medical student attitudes about feedback requires further investigation.

The educational impact of weekly e-mailed fast facts and concepts

PURPOSE

Educational interventions such as electives, didactics, and Web-based teaching have been shown to improve residents' knowledge, attitudes, and skills. However, integrating curricular innovations into residency training is difficult due to limited time, faculty, and cost. In this study, the authors assessed the educational impact of weekly Fast Facts and Concept (FFAC) e-mails on residents' knowledge of palliative care topics, self-reported preparedness in palliative care skills, and satisfaction with palliative care education.

METHOD

Internal medicine interns at the University of Pittsburgh and Medical College of Wisconsin were randomized to control and intervention groups in July 2009. Pretests and posttests assessed medical knowledge through 24 multiple choice questions, preparedness on 14 skills via a 4-point Likert scale and satisfaction based on ranking of education quality. The intervention group received 32 weekly e-mails.

RESULTS

The study group included 82 interns with a pretest response rate of 100% and posttest response rate of 70%. The intervention group showed greater improvement in knowledge than the control (18% increase compared to 8% in the control group, p = 0.005). Preparedness in symptom management skills (converting between opioids, differentiating types of pain, treating nausea) improved in the intervention group more than the control group (p = 0.04, 0.01, and 0.02, respectively). There were no differences in preparedness in communication skills or satisfaction between the control and intervention groups.

CONCLUSIONS

E-mailed FFAC are an educational intervention that increases intern medical knowledge and self-reported preparedness in symptom management skills but not preparedness in communication skills or satisfaction with palliative care education.

Is physician engagement with Web-based CME associated with patients' baseline hemoglobin A1c levels? The Rural Diabetes Online Care study

PURPOSE

To investigate the association between physician participants' levels of engagement in a Web-based educational intervention and their patients' baseline diabetes measures.

METHOD

The authors conducted a randomized trial of online CME activities designed to improve diabetes care provided by family, general, and internal medicine physicians in rural areas of 11 southeastern states between September 2006 and July 2008. Using incidence rate ratios derived from negative binomial models, the relationship between physicians' engagement with the study Web site and baseline proportion of their patients having controlled diabetes (hemoglobin A1c < or = 7%) was explored.

RESULTS

One hundred thirty-three participants (intervention = 64; control = 69) provided information for 1,637 patients with diabetes. In the intervention group, physicians in practices in the worst quartiles of A1c control were least engaged with the study Web site in nearly all dimensions. Total number of pages viewed decreased as quartile of A1c control worsened (137, 73, 68, 57; P = .007); similarly, for a given 10% increase in proportion of patients with controlled A1c, participants viewed 1.13 times more pages (95% CI: 1.02-1.26, P = .02). In the control group, engagement was neither correlated with A1c control nor different across quartiles of A1c control.

CONCLUSIONS

Engagement in Web-based interventions is measurable and has important implications for research and education. Because physicians of patients with the greatest need for improvement in A1c control may not use online educational resources as intensely as others, other strategies may be necessary to engage these physicians in professional development activities.

Interactive spaced education to assess and improve knowledge of clinical practice guidelines: a randomized controlled trial

OBJECTIVE

To determine whether Interactive Spaced Education (ISE) is an effective and acceptable form of graduate and continuing medical education (GME/CME), using clinical practice guideline (CPG) education as an experimental system.

SUMMARY BACKGROUND DATA

ISE is a novel form of online education, which combines the pedagogical merits of the spacing and testing effects. Its efficacy for GME and CME is not known.

METHODS

One-hundred sixty urologists and 320 urology residents were randomized to 1 of 2 cohorts. We developed and validated 48 ISE items (questions and answers) on 5 urology CPGs (hematuria and priapism [HP]; staghorn calculi, infertility, and antibiotic use [SIA]). Physicians were sent 3 emails a week, each containing 2 questions. Content was repeated 3 times over 20 weeks. Cohort A physicians received the 3-cycle ISE course on HP, with 24 control items on SIA in cycle 3. Cohort B physicians received the 3-cycle ISE course on SIA, with 24 control items on HP in cycle 3.

RESULTS

The ISE program was completed by 71% urologists and 83% residents. Cohort A scores on HP increased from mean 44.9% in cycle 1% to 75.7% in cycle 3, a 57% relative increase compared with controls (P < 0.001; Cohen effect size, 2.2). Similarly, cohort B scores on SIA increased from 45.2% in cycle 1% to 69.5% in cycle 3, a 56% relative increase compared with controls (P < 0.001; effect size, 2.2). Eighty-four percent of all participants requested to enroll in further ISE programs.

CONCLUSIONS

ISE is an effective and well-accepted form of GME and CME and is a promising new methodology to improve CPG knowledge.