Teaching residents evidence-based medicine skills: a controlled trial of effectiveness and assessment of durability

Physician interpretation of information about prescription drugs in scientific publications vs. promotional pieces

BACKGROUND

Physicians gain knowledge about medical product uses from a variety of information vehicles including FDA-approved labeling, peer-reviewed journal articles, compendia, continuing medical education (CME), and physician-directed promotion. The source of this information, the quality of the information, and environmental pressures such as lack of time may impact perceptions.

OBJECTIVE

The authors tested the effect of three types of information sources (journal abstract, sales aid without graphics, sales aid with graphics), the presence or absence of time pressure to read the information, and two levels of methodological rigor (high, low) on perceptions of study quality, perceptions of product effectiveness and riskiness, and prescribing likelihood.

METHODS

Primary care physicians (n = 630) were randomly assigned to view one version of a study abstract and then answered questions.

RESULTS

Participants who viewed a high-methodological rigor study reported more perceived credibility and importance of the data (ps < .05), and less need for interpreting the study data with caution and less bias than those who viewed a low-rigor study. Those who were not under time pressure to read the stimuli rated the fictitious study description as more credible, rigorous, important, and had more confidence in study data than those who were under time pressure. Participants who had less time to review high-rigor journal abstracts and sales aids with graphics were less likely to agree the study data should be interpreted with caution than doctors who had more time with the stimuli. No effects of source type were observed.

CONCLUSIONS

The results suggest that prominently disclosing methodological rigor helps the audience form an accurate perception of the presented information. This also further highlights the importance that any promotional communications should be truthful and non-misleading.

Multisite assessment of emergency medicine resident knowledge of evidence-based medicine as measured by the Fresno Test of Evidence-Based Medicine

CONTEXT

Evidence-based medicine (EBM) is the application of scientific evidence while treating a patient. To date, however, there is very little evidence describing how residents in emergency medicine understand and incorporate EBM into practice.

OBJECTIVES

The aim of this study was to determine EBM theoretical and quantitative knowledge in emergency medicine residents in community hospital-based training programs.

METHODS

A sample of emergency medicine residents from nine hospitals was enrolled to complete a cross-sectional assessment of EBM skills from April 2021 through June 2021. Performance on the Fresno Test of Evidence-Based Medicine (FTEBM) was assessed utilizing descriptive statistics, t tests, and one-way analysis of variance.

RESULTS

A total of 50.8% (124/244) of current emergency medicine residents completed the FTEBM during the study period. No significant difference on FTEBM scores was noted between the different types of medical degrees (DO vs. MD) (p=0.511), holding an advanced research degree (p=0.117), or between each postgraduate year of training (p=0.356). The mean score of those residents who rated their knowledge of EBM as average or higher was 36.0% (32.8-39.1%). The mean score of those residents who rated their programs as having an "average" or higher institutional focus on EBM was 34.9% (32.2-37.6%).

CONCLUSIONS

Participating emergency medicine residents show an incomplete understanding of EBM both in theory and applied computations despite rating themselves as having an average understanding. Emergency medicine residencies would be well suited to implement a standardized EBM curriculum that focuses on longitudinal reinforcement of key concepts needed for the practicing physician.

Effect of training on evidence-based practice to undergraduate dental students: pre and postexperimental study

BACKGROUND

Evidence-based practice (EBP) training is essential for undergraduate students in making sound clinical decisions during patient care. However, EBP training is not included in the curriculum of undergraduate dental students in India and there is a dearth of research that assessed the effectiveness of EBP training in India. Hence, this study aimed to assess the effectiveness of EBP training to dental undergraduate students.

METHODS

Final year undergraduate students and interns of a dental institution participated in this pre and postexperimental study. All the participants received a 2-day workshop (didactic lectures, hands-on and group activities) in a classroom setting. A validated 35 item instrument titled EBP Knowledge, Attitudes, Access and Confidence questionnaire was used for the outcome assessment of EBP training.

RESULTS

Fifty undergraduate students (33 interns and 17 final year) participated in the study. Statistically significant improvement was observed in six out of 10 items related to EBP knowledge. All the items related to attitude showed a significant positive change in scores. Only four out of nine items related to accessing evidence showed a statistically significant change. Participants reported moderate confidence gain in critical appraisal skills.

CONCLUSION

The current study demonstrated moderate improvement in the effectiveness of EBP training in improving EBP knowledge, accessing evidence and critical appraisal skills.

Educational Strategies for Training in Quality Improvement and Implementation Medicine

Education in quality improvement (QI) is endorsed by the Association of American Medical Colleges across the spectrum of undergraduate, graduate, and postgraduate training. QI training is also a required component of graduate medical training per the American College of Graduate Medical Education. Despite widespread recognition of the importance of QI education and high levels of trainee involvement in QI as reported by pulmonary and critical care fellowship program directors, significant barriers to the implementation of effective and meaningful QI education during training exist. This creates an opportunity for the promotion of successfully implemented QI programs. Research demonstrates that successful QI educational programs involve the teaching of key QI concepts, participation in QI projects, protected time for QI project development, and institutional support. Using QI models such as the Plan-Do-Study-Act cycle and the Standards for Quality Improvement Reporting Excellence framework for reporting new knowledge about healthcare improvements also enhances both the educational value of the QI project and prospects for wider scholarly dissemination. In this perspective article, three examples of QI projects are discussed that serve to illustrate effective strategies of QI implementation.

Teaching Real-World Evidence: Protocol for a Systematic Review

Background

Real-world evidence (RWE) refers to observational health care data beyond clinical trial data. It holds the promise of transforming health care as a new form of evidence to support decision makers in making decisions when developing and regulating medicines. As the importance of RWE is recognized by industry and regulatory bodies, teaching RWE becomes an important matter to evaluate and refine in order to develop future researchers and stakeholders who can better integrate RWE into the routine development of medicine.

Objective

The aim of this review is to understand how RWE is currently being taught. From this landscape study, the insufficiencies of the current education of RWE can be identified and subsequently inform future education policies around RWE and its subfacets.

Methods

We will search MEDLINE, EMBASE, PsycINFO, Healthcare Management Information Consortium, Cochrane, and Web of Science for published studies using a combination of keywords and subject headings related to RWE and education. In addition, a Google search to identify grey literature will be conducted. Two authors will independently screen the titles and abstracts identified from the search and accept or reject the studies according to the study inclusion criteria; any discrepancies will be discussed and resolved. The quality of the included literature will be assessed using the Critical Appraisal Skills Programme systematic review checklist.

Results

Data from eligible publications will be abstracted into a predesigned form in order to better understand the current state of education of RWE and inform future RWE education directions and policies.

Conclusions

The subsequent systematic review will be published in a peer-reviewed journal.

International Registered Report Identifier (IRRID)

PRR1-10.2196/16941

Teaching Real-World Evidence: Protocol for a Systematic Review (Preprint).. [DOI: 10.2196/preprints.16941] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

BACKGROUND

Real-world evidence (RWE) refers to observational health care data beyond clinical trial data. It holds the promise of transforming health care as a new form of evidence to support decision makers in making decisions when developing and regulating medicines. As the importance of RWE is recognized by industry and regulatory bodies, teaching RWE becomes an important matter to evaluate and refine in order to develop future researchers and stakeholders who can better integrate RWE into the routine development of medicine.

OBJECTIVE

The aim of this review is to understand how RWE is currently being taught. From this landscape study, the insufficiencies of the current education of RWE can be identified and subsequently inform future education policies around RWE and its subfacets.

METHODS

We will search MEDLINE, EMBASE, PsycINFO, Healthcare Management Information Consortium, Cochrane, and Web of Science for published studies using a combination of keywords and subject headings related to RWE and education. In addition, a Google search to identify grey literature will be conducted. Two authors will independently screen the titles and abstracts identified from the search and accept or reject the studies according to the study inclusion criteria; any discrepancies will be discussed and resolved. The quality of the included literature will be assessed using the Critical Appraisal Skills Programme systematic review checklist.

RESULTS

Data from eligible publications will be abstracted into a predesigned form in order to better understand the current state of education of RWE and inform future RWE education directions and policies.

CONCLUSIONS

The subsequent systematic review will be published in a peer-reviewed journal.

INTERNATIONAL REGISTERED REPORT

PRR1-10.2196/16941

Does evidence-based medicine training improve doctors' knowledge, practice and patient outcomes? A systematic review of the evidence

Courses in Evidence-Based Medicine (EBM) for doctors have consistently demonstrated short-term improvements in knowledge. However, there is no strong evidence linking EBM training to changes in clinical practice or patient outcomes. This systematic review investigates whether EBM training leads to sustained improvements in doctors' knowledge and practice behaviors that may also facilitate changes in patient outcomes and experiences. A literature search was undertaken in Ovid Medline, Ovid Embase, The Cochrane Library, ERIC and Scopus. Studies published from 1997 to 2016 that assessed outcomes of EBM educational interventions amongst doctors and used measures of knowledge, skills, attitudes, practice or patient outcomes were included. Fifteen studies were included in the analysis: four randomized controlled trials (RCTs), three non-RCTs, and eight before-after (longitudinal cohort) studies. Heterogeneity among studies prevented meaningful comparisons. Varying degrees of bias due to the use of subjective measures were identified, limiting study validity. Results showed that EBM interventions can improve short-term knowledge and skills, but there is little reliable evidence of changes in long-term knowledge, attitudes, and clinical practice. No study measured improvement in patient outcomes or experiences. EBM training for medical practitioners needs to incorporate measures of behavioral changes while incorporating patient outcomes and experience measures.

Effects of an evidence-based practice education program using multifaceted interventions: a quasi-experimental study with undergraduate nursing students

BACKGROUND

Although Evidence-Based Practice (EBP) should be introduced early on in nursing education to develop students' independence and self-learning ability, there are few such courses for undergraduate nursing students in Korea. This study examined the effects of the EBP education program for undergraduate nursing students (EBP-EPUNS) on nursing students' knowledge, skills, attitudes, competencies, and future use of EBP.

METHODS

A quasi-experimental study design with pre-test, intervention, and post-test was used. The participants were 44 nursing students (experimental: 22, control: 22). A 20-h long EBP-EPUNS consisting of 5-step EBP components was provided through 8 sessions spread across 4 weeks.

RESULTS

An independent t-test and a repeated-measures ANOVA showed that the experimental group had statistically significant higher post-test scores on EBP knowledge (p < 0.001), skills (p < 0.001), attitudes (p < 0.001), competencies (p < 0.001), future use of EBP (p = 0.001), and critical thinking (p < 0.001), compared to the control group.

CONCLUSION

The EBP education program was effective in improving the knowledge, skills, attitudes, competencies, and future use of EBP among nursing students. Hence, we recommend the EBP education program as a general education course for undergraduate nursing students to promote needed proficiency in EBP.

Determining effectiveness of EBM education in otolaryngology residents using modified fresno test

OBJECTIVE

To determine the effectiveness of education in evidence-based medicine (EBM) on the knowledge, competency, and skills of otolaryngology residents of Tehran University of Medical Sciences (TUMS) at Amir-Alam and Imam Khomeini Hospitals.

METHODS

In a quasi-experimental (before-and-after) study, all ear, nose, and throat residents of TUMS (n = 41) entered the study. The residents underwent the modified Fresno test. Then, two EBM workshops with a similar content were held on 2 separate days in each hospital, with each session lasting 6 hours. The learned material was practiced in weekly journal clubs. Six months after the workshop, the modified Fresno test was applied again, and the results were analyzed.

RESULTS

A significant improvement in the modified Fresno test score was observed. The mean score of the modified Fresno test was 57.43 ± 22.07 before the workshop and 79.26 ± 22.48 after the workshop (P < 0.001).

CONCLUSION

The results of the study show that EBM education and practice of the learned materials in journal clubs can improve the knowledge and skills of residents. Further research with larger samples is needed to improve the precision of our findings and to increase confidence in the results.

LEVEL OF EVIDENCE

2 Laryngoscope, 129:2291-2294, 2019.

The impact of critical appraisal workshops on residents' evidence based medicine skills and knowledge

OBJECTIVE

To assess the impact of four evidence based medicine (EBM) critical appraisal education workshops in improving residents' EBM knowledge and skills.

METHODS

The eligible participants in the workshops were 88 residents-in-training, postgraduate years one through four, rotating through the outpatient internal medicine clinic. Four EBM workshops, consisting of 3 days each (30 minutes daily), were taught by our faculty. Topics covered included critical appraisal of randomized controlled trials, case-control and cohort studies, diagnosis studies, and systematic reviews.

RESULTS

As a program evaluation, anonymous pre-workshop and post-workshop tests were administered. Each of the four sets of tests showed improvement in scores: therapy from 58% to 77% (42% response rate), harm from 65% to 73% (38% response rate), diagnosis from 49% to 68% (49% response rate), and systematic review from 57% to 72% (30% response rate).

CONCLUSION

We found that teaching EBM in four short workshops improved EBM knowledge and critical appraisal skills related to the four topics.

Focused Evidence-Based Medicine Curriculum for Trainees in Neonatal-Perinatal Medicine

INTRODUCTION

While evidence-based medicine (EBM) is an Accreditation Council for Graduate Medical Education core competency, EBM teaching in pediatric subspecialties is rarely reported. Therefore, we designed, implemented, and evaluated this focused EBM curriculum for trainees in neonatal-perinatal medicine.

METHODS

This EBM curriculum consists of seven weekly 1-hour sessions. Specific EBM skills taught in the sessions include formulating a structured clinical question, conducting an efficient literature search, critically appraising published literature in both intervention and diagnostic studies, and incorporating evidence into clinical decision-making. The course was evaluated by a neonatology-adapted Fresno test (NAFT) and neonatology case vignettes, which were administered to learners before and after the curriculum. This publication includes the needs assessment survey, PowerPoint slides for the seven sessions, the NAFT, and the scoring rubric for the test.

RESULTS

The NAFT was internally reliable, with a Cronbach's alpha of .74. The intraclass correlation coefficient of the three raters' variability in assessment of learners was excellent at .98. Mean test scores increased significantly (54 points, p < .001) in 14 learners after the EBM curriculum, indicating an increase in EBM-related knowledge and skills.

DISCUSSION

This focused EBM curriculum enhances trainees' knowledge and skills and fosters evidence-based practice. The curriculum can be easily adapted for learners in pediatrics, as well as family medicine, in order to enhance trainees' EBM skills and knowledge.

Another Time, Another Space: The Evolution of the Virtual Journal Club

Virtual journal clubs (VJCs) provide a standardized, easily accessible forum for evidence-based discussion. The new virtual reality setting in which journal clubs and other online education events now take place offers great advantages and new opportunities for radiologists in academic medicine and private practice. VJCs continue to evolve, largely due to many emerging technologies and platforms. VJCs will continue to play an increasingly important role in medical education, interdisciplinary interaction, and multi-institutional collaboration. In this article, we discuss how to conduct and lead a critical review of medical literature in the setting of a virtual or traditional journal club. We discuss the current applications of VJCs in medical and graduate medical education and continued lifelong learning. We also explain the advantages and disadvantages of VJCs over traditional venues. Finally, the reader will be given the tools to successfully implement and run a VJC.

A systematic review of how studies describe educational interventions for evidence-based practice: stage 1 of the development of a reporting guideline

BACKGROUND

The aim of this systematic review was to identify which information is included when reporting educational interventions used to facilitate foundational skills and knowledge of evidence-based practice (EBP) training for health professionals. This systematic review comprised the first stage in the three stage development process for a reporting guideline for educational interventions for EBP.

METHODS

The review question was 'What information has been reported when describing educational interventions targeting foundational evidence-based practice knowledge and skills?'MEDLINE, Academic Search Premier, ERIC, CINAHL, Scopus, Embase, Informit health, Cochrane Library and Web of Science databases were searched from inception until October - December 2011. Randomised and non-randomised controlled trials reporting original data on educational interventions specific to developing foundational knowledge and skills of evidence-based practice were included.Studies were not appraised for methodological bias, however, reporting frequency and item commonality were compared between a random selection of studies included in the systematic review and a random selection of studies excluded as they were not controlled trials. Twenty-five data items were extracted by two independent reviewers (consistency > 90%).

RESULTS

Sixty-one studies met the inclusion criteria (n = 29 randomised, n = 32 non-randomised). The most consistently reported items were the learner's stage of training, professional discipline and the evaluation methods used (100%). The least consistently reported items were the instructor(s) previous teaching experience (n = 8, 13%), and student effort outside face to face contact (n = 1, 2%).

CONCLUSION

This systematic review demonstrates inconsistencies in describing educational interventions for EBP in randomised and non-randomised trials. To enable educational interventions to be replicable and comparable, improvements in the reporting for educational interventions for EBP are required. In the absence of a specific reporting guideline, there are a range of items which are reported with variable frequency. Identifying the important items for describing educational interventions for facilitating foundational knowledge and skills in EBP remains to be determined. The findings of this systematic review will be used to inform the next stage in the development of a reporting guideline for educational interventions for EBP.

Is literature search training for medical students and residents effective? a literature review

OBJECTIVES

This literature review examines the effectiveness of literature searching skills instruction for medical students or residents, as determined in studies that either measure learning before and after an intervention or compare test and control groups. The review reports on the instruments used to measure learning and on their reliability and validity, where available. Finally, a summary of learning outcomes is presented.

METHODS

Fifteen studies published between 1998 and 2011 were identified for inclusion in the review. The selected studies all include a description of the intervention, a summary of the test used to measure learning, and the results of the measurement.

RESULTS

Instruction generally resulted in improvement in clinical question writing, search strategy construction, article selection, and resource usage.

CONCLUSION

Although the findings of most of the studies indicate that the current instructional methods are effective, the study designs are generally weak, there is little evidence that learning persists over time, and few validated methods of skill measurement have been developed.

Increasing utilization of Internet-based resources following efforts to promote evidence-based medicine: a national study in Taiwan

Background

Since the beginning of 2007, the National Health Research Institutes has been promoting the dissemination of evidence-based medicine (EBM). The current study examined longitudinal trends of behaviors in how hospital-based physicians and nurses have searched for medical information during the spread of EBM.

Methods

Cross-sectional postal questionnaire surveys were conducted in nationally representative regional hospitals of Taiwan thrice in 2007, 2009, and 2011. Demographic data were gathered concerning gender, age, working experience, teaching appointment, academic degree, and administrative position. Linear and logistic regression models were used to examine predictors and changes over time.

Results

Data from physicians and nurses were collected in 2007 (n = 1156), 2009 (n = 2975), and 2011 (n = 3999). There were significant increases in the use of four Internet-based resources – Web portals, online databases, electronic journals, and electronic books – across the three survey years among physicians and nurses (p < 0.001). Access to textbooks and printed journals, however, did not change over the 4-year study period. In addition, there were significant relationships between the usage of Internet-based resources and users’ characteristics. Age and faculty position were important predictors in relation to the usage among physicians and nurses, while academic degree served as a critical factor among nurses only.

Conclusions

Physicians and nurses used a variety of sources to look for medical information. There was a steady increase in use of Internet-based resources during the diffusion period of EBM. The findings highlight the importance of the Internet as a prominent source of medical information for main healthcare professionals.

Principles and practice of clinical research course for surgeons: an evaluation of knowledge transfer and perceptions

BACKGROUND

Knowledge and training in evidence-based medicine (EBM) and clinical research is under-represented in most surgical training programs in North America. To address a lack of resources for surgeons, trainees and related specialties, we developed a Principles and Practice of Clinical Research (PPCR) course. The current study evaluated transfer of knowledge and perceptions about the course.

METHODS

The course was an intensive 2.5-day workshop consisting of interactive lectures and small group breakout sessions. Pre- and postcourse tests were completed by participants. The Fresno test, questions from the Centre of Applied Medical Statistics (CAMS) test and questions developed by the course chairs were used to determine if participants' knowledge of EBM, clinical research methodology and statistics improved. We also elicited participant perceptions of the course.

RESULTS

Overall participant knowledge about EBM and clinical research methods improved significantly from the pre- to the postcourse test (mean improvement inscore 13.5%, relative increase 35.3%, p < 0.001). Specifically, improvements were demonstrated on the Fresno test (mean improvement 13.5%, relative increase 36.1%, p< 0.001) and the CAMS test (mean improvement 11.4%, relative increase 20.1%, p = 0.001). Participants showed the greatest improvement in general knowledge about clinical research (mean improvement 15.4%, relative increase 46.5%, p < 0.001). The PPCR course was well received; 30 (81.1%) participants who completed the course evaluation gave it a positive rating.

CONCLUSION

Participants in a short course focusing on EBM and clinical research methodology had significant improvements in scores on tests of knowledge gained. Widespread implementation of similar courses may bridge knowledge gaps for surgeons, surgical trainees and health professionals. Whether shorter knowledge gain sare retained in the longer term remains unknown.

Two Strategies to Intensify Evidence-based Medicine Education of Undergraduate Students: A Randomised Controlled Trial

Introduction: Undergraduate evidence-based practice (EBP) is usually taught through standalone courses and workshops away from clinical practice. This study compared the effects of 2 clinically integrated educational strategies on final year medical students. Materials and Methods: Final year medical students rotating to the general medicine service for a 2-week internship were randomly assigned to participate in a weekly EBP-structured case conference focusing on students’ primary care patients (Group A, n = 47), or to receive a weekly didactic lecture about EBP (Group B, n = 47). The teaching effects of these 2 interventions were evaluated by a validated instrument for assessment of EBP related knowledge (EBP-K), attitude (EBP-A), personal application (EBP-P), and anticipated future use (EBP-F) on the first and last days of rotation. Results: All scores improved significantly after the 2-week EBM-teaching for both groups. When compared to Group B, students in Group A had significantly higher post-intervention scores of EBP-K (21.2 ± 3.5 vs 19.0 ± 4.6; ie. 57.8 ± 72.9% vs 29.1 ± 39.1%; P <0.01) and EBP-P (18.7 ± 4.3 vs 15.3 ± 3.9; ie. 28.5 ± 25.5 % vs 14.1 ± 18.7 %; P <0.001). In contrast, the scores of EBP-A and EBP-F were similar between the 2 groups. Conclusion: Structured case conference, when compared to the didactic lectures, significantly improved EBP-K and EBP-P for final year medical students. Key words: Evidence-based medicine, Medical education, Preclinical medical student

Senior internal medicine residents' confidence with essential topics in evidence-based medicine taught during internship

BACKGROUND

Few studies have examined residents' retained knowledge and confidence regarding essential evidence-based medicine (EBM) topics.

OBJECTIVE

To compare postgraduate year-3 (PGY-3) residents' confidence with EBM topics taught during internship with that of PGY-1 residents before and after exposure to an EBM curriculum.

METHODS

All residents participated in an EBM curriculum during their intern year. We surveyed residents in 2009. PGY-1 residents completed a Likert-scale type survey (which included questions from the validated Berlin questionnaire and others, developed based on input from local EBM experts). We administered the Berlin questionnaire to a subset of PGY-3 residents.

RESULTS

Forty-five PGY-3 (88%; n  =  51) and 42 PGY-1 (91%; n  =  46) residents completed the survey. Compared with PGY-1 residents pre-curriculum, PGY-3 residents were significantly more confident in their knowledge of pre- and posttest probability (mean difference, 1.14; P  =  .002), number needed to harm (mean difference, 1.09; P  =  .002), likelihood ratio (mean difference, 1.01; P  =  .003), formulation of a focused clinical question (mean difference, 0.98; P  =  .001), and critical appraisal of therapy articles (mean difference, 0.91; P  =  .002). Perceived confidence was significantly lower for PGY-3 than post-curriculum PGY-1 residents on relative risk (mean difference, -0.86; P  =  .002), study design for prognosis questions (mean difference, -0.75; P  =  .004), number needed to harm (mean difference, -0.67; P  =  .01), ability to critically appraise systematic reviews (mean difference, -0.65, P  =  .009), and retrieval of evidence (mean difference, -0.56; P  =  .008), among others. There was no relationship between confidence with and actual knowledge of EBM topics.

CONCLUSIONS

Our findings demonstrate lower confidence among PGY-3 than among PGY-1 internal medicine residents for several EBM topics. PGY-3 residents demonstrated poor knowledge of several core topics taught during internship. Longitudinal EBM curricula throughout residency 5 help reinforce residents' EBM knowledge and their confidence.

Teaching evidence based medicine literature searching skills to medical students during the clinical years - a protocol for a randomised controlled trial

BACKGROUND

Two of the key steps in evidence based medicine (EBM) are being able to construct a clinical question and effectively search the literature to source relevant information. No evidence currently exists that informs whether such skills should be taught to medical students during their pre-clinical years, or delivered to include both the pre-clinical and clinical years of study. This is an important component of curriculum design as the level of clinical maturity of students can affect their perception of the importance and uptake of EBM principles in practice.

METHODS/DESIGN

A randomised controlled trial will be conducted to identify the effectiveness of delivering a formal workshop in EBM literature searching skills to third year medical students entering their clinical years of study. The primary outcome of EBM competency in literature searching skills will be evaluated using the Fresno tool.

DISCUSSION

This trial will provide novel information on the effectiveness of delivering a formal education workshop in evidence based medicine literature searching skills during the clinical years of study. The result of this study will also identify the impact of teaching EBM literature searching skills to medical students during the clinical years of study.

Effectiveness of a short-course in improving knowledge and skills on evidence-based practice

Background

To assess the effectiveness (change in knowledge and skills measured by the Fresno test) of a short course in Evidence Based Practice (EBP) carried out in a group of family medicine residents

Methods

Before-after study. Participants' were 152 Family Medicine residents in their second year of the training programme. Settings were Primary Care Teaching Units in Catalonia. Intervention was comprised of a four half-day training course designed to develop the knowledge and skills required to practice evidence-based care. The main outcome measure was change in EBP knowledge and skills, measured using the Spanish version of the Fresno test (score range, 0-212)

Results

The mean difference between pre-test and post-test was 47.7, a statistically significant result with 95% CI of 42.8-52.5 (p < 0.0001). An important improvement was observed in the questions related to calculations such as sensitivity, specificity, the absolute risk reduction or the number needed to treat. A more modest increase was found in the residents' knowledge and skills in finding the best clinical evidence, and appraising the validity and applicability of an article. Finally, a weak and non-statistically significant improvement was found in formulating a clinical question.

Conclusions

The study provides evidence for responsiveness to changes in knowledge and skills in EBP after an educational intervention.

Extended evaluation of a longitudinal medical school evidence-based medicine curriculum

BACKGROUND

Evidence-based medicine (EBM) is an important element of medical education. However, limited information is available on effective curricula.

OBJECTIVE

To evaluate a longitudinal medical school EBM curriculum using validated instruments.

DESIGN, PARTICIPANTS, MEASUREMENTS

We evaluated EBM attitudes and knowledge of medical students as they progressed through an EBM curriculum. The first component of the curriculum was an EBM "short course" with didactic and small-group sessions occurring at the end of the second year. The second component integrated EBM assignments with third-year clinical rotations. The 15-point Berlin Questionnaire was administered before the course in 2006 and 2007, after the short course, and at the end of the third year. The 212-point Fresno Test was administered before the course in 2007 and 2008, after the short course, and at the end of the third year. Self-reported knowledge and attitudes were also assessed in all three classes of medical students.

RESULTS

EBM knowledge scores on the 15-point Berlin Questionnaire increased from baseline by 3.0 points (20.0%) at the end of the second year portion of the course (p < 001) and by 3.4 points (22.7%) at the end of the third year (p < 001). EBM knowledge scores on the 212-point Fresno Test increased from baseline by 39.7 points (18.7%) at the end of the second year portion of the course (p < 001) and by 54.6 points (25.8%) at the end of the third year (p < 001). On a 5-point scale, self-rated EBM knowledge increased from baseline by 1.0 and 1.4 points, respectively (both p < 001). EBM was felt to be of high importance for medical education and clinical practice at all time points, with increases noted after both components of the curriculum.

CONCLUSIONS

A longitudinal medical school EBM was associated with markedly increased EBM knowledge on two validated instruments. Both components of the curriculum were associated with gains in knowledge. The curriculum was also associated with increased perceptions of the importance of EBM for medical education and clinical practice.

Validation of an instrument to assess evidence-based practice knowledge, attitudes, access, and confidence in the dental environment

This article reports the validation of an assessment instrument designed to measure the outcomes of training in evidence-based practice (EBP) in the context of dentistry. Four EBP dimensions are measured by this instrument: 1) understanding of EBP concepts, 2) attitudes about EBP, 3) evidence-accessing methods, and 4) confidence in critical appraisal. The instrument-the Knowledge, Attitudes, Access, and Confidence Evaluation (KACE)-has four scales, with a total of thirty-five items: EBP knowledge (ten items), EBP attitudes (ten), accessing evidence (nine), and confidence (six). Four elements of validity were assessed: consistency of items within the KACE scales (extent to which items within a scale measure the same dimension), discrimination (capacity to detect differences between individuals with different training or experience), responsiveness (capacity to detect the effects of education on trainees), and test-retest reliability. Internal consistency of scales was assessed by analyzing responses of second-year dental students, dental residents, and dental faculty members using Cronbach coefficient alpha, a statistical measure of reliability. Discriminative validity was assessed by comparing KACE scores for the three groups. Responsiveness was assessed by comparing pre- and post-training responses for dental students and residents. To measure test-retest reliability, the full KACE was completed twice by a class of freshman dental students seventeen days apart, and the knowledge scale was completed twice by sixteen faculty members fourteen days apart. Item-to-scale consistency ranged from 0.21 to 0.78 for knowledge, 0.57 to 0.83 for attitude, 0.70 to 0.84 for accessing evidence, and 0.87 to 0.94 for confidence. For discrimination, ANOVA and post hoc testing by the Tukey-Kramer method revealed significant score differences among students, residents, and faculty members consistent with education and experience levels. For responsiveness to training, dental students and residents demonstrated statistically significant changes, in desired directions, from pre- to post-test. For the student test-retest, Pearson correlations for KACE scales were as follows: knowledge 0.66, attitudes 0.66, accessing evidence 0.74, and confidence 0.76. For the knowledge scale test-retest by faculty members, the Pearson correlation was 0.79. The construct validity of the KACE is equivalent to that of instruments that assess similar EBP dimensions in medicine. Item consistency for the knowledge scale was more variable than for other KACE scales, a finding also reported for medically oriented EBP instruments. We conclude that the KACE has good discriminative validity, responsiveness to training effects, and test-retest reliability.

Evidence-based medicine training during residency: a randomized controlled trial of efficacy

BACKGROUND

Evidence-based medicine (EBM) has been widely integrated into residency curricula, although results of randomized controlled trials and long term outcomes of EBM educational interventions are lacking. We sought to determine if an EBM workshop improved internal medicine residents' EBM knowledge and skills and use of secondary evidence resources.

METHODS

This randomized controlled trial included 48 internal medicine residents at an academic medical center. Twenty-three residents were randomized to attend a 4-hour interactive workshop in their PGY-2 year. All residents completed a 25-item EBM knowledge and skills test and a self-reported survey of literature searching and resource usage in their PGY-1, PGY-2, and PGY-3 years.

RESULTS

There was no difference in mean EBM test scores between the workshop and control groups at PGY-2 or PGY-3. However, mean EBM test scores significantly increased over time for both groups in PGY-2 and PGY-3. Literature searches, and resource usage also increased significantly in both groups after the PGY-1 year.

CONCLUSIONS

We were unable to detect a difference in EBM knowledge between residents who did and did not participate in our workshop. Significant improvement over time in EBM scores, however, suggests EBM skills were learned during residency. Future rigorous studies should determine the best methods for improving residents' EBM skills as well as their ability to apply evidence during clinical practice.

Undergraduate medical student perceptions and use of Evidence Based Medicine: a qualitative study

BACKGROUND

Many medical schools teach the principles of Evidence Based Medicine (EBM) as a subject within their medical curriculum. Few studies have explored the barriers and enablers that students experience when studying medicine and attempting to integrate EBM in their clinical experience. The aim of this study was to identify undergraduate medical student perceptions of EBM, including their current use of its principles as students and perceived future use as clinicians.

METHODS

Third year medical students were recruited via email to participate in focus group discussions. Four focus groups were conducted separately across four hospital sites. All focus groups were conducted by the same facilitator. All discussions were transcribed verbatim, and analysed independently by the two authors according to the principles of thematic analysis.

RESULTS

Focus group discussions were conducted with 23 third-year medical students, representing three metropolitan and one rural hospital sites. Five key themes emerged from the analysis of the transcripts: (1) Rationale and observed use of EBM in practice, (2) Current use of EBM as students, (3) Perceived use of EBM as future clinicians, (4) Barriers to practicing EBM, and (5) Enablers to facilitate the integration of EBM into clinical practice. Key facilitators for promoting EBM to students include competency in EBM, mentorship and application to clinical disciplines. Barriers to EBM implementation include lack of visible application by senior clinicians and constraints by poor resourcing.

CONCLUSIONS

The principles and application of EBM is perceived by medical students to be important in both their current clinical training and perceived future work as clinicians. Future research is needed to identify how medical students incorporate EBM concepts into their clinical practice as they gain greater clinical exposure and competence.

Interventions to improve question formulation in professional practice and self-directed learning

BACKGROUND

Formulating questions is fundamental to the daily life of a healthcare worker and is a defining characteristic of professional competence and meaningful learning. With high expectations for healthcare providers to remain up-to-date with current evidence and the movement towards formalizing reflective practice as part of physician revalidation, it is important that curricula developed for improving the ability to formulate well-constructed questions are based on the best evidence.

OBJECTIVES

To assess the effectiveness of interventions for increasing the frequency and quality of questions formulated by healthcare providers in practice and the context of self-directed learning.

SEARCH STRATEGY

We obtained studies from searches of electronic bibliographic databases, and supplemented these with handsearching, checking reference lists, and consultation with experts.

SELECTION CRITERIA

We considered published and unpublished randomized controlled trials (RCTs), controlled clinical trials (CCTs), interrupted time-series (ITS), and controlled before-after (CBA) studies of any language examining interventions for increasing the quality and frequency of questions formulated by health professionals involved with direct patient care.

DATA COLLECTION AND ANALYSIS

Two review authors independently undertook all relevancy screening and 'Risk of bias' assessment in duplicate. Intervention characteristics, follow-up intervals, and measurement outcomes were diverse and precluded quantitative analysis. We have summarized data descriptively.

MAIN RESULTS

Searches identified four studies examining interventions to improve question formulation in healthcare professionals. Interventions were mostly multi-component, limited within the context of EBM and primarily in physician and resident populations. We did not identify studies examining changes in frequency of questions formulated or those within the context of reflection. Risk of bias was generally rated to be 'high risk'. Three of the four studies showed improvements in question formulation in physicians, residents, or mixed allied health populations in the short- to moderate term follow up. Only one study examined sustainability of effects at one year and reported that skills had eroded over time.

AUTHORS' CONCLUSIONS

Evidence from our review suggests that interventions to increase the quality of questions formulated in practice produce mixed results at both short- (immediately following intervention), and moderate-term follow up (up to nine months), comparatively. Although three studies reported effectiveness estimates of an educational intervention for increasing the quality of question formulation within the short term, only one study examined the effectiveness in the longer term (one year) and revealed that search skills had eroded over time. Data suggests that sustainability of effects from educational interventions for question formulation are unknown.

Physicians' characteristics in the usage of online database: a representative nationwide survey of regional hospitals in Taiwan

Physicians have to deal with a broad range of medical problems in clinical practice, thus making the timely acquisition of relevant information is a critical skill for physicians to improve care quality. The current national study investigates how physicians search for medical information and analyses how they use online medical databases. A structured questionnaire survey was conducted, with 457 valid returns collected. Internet-based resources (Web portals, online databases, and electronic journals) were more often accessed by physicians to look for medical information than personal or paper ones. Almost universally, physicians have accessed online databases. MEDLINE was the most frequently accessed database. Furthermore, physicians under 50 years old tended to access online databases more often than their elder colleagues (OR = 5.27, 95% CI = 1.96-14.14 for age <35; OR = 4.68, 95% CI = 2.07-10.60 for ages 35-50). In addition, physicians with faculty position were more often accessing online databases (OR = 3.32; 95% CI = 1.75-6.30). Other factors - including clinical experience, administrative position, gender, academic degree, and professional specialty - carried no significant differences. These data may assist in determining how to promote the use of online evidence-based medical information for clinical services.

Do workshops in evidence-based practice equip participants to identify and answer questions requiring consideration of clinical research? A diagnostic skill assessment

Evidence-based practice (EBP) requires practitioners to identify and formulate questions in response to patient encounters, and to seek, select, and appraise applicable clinical research. A standardized workshop format serves as the model for training of medical educators in these skills. We developed an evaluation exercise to assess the ability to identify and solve a problem requiring the use of targeted skills and administered it to 47 North American junior faculty and residents in various specialties at the close of two short workshops in EBP. Prior to the workshop, subjects reported prior training in EBP and completed a previously validated knowledge test. Our post-workshop exercise differed from the baseline measures and required participants to spontaneously identify a suitable question in response to a simulated clinical encounter, followed by a description of a stepwise approach to answering it. They then responded to successively more explicitly prompted queries relevant to their question. We analyzed responses to identify areas of skill deficiency and potential reasons for these deficiencies. Twelve respondents (26%) initially failed to identify a suitable question in response to the clinical scenario. Ability to choose a suitable question correlated with the ability to connect an original question to an appropriate study design. Prior EBP training correlated with the pretest score but not with performance on our exercise. Overall performance correlated with ability to correctly classify their questions as pertaining to therapy, diagnosis, prognosis, or harm. We conclude that faculty and residents completing standard workshops in EBP may still lack the ability to initiate and investigate original clinical inquiries using EBP skills.

Development and validation of a questionnaire to evaluate medical students' evidence-based medicine competencies

PURPOSE

The purpose of the study was to develop and validate a questionnaire to evaluate medical students' knowledge of, attitude towards and practice of evidence-based medicine (EBM).

METHODS

The participants of the study were 418 medical students enrolled in the Kyung Hee University School of Medicine in Seoul, Korea. To examine construct validity of the questionnaire, an exploratory factor analysis (EFA) was performed with 118 participants; a confirmatory factor analysis (CFA) was conducted with the remaining 281 participants. We developed 41 items with a 4-point Likert scale. An EFA was performed to verify the emergence of four dimensions of EBM competencies. The principal axis factoring method and the direct oblimin rotation method were used. To confirm construct validity, a CFA was conducted with the remaining 281 participants. To evaluate model fitness, root mean square error of approximation (RMSEA) and comparative fit index (CFI) were used as fit indices. We conducted ANOVA with Scheffe as discriminant validation, and calculated Cronbach's alpha of 4 subscales as reliability checkup.

RESULTS

After refinement procedure, factor analysis of the 32 items in therevised questionnaire yielded 4 factors. The Scree plot supported a 4 factor solution explaining 53.5% of the variance. The 4 components derived were: factor 1_knowledge on EBM (11 items; Cronbach's alpha=0.92); factor 2_ pursuit towards EBM (10; 0.88); factor 3_reluctance on EBM (7; 0.78); factor 4_practice of EBM (4; 0.75). The questionnaire could discriminate competence differences among 1-3 yr students. Satisfactory Cronbach's alpha scores were noted for each factor as well.

CONCLUSION

The EBM competency questionnaire was validated.

Prospective, controlled assessment of the impact of formal evidence-based medicine teaching workshop on ability to appraise the medical literature

BACKGROUND

The ability to critically appraise the calibre of studies in medical literature is increasingly important for medical professionals.

AIM

This prospective controlled study evaluated the impact of a 6-h Evidence Based Medicine (EBM) Workshop on the critical appraisal skills of medical trainees.

METHODS

Individuals attended three 2-h workshops over a 3-week period, incorporating didactic lectures in statistics, clinical trial design, appraising research papers and practical examples. Appraisal skills were assessed pre- and post-training based on grading the quality of randomised control studies (level 1 evidence), cohort studies (level 2 evidence) and case-control studies (level 3 evidence) [From Oxford Centre for Evidence Based Medicine Levels of Evidence (2001), http://www.cebm.net/critical_appraisal.asp ].

RESULTS

Overall grading improved from 39% (pre-course) to 74% (post-course), P = 0.002, with grading of levels 1, 2 and 3 studies improving from 42 to 75%, 53 to 61% and 21 to 84%, respectively.

CONCLUSIONS

We conclude that a 6-h formal EBM workshop is effective in enhancing the critical appraisal skills of medical trainees.

Teaching Evidence-based Practice: Perspectives from the Undergraduate and Post-graduate Viewpoint

Introduction: Evidence-based practice (EBP) involves making clinical decisions informed by the most relevant and valid evidence available. It has been suggested that the outcomes of teaching EBP skills may differ between undergraduates and post-graduates due to different determinants in learning. This paper reviews the current literature and discusses the impact of the teaching environment (undergraduate or post-graduate) for teaching EBP and its impact on EBP competency. Methods: A search of the literature was performed across the MEDLINE, CINAHL, PsychInfo and ERIC databases. Randomised controlled trials (RCTs) and non-randomised trials were eligible for inclusion in the paper. Studies were included for review if they explored the impact of teaching on participants’ EBP competency, consisting of critical appraisal skills, knowledge and/or behaviour. Results: Ten articles were eligible for inclusion for this review, of which 7 met all inclusion criteria. EBP competency was shown to increase regardless of whether EBP is delivered to medical students at an undergraduate or post-graduate level. EBP taught to a non-medical undergraduate audience did not modify participants’ EBP competency. No study directly compared teaching EBP to an undergraduate and post-graduate audience. Conclusions: Given the limited amount of studies included in this review, further research incorporating high- level methodologies is required to establish a clear recommendation on the research question. Key words: Competency-based education, Evidence-based practice, Graduate medical educa- tion, Undergraduate medical education

Impact of an evidence-based medicine curriculum on resident use of electronic resources: a randomized controlled study

BACKGROUND

Evidence-based medicine (EBM) is widely taught in residency, but evidence for effectiveness of EBM teaching on changing residents' behavior is limited.

OBJECTIVE

To investigate the impact of an EBM curriculum on residents' use of evidence-based resources in a simulated clinical experience.

DESIGN/PARTICIPANTS

Fifty medicine residents randomized to an EBM teaching or control group.

MEASUREMENTS

A validated test of EBM knowledge (Fresno test) was administered before and after intervention. Post intervention, residents twice completed a Web-based, multiple-choice instrument (15 items) comprised of clinical vignettes, first without then with access to electronic resources. Use of electronic resources was tracked using ProxyPlus software. Within group pre-post differences and between group post-test differences were examined.

RESULTS

There was more improvement in EBM knowledge (100-point scale) for the intervention group compared to the control group (mean score increase 22 vs. 12, p = 0.012). In the simulated clinical experience, the most commonly accessed resources were Ovid (71% of residents accessed) and InfoPOEMs (62%) for the EBM group and UptoDate (67%) and MDConsult (58%) for the control group. Residents in the EBM group were more likely to use evidence-based resources than the control group. Performance on clinical vignettes was similar between the groups both at baseline (p = 0.19) and with access to information resources (p = 0.89).

CONCLUSIONS

EBM teaching improved EBM knowledge and increased use of evidence-based resources by residents, but did not improve performance on Web-based clinical vignettes. Future studies will need to examine impact of EBM teaching on clinical outcomes.

The effect of training on question formulation among public health practitioners: results from a randomized controlled trial

OBJECTIVE

To improve understanding of the information-seeking behaviors of public health professionals, the authors conducted this randomized controlled trial involving sixty participants to determine whether library and informatics training, with an emphasis on PubMed searching skills, increased the frequency and sophistication of participants' practice-related questions.

METHODS

The intervention group (n = 34) received evidence-based public health library and informatics training first, whereas the control group (n = 26) received identical training two weeks later. The frequency and sophistication of the questions generated by both intervention and control groups during the interim two-week period served as the basis for comparison.

RESULTS

The intervention group reported an average of almost 1.8 times more questions than those reported by the control group (1.24 vs. 0.69 questions per participant); however, this difference did not reach statistical significance. The intervention group overall produced more sophisticated (foreground) questions than the control group (18 vs. 9); however, this difference also did not reach statistical significance.

CONCLUSION

The training provided in the current study seemed to prompt public health practitioners to identify and articulate questions more often. Training appears to create the necessary precondition for increased information-seeking behavior among public health professionals.

Analysis of queries sent to PubMed at the point of care: observation of search behaviour in a medical teaching hospital

BACKGROUND

The use of PubMed to answer daily medical care questions is limited because it is challenging to retrieve a small set of relevant articles and time is restricted. Knowing what aspects of queries are likely to retrieve relevant articles can increase the effectiveness of PubMed searches. The objectives of our study were to identify queries that are likely to retrieve relevant articles by relating PubMed search techniques and tools to the number of articles retrieved and the selection of articles for further reading.

METHODS

This was a prospective observational study of queries regarding patient-related problems sent to PubMed by residents and internists in internal medicine working in an Academic Medical Centre. We analyzed queries, search results, query tools (Mesh, Limits, wildcards, operators), selection of abstract and full-text for further reading, using a portal that mimics PubMed.

RESULTS

PubMed was used to solve 1121 patient-related problems, resulting in 3205 distinct queries. Abstracts were viewed in 999 (31%) of these queries, and in 126 (39%) of 321 queries using query tools. The average term count per query was 2.5. Abstracts were selected in more than 40% of queries using four or five terms, increasing to 63% if the use of four or five terms yielded 2-161 articles.

CONCLUSION

Queries sent to PubMed by physicians at our hospital during daily medical care contain fewer than three terms. Queries using four to five terms, retrieving less than 161 article titles, are most likely to result in abstract viewing. PubMed search tools are used infrequently by our population and are less effective than the use of four or five terms. Methods to facilitate the formulation of precise queries, using more relevant terms, should be the focus of education and research.

Evaluation of a longitudinal medical school evidence-based medicine curriculum: a pilot study

BACKGROUND

Evidence-based medicine (EBM) is increasingly taught in medical schools, but few curricula have been evaluated using validated instruments.

OBJECTIVE

To evaluate a longitudinal medical school EBM curriculum using a validated instrument.

DESIGN, PARTICIPANTS, MEASUREMENTS

We evaluated EBM attitudes and knowledge of 32 medical students as they progressed through an EBM curriculum. The first part was an EBM "short course" with didactic and small-group sessions occurring at the end of the second year. The second part integrated EBM assignments with third-year clinical rotations. The validated 15-item Berlin Questionnaire was administered before the course, after the short course, and at the end of the third year.

RESULTS

EBM knowledge scores increased from baseline by 2.8 points at the end of the second year portion of the course (p = .0001), and by 3.7 points at the end of the third year (p < .0001). Self-rated EBM knowledge increased from baseline by 0.8 and 1.1 points, respectively (p = .0006 and p < .0001, respectively). EBM was felt to be of high importance for medical education and clinical practice at all time points, peaking after the short course.

CONCLUSIONS

A longitudinal medical school EBM curriculum was associated with increased EBM knowledge. This knowledge increase was sustained throughout the curriculum.

Developing an integrated evidence-based medicine curriculum for family medicine residency at the University of Alberta

There is general consensus in the academic community that evidence-based medicine (EBM) teaching is essential. Unfortunately, many postgraduate programs have significant weakness in their EBM programs. The Family Medicine Residency committee at the University of Alberta felt their EBM curriculum would benefit from critical review and revision. An EBM Curriculum Committee was created to evaluate previous components and develop new strategies as needed. Input from stakeholders including faculty and residents was sought, and evidence regarding the teaching and practical application of EBM was gathered. The committee drafted goals and objectives, the primary of which were to assist residents to (1) become competent self-directed, lifelong learners with skills to effectively and efficiently keep up to date, and 2) develop EBM skills to solve problems encountered in daily practice. New curriculum components, each evidence based, were introduced in 2005 and include a family medicine EBM workshop to establish basic EBM knowledge; a Web-based Family Medicine Desktop promoting easier access to evidence-based Internet resources; a brief evidence-based assessment of the research project enhancing integration of EBM into daily practice; and a journal club to support peer learning and growth of rapid appraisal skills. Issues including time use, costs, and change management are discussed. Ongoing evaluation of the curriculum and its components is a principal factor of the design, allowing critical review and adaptation of the curriculum. The first two years of the curriculum have yielded positive feedback from faculty and statistically significant improvement in multiple areas of residents' opinions of the curriculum and comfort with evidence-based practice.

A training intervention to improve information management in primary care

BACKGROUND AND OBJECTIVES

Training programs designed to improve information management have been implemented but not adequately tested. Three critical components for information management were tested in a randomized control study: (1) knowledge of valid, synthesized summary information, (2) skills to use Web-based resources that provide access to these summaries, and (3) use of Web-based resources in clinical practice.

METHODS

Twenty-four primary care practices were provided with computers and high-speed Internet access and then matched, with half randomly assigned to receive training and half to receive training at a later date. Training was designed to address knowledge, skills, and use of Web-based information. Outcomes were assessed by comparing baseline and follow-up questionnaires that focused on five conceptual domains related to Web-based resource use for patient care decisions and patient education.

RESULTS

Compared to the delayed training group, the initial training group increased their knowledge and skill of Web-based resources and use for patient care decisions. Some measures of communication with patients about using Web-based resources and of incorporating use of Web-based resources into daily practice increased from baseline to follow-up for all participants.

CONCLUSIONS

Our findings suggest that training and providing computers and Internet connections have measurable effects on information management behaviors.

Real-time EBM: from bed board to keyboard and back

BACKGROUND

To practice Evidence-Based Medicine (EBM), physicians must quickly retrieve evidence to inform medical decisions. Internal Medicine (IM) residents receive little formal education in electronic database searching, and have identified poor searching skills as a barrier to practicing EBM.

OBJECTIVE

To design and implement a database searching tutorial for IM residents on inpatient rotations and to evaluate its impact on residents' skill and comfort searching MEDLINE and filtered EBM resources.

DESIGN

Randomized controlled trial. Residents randomized to the searching tutorial met for up to 6 1-hour small group sessions to search for answers to questions about current hospitalized patients.

PARTICIPANTS

Second- and 3rd-year IM residents.

MEASUREMENTS

Residents in both groups completed an Objective Structured Searching Evaluation (OSSE), searching for primary evidence to answer 5 clinical questions. OSSE outcomes were the number of successful searches, search times, and techniques utilized. Participants also completed self-assessment surveys measuring frequency and comfort using EBM databases.

RESULTS

During the OSSE, residents who participated in the intervention utilized more searching techniques overall (p < .01) and used PubMed's Clinical Queries more often (p < .001) than control residents. Searching "success" and time per completed search did not differ between groups. Compared with controls, intervention residents reported greater comfort using MEDLINE (p < .05) and the Cochrane Library (p < .05) on post-intervention surveys. The groups did not differ in comfort using ACP Journal Club, or in self-reported frequency of use of any databases.

CONCLUSIONS

An inpatient EBM searching tutorial improved searching techniques of IM residents and resulted in increased comfort with MEDLINE and the Cochrane Library, but did not impact overall searching success.

Teaching evidence-based practice: the teachers consider the content

OBJECTIVE

To identify the views of evidence-based practice (EBP) experts about information that should be included in EBP courses.

DESIGN

A two-round Delphi process.

SETTING AND SUBJECT

The panel of experts in teaching EBP residing in developed and developing countries identified through purposive and snowball sampling.

MAIN OUTCOME MEASURES

We developed a list of 49 topics representing key educational topics. The panel scored the importance of topics for inclusion in introductory and advanced courses. In the second round, we sent a summary of results to the panel and asked them to re-score in light of the group's responses.

RESULTS

We used email to invite 105 EBP teachers to participate in the study. Fifty-one people from 15 countries agreed to participate, and 40 completed the second round. We achieved consensus that formulating clinical questions, searching pre-appraised resources, introduction to systematic reviews and critical appraisal of studies about therapy should be covered earlier in EBP courses while other critical appraisal topics and quantitative decision-making techniques should be left to more advanced levels.

CONCLUSION

Experts concur that introductory EBP courses should be simple and certain topics should be avoided. Specifically, critical appraisal and statistical methods should be left to advanced courses.

Evaluating primary care doctors' evidence-based medicine skills in a busy clinical setting

RATIONALE, AIMS AND OBJECTIVES

To date, primary care doctors' (PCDs) evidence-based medicine (EBM) skills have rarely been studied. We conducted a cross-sectional study to evaluate PCDs' practical EBM skills and to determine risk markers associated with these skills.

METHODS

The study sample consisted of 70 PCDs (70.7% response rate) practising in a busy urban setting from a large health maintenance organization. Participants were given a short validated questionnaire gauging attitudes, barriers, online medical resources utilization, as well as demographic and personal characteristics. Additionally, doctors completed an online and written exam evaluating their ability to formulate clinical questions, and retrieve medical information efficiently. Data analysis was performed using both bivariate and multivariate analysis (linear regression).

RESULTS

PCDs found it difficult to formulate clinical questions both in the written and online exam, mostly neglecting to mention the Patient and Comparison components of PICO (patient, intervention, comparison and outcome). Search strategies primarily dispensed with the use of MeSH terms, ignoring appropriate limits. Doctors final scores were low (score = 41.5/100, SD = 16.2). In bivariate analysis clinical experience was negatively correlated with the final score (r = -0.36, P = 0.01), and specialists' scores were significantly higher than general practitioners' scores (46.7/100 and 31.5/100 respectively, P < 0.001). In multivariate analysis, doctors specialization was the only statistically significant predictor of the final score (B = 12.74, P = 0.002), while controlling for participating in a prior EBM course.

CONCLUSIONS

This study emphasizes the need for enhancing PCDs practical EBM skills. Future research and interventions should focus on this population emphasizing the specific needs of subpopulations (i.e. general practitioners and doctors without previous EBM training).

Evaluating the impact of an evidence-based medicine educational intervention on primary care doctors' attitudes, knowledge and clinical behaviour: a controlled trial and before and after study

RATIONALE, AIMS AND OBJECTIVES

Traditional continuing medical education programmes that offer passive learning have been shown to be poorly effective at changing doctors' clinical behaviour. A multifaceted evidence-based medicine (EBM) intervention was conducted at the largest health maintenance organization (HMO) in Israel, attempting to facilitate a change in doctors' attitudes, knowledge and clinical behaviour. No study thus far has examined the association between the teaching of EBM principles and doctors' clinical behaviour. This study evaluated the intervention programme through a controlled trial and before and after study. The objective of the evaluation is binary: first, to examine the impact of an educational intervention on family doctors' test ordering performance and drug utilization by their patients; and second, to assess the impact of the intervention on attitudes towards evidence-based practice and knowledge.

METHODS

Controlled trial and before and after study. Primary care clinics comprising similar patient characteristics were randomly allocated to the experimental or to the control group. Doctors in the experimental group participated in an EBM educational intervention, while the control group did not take part in the intervention. Clinicians' test ordering performance and their patients' drug utilization were derived from the HMO's database before intervention, after workshops and after intervention. Participants in the controlled trial consisted of 75 doctors and their 106 349 patients. The before and after study evaluated intervention doctors' (n = 70) EBM attitudes and knowledge through a validated questionnaire before and after workshops.

RESULTS

EBM workshops enhanced intervention doctors' EBM knowledge scores from 22.4/100 before workshops to 40.8/100 after workshops (P = 0.000). Doctors improved their ability to formulate clinical questions while enhancing their search strategy using Medline. In a linear regression model, two covariates, specialization (B = 12.59; P = 0.001) and habitually reading medical journals (B = 6.45; P = 0.052), best explained the variance in doctors' EBM knowledge scores, while controlling for pre-intervention scores (R(2) = 0.569; P = 0.000). Results from the controlled trial indicated that no statistically significant differences were found between intervention and control doctors' test ordering performances, and their patients' drug utilization.

CONCLUSIONS

The results of the study suggest that the intervention positively influenced attitudes and knowledge; however, no statistically significant impact was found on doctors' test ordering performance and on their patients' drug utilization. The intervention's inability to change doctors' clinical behaviour might be remedied by improving future interventions through adding additional facets to the educational intervention, such as social marketing techniques and personal feedback. A longer and more extensive intervention might be more effective but is extremely difficult to execute as we found in this study. Future larger-scale interventions must incorporate the intervention into the routines of the organization, thus minimizing barriers towards EBM implementation.

Evidence based practice in postgraduate healthcare education: a systematic review

BACKGROUND

Training in Evidence-Based Practice (EBP) has been widely implemented throughout medical school and residency curricula. The aim of this study is to systematically review studies that assessed the effectiveness of EBP teaching to improve knowledge, skills, attitudes and behavior of postgraduate healthcare workers, and to describe instruments available to evaluate EBP teaching.

METHODS

The design is a systematic review of randomized, non-randomized, and before-after studies. The data sources were MEDLINE, Cochrane Library, EMBASE, CINAHL and ERIC between 1966 and 2006. Main outcomes were knowledge, skills, attitudes and behavior towards EBP. Standardized effect sizes (E-S) were calculated. The E-S was categorized as small (E-S < 0.2), small to moderate (E-S between 0.2 and 0.5), moderate to large (E-S between 0.51 and 0.79), large (E-S > 0.79). Reliability and validity of instruments for evaluating education were assessed. Studies excluded were those that were not original, performed in medical students, focused on prescribing practices, specific health problems, theoretical reviews of different components of EBP, continuing medical education, and testing the effectiveness of implementing guidelines.

RESULTS

Twenty-four studies met our inclusion criteria. There were 15 outcomes within the 10 studies for which E-S could be calculated. The E-S ranged from 0.27 (95%CI: -0.05 to 0.59) to 1.32 (95%CI: 1.11 to 1.53). Studies assessing skills, behavior and/or attitudes had a "small to moderate" E-S. Only 1 of the 2 studies assessing knowledge had E-S of 0.57 (95 CI: 0.32 to 0.82) and 2 of the 4 studies that assessed total test score outcomes had "large" E-S. There were 22 instruments used, but only 10 had 2 or more types of validity or reliability evidence.

CONCLUSION

Small improvements in knowledge, skills, attitudes or behavior are noted when measured alone. A large improvement in skills and knowledge in EBP is noted when measured together in a total test score. Very few studies used validated measures tests.

Evidence-based practice in radiology education: why and how should we teach it?

Evidence-based practice in radiology (EBR) is becoming firmly established as a method to enhance the knowledge and fill the need for further knowledge within the profession. It has been identified as an essential part of the training program, both at medical school and at the resident and/or fellowship level. This review aims to describe the essential components of EBR, to describe the educational theories that may be applied to developing EBR within the curriculum, and to give several models that have been shown to result in greater student and resident participation in the search and exploration of evidence-based practice.

Evidence-based medicine: discrepancy between perceived competence and actual performance among graduating medical students

Since at the time of graduation from medical school physicians are expected to demonstrate adequate professional competence including mastery of critical appraisal skills, we conducted a preliminary, cross-sectional, web-based study to examine the extent to which fourth year medical students in the US are competent in core areas of evidence-based medicine (EBM). Using self-assessment instruments, subjects (n = 150) were asked to demonstrate their ability to understand the practical meaning of key methodological and data analysis constructs as they relate to patient care, to rate their perceived competence in core areas of EBM and to disclose their attitudes toward critical appraisal of the literature and EBM. The mean score in our cohort was 55% suggesting that students may have knowledge gaps that interfere with their ability to critically appraise the medical literature. There was an apparent chasm between subjects' perceived competence and their actual performance on the assessment instrument. These findings, if corroborated in larger studies, (1) suggest that better education in EBM is needed so as to avoid the possibility that patient care may inadvertently be jeopardized; and (2) cast doubt on the use of self-assessed knowledge as a proxy for actual skills with respect to EBM and medical decision-making.

Effect of a brief intervention on evidence-based medicine skills of pediatric residents

BACKGROUND

While Evidence-Based Medicine (EBM) skills are increasingly being taught in medical schools, teaching quality has been insufficient, so that incoming pediatric residents lack adequate EBM skills required for patient care. The objective of this study was to evaluate the effectiveness of a brief teaching module developed to improve EBM skills of pediatric residents.

METHODS

With-in subjects study design with pre- and post-test evaluation was performed in a large urban pediatric residency training program in Brooklyn, New York. We included PGY-1s during intern orientation, while second and third year pediatric residents were selected based on schedule availability. Sixty-nine residents were enrolled into the study, 60 (87%) completed the training. An EBM training module consisting of three or four weekly two-hour seminars was conducted. The module was designed to teach core EBM skills including (1) formulating answerable clinical questions, (2) searching the evidence, (3) critical appraisal skills including validity and applicability, and (4) understanding levels of evidence and quantitative results for therapy articles. A portion of the Fresno test of competence in EBM was used to assess EBM skills. The test presented a clinical scenario that was followed by nine short answer questions. One to three questions were used to assess EBM skills for each of the four core skills. The kappa co-efficient for inter-rater reliability was 0.74 (95% CI: 0.56-0.92).

RESULTS

Prior to the training module, the residents achieved a mean score of 17% correct overall. Post intervention, the mean score increased to 63% with improvement in each EBM category. A mean of 4.08 more questions (out of 9) were answered correctly after the training (95% CI of 3.44-4.72).

CONCLUSION

A brief training module was effective in improving EBM skills of pediatric residents.