Assessing competency in Evidence Based Practice: strengths and limitations of current tools in practice

Developing multisectoral strategies to promote evidence-based practice in rehabilitation: findings from an end-of-grant knowledge translation symposium

PURPOSE

Following a longitudinal study to understand how evidence-based practice evolves during the initial years of occupational therapy (OT) and physiotherapy (PT) practice, we held an end-of-grant symposium with representatives from education, practice, research, and policy. The objectives were to: (1) elicit feedback on the implications of the study results; and (2) co-develop a list of actionable recommendations for each sector.

METHODS

Qualitative participatory approach. The symposium was held over two half days and consisted of a presentation of study findings, a discussion on the implications of the research for each sector and future recommendations. Discussions were audio recorded, transcribed verbatim and analyzed using qualitative thematic analysis.

RESULTS

The themes related to implications of the longitudinal study included: (1) A need to rethink what evidence-based practice (EBP) really is; (2) How to practice EBP; and (3) The continuing challenge of measuring EBP. The co-development of actionable recommendations resulted in nine strategies.

CONCLUSIONS

This study highlighted how we may collectively promote EBP competencies in future OTs and PTs. We generated sector-specific avenues that may be pursued to promote EBP and argued for the importance of pooling efforts from the four sectors so that we may achieve the intended ethos of EBP.IMPLICATIONS FOR REHABILITATIONThere is a need to revisit the definition of evidence-based practice (EBP) and the traditional 3-circle model in rehabilitation to include a broader conceptualization of what constitutes evidence.We recommend using EBP measures as tools for self-reflection and professional development that can support practitioners to be reflective and accountable evidence-based practitioners.Optimal promotion of EBP competencies in occupational therapists and physiotherapists should rest upon collaborative efforts from the education, practice, research, and policy sectors.

Instruments to Assess Evidence-Based Practice Among Health Care Professionals: A Systematic Review

BACKGROUND

The use of measurement instruments to assess the use of Evidence-Based Practice by health professionals has been frequently reported in studies.

AIMS

This systematic review aimed to summarize, describe, and evaluate the measurement properties of the instruments that evaluate the use of Evidence-Based Practice in health professionals.

METHODS

The search was carried out in four databases considering three groups of search terms: evidence-based practice, evaluation, and measurement proprieties. Studies were included that described the use of instruments to assess Evidence-Based Practice in health professionals, with the full-text publication, which analyzed the measurement properties, in English. The methodological quality of the studies was evaluated using COnsensus-based Standards for the selection of health Measurement INstruments.

RESULTS

In total, 6,429 were found and only 92 were eligible for data analysis. Forty new instruments were identified most were developed for nursing and physical therapist. The investigators performed at least 1 type of validity test on 73% of the instruments. Reliability was tested at 90%, through internal consistency. Responsiveness was tested on less than half of the instruments (30%). Most of the instruments identified are reliable and valid to measure evidence-based practice in health professionals.

CONCLUSION

Although the Fresno Test remains the most complete instrument, and adequate for use. The COnsensus-based Standards for the selection of health Measurement INstruments checklist classified 7 (seven) instruments as suitable for the target audience.

Effectiveness of an Evidence-Based Practice Education Program for Undergraduate Nursing Students: A Systematic Review and Meta-Analysis

This systematic review and meta-analysis examined the impact of evidence-based practice (EBP) education programs on undergraduate nursing students, focusing on enhancing EBP competency, critical thinking, and problem-solving ability.

METHODS

The search, conducted through PubMed, Cochrane Library, EMBASE, CINAHL, and Web of Science up to December 2023, included studies published in English and Korean and adhered to PRISMA guidelines. Qualitative appraisal of the studies was conducted using the revised ROB II for randomized trials and the ROBINS-I for non-randomized trials. For the meta-analysis, the effect size of the intervention was calculated as a standardized mean difference.

RESULTS

In our study, 11 studies met our inclusion criteria, and 8 studies of those were included in the meta-analysis. The effect sizes for EBP competency, critical thinking, and problem-solving ability were 1.55, 1.29, and 0.65, respectively. The meta-regression analysis indicated that tailored education programs of 4-7 weeks and being in the 4th grade significantly enhanced EBP competency.

CONCLUSION

These findings support the development of a customizable and applied EBP education actively for students, preparing nursing students to effectively implement EBP in clinical settings after graduation. Despite the significant effect size of the outcome variables, the high heterogeneity suggests the need for further investigation to validate the EBP educational outcomes for nursing students.

Interpretation and use of a decision support tool for multiple treatment options: a combined randomised controlled trial and survey of medical students

OBJECTIVES

To investigate medical students' ability to interpret evidence, as well as their self-assessed understandability, perceived usefulness and preferences for design alternatives in an interactive decision support tool, displaying GRADE evidence summaries for multiple treatment options (Making Alternative Treatment CHoices Intuitive and Trustworthy, MATCH-IT).

DESIGN

A combined randomised controlled trial and survey. Participants were presented with a clinical scenario and randomised to one of two versions of the MATCH-IT tool (A/B), instructed to explore the evidence and decide on a recommendation. Participants answered a questionnaire assessing interpretation, treatment recommendation self-assessed understandability and perceived usefulness before exposure to the other MATCH-IT version and asked questions on design preferences.

SETTING

Online lecture in an evidence-based medicine (EBM) introductory course.

PARTICIPANTS

149 third-year medical students. 52% (n=77) had 6 months of clinical training and 48% (n=72) had preclinical training only.

INTERVENTIONS

The MATCH-IT tool version A uses colour coding to categorise interventions by magnitude and direction of effects and displays all outcomes in a table on entry. Version B has no colour coding, and the user must decide which outcomes to display in the table.

MAIN OUTCOME MEASURES

Interpretation of evidence, treatment recommendation, perceived usefulness and understandability, preference for format and design alternatives.

RESULTS

82.5% (n=123) of medical students correctly answered ≥4 out of 5 multiple choice questions assessing interpretation of data. 75.8% (n=114) of students made a treatment recommendation in accordance with an expert panel for the same clinical scenario. 87.2% (n=130) found the tool understandable while 91.9% perceived the tool as useful in addressing the clinical scenario.

CONCLUSION

Medical students with no prior training in EBM can interpret and use the MATCH-IT tool. Certain design alternatives were preferred but had no bearing on interpretation of evidence or understandability of the tool.

Evidence for commonly used teaching, learning and assessment methods in contact lens clinical skills education

INTRODUCTION

Evidence based practice is now an important part of healthcare education. The aim of this narrative literature review was to determine what evidence exists on the efficacy of commonly used teaching and learning and assessment methods in the realm of contact lens skills education (CLE) in order to provide insights into best practice. A summary of the global regulation and provision of postgraduate learning and continuing professional development in CLE is included.

METHOD

An expert panel of educators was recruited and completed a literature review of current evidence of teaching and learning and assessment methods in healthcare training, with an emphasis on health care, general optometry and CLE.

RESULTS

No direct evidence of benefit of teaching and learning and assessment methods in CLE were found. There was evidence for the benefit of some teaching and learning and assessment methods in other disciplines that could be transferable to CLE and could help students meet the intended learning outcomes. There was evidence that the following teaching and learning methods helped health-care and general optometry students meet the intended learning outcomes; clinical teaching and learning, flipped classrooms, clinical skills videos and clerkships. For assessment these methods were; essays, case presentations, objective structured clinical examinations, self-assessment and formative assessment. There was no evidence that the following teaching and learning methods helped health-care and general optometry students meet the intended learning outcomes; journal clubs and case discussions. Nor was any evidence found for the following assessment methods; multiple-choice questions, oral examinations, objective structured practical examinations, holistic assessment, and summative assessment.

CONCLUSION

Investigation into the efficacy of common teaching and learning and assessment methods in CLE are required and would be beneficial for the entire community of contact lens educators, and other disciplines that wish to adapt this approach of evidence-based teaching.

Translating and Validating the Vietnamese Version of the Health Sciences Evidence-Based Practice Questionnaire

No validated instrument is available for assessing the evidence-based practice capacity of Vietnamese health professionals. This study aimed to translate and validate the Health Sciences Evidence-Based Practice questionnaire (HS-EBP) from English to Vietnamese and ascertain its psychometric properties. Data were collected from two obstetric hospitals in Vietnam. Participants: A total of 343 midwives were randomly selected. The HS-EBP questionnaire was translated by a group of bilingual experts into Vietnamese (HS-EBP-V). Content validity was assessed by two experts. Internal consistency and test-retest reliabilities were assessed using Cronbach's α and intraclass correlation (ICC), respectively. Construct validity was assessed using the contrasted groups approach. As a result, the content validity index of the HS-EBP-V reached 1.0. For the individual subscales, Cronbach's α was 0.92-0.97 and ICC was between 0.45 and 0.66. The validity of the contrasted-groups approach showed discrimination by a significant difference in the subscale scores among diploma holders compared with bachelor's degree holders (p < 0.001). The validation of the HS-EBP questionnaire indicated satisfactory psychometric properties. The results indicate that the HS-EBP is a reliable and valid instrument which assesses the competencies of as well as facilitators of and barriers to the five steps of EBP among midwives. The HS-EBP-V was deemed a reliable and validated tool for assessing the competency and application of EBP among Vietnamese healthcare professionals.

Autonomic stress response of physiotherapy student in the different scenarios of an objective structured clinical examination

The aim of the present research was to analyse modifications in the autonomic stress response of Physiotherapy students undergoing a 12-scenario Objective Structured Clinical Evaluation (OSCE). A total of 86 last year students of the Physiotherapy bachelor's degree (27.29 years (SD = 6.66); 36 females and 50 males) randomly assigned were monitored during the complete OSCE to measure heart rate variability (HRV) in temporal, frequency, and non-linear domains. The HRV analysed showed a large anticipatory stress response of students maintained during the entire evaluation. The stress response varied regarding OSCE station complexity and demands and the highest sympathetic response was not found in higher emotional scenarios.The autonomic modulation monitoring allows teachers to design OSCE scenarios more adapted to the students, limiting the effect of the stress response to allow a better performance.

Developing the Heart Failure Specialist Nurse Competency Framework

The aim of this article is to outline the underpinning conceptual principles of advanced nursing and competency embedded in the Heart Failure Specialist Nurse Competency Framework, launched in January 2021. The authors refer to Benner's novice to expert pedagogy and explore how this provides a robust framework on which to assess the progress of the heart failure specialist nurse. Some key considerations are discussed, for example the importance of constructive feedback and feed-forward in the individual's learning cycle. Finally, plans for the future are discussed and the importance of evaluation in the authors' ongoing commitment to improving the learning experience.

Did conversion to thoracotomy during thoracoscopic lobectomy increase post-operative complications and prejudice survival? Results of best evidence topic analysis

The potential complications related to unplanned conversion to thoracotomy remains a major concern in thoracoscopic lobectomy and may limit the wide adoption of this strategy. We reviewed the literature from 1990 until February 2022, analyzing all papers comparing successful thoracoscopic lobectomy versus converted thoracoscopic lobectomy and/or upfront thoracotomy lobectomy to establish whether unplanned conversion negatively affected outcomes. Thirteen studies provided the most applicable evidence to evaluate this issue. Conversion to thoracotomy was reported to occur in up to 23% of cases (range, 5%–16%). Vascular injury, calcified lymph nodes, and dense adhesions were the most common reasons for conversion. Converted thoracoscopic lobectomy compared to successful thoracoscopic lobectomy was associated with longer operative time and hospital stay in all studies, with higher postoperative complication rates in seven studies, and with higher perioperative mortality rates in four studies. No significant differences were found between converted thoracoscopic lobectomy and upfront thoracotomy lobectomy. Five studies evaluated long‐term survival, and in all papers conversion did not prejudice survival. Surgeons should not fear unplanned conversion during thoracoscopic lobectomy, but to avoid unexpected conversion that may negatively impact surgical outcome, a careful selection of patients is recommended–especially for frail patients.

Multisite assessment of emergency medicine resident knowledge of evidence-based medicine as measured by the Fresno Test of Evidence-Based Medicine

CONTEXT

Evidence-based medicine (EBM) is the application of scientific evidence while treating a patient. To date, however, there is very little evidence describing how residents in emergency medicine understand and incorporate EBM into practice.

OBJECTIVES

The aim of this study was to determine EBM theoretical and quantitative knowledge in emergency medicine residents in community hospital-based training programs.

METHODS

A sample of emergency medicine residents from nine hospitals was enrolled to complete a cross-sectional assessment of EBM skills from April 2021 through June 2021. Performance on the Fresno Test of Evidence-Based Medicine (FTEBM) was assessed utilizing descriptive statistics, t tests, and one-way analysis of variance.

RESULTS

A total of 50.8% (124/244) of current emergency medicine residents completed the FTEBM during the study period. No significant difference on FTEBM scores was noted between the different types of medical degrees (DO vs. MD) (p=0.511), holding an advanced research degree (p=0.117), or between each postgraduate year of training (p=0.356). The mean score of those residents who rated their knowledge of EBM as average or higher was 36.0% (32.8-39.1%). The mean score of those residents who rated their programs as having an "average" or higher institutional focus on EBM was 34.9% (32.2-37.6%).

CONCLUSIONS

Participating emergency medicine residents show an incomplete understanding of EBM both in theory and applied computations despite rating themselves as having an average understanding. Emergency medicine residencies would be well suited to implement a standardized EBM curriculum that focuses on longitudinal reinforcement of key concepts needed for the practicing physician.

Evaluation of an evidence-based veterinary medicine exercise for instruction in clinical year of veterinary medicine program

INTRODUCTION

Evidence-based veterinary medicine (EBVM) is a fundamental core competency for new graduates. Our objectives were to evaluate clinical students' use, understanding of, and confidence in EBVM before undertaking an exercise consisting of an hour seminar in conjunction with a medical librarian, followed by a journal club and to evaluate students' knowledge of and ability to apply EBVM following the exercise.

METHODS

In this cohort study, students undertaking a large animal ambulatory rotation completed questionnaires before the seminar, following the journal club, and at graduation. Students took a Fresno test evaluating EBVM knowledge at the end of the rotation.

RESULTS

Prior to the seminar, 94% of students defined EBVM at least somewhat accurately, and 51% reported EBVM was at least 'quite important'. During previous rotations, 71% of students had performed literature searches to answer a clinician-assigned question; 89% had done this to answer their own questions. Students with previous research or EBVM training were more likely to perform self-directed literature searches. The most frequently used resource was textbooks. Eighty-seven per cent and 90% of students found the seminar and journal club, respectively, at least moderately useful in improving EBVM knowledge.

CONCLUSIONS

Our results support the inclusion of an EBVM exercise during the clinical year and suggest spaced repetition may be helpful in teaching this topic.

Evaluation of evidence-based practice learning among undergraduate nursing students: Relationship between self-reported and objective assessment

BACKGROUND

Self-reported scales and objective measurement tools are used to assess Evidence-based Practice (EBP) learning. An agreement between these measures has not been widely investigated among nursing students.

OBJECTIVE

The aim of the present study was to examine the relationship between EBP competency in undergraduate nursing students, defined as the capability to choose and use an integrated combination of EBP knowledge, skills and attitudes, evaluated with the self-reported EBP-COQ questionnaire and the final test score of an EBP course included in a Nursing Degree Program.

DESIGN

A cross-sectional study was carried out.

SETTINGS

University of Murcia (Spain), Nursing Degree, academic year 2019/2020.

PARTICIPANTS

The study population was comprised by 210 4th-year undergraduate nursing students enrolled in the Clinical Practice and Evidence-based Nursing course. All the students were invited to participate.

METHODS

A validated EBP-COQ questionnaire (1 "lowest score" to 5 "highest score") and an objective test (score from 0 to 10) were used to assess EBP competency. The SPSS 26.0 program was used to perform descriptive, bivariate and multivariate analyses.

RESULTS

152 students participated (response rate 72.4%), the sample was 81.4% female, and the median age was 20 years old; 76.3% attended more than 75% of the class hours. After the EBP course, the students obtained a high overall EBP competency score (mean = 4.21; SD = 0.26). The mean score on the objective test was 6.86 (SD = 1.36). A moderate positive correlation was found between the subjective and objective EBP competency measurements (Pearson's correlation coefficient 0.5; p < 0.0001).

CONCLUSIONS

Our findings contribute with new and significant evidence of the convergent validity of the EBP-COQ questionnaire, which increases the value of this instrument for evaluating the EBP competency of Nursing Degree students.

Good practices in health promotion for older people - Significance for evidence in health policy

This article is devoted to convincing policy makers to use good practices in encouraging older people to pursue adequate and effective health policies. Long-term scientific research focused on the effects of health promotion programmes is rarely undertaken, although its scope is still expanding. At the same time, it is strongly desirable to form health policy based on scientific evidence. In this situation, an indication of good practices characterised by precisely defined features and their systematic evaluation could be an alternative to an insufficient number of empirical studies. The first step of the methodology was a literature review on health promotion for older people, aimed at defining good practices and criteria used for their selection. The authors searched the following databases: PubMED, Embase and Cochrane Library, as well as international databases dedicated to health promotion programmes for older people (e.g. Age-friendly World (https://extranet.who.int/agefriendlyworld/age-friendly-practice-database-launched); HealthProElderly (www.healthproelderly.com/database/index.php?id=16); JA-CHRODIS (www.chrodis.eu); EuroHealthNet (www.eurohealthnet.eu) and ProFouND; (www.profound.eu.com). As relevant health policy information is usually available in national languages, the authors then approached national experts in 10 European countries, who filled in a dedicated survey on health promotion programmes for older people and indicated examples of good practices from their countries. Practical evidence, based on real implemented programmes, is valuable as inspiration for health promotion programmes, their planning and management. Selecting good practices from among implemented and evaluated actions makes it possible to establish their value. The significance of good practices in health promotion is to deliver real benefits and health effects for a target group, which, in the case of evident benefits, renders the practices credible and worthy of further dissemination. The EU already successfully shares good practices in migrant health and environmental protection. Creating databases on good practices helps policy makers promote the sustainability of already implemented activities and enhances their applicability by other organisations and in different settings.

Is video-assisted thoracoscopic lobectomy associated with higher overall costs compared with open surgery? Results of best evidence topic analysis

Thoracoscopic lobectomy has become the preferred approach for surgical management of early stage lung cancer, but the potential higher operative costs limit its widespread use. Theoretically, higher direct costs may be significantly counterbalanced by lower indirect costs, resulting in lower overall costs for thoracoscopic than for open lobectomy. To support this hypothesis, we reviewed the literature until May 2020, analyzing all papers comparing the cost of thoracoscopic versus open lobectomy.A total of 20 studies provided the most applicable evidence to evaluate this issue. In all the studies apart from one, thoracoscopic lobectomy was associated with higher operative costs due to the increased use of disposable instruments, and prolonged operative time. By contrast, in 17 studies the increased operative costs were significantly offset by indirect costs which were lower in thoracoscopic than in open lobectomy due to fewer postoperative complications, faster recovery, and lower readmission rates. It translated into lower overall costs for thoracoscopic than for open lobectomy in 10 studies, similar costs in seven, and higher in three, despite the lower hospitalization costs. The low bed fees and high prices of disposable instruments in these three studies may explain the discordance. The careful use of disposable instruments, and the minimizing hospitalization costs can reduce the total costs of thoracoscopic lobectomy to levels similar or to below those of open lobectomy. The worry that video‐assisted thoracoscopic surgery lobectomy (VATSL) might be associated with an increased overal cost is thus not warranted, and should not be used as an excuse against the use of VATS in surgery for early stage lung cancers.

Evidence-Based Practice Competency and Related Factors Among Nurses Working in Public Hospitals

This study aimed to describe the status and related factors of evidence-based practice (EBP) competency among nurses in China. A convenience sample of 1036 nurses were recruited from 13 public hospitals in Shanghai through an anonymous online survey. Nurses’ demographic data, working data, and evidence-based nursing (EBN)-related conditions were collected by a structured questionnaire and EBP competency was assessed using the Chinese version of Evidence-based Nursing Competency Rating Scale (EBNCRS). The original score of EBNCRS among nurses was not good. The original score of evidence searching, critical appraisal, and synthesis subscale was lower than the original score of evidence transfer, situation assessment, and evidence implementation subscale. Nurses’ age, hospital level, and perceived EBN knowledge were identified as significant related factors of EBP competency. There is an urgent need to upgrade the EBP competency of nurses in Shanghai, especially competency in evidence transfer, situation assessment, and evidence implementation.

Development and evaluation of an evidence-based medicine module in the undergraduate medical curriculum

BACKGROUND

Evidence-based medicine (EBM) is a core competence in both undergraduate and postgraduate medical curricula. However, its integration into curricula varies widely. Our study will help medical colleges develop, implement and evaluate their EBM courses. We assessed the effectiveness of workshops in improving critical appraisal skills among medical students.

METHODS

A before-and-after study design without a control group was used. A 5-week short EBM module including lectures, workshops, and online search sessions was conducted with 52 fourth-year medical students during their primary healthcare course at the College of Medicine, Princess Nourah bint Abdulrahman University. Statistical analysis was performed using SPSS statistical software (version 20, SPSS Inc., Chicago, US). Parametric tests as well as Student's paired t-test for pre- and post-test comparisons were used.

RESULTS

Forty-nine (49) participants completed the pre- and post-training Fresno tests, and 44.9% of the participants had a GPA of 4.0 or higher. The mean Fresno test score increased from 45.63 (SD 21.89) on the pre-test to 64.49 (SD 33.31) on the post-test, with significant improvements in the following items: search strategies, relevance, internal validity, magnitude and significance of results, statistical values of diagnosis studies (sensitivity, specificity, and LR), statistical values of therapy studies (ARR, RRR, and NNT), and best study design for diagnosis and prognosis (P < 0.05).

CONCLUSION

This study supports that a short course in EBM that is incorporated into the undergraduate curriculum, especially in the clinical years, might be effective in improving medical students' knowledge and skills in EBM. However, prospective studies are necessary to assess the long-term impact of these interventions and ultimately their effectiveness for clinical decision making.

Development and Validation of a Test for Competence in Evidence-Based Medicine

BACKGROUND

Medical educators need valid, reliable, and efficient tools to assess evidence-based medicine (EBM) knowledge and skills. Available EBM assessment tools either do not assess skills or are laborious to grade.

OBJECTIVE

To validate a multiple-choice-based EBM test-the Resident EBM Skills Evaluation Tool (RESET).

DESIGN

Cross-sectional study.

PARTICIPANTS

A total of 304 medicine residents from five training programs and 33 EBM experts comprised the validation cohort.

MAIN MEASURES

Internal reliability, item difficulty, and item discrimination were assessed. Construct validity was assessed by comparing mean total scores of trainees to experts. Experts were also asked to rate importance of each test item to assess content validity.

KEY RESULTS

Experts had higher total scores than trainees (35.6 vs. 29.4, P < 0.001) and also scored significantly higher than residents on 11/18 items. Cronbach's alpha was 0.6 (acceptable), and no items had a low item-total correlation. Item difficulty ranged from 7 to 86%. All items were deemed "important" by > 50% of experts.

CONCLUSIONS

The proposed EBM assessment tool is a reliable and valid instrument to assess competence in EBM. It is easy to administer and grade and could be used to guide and assess interventions in EBM education.

A systematic review and taxonomy of tools for evaluating evidence-based medicine teaching in medical education

BACKGROUND

The importance of teaching the skills and practice of evidence-based medicine (EBM) for medical professionals has steadily grown in recent years. Alongside this growth is a need to evaluate the effectiveness of EBM curriculum as assessed by competency in the five 'A's': asking, acquiring, appraising, applying and assessing (impact and performance). EBM educators in medical education will benefit from a compendium of existing assessment tools for assessing EBM competencies in their settings. The purpose of this review is to provide a systematic review and taxonomy of validated tools that evaluate EBM teaching in medical education.

METHODS

We searched MEDLINE, EMBASE, Cochrane library, Educational Resources Information Centre (ERIC), Best Evidence Medical Education (BEME) databases and references of retrieved articles published between January 2005 and March 2019. We have presented the identified tools along with their psychometric properties including validity, reliability and relevance to the five domains of EBM practice and dimensions of EBM learning. We also assessed the quality of the tools to identify high quality tools as those supported by established interrater reliability (if applicable), objective (non-self-reported) outcome measures and achieved ≥ 3 types of established validity evidence. We have reported our study in accordance with the PRISMA guidelines.

RESULTS

We identified 1719 potentially relevant articles of which 63 full text articles were assessed for eligibility against inclusion and exclusion criteria. Twelve articles each with a unique and newly identified tool were included in the final analysis. Of the twelve tools, all of them assessed the third step of EBM practice (appraise) and four assessed just that one step. None of the twelve tools assessed the last step of EBM practice (assess). Of the seven domains of EBM learning, ten tools assessed knowledge gain, nine assessed skills and-one assessed attitude. None addressed reaction to EBM teaching, self-efficacy, behaviours or patient benefit. Of the twelve tools identified, six were high quality. We have also provided a taxonomy of tools using the CREATE framework, for EBM teachers in medical education.

CONCLUSIONS

Six tools of reasonable validity are available for evaluating most steps of EBM and some domains of EBM learning. Further development and validation of tools that evaluate all the steps in EBM and all educational outcome domains are needed.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42018116203.

The IASP pain curriculum for undergraduate allied health professionals: educators defining competence level using Dublin descriptors

BACKGROUND

Improving pain education for undergraduate health professionals is hampered by lacking shared education outcomes. This study describes how educators and pain experts operationalize content and competency levels deemed necessary for a undergraduate pain education core curriculum for health professionals (physical and occupational therapists, nurses, psychologists).

METHODS

Educators and experts on pain and pain education gave their opinion on content and competency level for each individual item of the International Association for the Study of Pain (IASP) inter professional curriculum. Participants decided whether or not to include each item in the undergraduate curriculum. Items were included when > 70% of the respondents agreed. The required competency for each item was rated using ordinal Dublin Descriptors.

RESULTS

Overall, 22 experts rated the curriculum, with > 70% agreement on inclusion on 62% of the IASP items. Within the IASP domain 'Multidimensional nature of pain' there was full agreement on the inclusion of 12 items. 'Ethics' was considered less important with only 1 item deemed necessary. There is a high number of items selected within the domains 'Pain Assessment and measurement' (78%) and 'Management of Pain' (74%). Considerably less items were chosen in the domain 'Clinical Conditions' (41%). For most items the median required skills and competency level was either Knowledge and Understanding, or Applying Knowledge and Understanding.

CONCLUSION

Overall, educators and experts in pain agreed on content and competency levels for an undergraduate pain curriculum based on the IASP. Defining a shared competency level will help improve definition of education outcome.

A Multicountry Study on Nursing Students' Self-Perceived Competence and Barriers to Evidence-Based Practice

BACKGROUND

Nursing education and training are essential in the attainment of evidence-based practice (EBP) competence in nursing students. Although there is a growing literature on EBP among nursing students, most of these studies are confined to a single cultural group. Thus, cross-cultural studies may provide shared global perspectives and theoretical understandings for the advancement of knowledge in this critical area.

AIMS

This study compared self-perceived EBP competence among nursing students in four selected countries (India, Saudi Arabia, Nigeria, and Oman) as well as perceived barriers to EBP adoption.

METHODS

A descriptive, cross-sectional, and comparative survey of 1,383 nursing students from India, Saudi Arabia, Nigeria, and Oman participated in the study. The Evidence-Based Practice Questionnaire (EBP-COQ) and the BARRIERS scale were used to collect data during the months of January 2016 to August 2017.

RESULTS

Cross-country comparisons revealed significant differences in EBP competence (F = 24.437, p < .001), knowledge (F = 3.621, p = .013), skills (F = 9.527, p < .001), and attitudes (F = 74.412, p < .001) among nursing students. Three variables including nursing students' gender (β = .301, p < .001), type of institution, (β = -0.339, p = .001), and type of nursing student (β = .321, p < .001) were associated with EBP competence. Barriers to EBP adoption included having no authority to change patient care policies (M = 1.65, SD = 1.05), slow publication of evidence (M = 1.59, SD = 1.01), and paucity of time in the clinical area to implement the evidence (M = 1.59, SD = 1.05).

LINKING EVIDENCE TO ACTION

Both academe and hospital administration can play a pivotal role in the successful acquisition of EBP competence in nursing students.

Student pharmacists' performance and perceptions on an evidence-based medicine objective structured clinical examination

BACKGROUND AND PURPOSE

Studies have examined evidence-based medicine (EBM) focused objective structured clinical examinations (OSCEs) in medical training, but data are lacking in pharmacy trainees. This study sought to assess student pharmacists' performance on and perceptions of a novel EBM OSCE.

EDUCATIONAL ACTIVITY AND SETTING

This EBM OSCE included answering a drug-information inquiry, researching background questions, calling a simulated provider to acquire specific patient information, developing a foreground clinical question, reviewing pre-appraised trial synopses, and applying evidence to write a recommendation. Pharmacy faculty served as simulated providers and assessed students on knowledge/analytical (AC) and global communication (GC) skills. Students completed a worksheet (WS) that included developing a patient, intervention, comparison, outcome (PICO) statement, trial selection, and clinical recommendation. After OSCE completion, students were surveyed regarding perceptions of their performance and OSCE applicability. Outcomes assessed were performance scores (AC, GC, WS) and student perceptions.

FINDINGS

One-hundred twenty-nine students completed the survey and were included in analysis. AC, WS, and GC performance [median (IQR)] were 75.0 (37.8), 86.4 (36.9), and 88.9 (22.2), respectively, on a 100-point scale. On the WS, 89% of students developed a suitable searchable clinical question and 61% selected the correct trial synopsis to apply to the case. Students felt literature application and WS development were most challenging. A majority of students felt this OSCE increased comfort in engaging with providers (74%) and that these skills correlate with real clinical scenarios (77%).

SUMMARY

OSCEs can be a valuable tool for simulating clinical scenarios and assessing student pharmacists' EBM skills.

Development and pilot testing of a tool to assess evidence-based practice skills among French general practitioners

BACKGROUND

There is currently an absence of valid and relevant instruments to evaluate how Evidence-based Practice (EBP) training improves, beyond knowledge, physicians' skills. Our aim was to develop and test a tool to assess physicians' EBP skills.

METHODS

The tool we developed includes four parts to assess the necessary skills for applying EBP steps: clinical question formulation; literature search; critical appraisal of literature; synthesis and decision making. We evaluated content and face validity, then tested applicability of the tool and whether external observers could reliably use it to assess acquired skills. We estimated Kappa coefficients to measure concordance between raters.

RESULTS

Twelve general practice (GP) residents and eleven GP teachers from the University of Bordeaux, France, were asked to: formulate four clinical questions (diagnostic, prognosis, treatment, and aetiology) from a proposed clinical vignette, find articles or guidelines to answer four relevant provided questions, analyse an original article answering one of these questions, synthesize knowledge from provided synopses, and decide about the four clinical questions. Concordance between two external raters was excellent for their assessment of participants' appraisal of the significance of article results (K = 0.83), and good for assessment of the formulation of a diagnostic question (K = 0.76), PubMed/Medline (K = 0.71) or guideline (K = 0.67) search, and of appraisal of methodological validity of articles (K = 0.68).

CONCLUSIONS

Our tool allows an in-depth analysis of EBP skills, thus could supplement existing instruments focused on knowledge or specific EBP step. The actual usefulness of such tools to improve care and population health remains to be evaluated.

Korean Nursing Students' Acquisition of Evidence-Based Practice and Critical Thinking Skills

BACKGROUND

Evidence-based practice (EBP) is essential for enhancing nurses' quality of care. We identified Korean nursing students' practices, attitudes, and knowledge concerning EBP, as well as their critical thinking disposition (CTD).

METHOD

The EBP Questionnaire (EBPQ) was administered to a convenience sample of 266 nursing students recruited from four nursing schools in Seoul and its metropolitan area.

RESULTS

Average EBPQ and CTD total scores were 4.69 ± 0.64 and 3.56 ± 0.32, respectively. Students who were ages ⩾23 years, male, and satisfied with their major demonstrated higher EBPQ and CTD scores. EBPQ scores were significantly correlated with CTD scores (r = .459, p < .01), and CTD was an explanatory factor of EBP (adjusted R² = 0.200).

CONCLUSION

It is necessary to develop comprehensive teaching strategies to help nursing students improve their CTD and information utilization skills, as well as integrate EBP in undergraduate programs to enhance nurses' EBP abilities. [J Nurs Educ. 2018;57(1):21-27.].

Psychometric Evaluation of the 'Evidence Based Practice Competencies Questionnaire - Cerebral Palsy'

AIM

To understand whether knowledge translation activities are effective, good measurement of practice is required. This study investigated the psychometric properties of a self-report measure of allied health practitioner (AHP) evidence-based behaviors when working with children with cerebral palsy.

METHODS

Construct validity and reliability studies were undertaken for the 12-item Evidence Based Practice Competency Questionnaire - Cerebral Palsy (EBP-CQ-CP) using the Consensus-based Standards of Measurement Instruments methods. Factor analysis tested construct validity. Weighted Kappa tested chance-corrected agreement for each item and intra-class correlation coefficients (ICC) the reliability of factors derived in the validity study.

RESULTS

In the validity study 259 AHP completed the EBP-CQ-CP on occasion 1, and 228 on occasion 2. In the reliability study 46 AHP completed the questionnaire twice. Exploratory factor analysis determined the EBP-CQ-CP contained two scales: 'communicating evidence based expectations' and 'evidence based assessment practices'. Confirmatory factor analysis using data from the second occasion of assessment supported the findings. Excellent consistency in ratings across factor scores were obtained from 46 pairs of raters: Factor 1, ICC = 0.93 (95% Confidence Interval 0.88-0.96); Factor 2, ICC = 0.94 (95% Confidence Interval 0.88-0.97).

CONCLUSIONS

This study supports the interpretation of the EBP-CQ-CP in a clinically meaningful and psychometrically robust manner.

Evidence-based dentistry skill acquisition by second-year dental students

INTRODUCTION

Identification and assessment of Evidence-based dentistry (EBD) outcomes have been elusive. Our objective was to describe EBD skill acquisition during the second (D2) year of pre-doctoral dental education and student competency at the end of the year.

METHODS

The first and fourth (final) curricular-required EBD Exercises (ie, application of the first 4 steps of the 5-Step evidence-based practice process applied to a real or hypothetical situation) completed by D2 students (n = 151) during 2014-2015 and 2015-2016 were evaluated to measure skill acquisition through use of a novel rubric with measures of performance from novice to expert. Exercises were evaluated on the performance for each step, identification of manuscript details and reflective commentary on manuscript components. Changes in performance were evaluated using the chi-square test for trend and the Wilcoxon signed-rank test.

RESULTS

Seventy-eight per cent of students scored competent or higher on the Ask step at the beginning of the D2 year; scores improved with 58% scoring proficient or expert on the fourth Exercise (P < .001). Most students were advanced beginners or higher in the Acquire, Appraise and Apply steps at the beginning of the D2 year, with minimal growth observed during the year. Identification of manuscript details improved between the first and fourth Exercises (P = .015); however, depth of commentary skills did not change.

DISCUSSION

Unlike previous investigations evaluating EBD knowledge or behaviour in a testing situation, we evaluated skill acquisition using applied Exercises.

CONCLUSION

Consistent with their clinical and scientific maturity, D2 students minimally performed as advanced beginners at the end of their D2 year.

Healthcare professionals' behavior, skills, knowledge and attitudes on evidence-based health practice: a protocol of cross-sectional study

BACKGROUND

In Brazil, as in most countries nowadays, there is a pursuit for healthcare quality improvement and sustainability in public and private systems. Healthcare professionals' perceptions, knowledge and attitudes determine evidence-based practice (EBP), which remain uncertain among Brazilian practitioners. A standardised national instrument whose wide use will identify gaps and flaws in establishing an EBP could contribute to an effective resources allocation from health professionals willing to use an EBP.

OBJECTIVES

To present a study protocol on the development and validation of an instrument to measure Brazilian healthcare professionals' behaviour, skills, self-efficacy, knowledge and attitudes towards EBP.

METHODS

This is a validation study with Brazilian healthcare professionals to develop a valid and reliable questionnaire, including selection of domains and formulation of questions. Construct and content validity will be assess by a panel of experts, with data collection and analysis following a Delphi-like methodology. Further, a pilot survey will be accomplished with a representative sample of different healthcare professionals from all main Brazilian regions. An exploratory factor analysis and a confirmatory factor analysis will be conducted afterwards. The ratio of χ² and df (χ²/df), comparative fit index, goodness of fit index and root mean square error of approximation will be used for assessing the model fit. In addition, the reliability of the instrument will be estimated by test-retest reproducibility and Cronbach's alpha coefficient (α).

ETHICS AND DISSEMINATION

This study has received ethical approval from the Pharmaceutical Sciences Faculty of the São Paulo State University (1.425.808). The use among a wide national sample is expected to promote an extensive view of evidence-based decision-making, identifying the knowledge gaps in this area. Study findings will be circulated to healthcare professionals and scientists in the field through the publication in peer-reviewed journals and conference presentations.

An approach on defining competency in evidence-based dentistry

INTRODUCTION

Assessment of evidence-based dentistry (EBD) knowledge and behaviour is hampered by lack of explicit competency criteria. This void impedes instructional design and assessment of student growth during the educational process.

METHODS

Knowledge and cognitive domains supporting educational objectives in a pre-doctoral dental programme were identified for each level of the EBD five-step process. We articulated educational objectives with behavioural expectations for each level of skill acquisition at each step of the EBD process. Outcome evaluation criteria identify students' progressive level of skill acquisition from novice to expert.

RESULTS

The educational objectives, type of knowledge, and nature of the cognitive process supporting these objectives are presented for each step of the EBD process. For example, educational objectives of the "Ask" step include (i) to construct a question from the patient presentation and knowledge limitations that addresses the clinical problem and (ii) to articulate the Problem, Intervention/Exposure, Comparison, Outcome (PICO) components. Achievement of these objectives requires both factual information regarding the PICO format and the cognitive process of understanding. Educational outcome criteria consistent with a competent clinician include clear articulation of the PICO with identifiable pieces that relate to the clinical situation.

DISCUSSION

Assessment strategies for progression towards EBD competency are limited due to the complexity associated with evaluating EBD knowledge and behaviours. To evaluate performance, the EBD academic community must define competency expectations for entry into unsupervised general dental practice.

CONCLUSION

This framework offers measurable outcome evaluation criteria to initiate a conversation with academic peers regarding current gaps in EBD assessment.

Assessing the evidence based medicine educational needs of community pharmacy preceptors within an experiential program in Qatar

INTRODUCTION

Observations have shown a lack of evidence-based medicine (EBM) knowledge and skills among community pharmacy preceptors in Qatar. These skills are important when delivering evidence-based recommendations to healthcare providers and patients. The aim of this study was to explore the community pharmacy preceptors' knowledge and understanding of EBM and to identify challenges in applying this approach to community practice.

METHODS

A survey instrument was developed to assess the knowledge of community pharmacy preceptors related to EBM. Knowledge-based questions were mapped according to National Association of Pharmacy Regulatory Authorities professional competencies for practicing pharmacists. The survey tool was administered to community pharmacy preceptors involved in mentoring Qatar University College of Pharmacy students.

RESULTS

Twenty-six community pharmacy preceptors completed the survey with a response rate of 65%. The knowledge-based questions linked to drug information resulted in 18%, 36%, and 46% of correct, incorrect, and unsure responses respectively while the critical appraisal questions showed 18%, 27%, 55% of correct, incorrect, and unsure responses respectively. Major barriers to practicing EBM included lack of EBM training (81%) and interpreting study results (80%).

CONCLUSIONS

Community pharmacy preceptors showed poor knowledge and understanding of EBM. These results and expressed needs by pharmacists support the development of a supplementary course to improve EBM skills.

Continuing Professional Development via Social Media or Conference Attendance: A Cost Analysis

BACKGROUND

Professional development is essential in the health disciplines. Knowing the cost and value of educational approaches informs decisions and choices about learning and teaching practices.

OBJECTIVE

The primary aim of this study was to conduct a cost analysis of participation in continuing professional development via social media compared with live conference attendance.

METHODS

Clinicians interested in musculoskeletal care were invited to participate in the study activities. Quantitative data were obtained from an anonymous electronic questionnaire.

RESULTS

Of the 272 individuals invited to contribute data to this study, 150 clinicians predominantly from Australia, United States, United Kingdom, India, and Malaysia completed the outcome measures. Half of the respondents (78/150, 52.0%) believed that they would learn more with the live conference format. The median perceived participation costs for the live conference format was Aus $1596 (interquartile range, IQR 172.50-2852.00). The perceived cost of participation for equivalent content delivered via social media was Aus $15 (IQR 0.00-58.50). The majority of the clinicians (114/146, 78.1%, missing data n=4) indicated that they would pay for a subscription-based service, delivered by social media, to the median value of Aus $59.50.

CONCLUSIONS

Social media platforms are evolving into an acceptable and financially sustainable medium for the continued professional development of health professionals. When factoring in the reduced costs of participation and the reduced loss of employable hours from the perspective of the health service, professional development via social media has unique strengths that challenge the traditional live conference delivery format.

Evidence-based medicine teaching requirements in the USA: taxonomy and themes

BACKGROUND

In the USA, recent changes to oversight of residency training codify the requirements for teaching evidence-based medicine and information mastery (lifelong learning) knowledge, skills, and attitudes.

OBJECTIVE

The goal of this project is to determine current requirements for teaching of evidence-based medicine (EBM) and information mastery (IM) in specialty residency education in the USA.

METHODS

The project was a qualitative thematic analysis using content analysis. The source of the requirements for EBM and IM were the "milestone" statements for all 28 major specialties and transition year programs. Milestone descriptors related to EBM/IM were extracted and codes were developed and applied to each descriptor by four researchers. The resulting codes were coalesced into themes and tested against the milestone descriptors.

RESULTS

The coding process identified 15 content areas comprising five themes. Two themes related to the knowledge and skills of EBM and three themes related to the knowledge, skills, and attitudes of IM. EBM themes encompassed basic critical appraisal skills and knowledge of clinical epidemiology principles and statistics. IM themes centered on identifying one's information needs for patient-specific information, using information sources, and using current awareness services to remain abreast of changes in medicine. In general, they align well with the Sicily Statement on Evidence-Based Practice. No specialty required competence in all areas.

CONCLUSION

New training requirements for specialties in the USA require the development of both classic EBM skills as well as skills for managing information. However, there is marked variation in the requirements among specific specialties.

Refining the Journal Club Presentations of Postgraduate Students in Seven Clinical Departments for Better Evidence-based Practice

BACKGROUND

A gap between best practice and actual clinical care exists and this can be overcome by evidence-based practice (EBP), which is essential to improve the clinical decision making. A strategy to reduce deficits in care provision is to train the postgraduate students in the practice of EBP in the journal clubs as evidence from medical colleges in India reveals that current format of journal club presentations is unsatisfactory.

AIM

The aim of the present study was to refine the journal club presentations of postgraduate students of clinical departments and to study the effectiveness of EBP training in them for better EBP.

SUBJECTS AND METHODS

This study was conducted in S. Nijalingappa Medical College, Bagalkot, Karnataka, India, and it was a pre- and post-trial. This study was a pre- and post-trial done during the journal club presentations of postgraduate students from clinical departments. Postgraduate students' understanding of concepts about EBP was assessed using Fresno test questionnaire in traditional journal club presentation. A hands-on session incorporating steps of EBP was imparted to them. Soon after the session, each student was assessed. In the next journal club presentation, 1 week later, the students were assessed again with the same questionnaire by the same faculty. Scores of the postgraduate students, before and after intervention (immediate and 1 week later), were compared. Data were analyzed by paired t-test using SPSS.

RESULTS

An increase in mean posttest scores was seen immediately and also 1 week later as compared to the pretest scores. The scores also increased significantly, when each step of EBP was considered.

CONCLUSIONS

Incorporating teaching of EBP in journal club presentations improved the competencies of postgraduate students in clinical decision making.

Portfolio as a tool to evaluate clinical competences of traumatology in medical students

This article investigates whether a reflexive portfolio is instrumental in determining the level of acquisition of clinical competences in traumatology, a subject in the 5th year of the degree of medicine. A total of 131 students used the portfolio during their clinical rotation of traumatology. The students' portfolios were blind evaluated by four professors who annotated the existence (yes/no) of 23 learning outcomes. The reliability of the portfolio was moderate, according to the kappa index (0.48), but the evaluation scores between evaluators were very similar. Considering the mean percentage, 59.8% of the students obtained all the competences established and only 13 of the 23 learning outcomes (56.5%) were fulfilled by >50% of the students. Our study suggests that the portfolio may be an important tool to quantitatively analyze the acquisition of traumatology competences of medical students, thus allowing the implementation of methods to improve its teaching.

Creation of a Tool for Assessing Knowledge in Evidence-Based Decision-Making in Practicing Health Care Providers

INTRODUCTION

Evidence-based decision-making (EBDM) is the process health care providers (HCPs) use to identify and appraise potential evidence. It supports the integration of best research evidence with clinical expertise and patient values into the decision-making process for patient care. Competence in this process is essential to delivery of optimal care. There is no objective tool that assesses EBDM across HCP groups. This research aimed to develop a content valid tool to assess knowledge of the principles of evidence-based medicine and the EBDM process, for use with all HCPs.

METHODS

A Delphi process was used in the creation of the tool. Pilot testing established its content validity with the added benefit of evaluating HCPs' knowledge of EBDM. Descriptive statistics and multivariate mixed models were used to evaluate individual survey responses in total, as well as within each EBDM component.

RESULTS

The tool consisted of 26 multiple-choice questions. A total of 12,884 HCPs in Nova Scotia were invited to participate in the web-based validation study, yielding 818 (6.3%) participants, 471 of whom completed all questions. The mean overall score was 68%. Knowledge in one component, integration of evidence with clinical expertise and patient preferences, was identified as needing development across all HCPs surveyed.

DISCUSSION

A content valid tool for assessing HCP EBDM knowledge was created and can be used to support the development of continuing education programs to enhance EBDM competency.

Cross-cultural adaptation and reproducibility of the Brazilian-Portuguese version of the modified FRESNO Test to evaluate the competence in evidence based practice by physical therapists

BACKGROUND

The Modified Fresno Test was developed to assess knowledge and skills of both physical therapy (PT) professionals and students to use evidence-based practice (EBP).

OBJECTIVES

To translate the Modified Fresno Test into Brazilian-Portuguese and to evaluate the test's reproducibility.

METHOD

The first step consisted of adapting the instrument into the Brazilian-Portuguese language. Then, a total of 57 participants, including PT students, PT professors and PT practitioners, completed the translated instrument. The responses from the participants were used to evaluate reproducibility of the translated instrument. Internal consistency was calculated using the Cronbach's alpha. Reliability was calculated using the intraclass correlation coefficient (ICC) for continuous variables, and the Kappa coefficient (K) for categorical variables. The agreement was assessed using the standard error of the measurement (SEM).

RESULTS

The cross-cultural adaptation process was appropriate, providing an adequate Brazilian-Portuguese version of the instrument. The internal consistency was good (α=0.769). The reliability for inter- and intra-rater assessment were ICC=0.89 (95% CI 0.82 to 0.93); for evaluator 1 was ICC=0.85 (95% CI 0.57 to 0.93); and for evaluator 2 was ICC=0.98 (95% CI 0.97 to 0.99). The SEM was 13.04 points for inter-rater assessment, 12.57 points for rater 1 and 4.59 points for rater 2.

CONCLUSION

The Brazilian-Portuguese language version of the Modified Fresno Test showed satisfactory results in terms of reproducibility. The Modified Fresno Test will allow physical therapy professionals and students to be evaluated on the use of understanding EBP.

Measuring evidence-based practice knowledge and skills in occupational therapy--a brief instrument

BACKGROUND

Valid and reliable instruments are required to measure the effect of educational interventions to improve evidence-based practice (EBP) knowledge and skills in occupational therapy. The aims of this paper are to: 1) describe amendments to the Adapted Fresno Test of Competence in EBP (AFT), and 2) report the psychometric properties of the modified instrument when used with South African occupational therapists.

METHODS

The clinical utility of the AFT was evaluated for use with South African occupational therapists and modifications made. The modified AFT was used in two studies to assess its reliability and validity. In Study 1 a convenience sample of 26 occupational therapists in private practice or government-funded health facilities in a South African province were recruited to complete the modified AFT on two occasions 1 week apart. Completed questionnaires were scored independently by two raters. Inter-rater, test-retest reliability and internal consistency were determined. Study 2 was a pragmatic randomised controlled trial involving occupational therapists in four Western Cape Department of Health district municipalities (n = 58). Therapists were randomised in matched pairs to one of two educational interventions (interactive or didactic), and completed the modified AFT at baseline and 12 weeks after the intervention. An intention-to-treat analysis was performed. Data were not normally distributed, thus non-parametric statistics were used.

RESULTS

In Study 1, 21 of 26 participants completed the questionnaire twice. Test-retest (ICC = 0.95, 95 % CI = 0.88-0.98) and inter-rater reliability (Time 1: ICC = 0.995, 95 % CI = 0.99-0.998; Time 2: ICC = 0.99, 95 % CI = 0.97-0.995) were excellent for total scores. Internal consistency based on time 1 scores was satisfactory (α = 0.70). In Study 2, 28 participants received an interactive educational intervention and completed the modified AFT at baseline and 12 weeks later. Median total SAFT scores increased significantly from baseline to 12-weeks (Z = -4.078, p < 0.001) with a moderate effect size (r = 0.55).

CONCLUSION

The modified AFT has demonstrated validity for detecting differences in EBP knowledge between two groups. It also has excellent test-retest and inter-rater reliability. The instrument is recommended for contexts where EBP is an emerging approach and time is at a premium.

TRIAL REGISTRATION

Pan African Controlled Trials Register PACTR201201000346141 . Registered 31 January 2012. Clinical Trials NCT01512823 . Registered 1 February 2012. South African National Clinical Trial Register DOH2710093067 . Registered 27 October 2009.

A randomised controlled trial of a blended learning education intervention for teaching evidence-based medicine

BACKGROUND

Few studies have been performed to inform how best to teach evidence-based medicine (EBM) to medical trainees. Current evidence can only conclude that any form of teaching increases EBM competency, but cannot distinguish which form of teaching is most effective at increasing student competency in EBM. This study compared the effectiveness of a blended learning (BL) versus didactic learning (DL) approach of teaching EBM to medical students with respect to competency, self-efficacy, attitudes and behaviour toward EBM.

METHODS

A mixed methods study consisting of a randomised controlled trial (RCT) and qualitative case study was performed with medical students undertaking their first clinical year of training in EBM. Students were randomly assigned to receive EBM teaching via either a BL approach or the incumbent DL approach. Competency in EBM was assessed using the Berlin questionnaire and the 'Assessing Competency in EBM' (ACE) tool. Students' self-efficacy, attitudes and behaviour was also assessed. A series of focus groups was also performed to contextualise the quantitative results.

RESULTS

A total of 147 students completed the RCT, and a further 29 students participated in six focus group discussions. Students who received the BL approach to teaching EBM had significantly higher scores in 5 out of 6 behaviour domains, 3 out of 4 attitude domains and 10 out of 14 self-efficacy domains. Competency in EBM did not differ significantly between students receiving the BL approach versus those receiving the DL approach [Mean Difference (MD)=-0.68, (95% CI-1.71, 0.34), p=0.19]. No significant difference was observed between sites (p=0.89) or by student type (p=0.58). Focus group discussions suggested a strong student preference for teaching using a BL approach, which integrates lectures, online learning and small group activities.

CONCLUSIONS

BL is no more effective than DL at increasing medical students' knowledge and skills in EBM, but was significantly more effective at increasing student attitudes toward EBM and self-reported use of EBM in clinical practice. Given the various learning styles preferred by students, a multifaceted approach (incorporating BL) may be best suited when teaching EBM to medical students. Further research on the cost-effectiveness of EBM teaching modalities is required.

Graduate nursing students' evaluation of EBP courses: a cross-sectional study

BACKGROUND

There is a lack of appropriate tools for assessing the effectiveness of teaching evidence-based practice in nursing.

OBJECTIVES

The objective of the study was to develop the instrument evaluating the students' perception of the effectiveness of EBP courses and to verify its psychometric properties.

DESIGN

A descriptive cross-sectional study design was used to verify psychometric properties of the questionnaire measuring the students' perception of the effectiveness of EBP courses.

PARTICIPANTS

The psychometric properties were evaluated in a group of 129 graduate nursing students who completed EBP courses.

METHODS

The instrument for measuring the students' perception of the effectiveness of EBP courses was inspired by Kirkpatrick's evaluation model, which advocates evaluating interventions at four levels - reaction (satisfaction), learning, behavior change (transfer) and results (benefits). A web-based survey was used for data collection. Data was collected from the middle of January 2013 through the end of March 2013.

RESULTS

A thirteen item instrument was developed for measuring the students' perception of the effectiveness of EBP courses. The internal consistency of the scale, based on standardized Cronbach's alpha, was .93. The results of factor analysis identified three factors of the instrument. The highest rated items on a scale of 1 (strongly disagree) to 7 (strongly agree) were 'implementation of EBP can improve clinical care' (mean 6.16), 'EBP instructors had a thorough knowledge of EBP' (6.13), 'EBP instructors were enthusiastic about teaching EBP' (5.65), and 'I can use my EBP knowledge and skills in my practice' (5.58).

CONCLUSIONS

The results of testing of the psychometric properties of the questionnaire showed at least satisfactory validity and reliability. The majority of students perceived EBP courses as effective. The instrument may be used to assess the students' perception of the effectiveness of EBP courses.

Objective consensus from decision trees

BACKGROUND

Consensus-based approaches provide an alternative to evidence-based decision making, especially in situations where high-level evidence is limited. Our aim was to demonstrate a novel source of information, objective consensus based on recommendations in decision tree format from multiple sources.

METHODS

Based on nine sample recommendations in decision tree format a representative analysis was performed. The most common (mode) recommendations for each eventuality (each permutation of parameters) were determined. The same procedure was applied to real clinical recommendations for primary radiotherapy for prostate cancer. Data was collected from 16 radiation oncology centres, converted into decision tree format and analyzed in order to determine the objective consensus.

RESULTS

Based on information from multiple sources in decision tree format, treatment recommendations can be assessed for every parameter combination. An objective consensus can be determined by means of mode recommendations without compromise or confrontation among the parties. In the clinical example involving prostate cancer therapy, three parameters were used with two cut-off values each (Gleason score, PSA, T-stage) resulting in a total of 27 possible combinations per decision tree. Despite significant variations among the recommendations, a mode recommendation could be found for specific combinations of parameters.

CONCLUSION

Recommendations represented as decision trees can serve as a basis for objective consensus among multiple parties.

Faculty perception of the effectiveness of EBP courses for graduate nursing students

BACKGROUND

Effective teaching is key in preparing students to become successful evidence-based healthcare professionals. The effectiveness of graduate evidence-based practice (EBP) pedagogy is not often a subject of research studies.

PURPOSE

The purpose of this study was to determine how faculty from the 50 top graduate nursing schools in the United States perceived the effectiveness of EBP courses for graduate nursing students.

METHODS

A descriptive cross-sectional design was used to explore faculty perception of the effectiveness of EBP courses. A web-based survey was used for data collection. A total of 45 questionnaires were subjected to statistical analysis.

RESULTS

The mean perception of the effectiveness of EBP courses for the whole sample, on a scale from 1 to 7, was 5.58 (min. 4.29; max. 6.73), a higher score signifying higher perceived effectiveness. The highest rated item concerned a school's access to different databases. The strongest correlations were found between the total score and the scores for items describing students' opportunities to strengthen and apply their EBP skills (rs = .66). The internal consistency of the Perception of Effectiveness of EBP Courses scale, based on standardized Cronbach's alpha, was .84, which signifies strong internal consistency. Faculty perceived themselves as most competent at the following EBP skills: (a) "Asking questions regarding patients' care" (6.56), (b) "Considering patient preferences when implementing EBP" (6.40), and (c) "Critically appraising the relevant body of evidence to address clinical questions" (6.40).

DISCUSSION

To strengthen the effectiveness of EBP courses, students should have more opportunities to implement their EBP knowledge and skills after completing EBP courses.

LINKING EVIDENCE TO ACTION

Evaluation of faculty perceptions of the effectiveness of EBP courses can help to guide the development of nursing school curricula that better integrate EBP. Further evaluation of the psychometric properties of the instrument used to measure perception of the effectiveness of EBP courses is required along with objective measures of faculty knowledge and skills in teaching EBP.

Online training course on critical appraisal for nurses: adaptation and assessment

BACKGROUND

Research is an essential activity for improving quality and efficiency in healthcare. The objective of this study was to train nurses from the public Basque Health Service (Osakidetza) in critical appraisal, promoting continuous training and the use of research in clinical practice.

METHODS

This was a prospective pre-post test study. The InfoCritique course on critical appraisal was translated and adapted. A sample of 50 nurses and 3 tutors was recruited. Educational strategies and assessment instruments were established for the course. A course website was created that contained contact details of the teaching team and coordinator, as well as a course handbook and videos introducing the course. Assessment comprised the administration of questionnaires before and after the course, in order to explore the main intervention outcomes: knowledge acquired and self-learning readiness. Satisfaction was also measured at the end of the course.

RESULTS

Of the 50 health professionals recruited, 3 did not complete the course for personal or work-related reasons. The mean score on the pre-course knowledge questionnaire was 70.5 out of 100, with a standard deviation of 11.96. In general, participants' performance on the knowledge questionnaire improved after the course, as reflected in the notable increase of the mean score, to 86.6, with a standard deviation of 10.00. Further, analyses confirmed statistically significant differences between pre- and post-course results (p < 0.001). With regard to self-learning readiness, after the course, participants reported a greater readiness and ability for self-directed learning. Lastly, in terms of level of satisfaction with the course, the mean score was 7 out of 10.

CONCLUSIONS

Participants significantly improved their knowledge score and self-directed learning readiness after the educational intervention, and they were overall satisfied with the course. For the health system and nursing professionals, this type of course has the potential to provide methodological tools for research, promote a research culture, and encourage critical thinking for evidence-based decision making.