The Utrecht questionnaire (U-CEP) measuring knowledge on clinical epidemiology proved to be valid

Translating and Validating the Vietnamese Version of the Health Sciences Evidence-Based Practice Questionnaire

No validated instrument is available for assessing the evidence-based practice capacity of Vietnamese health professionals. This study aimed to translate and validate the Health Sciences Evidence-Based Practice questionnaire (HS-EBP) from English to Vietnamese and ascertain its psychometric properties. Data were collected from two obstetric hospitals in Vietnam. Participants: A total of 343 midwives were randomly selected. The HS-EBP questionnaire was translated by a group of bilingual experts into Vietnamese (HS-EBP-V). Content validity was assessed by two experts. Internal consistency and test-retest reliabilities were assessed using Cronbach's α and intraclass correlation (ICC), respectively. Construct validity was assessed using the contrasted groups approach. As a result, the content validity index of the HS-EBP-V reached 1.0. For the individual subscales, Cronbach's α was 0.92-0.97 and ICC was between 0.45 and 0.66. The validity of the contrasted-groups approach showed discrimination by a significant difference in the subscale scores among diploma holders compared with bachelor's degree holders (p < 0.001). The validation of the HS-EBP questionnaire indicated satisfactory psychometric properties. The results indicate that the HS-EBP is a reliable and valid instrument which assesses the competencies of as well as facilitators of and barriers to the five steps of EBP among midwives. The HS-EBP-V was deemed a reliable and validated tool for assessing the competency and application of EBP among Vietnamese healthcare professionals.

Quality, methods, and recommendations of systematic reviews on measures of evidence-based practice: an umbrella review

OBJECTIVES

The objective of the review was to estimate the quality of systematic reviews on evidence-based practice measures across health care professions and identify differences between systematic reviews regarding approaches used to assess the adequacy of evidence-based practice measures and recommended measures.

INTRODUCTION

Systematic reviews on the psychometric properties of evidence-based practice measures guide researchers, clinical managers, and educators in selecting an appropriate measure for use. The lack of psychometric standards specific to evidence-based practice measures, in addition to recent findings suggesting the low methodological quality of psychometric systematic reviews, calls into question the quality and methods of systematic reviews examining evidence-based practice measures.

INCLUSION CRITERIA

We included systematic reviews that identified measures that assessed evidence-based practice as a whole or of constituent parts (eg, knowledge, attitudes, skills, behaviors), and described the psychometric evidence for any health care professional group irrespective of assessment context (education or clinical practice).

METHODS

We searched five databases (MEDLINE, Embase, CINAHL, PsycINFO, and ERIC) on January 18, 2021. Two independent reviewers conducted screening, data extraction, and quality appraisal following the JBI approach. A narrative synthesis was performed.

RESULTS

Ten systematic reviews, published between 2006 and 2020, were included and focused on the following groups: all health care professionals (n = 3), nurses (n = 2), occupational therapists (n = 2), physical therapists (n = 1), medical students (n = 1), and family medical residents (n = 1). The overall quality of the systematic reviews was low: none of the reviews assessed the quality of primary studies or adhered to methodological guidelines, and only one registered a protocol. Reporting of psychometric evidence and measurement characteristics differed. While all the systematic reviews discussed internal consistency, feasibility was only addressed by three. Many approaches were used to assess the adequacy of measures, and five systematic reviews referenced tools. Criteria for the adequacy of individual properties and measures varied, but mainly followed standards for patient-reported outcome measures or The Standards of Educational and Psychological Testing. Two hundred and four unique measures were identified across 10 reviews. One review explicitly recommended measures for occupational therapists, and four reviews identified adequate measures for all health care professionals (n = 3) and medical students (n = 1). The 27 measures deemed adequate by these five systematic reviews are described.

CONCLUSIONS

Our results suggest a need to improve the overall methodological quality and reporting of systematic reviews on evidence-based practice measures to increase the trustworthiness of recommendations and allow comprehensive interpretation by end-users. Risk of bias is common to all the included systematic reviews as the quality of primary studies was not assessed. The diversity of tools and approaches used to evaluate the adequacy of evidence-based practice measures reflects tensions regarding the conceptualization of validity, suggesting a need to reflect on the most appropriate application of validity theory to evidence-based practice measures.

SYSTEMATIC REVIEW REGISTRATION NUMBER

PROSPERO CRD42020160874.

Multi-dimensionality of patient reported outcome measures in rhinoplasty satisfaction

Background The FACE-Q rhinoplasty module (nose and nostrils), Utrecht Questionnaire and NOSE-scale are validated Dutch patient-reported outcome measures (PROMs) to evaluate rhinoplasty satisfaction. The objective of this study was to analyze the dimensionality of the measured variables in these four existing questionnaires. Additionally, we investigated the ability of the PROMS to measure change. Methods A prospective single center study was performed in a consecutive cohort of 106 Dutch-speaking patients. Patients were invited to fill in four PROMs: FACE-Q rhinoplasty module (nose and nostrils), Utrecht Questionnaire and NOSE-scale, preoperatively and 3 months postoperatively. Item quality was calculated in all four questionnaires. The ability of the questionnaires to differentiate between pre-and postoperative patients was determined with a binary logistic regression. Exploratory factor analysis was performed to determine the latent dimensions. Results Item quality was confirmed in all questionnaires. Backward binary logistic regression revealed that NOSE and FACE-Q nose module were the best discriminant factors pre- and postoperatively. Combination of these two questionnaires gave a specificity of 97,33% and a sensitivity of 94.52% to discriminate between pre-and post-operative cases. Exploratory factor analysis identified the presence of 4 dimensions: 1) cosmesis of the nose 2) cosmesis of the nostrils 3) nasal function and 4) psychosocial well-being in rhinoplasty patients. Lack of factorial invariance in the pre- as compared to the postoperative phase was detected, especially with the FACE-Q nose and to a lesser extent with the Utrecht questionnaire.

Adaptation and validation of the Berlin questionnaire of competence in evidence-based dentistry for dental students: a pilot study

BACKGROUND

The purpose of this pilot study was to create a valid and reliable set of assessment questions for examining Evidence-based Dentistry (EbD) knowledge. For this reason, we adapted and validated for dental students the Berlin Questionnaire (BQ), which assesses Evidence-based Medicine (EbM) abilities.

METHODS

The Berlin Questionnaire was validated with medical residents. We adapted it for use in a dentistry setting. An expert panel reviewed the adapted BQ for content validity. A cross-sectional cohort representing four training levels (EbD-novice dental students, EbD-trained dental students, dentists, and EbM-/EbD-expert faculty) completed the questionnaire. A total of 140 participants comprised the validation set. Internal reliability, item difficulty and item discrimination were assessed. Construct validity was assessed by comparing the mean total scores of students to faculty and comparing proportions of students and faculty who passed each item.

RESULTS

Among the 133 participants (52 EbD-novice dental students, 53 EbD-trained dental students, 12 dentists, and 16 EbM-/ EbD-expert faculty), a statistically significant (p < 0.001) difference was evident in the total score corresponding to the training level. The total score reliability and psychometric properties of items modified for discipline-specific content were acceptable. Cronbach's alpha was 0.648.

CONCLUSION

The adapted Berlin Questionnaire is a reliable and valid instrument to assess competence in Evidence-based Dentistry in dental students. Future research will focus on refining the instrument further.

A systematic review and taxonomy of tools for evaluating evidence-based medicine teaching in medical education

BACKGROUND

The importance of teaching the skills and practice of evidence-based medicine (EBM) for medical professionals has steadily grown in recent years. Alongside this growth is a need to evaluate the effectiveness of EBM curriculum as assessed by competency in the five 'A's': asking, acquiring, appraising, applying and assessing (impact and performance). EBM educators in medical education will benefit from a compendium of existing assessment tools for assessing EBM competencies in their settings. The purpose of this review is to provide a systematic review and taxonomy of validated tools that evaluate EBM teaching in medical education.

METHODS

We searched MEDLINE, EMBASE, Cochrane library, Educational Resources Information Centre (ERIC), Best Evidence Medical Education (BEME) databases and references of retrieved articles published between January 2005 and March 2019. We have presented the identified tools along with their psychometric properties including validity, reliability and relevance to the five domains of EBM practice and dimensions of EBM learning. We also assessed the quality of the tools to identify high quality tools as those supported by established interrater reliability (if applicable), objective (non-self-reported) outcome measures and achieved ≥ 3 types of established validity evidence. We have reported our study in accordance with the PRISMA guidelines.

RESULTS

We identified 1719 potentially relevant articles of which 63 full text articles were assessed for eligibility against inclusion and exclusion criteria. Twelve articles each with a unique and newly identified tool were included in the final analysis. Of the twelve tools, all of them assessed the third step of EBM practice (appraise) and four assessed just that one step. None of the twelve tools assessed the last step of EBM practice (assess). Of the seven domains of EBM learning, ten tools assessed knowledge gain, nine assessed skills and-one assessed attitude. None addressed reaction to EBM teaching, self-efficacy, behaviours or patient benefit. Of the twelve tools identified, six were high quality. We have also provided a taxonomy of tools using the CREATE framework, for EBM teachers in medical education.

CONCLUSIONS

Six tools of reasonable validity are available for evaluating most steps of EBM and some domains of EBM learning. Further development and validation of tools that evaluate all the steps in EBM and all educational outcome domains are needed.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42018116203.

How to choose an evidence-based medicine knowledge test for medical students? Comparison of three knowledge measures

BACKGROUND

There are a few studies of alignment between different knowledge-indices for evidence-based medicine (EBM). The aim of this study was to investigate whether the type of test used to assess knowledge of EBM affects the estimation of this knowledge in medical students.

METHODS

Medical students enrolled in 1-week EBM course were tested with the Fresno, Berlin, and ACE tests at the beginning and the end of the course. We evaluated the ability of these tests to detect a change in the acquired level of EBM knowledge and compared the estimates of change with those of the Control group that was tested with the ACE and Berlin tests before and after an unrelated non-EBM course. The distributions of test scores and average item difficulty indices were compared among the tests and the groups.

RESULTS

Test scores improved on all three tests when compared with their pre-test results and the control. Students had on average a "good" performance on the ACE test, "sufficient" performance on the Berlin test, and "insufficient" performance or have "not passed" on the Fresno test. The post-test improvements in performance on the Fresno test (median 31% increase in percent scores, 95% confidence interval (CI) 25-42%) outperformed those on the ACE (13, 95% CI 13-20%) and Berlin tests (13, 95% CI 7-20%). Post-test score distributions demonstrated that the ACE test had less potential to discriminate between levels of EBM knowledge than other tests.

CONCLUSION

The use of different EBM tests resulted in different assessment of general EBM knowledge in a sample of graduate medical students, with lowest results on the Fresno and highest on the ACE test. In the light of these findings, EBM knowledge assessment should be based on the course's content and learning objectives.

Evidence-based practice educational intervention studies: a systematic review of what is taught and how it is measured

BACKGROUND

Despite the established interest in evidence-based practice (EBP) as a core competence for clinicians, evidence for how best to teach and evaluate EBP remains weak. We sought to systematically assess coverage of the five EBP steps, review the outcome domains measured, and assess the properties of the instruments used in studies evaluating EBP educational interventions.

METHODS

We conducted a systematic review of controlled studies (i.e. studies with a separate control group) which had investigated the effect of EBP educational interventions. We used citation analysis technique and tracked the forward and backward citations of the index articles (i.e. the systematic reviews and primary studies included in an overview of the effect of EBP teaching) using Web of Science until May 2017. We extracted information on intervention content (grouped into the five EBP steps), and the outcome domains assessed. We also searched the literature for published reliability and validity data of the EBP instruments used.

RESULTS

Of 1831 records identified, 302 full-text articles were screened, and 85 included. Of these, 46 (54%) studies were randomised trials, 51 (60%) included postgraduate level participants, and 63 (75%) taught medical professionals. EBP Step 3 (critical appraisal) was the most frequently taught step (63 studies; 74%). Only 10 (12%) of the studies taught content which addressed all five EBP steps. Of the 85 studies, 52 (61%) evaluated EBP skills, 39 (46%) knowledge, 35 (41%) attitudes, 19 (22%) behaviours, 15 (18%) self-efficacy, and 7 (8%) measured reactions to EBP teaching delivery. Of the 24 instruments used in the included studies, 6 were high-quality (achieved ≥3 types of established validity evidence) and these were used in 14 (29%) of the 52 studies that measured EBP skills; 14 (41%) of the 39 studies that measured EBP knowledge; and 8 (26%) of the 35 studies that measured EBP attitude.

CONCLUSIONS

Most EBP educational interventions which have been evaluated in controlled studies focus on teaching only some of the EBP steps (predominantly critically appraisal of evidence) and did not use high-quality instruments to measure outcomes. Educational packages and instruments which address all EBP steps are needed to improve EBP teaching.

Does integrated training in evidence-based medicine (EBM) in the general practice (GP) specialty training improve EBM behaviour in daily clinical practice? A cluster randomised controlled trial

OBJECTIVES

Evidence-based medicine (EBM) is an important element in the general practice (GP) specialty training. Studies show that integrating EBM training into clinical practice brings larger benefits than stand-alone modules. However, these studies have neither been performed in GP nor assessed EBM behaviour of former trainees in daily clinical practice.

SETTING

GP specialty training in the Netherlands.

PARTICIPANTS

All 82 third year GP trainees who started their final third year in 2011 were approached for inclusion, of whom 79 (96%) participated: 39 in the intervention group and 40 in the control group.

INTERVENTION

Integrated EBM training, in which EBM is embedded closely within the clinical context by joint assignments for the trainee and supervisor in daily practice, and teaching sessions based on dilemmas from actual patient consultations.

COMPARISON

Stand-alone EBM training at the institute only.

PRIMARY AND SECONDARY OUTCOMES

Our primary outcome was EBM behaviour, assessed by measuring guideline adherence (incorporating rational, motivated deviation) and information-seeking behaviour. Our secondary outcomes were EBM attitude and EBM knowledge. Data were acquired using logbooks and questionnaires, respectively. Analyses were performed using mixed models.

RESULTS

Logbook data were available from 76 (96%) of the participating trainees at baseline (7614 consultations), 60 (76%) at the end of the third year (T1, 4973 consultations) and 53 (67%) 1 year after graduation (T2, 3307 consultations). We found no significant differences in outcomes between the 2 groups, with relative risks for guideline adherence varying between 0.96 and 0.99 (95% CI 0.86 to 1.11) at T1, and 0.99 and 1.10 (95% CI 0.92 to 1.25) at T2, and for information-seeking behaviour between 0.97 and 1.16 (95% CI 0.70 to 1.91) and 0.90 and 1.10 (95% CI 0.70 to 1.32), respectively.

CONCLUSIONS

Integrated EBM training compared with stand-alone EBM training does not improve EBM behaviour, attitude or knowledge of (future) GPs.