Corroborating evidence-based medicine

Challenges and future directions in the measurement of evidence-based practice: Qualitative analysis of umbrella review findings

RATIONALE, AIMS AND OBJECTIVES: An important aspect of scholarly discussions about evidence-based practice (EBP) is how EBP is measured. Given the conceptual and empirical developments in the study of EBP over the last 3 decades, there is a need to better understand how to best measure EBP in educational and clinical contexts. The aim of this study was to identify and describe the main challenges, recommendations for practice, and areas of future research in the measurement of EBP across the health professions as reported by systematic reviews (SRs).

METHODS

We conducted a secondary analysis of qualitative data obtained in the context of a previously published umbrella review that aimed to compare SRs on EBP measures. Two reviewers independently extracted excerpts from the results and discussion/conclusion sections of the 10 included SRs that aligned with the three research aims. An iterative six-phase reflexive thematic analysis according to Braun and Clarke was conducted.

RESULTS

Our thematic analysis produced five themes describing the main challenges associated with measuring EBP, four themes outlining main recommendations for practice, and four themes representing areas of future research. Challenges include limited psychometric testing and validity evidence for existing EBP measures; limitations with the self-report format; lack of construct clarity of EBP measures; inability to capture the complexity of the EBP process and outcomes; and the context-specific nature of EBP measures. Reported recommendations for practice include acknowledging the multidimensionality of EBP; adapting EBP measures to the context and re-examining the validity argument; and considering the feasibility and acceptability of measures. Areas of future research included the development of comprehensive, multidimensional EBP measures and the need for expert consensus on the operationalization of EBP.

CONCLUSIONS

This study suggests that existing measures may be insufficient in capturing the multidimensional, contextual and dynamic nature of EBP. There is a need for a clear operationalization of EBP and an improved understanding and application of validity theory.

Evidence-based medicine as science

Evidence-based medicine has claimed to be science on a number of occasions, but it is not clear that this status is deserved. Within the philosophy of science, four main theories about the nature of science are historically recognized: inductivism, falsificationism, Kuhnian paradigms, and research programmes. If evidence-based medicine is science, knowledge claims should be derived using a process that corresponds to one of these theories. This paper analyses whether this is the case. In the first section, different theories about the nature of science are introduced. In the second section, the claim that evidence-based medicine is science is reinterpreted as the claim that knowledge claims derived from randomized controlled trails and meta-analyses are science. In the third section, the knowledge claims valued within evidence-based medicine are considered from the perspective of inductivism, falsificationism, Kuhnian paradigms, and research programmes. In the final section, possible counter arguments are considered. It is argued that the knowledge claims valued by evidence-based medicine are not justified using inductivism, falsificationism, Kuhnian paradigms, or research programmes. If these are the main criteria for evaluating if something is science or not, evidence-based medicine does not meet these criteria.

Exclusion of 'nonRCT evidence' in guidelines for chronic diseases - is it always appropriate? The Look AHEAD study

Evidence-based medicine (EBM) is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. The introduction of EBM was a conceptual and practical milestone in the history of medicine, with far-reaching impact yet to be fully realized. EBM has limitations, including inapplicability to populations dissimilar to those in studies, and may not reflect duration of exposure to risk factors, details of lifestyle, incubation period, latency, or environmental changes during chronic diseases. Routine exclusion of evidence other than randomized controlled trials (RCTs) or meta-analyses from consideration in treatment may not always be wise. This review is not a result of a search, but rather a conceptual unification of (a) the increasing restrictions in guideline-writing favoring more RCTs, and rejecting observational studies when chronic diseases with a long incubation period may sometimes be best probed by the latter; (b) the possibility RCTs may be inconclusive, nonapplicable, or result in 'negative' results which may misdirect future therapy by physicians and undermine adherence by patients; (c) the potential improvement in patient care from having all available information evaluated (especially epidemiological studies of chronic diseases) and synthesized in guidelines. The example of the Look AHEAD study is chosen - a 'negative' RCT with significant information overlooked by reviewers, who initially declared that weight loss and physical activity were ineffective in treating diabetes, or in preventing cardiovascular complications. In this review, placing this study in perspective, among others, suggests the opposite - exercise and weight loss are effective if done early and sufficiently. Synthesizing worthy data from many sources, including prospective and pathophysiological studies, particularly when RCTs are unavailable, has the potential to add depth and expand the understanding of disease. In addition, integrated data may generate useful, rich material for use during shared decision making discussions with patients, and clarify future hypotheses.