Evidence based medicine manifesto for better healthcare

Evidence-based research impact praxis: Integrating scholarship and practice to ensure research benefits society

Effective research impact development is essential to address global challenges. This commentary highlights key issues facing research impact development as a nascent professional field of practice. We argue that those working on research impact should take a strategic, 'evidence-based' approach to maximize potential research benefits and minimize potential harms. We identify key features of evidence-based good practice in the context of research impact work. This includes integrating relevant research and theory into professional decision-making, drawing on a diversity of academic disciplines offering pertinent insights. Such an integration of scholarship and practice will improve the capacity of research impact work to make a positive difference for society. Moving the focus of research impact work to earlier stages in the research and innovation process through stakeholder engagement and anticipatory research can also boost its effectiveness. The research impact evidence base should be combined with the right kind of professional capacities and practical experience to enhance positive impact. Such capacities need to be developed through relevant education and training, for example, in participatory methods and social inclusion. Such training for research impact work needs to forge strong links between research impact scholarship and practice. Finally, there is a need for improvements in the evidence base for research impact to make it more practically useful.

Project rebuild the evidence base (REB): A method to interpret randomised clinical trials and their meta-analysis to present solid benefit-risk assessments to patients

Evidence-based medicine is the cornerstone of shared-decision making in healthcare today. The public deserves clear, transparent and trust-worthy information on drug efficacy. Yet today, many drugs are prescribed and used without solid evidence of efficacy. Clinical trials and randomised clinical trials (RCTs) are the best method to evaluate drug efficacy and side effects. In a shared medical decision-making approach, general practitioners need drug assessment based on patient-important outcomes. The aim of project rebuild the evidence base (REB) is to bridge the gap between the data needed in clinical practice and the data available from clinical research. The drugs will be assessed on clinical patient important outcomes and for a population. Using the Cochrane tools, we propose to analyse for each population and outcome: 1) a meta-analysis based on RCTs with a low risk of bias overall; 2) an evaluation of results of confirmatory RCTs; 3) a statistical analysis of heterrogeneity between RCTs and 4) an analysis of publication bias. Depending on the results of these analyses, the evidence will be categorized in 4 different levels: firm evidence, evidence (to be confirmed), signal or absence of evidence. Project REB proposes a method for reading and interpreting RCTs and their meta-analysis to produce quality data for general practitioners to focus on risk-benefit assessment in the interest of patients. If this data does not exist, it could enable clinical research to better its aim.

Tolerating bad health research: the continuing scandal

BACKGROUND

At the 2015 REWARD/EQUATOR conference on research waste, the late Doug Altman revealed that his only regret about his 1994 BMJ paper 'The scandal of poor medical research' was that he used the word 'poor' rather than 'bad'. But how much research is bad? And what would improve things?

MAIN TEXT

We focus on randomised trials and look at scale, participants and cost. We randomly selected up to two quantitative intervention reviews published by all clinical Cochrane Review Groups between May 2020 and April 2021. Data including the risk of bias, number of participants, intervention type and country were extracted for all trials included in selected reviews. High risk of bias trials was classed as bad. The cost of high risk of bias trials was estimated using published estimates of trial cost per participant. We identified 96 reviews authored by 546 reviewers from 49 clinical Cochrane Review Groups that included 1659 trials done in 84 countries. Of the 1640 trials providing risk of bias information, 1013 (62%) were high risk of bias (bad), 494 (30%) unclear and 133 (8%) low risk of bias. Bad trials were spread across all clinical areas and all countries. Well over 220,000 participants (or 56% of all participants) were in bad trials. The low estimate of the cost of bad trials was £726 million; our high estimate was over £8 billion. We have five recommendations: trials should be neither funded (1) nor given ethical approval (2) unless they have a statistician and methodologist; trialists should use a risk of bias tool at design (3); more statisticians and methodologists should be trained and supported (4); there should be more funding into applied methodology research and infrastructure (5).

CONCLUSIONS

Most randomised trials are bad and most trial participants will be in one. The research community has tolerated this for decades. This has to stop: we need to put rigour and methodology where it belongs - at the centre of our science.

Limited engagement with transparent and open science standards in the policies of pain journals: a cross-sectional evaluation

Scientific progress requires transparency and openness. The ability to critique, replicate and implement scientific findings depends on the transparency of the study design and methods, and the open availability of study materials, data and code. Journals are key stakeholders in supporting transparency and openness. This study aimed to evaluate 10 highest ranked pain journals' authorship policies with respect to their support for transparent and open research practices. Two independent authors evaluated the journal policies (as at 27 May 2019) using three tools: the self-developed Transparency and Openness Evaluation Tool, the Centre for Open Science (COS) Transparency Factor and the International Committee of Medical Journal Editors (ICMJE) requirements for disclosure of conflicts of interest. We found that the journal policies had an overall low level of engagement with research transparency and openness standards. The median COS Transparency Factor score was 3.5 (IQR 2.8) of 29 possible points, and only 7 of 10 journals' stated requirements for disclosure of conflicts of interest aligned fully with the ICMJE recommendations. Improved transparency and openness of pain research has the potential to benefit all that are involved in generating and using research findings. Journal policies that endorse and facilitate transparent and open research practices will ultimately improve the evidence base that informs the care provided for people with pain.

The iatrogenic opioid crisis: An example of 'institutional corruption of pharmaceuticals'?

RATIONALE

Prescribed opioids are major contributors to the international public health opioid crisis. Such widespread iatrogenic harms usually result from collective decision failures of healthcare organizations rather than solely of individual organizations or professionals. Findings from a system-wide safety analysis of the iatrogenic opioid crisis that includes roles of pertinent healthcare organizations may help avoid or mitigate similar future iatrogenic consequences. In this retrospective exploratory study, we report such an analysis.

METHODS

The study population encompassed the entire age spectrum and included those in whom opioids prescribed for chronic pain (unrelated to malignancy) were associated with death or morbidity. Root cause analysis, incorporating recent suggestions for improvement, was used to identify possible contributory factors from the literature. Based on their mandated roles and potential influences to prevent or mitigate the iatrogenic crisis, relevant organizations were grouped and stratified from most to least influential.

RESULTS

The analysis identified a chain of multiple interrelated causal factors within and between organizations. The most influential organizations were pharmaceutical, political, and drug regulatory; next: experts and their related societies, and publications. Less influential: accreditation, professional licensing and regulatory, academic and healthcare funding bodies. Collectively, their views and decisions influenced prescribing practices of frontline healthcare professionals and advocacy groups. Financial associations between pharmaceutical and most other organizations/groups were common. Ultimately, patients were adversely affected. There was a complex association with psychosocial variables.

LIMITATIONS

The analysis suggests associations not causality.

CONCLUSION

The iatrogenic crisis has multiple intricately linked roots. The major catalyst: pervasive pharma-linked financial conflicts of interest (CoIs) involving most other healthcare organizations. These extensive financial CoIs were likely triggers for a cascade of erroneous decisions and actions that adversely affected patients. The actions and decisions of pharma ranged from unethical to illegal. The iatrogenic opioid crisis may exemplify 'institutional corruption of pharmaceuticals'.

Clinical practice guidelines for recurrent miscarriage in high-income countries: a systematic review

Recurrent miscarriage affects 1-2% of women of reproductive age, depending on the definition used. A systematic review was conducted to identify, appraise and describe clinical practice guidelines (CPG) published since 2000 for the investigation, management, and/or follow-up of recurrent miscarriage within high-income countries. Six major databases, eight guideline repositories and the websites of 11 professional organizations were searched to identify potentially eligible studies. The quality of eligible CPG was assessed using the Appraisal of Guidelines for Research and Evaluation (AGREE II) Tool. A narrative synthesis was conducted to describe, compare and contrast the CPG and recommendations therein. Thirty-two CPG were included, from which 373 recommendations concerning first-trimester recurrent miscarriage were identified across four sub-categories: structure of care (42 recommendations, nine CPG), investigations (134 recommendations, 23 CPG), treatment (153 recommendations, 24 CPG), and counselling and supportive care (46 recommendations, nine CPG). Most CPG scored 'poor' on applicability (84%) and editorial independence (69%); and to a lesser extent stakeholder involvement (38%) and rigour of development (31%). Varying levels of consensus were found across CPG, with some conflicting recommendations. Greater efforts are required to improve the quality of evidence underpinning CPG, the rigour of their development and the inclusion of multi-disciplinary perspectives, including those with lived experience of recurrent miscarriage.

Social Isolation and Cognitive Function in Later Life: A Systematic Review and Meta-Analysis

BACKGROUND

There is some evidence to suggest that social isolation may be associated with poor cognitive function in later life. However, findings are inconsistent and there is wide variation in the measures used to assess social isolation.

OBJECTIVE

We conducted a systematic review and meta-analysis to investigate the association between social isolation and cognitive function in later life.

METHODS

A search for longitudinal studies assessing the relationship between aspects of social isolation (including social activity and social networks) and cognitive function (including global measures of cognition, memory, and executive function) was conducted in PsycInfo, CINAHL, PubMed, and AgeLine. A random effects meta-analysis was conducted to assess the overall association between measures of social isolation and cognitive function. Sub-analyses investigated the association between different aspects of social isolation and each of the measures of cognitive function.

RESULTS

Sixty-five articles were identified by the systematic review and 51 articles were included in the meta-analysis. Low levels of social isolation characterized by high engagement in social activity and large social networks were associated with better late-life cognitive function (r = 0.054, 95% CI: 0.043, 0.065). Sub-analyses suggested that the association between social isolation and measures of global cognitive function, memory, and executive function were similar and there was no difference according to gender or number of years follow-up.

CONCLUSIONS

Aspects of social isolation are associated with cognitive function in later life. There is wide variation in approaches to measuring social activity and social networks across studies which may contribute to inconsistencies in reported findings.

[Quality deficits of drug trials for older patients: An analysis of a series of systematic reviews]

INTRODUCTION

The aim of this work is to present an exemplary methodological review of the quality of included studies on drug therapy in older patients, using a published series of six systematic reviews (SRs). These six SRs included 48 systematic reviews, 65 intervention studies and 33 observational studies. The series of SRs has been carried out in the PRIMA-eDS-project (www.prima-eds.eu) to develop recommendations for the treatment of elderly patients with polypharmacy.

METHODS

The research question was to which extent recommendations on drug therapy in older patients are based on sound evidence. To this purpose, we performed a quality assessment of all studies included using AMSTAR for systematic reviews, CASP for observational studies, and the Cochrane "Risk of Bias" tool for intervention studies.

RESULTS

The evidence base for commonly prescribed drugs in the elderly is weak. The studies identified by the systematic reviews revealed a significant lack of studies addressing the target population as well as a lack of high-quality evidence. Among the 33 observational studies, it was unclear in nearly half of the publications whether the follow-up was sufficiently long and complete. For one-third, the conclusions did not match the observed evidence. The greatest risk of bias in the intervention trials was due to selection and incorrect blinding. Quality deficits of the systematic reviews consisted in the provision of a complete study list and the lack of consideration of potential publication bias.

DISCUSSION

Overall, many methodological deficits were revealed, making it difficult or almost impossible to derive reliable recommendations.

CONCLUSION

Our work illustrates the immense need for research in the treatment of older patients as well as the importance of ensuring the highest quality standards when conducting intervention and observational studies or carrying out systematic reviews.

Clinicians' attitudes and perceived barriers and facilitators to cancer treatment clinical practice guideline adherence: a systematic review of qualitative and quantitative literature

Background

Clinical Practice Guidelines (CPGs) synthesize the best available evidence to guide clinician and patient decision making. There are a multitude of barriers and facilitators to clinicians adhering to CPGs; however, little is known about active cancer treatment CPG adherence specifically. This systematic review sought to identify clinician attitudes, and perceived barriers and facilitators to active cancer treatment CPG adherence.

Methods

A systematic search was undertaken of five databases; Ovid Medline, PsychInfo, Embase, Scopus, CINAHL, and PROQUEST. The retrieved abstracts were screened for eligibility against inclusion criteria, and a full text review was conducted of all eligible studies. Data were extracted, and a quality assessment was conducted of all included studies. The qualitative papers were thematically analyzed. Attitudes, barriers, and facilitating factors extracted from the quantitative papers were categorized within the qualitative thematic framework.

Results

The search resulted in the identification of 9676 titles. After duplicates were removed, abstracts screened, and full texts reviewed, 15 studies were included. Four themes were identified which related to negative clinician attitudes and barriers to active cancer treatment CPG adherence: (1) concern over CPG content and currency of CPGs; (2) concern about the evidence underpinning CPGs; (3) clinician uncertainty and negative perceptions of CPGs; and (4) organizational and patient factors. The review also identified four themes related to positive attitudes and facilitators to active cancer treatment CPG adherence: (5) CPG accessibility and ease of use; (6) endorsement and dissemination of CPGs and adequate access to treatment facilities and resources; (7) awareness of CPGs and belief in their relevance; and (8) belief that CPGs support decision making, improve patient care, reduce clinical variation, and reduce costs.

Conclusion

These results highlight that adherence to active cancer treatment CPG recommendations by oncology clinicians is influenced by multiple factors such as attitudes, practices, and access to resources. The review has also revealed many similarities and differences in the factors associated with general CPG, and active cancer treatment CPG, adherence. These findings will inform tailored implementation strategies to increase adherence to cancer treatment CPGs.

Trial registration

PROSPERO (2019) CRD42019125748.

An epistemological problem for integration in EBM

Evidence-based medicine (EBM) calls for medical practitioners to "integrate" our best available evidence into clinical practice. A significant amount of the literature on EBM takes this integration to be unproblematic, focusing on questions like how to interpret evidence and engage with patient values, rather than critically looking at how these features of EBM can be implemented together. Other authors have also commented on this gap in the literature, for example, identifying the lack of clarity about how patient preferences and evidence from trials is supposed to be integrated in practice. In this paper, I look at this issue from an epistemological perspective, (looking at how different types of knowledge in EBM can be used to make sounds judgements). In particular, I introduce an epistemological issue for this integration problem, which I call the epistemic integration problem. This is essentially the problem of how we can use information that is both general (eg, about a population sample) and descriptive (eg, about what expected outcomes are) to reach clinical judgements that are individualized (applying to a particular patient) and normative (about what is best for their health).

Consistent Definitions of Clinical Practice Guidelines, Consensus Statements, Position Statements, and Practice Alerts

An evidence-based approach to clinical decision-making for optimizing patient care is desirable because it promotes quality of care, improves patient safety, decreases medical errors, and reduces health care costs. Clinical practice recommendations are systematically developed documents regarding best practice for specific clinical management issues, which can assist care providers in their clinical decision-making. However, there is currently wide variation in the terminology used for such clinical practice recommendations. The aim of this article is to provide guidance to authors, reviewers, and editors on the definitions of terms commonly used for clinical practice recommendations. This is intended to improve transparency and clarity regarding the definitions of these terminologies.

Is Scientific Medical Literature Related to Endometriosis Treatment Evidence-Based? A Systematic Review on Methodological Quality of Randomized Clinical Trials

Background and objectives: Informed decision-making requires the ability to identify and integrate high-quality scientific evidence in daily practice. We aimed to assess whether randomized controlled trials (RCTs) on endometriosis therapy follow methodological criteria corresponding to the RCTs' specific level in the hierarchy of evidence in such details to allow the reproduction and replication of the study. Materials and Methods: Using the keywords "therapy" and "endometriosis" and "efficacy" three bibliographic databases were searched for English written scientific articles published from 1 January 2008 to 3 March 2018. Only the randomized clinical trials (RCTs) were evaluated in terms of whether they provided the appropriate level of scientific evidence, equivalent to level 1, degree 1b in the hierarchy of evidence. A list of criteria to ensure study replication and reproduction, considering CONSORT guideline and MECIR standards, was developed and used to evaluate RCTs' methodological soundness, and scores were granted. Three types of bias, namely selection bias (random sequence generation and allocation concealment), detection bias (blinding of outcome assessment), and attrition bias (incomplete outcome data) were also evaluated. Results: We found 387 articles on endometriosis therapy, of which 38 were RCTs: 30 double-blinded RCTs and 8 open-label RCTs. No article achieved the maximum score according to the evaluated methodological criteria. Even though 73.3% of the double-blinded RCTs had clear title, abstract, introduction, and objectives, only 13.3% provided precise information regarding experimental design and randomization, and also showed a low risk of bias. The blinding method was poorly reported in 43.3% of the double-blinded RCTs, while allocation concealment and random sequence generation were inadequate in 33.3% of them. Conclusions: None of the evaluated RCTs met all the methodological criteria, none had only a low risk of bias and provided sufficient details on methods and randomization to allow for the reproduction and replication of the study. Consequently, the appropriate level of scientific evidence (level 1, degree 1b) could not be granted. On endometriosis therapy, this study evaluated the quality of reporting in RCTs and not the quality of how the studies were performed.

Development of evidence-based recommendations for procedure-specific pain management: PROSPECT methodology

Effective peri‐operative pain management is a prerequisite for optimal recovery after surgery. Despite published evidence‐based guidelines from several professional groups, postoperative pain management remains inadequate. The procedure‐specific pain management (PROSPECT) collaboration consists of anaesthetists and surgeons with broad international representation that provide healthcare professionals with practical and evidence‐based recommendations formulated in a way that facilitates clinical decision‐making across all stages of the peri‐operative period on a procedure‐specific basis. The aim of this manuscript is to provide a detailed description of the current PROSPECT methodology with the intention of providing the rigour and transparency in which procedure‐specific pain management recommendations are developed. The high methodological standards of the recommendations should improve the quality of clinical practice.

Systematic Review and Meta-analysis: Sometimes Bigger Is Indeed Better

Clinicians encounter an ever increasing and frequently overwhelming amount of information, even in a narrow scope or area of interest. Given this enormous amount of scientific information published every year, systematic reviews and meta-analyses have become indispensable methods for the evaluation of medical treatments and the delivery of evidence-based best practice. The present basic statistical tutorial thus focuses on the fundamentals of a systematic review and meta-analysis, against the backdrop of practicing evidence-based medicine. Even if properly performed, a single study is no more than tentative evidence, which needs to be confirmed by additional, independent research. A systematic review summarizes the existing, published research on a particular topic, in a well-described, methodical, rigorous, and reproducible (hence "systematic") manner. A systematic review typically includes a greater range of patients than any single study, thus strengthening the external validity or generalizability of its findings and the utility to the clinician seeking to practice evidence-based medicine. A systematic review often forms the basis for a concomitant meta-analysis, in which the results from the identified series of separate studies are aggregated and statistical pooling is performed. This allows for a single best estimate of the effect or association. A conjoint systematic review and meta-analysis can provide an estimate of therapeutic efficacy, prognosis, or diagnostic test accuracy. By aggregating and pooling the data derived from a systemic review, a well-done meta-analysis essentially increases the precision and the certainty of the statistical inference. The resulting single best estimate of effect or association facilitates clinical decision making and practicing evidence-based medicine. A well-designed systematic review and meta-analysis can provide valuable information for researchers, policymakers, and clinicians. However, there are many critical caveats in performing and interpreting them, and thus, like the individual research studies on which they are based, there are many ways in which meta-analyses can yield misleading information. Creators, reviewers, and consumers alike of systematic reviews and meta-analyses would thus be well-served to observe and mitigate their associated caveats and potential pitfalls.

Psychometrics: Trust, but Verify

There is a continued mandate for practicing evidence-based medicine and the prerequisite rigorous analysis of the comparative effectiveness of alternative treatments. There is also an increasing emphasis on delivering value-based health care. Both these high priorities and their related endeavors require correct information about the outcomes of care. Accurately measuring and confirming health care outcomes are thus likely now of even greater importance. The present basic statistical tutorial focuses on the germane topic of psychometrics. In its narrower sense, psychometrics is the science of evaluating the attributes of such psychological tests. However, in its broader sense, psychometrics is concerned with the objective measurement of the skills, knowledge, and abilities, as well as the subjective measurement of the interests, values, and attitudes of individuals-both patients and their clinicians. While psychometrics is principally the domain and content expertise of psychiatry, psychology, and social work, it is also very pertinent to patient care, education, and research in anesthesiology, perioperative medicine, critical care, and pain medicine. A key step in selecting an existing or creating a new health-related assessment tool, scale, or survey is confirming or establishing the usefulness of the existing or new measure; this process conventionally involves assessing its reliability and its validity. Assessing reliability involves demonstrating that the measurement instrument generates consistent and hence reproducible results-in other words, whether the instrument produces the same results each time it is used in the same setting, with the same type of subjects. This includes interrater reliability, intrarater reliability, test-retest reliability, and internal reliability. Assessing validity is answering whether the instrument is actually measuring what it is intended to measure. This includes content validity, criterion validity, and construct validity. In evaluating a reported set of research data and its analyses, in a similar manner, it is important to assess the overall internal validity of the attendant study design and the external validity (generalizability) of its findings.

Research integrity and the law that never was

Evidence-based medicine is arguably among the most important innovations of the modern era, but publication bias and inadequate research transparency are serious issues affecting the very foundation of evidence-based practice. Despite this truth, these crucial issues have gone largely unaddressed or inadequately addressed for a distressingly long period of time. Regulatory efforts have thus far proven insufficient in eliminating these issues. Fortunately, the last 5 years in particular have seen developments that one hopes will contribute to the eradication of these issues and a future where we can look back on these issues as a sordid story of our past. However, much like the purported fixes of the past, time will be the final arbiter of the efficacy of remedial measures currently underway. This article chronicles the history of these issues, failed attempts to fix these issues, and what can be and is being done with the hope of bringing about true resolution.

Evidence-Based Pain Management: Building on the Foundations of Cochrane Systematic Reviews

We discuss the history and current status of evidence-based medicine for the prevention and treatment of acute and chronic pain as it has developed in the Cochrane Collaboration's Pain, Palliative and Supportive Care Review Group.To date, the Pain, Palliative and Supportive Care Review Group has published 277 reviews and a further 11 reviews of systematic reviews summarizing the evidence for interventions. The Cochrane Library has readily available high-quality summaries of evidence of pharmacological interventions especially for postsurgical pain but also for chronic musculoskeletal and neuropathic pain. The library covers all forms of intervention, not only pharmacological.The world of evidence-based medicine is changing: most historical trials have been entered into reviews, but the evidence is still not well disseminated and needs to be better translated into decision support. Evidence should be at the heart of policymaking. Much has been achieved in the past 21 years, but there are no grounds for complacency.

From MARS to MAGIC: The remarkable journey through time and space of the Grading of Recommendations Assessment, Development and Evaluation initiative

For over 30 years, "evidence-based" clinical guidelines remained entrenched in an oversimplified, design-based, framework for rating the strength of evidence supporting clinical recommendations. The approach frequently equated the rating of evidence with that of the recommendations themselves. "Grading Recommendations Assessment, Development and Evaluation (GRADE)" has emerged as a proposed antidote to obsolete guideline methodology. GRADE sponsors and collaborators are in the process of attempting to amplify and extend the framework to encompass implementation and adaptation of guidelines, above and beyond the evaluation and rating of clinical research. Alternative schemes and models for such extensions are beginning to appear. This commentary reviews the strengths and weaknesses of GRADE with reference to other recent critiques. It considers the GRADE Working Group's "evidence-to-decision" extension of the evidence rating framework, together with proposed alternatives. It identifies pitfalls of the GRADE system's cooptation of relational processes necessary to the interpretation and uptake of recommendations that properly belong to end-users. It also identifies dangers inherent in blurring important boundaries between clinical and policy applications of guidelines. Finally, it addresses criticisms regarding the lack of a theoretical framework supporting the different facets of the GRADE approach and proposes a social constructivist orientation to clinical guideline development and use. Recommendations are offered to potential guideline developers and users regarding how to draw upon the strengths of the GRADE framework without succumbing to its pitfalls.