The judgements that evidence-based medicine adopts

The failure of drug repurposing for COVID-19 as an effect of excessive hypothesis testing and weak mechanistic evidence

The current strategy of searching for an effective treatment for COVID-19 relies mainly on repurposing existing therapies developed to target other diseases. Conflicting results have emerged in regard to the efficacy of several tested compounds but later results were negative. The number of conducted and ongoing trials and the urgent need for a treatment pose the risk that false-positive results will be incorrectly interpreted as evidence for treatments' efficacy and a ground for drug approval. Our purpose is twofold. First, we show that the number of drug-repurposing trials can explain the false-positive results. Second, we assess the evidence for treatments' efficacy from the perspective of evidential pluralism and argue that considering mechanistic evidence is particularly needed in cases when the evidence from clinical trials is conflicting or of low quality. Our analysis is an application of the program of Evidence Based Medicine Plus (EBM+) to the drug repurposing trials for COVID. Our study shows that if decision-makers applied EBM+, authorizing the use of ineffective treatments would be less likely. We analyze the example of trials assessing the efficacy of hydroxychloroquine as a treatment for COVID-19 and mechanistic evidence in favor of and against its therapeutic power to draw a lesson for decision-makers and drug agencies on how excessive hypothesis testing can lead to spurious findings and how studying negative mechanistic evidence can be helpful in discriminating genuine from spurious results.

E-synthesis for carcinogenicity assessments: A case study of processed meat

RATIONALE, AIMS AND OBJECTIVES

Recent controversies about dietary advice concerning meat demonstrate that aggregating the available evidence to assess a putative causal link between food and cancer is a challenging enterprise.

METHODS

We show how a tool developed for assessing putative causal links between drugs and adverse drug reactions, E-Synthesis, can be applied for food carcinogenicity assessments. The application is demonstrated on the putative causal relationship between processed meat consumption and cancer.

RESULTS

The output of the assessment is a Bayesian probability that processed meat consumption causes cancer. This Bayesian probability is calculated from a Bayesian network model, which incorporates a representation of Bradford Hill's Guidelines as probabilistic indicators of causality. We show how to determine probabilities of indicators of causality for food carcinogenicity assessments based on assessments of the International Agency for Research on Cancer.

CONCLUSIONS

We find that E-Synthesis is a tool well-suited for food carcinogenicity assessments, as it enables a graphical representation of lines and weights of evidence, offers the possibility to make a great number of judgements explicit and transparent, outputs a probability of causality suitable for decision making and is flexible to aggregate different kinds of evidence.

Artificial intelligence methods for a Bayesian epistemology-powered evidence evaluation

RATIONALE, AIMS AND OBJECTIVES

The diversity of types of evidence (eg, case reports, animal studies and observational studies) makes the assessment of a drug's safety profile into a formidable challenge. While frequentist uncertain inference struggles in aggregating these signals, the more flexible Bayesian approaches seem better suited for this quest. Artificial Intelligence (AI) offers great promise to these approaches for information retrieval, decision support, and learning probabilities from data.

METHODS

E-Synthesis is a Bayesian framework for drug safety assessments built on philosophical principles and considerations. It aims to aggregate all the available information, in order to provide a Bayesian probability of a drug causing an adverse reaction. AI systems are being developed for evidence aggregation in medicine, which increasingly are automated.

RESULTS

We find that AI can help E-Synthesis with information retrieval, usability (graphical decision-making aids), learning Bayes factors from historical data, assessing quality of information and determining conditional probabilities for the so-called 'indicators' of causation for E-Synthesis. Vice versa, E-Synthesis offers a solid methodological basis for (semi-)automated evidence aggregation with AI systems.

CONCLUSIONS

Properly applied, AI can help the transition of philosophical principles and considerations concerning evidence aggregation for drug safety to a tool that can be used in practice.

On evidence fiascos and judgments in COVID-19 policy

Calls for evidence-based approaches to COVID-19 have sparked up discussions on the use of evidence for policy. In this note, we expand these discussions: while the debate has mostly focused on the types of evidence to be used for policy, we argue that the assessment of judgments involved in data practices and evidence production should play a central role in evaluating policy.

Underdetermination and evidence-based policy

Safety assessment of technologies and interventions is often underdetermined by evidence. For example, scientists have collected evidence concerning genetically modified plants for decades. This evidence was used to ground opposing safety protocols for "stacked genetically modified" plants, in which two or more genetically modified plants are combined. Evidence based policy would thus be rendered more effective by an approach that accounts for underdetermination. Douglas (2012) proposes an explanatory approach, based on the criteria of transparency, empirical competence, internal consistency of explanations, and predictive potency. However, sometimes multiple explanations can satisfy these criteria. We propose an additional criterion based on converse abduction, where explanations are selected on the basis of ontological background assumptions as well as by evidence. We then apply our proposed scheme to the case of the regulation of stacked genetically modified plants. We discuss the implications and suggest follow-up work concerning the generalizability of the approach.

Agent-based modelling for SARS-CoV-2 epidemic prediction and intervention assessment: A methodological appraisal

BACKGROUND

Our purpose is to assess epidemiological agent-based models-or ABMs-of the SARS-CoV-2 pandemic methodologically. The rapid spread of the outbreak requires fast-paced decision-making regarding mitigation measures. However, the evidence for the efficacy of non-pharmaceutical interventions such as imposed social distancing and school or workplace closures is scarce: few observational studies use quasi-experimental research designs, and conducting randomized controlled trials seems infeasible. Additionally, evidence from the previous coronavirus outbreaks of SARS and MERS lacks external validity, given the significant differences in contagiousness of those pathogens relative to SARS-CoV-2. To address the pressing policy questions that have emerged as a result of COVID-19, epidemiologists have produced numerous models that range from simple compartmental models to highly advanced agent-based models. These models have been criticized for involving simplifications and lacking empirical support for their assumptions.

METHODS

To address these voices and methodologically appraise epidemiological ABMs, we consider AceMod (the model of the COVID-19 epidemic in Australia) as a case study of the modelling practice.

RESULTS

Our example shows that, although epidemiological ABMs involve simplifications of various sorts, the key characteristics of social interactions and the spread of SARS-CoV-2 are represented sufficiently accurately. This is the case because these modellers treat empirical results as inputs for constructing modelling assumptions and rules that the agents follow; and they use calibration to assert the adequacy to benchmark variables.

CONCLUSIONS

Given this, we claim that the best epidemiological ABMs are models of actual mechanisms and deliver both mechanistic and difference-making evidence. Consequently, they may also adequately describe the effects of possible interventions. Finally, we discuss the limitations of ABMs and put forward policy recommendations.

Pharmacovigilance as Scientific Discovery: An Argument for Trans-Disciplinarity

Pharmacovigilance currently faces several unsolved challenges. Of particular importance are issues concerning how to ascertain, collect, confirm, and communicate the best evidence to assist the clinical choice for individual patients. Here, we propose that these practical challenges partially stem from deeper fundamental issues concerning the epistemology of pharmacovigilance. After reviewing some of the persistent challenges, recent measures, and suggestions in the current pharmacovigilance literature, we support the argument that the detection of potential adverse drug reactions ought to be seen as a serendipitous scientific discovery. We further take up recent innovations from the multidisciplinary field of serendipity research about the importance of networks, diversity of expertise, and plurality of methodological perspectives for cultivating serendipitous discovery. Following this discussion, we explore how pharmacovigilance could be systematized in a way that optimizes serendipitous discoveries of untargeted drug effects, emerging from the clinical application. Specifically, we argue for the promotion of a trans-disciplinary responsive network of scientists and stakeholders. Trans-disciplinarity includes extending the involvement of stakeholders beyond the regulatory community, integrating diverse methods and sources of evidence, and enhancing the ability of diverse groups to raise signals of harms that ought to be followed up by the network. Consequently, promoting a trans-disciplinary approach to pharmacovigilance is a long-term effort that requires structural changes in medical education, research, and enterprise. We suggest a number of such changes, discuss to what extent they are already in process, and indicate the advantages from both epistemological and ethical perspectives.

Causal Evidence and Dispositions in Medicine and Public Health

Since the introduction of evidence-based medicine, there have been discussions about the epistemic primacy of randomised controlled trials (RCTs) for establishing causality in medicine and public health. A growing movement within philosophy of science calls instead for evidential pluralism: that we need more than one single method to investigate health outcomes. How should such evidential pluralism look in practice? How useful are the various methods available for causal inquiry? Further, how should different types of causal evidence be evaluated? This paper proposes a constructive answer and introduces a framework aimed at supporting scientists in developing appropriate methodological approaches for exploring causality. We start from the philosophical tradition that highlights intrinsic properties (dispositions, causal powers or capacities) as essential features of causality. This abstract idea has wide methodological implications. The paper explains how different methods, such as lab experiments, case studies, N-of-1 trials, case control studies, cohort studies, RCTs and patient narratives, all have some strengths and some limitations for picking out intrinsic causal properties. We explain why considering philosophy of causality is crucial for evaluating causality in the health sciences. In our proposal, we combine the various methods in a temporal process, which could then take us from an observed phenomenon (e.g., a correlation) to a causal hypothesis and, finally, to improved theoretical knowledge.

Treating real people: Science and humanity

Something important is happening in applied, interdisciplinary research, particularly in the field of applied health research. The vast array of papers in this edition are evidence of a broad change in thinking across an impressive range of practice and academic areas. The problems of complexity, the rise of chronic conditions, overdiagnosis, co-morbidity, and multi-morbidity are serious and challenging, but we are rising to that challenge. Key conceptions regarding science, evidence, disease, clinical judgement, and health and social care are being revised and their relationships reconsidered: Boundaries are indeed being redrawn; reasoning is being made "fit for practice." Ideas like "person-centred care" are no longer phrases with potential to be helpful in some yet-to-be-clarified way: Theorists and practitioners are working in collaboration to give them substantive import and application.

What is the guidelines challenge? The CauseHealth perspective

This paper is an introduction to the conference, The Guidelines Challenge, held in Oxford in October 2017. My aim is to explain our motivation for organising this conference, as part of the research project Causation, Complexity, and Evidence in Health Sciences (CauseHealth). Depending on the professional starting point, the guidelines challenge can be interpreted in a number of ways. Our idea with this conference was to discuss guidelines from 3 overarching perspectives: practice, policy, and philosophy. In particular, we wanted to discuss some of the challenges that face anyone developing and implementing clinical guidelines in the evidence-based era of medicine. This introduction gives a brief overview of what CauseHealth sees as the guidelines challenge from these perspectives. More attention is given to the philosophical issues with which the CauseHealth project is particularly concerned, although a proper treatment or discussion of these issues naturally falls outside the scope of this introduction.

The Emperor's New Clothes: a Critical Appraisal of Evidence-based Medicine

Evidence-Based Medicine (EBM) is the way we are expected to deliver our healthcare in the 21st century. It has been described as the integration of information from best available evidence with the doctor's experience and the patient's point of view. Unfortunately, the original meaning of EBM has been lost and the worldwide medical community has shifted the paradigm to Guidelines-Based Medicine, that has displaced the figures of the doctor and the patient from the decision-making process and relegated them to mere executor and final target of decisions taken by someone else. Problems related to the reliability of evidence and to the way guidelines are constructed, implemented and followed are discussed in detail. It is mandatory that the whole medical community takes responsibility and tries to reverse this apparently inexorable process so to re-establish a proper evidence-based care, where patients and their healing relation with practitioners are at the centre and where doctors are able to critically evaluate the available evidence and use it in light of their personal experience and knowledge.