Shortcomings of the randomized controlled trial: a view from the boondocks

The need to reform our assessment of evidence from clinical trials: a commentary

The ideology of evidence-based medicine (EBM) has dramatically altered the way we think, conceptualize, philosophize and practice medicine. One of its major pillars is the appraisal and classification of evidence. Although important and beneficial, this process currently lacks detail and is in need of reform. In particular, it largely focuses on three key dimensions (design, [type I] alpha error and beta [type II] error) to grade the quality of evidence and often omits other crucial aspects of evidence such as biological plausibility, reproducibility, generalizability, temporality, consistency and coherence. It also over-values the randomized trial and meta-analytical techniques, discounts the biasing effect of single centre execution and gives insufficient weight to large and detailed observational studies. Unless these aspects are progressively included into systems for grading, evaluating and classifying evidence and duly empirically assessed (according to the EBM paradigm), the EBM process and movement will remain open to criticism of being more evidence-biased than evidence-based."All scientific work is incomplete--whether it be observational or experimental. All scientific work is liable to be upset or modified by advancing knowledge. That does not confer upon us a freedom to ignore the knowledge we already have, or to postpone the action that it appears to demand at a given time". (Sir Bradford Austin Hill 1).

[Measurement in public health: what pulmonary tuberculosis management has taught us]

Pulmonary tuberculosis rates are increasing worldwide, including in Spain. One of the main challenges when treating this disease is achieving treatment completion, since studies have shown that approximately 30-35% of all patients do not take their medications as intended. The present article explores a continuum of evaluation strategies and performance measures for assessing the effectiveness of community-based programs designed to enhance treatment completion in patients with active pulmonary tuberculosis. Four traditional evaluation strategies (case studies, retrospective and case-control studies, forecasting/modeling, and cost effectiveness analysis) and 2 emerging and promising approaches (quality of life assessment and indicators of the continuum of care) are presented. Several of the evaluation strategies reviewed indicate that treatment programs using directly observed therapy (DOT) that are comprehensive, community-based and patient-centered achieve the highest treatment completion rates. Combinations of these strategies are recommended to create a body of evidence capturing the impact and nuances of community-based public health interventions in improving health outcomes, in this case for patients with pulmonary tuberculosis.

Lay conceptions of the ethical and scientific justifications for random allocation in clinical trials

Randomised controlled trials (RCTs) play a central role in modern medical advance, and they require participants who understand and accept the procedures involved. Published evidence suggests that RCT participants often fail to understand that treatments are allocated at random and that clinicians are in equipoise about which treatment is best. We examine background assumptions that members of the public might draw upon if invited to take part in a RCT. Four studies (N=82; 67; 67; 128), in the UK, identified whether members of the public (i). accept that an individual clinician might be genuinely unsure which of two treatments was better; (ii). judge that when there is uncertainty it is acceptable to suggest deciding at random; (iii). recognise scientific benefits of random allocation to treatment conditions in a trial. Around half the participants were loathe to accept that a clinician could be completely uncertain, and this was no different whether the context was one of individual treatment or research. Most participants found it unacceptable to suggest allocating treatment at random, though there was weak evidence that a research context may reduce the unacceptability. Participants did not judge that more certain knowledge would be gained about which treatment was best when treatments were allocated at random rather than by patient/doctor choice: scientific benefits of randomisation were apparently not recognised. Judgements were no different in non-medical contexts. Results suggest a large mismatch between the assumptions underlying the trial design, and the assumptions that lay participants can bring to bear when they try to make sense of descriptive information about randomisation and equipoise. Previous attempts to improve understanding by improving the clarity or salience of trial information, or of making explicit the research context, while helpful, may need to be supplemented with accessible explanations for random allocation.

Why alternative medicine cannot be evidence-based

The concept of evidence-based medicine (EBM) has been widely adopted by orthodox Western medicine. Proponents of EBM have argued that complementary and alternative medicine (CAM) modalities ought to be subjected to rigorous, controlled clinical trials in order to assess their efficacy. However, this does not represent a scientific necessity, but rather is a philosophical demand: promoters of EBM seek to establish their particular epistemology as the primary arbiter of all medical knowledge. This claim is problematic. The methods for obtaining knowledge in a healing art must be coherent with that art's underlying understanding and theory of illness. Thus, the method of EBM and the knowledge gained from population-based studies may not be the best way to assess certain CAM practices, which view illness and healing within the context of a particular individual only. In addition, many alternative approaches center on the notion of non-measurable but perceptible aspects of illness and health (e.g., Qi) that preclude study within the current framework of controlled clinical trials. Still, the methods of developing knowledge within CAM currently have limitations and are subject to bias and varied interpretation. CAM must develop and defend a rational and coherent method for assessing causality and efficacy, though not necessarily one based on the results of controlled clinical trials. Orthodox medicine should consider abandoning demands that CAM become evidence-based, at least as "evidence" is currently narrowly defined, but insist instead upon a more complete and coherent description and defense of the alternative epistemic methods and tools of these disciplines.

Examining the assumptions of evidence-based medicine

Evidence-based medicine (EBM) has had enormous impact on clinical medicine, attracting both converts and detractors. The basic premise, that clinical decisions should be made on evidence which is as sound as possible, appears unassailable. Why, then, has the idea stimulated such controversy? In this article, I examine six assumptions of EBM, and point out how these assumptions are frequently open to challenge. Finally, I make specific recommendations for future activities to ensure that the strategy achieves an appropriate role in health care decision-making. These include: (1) conducting research to establish the effectiveness of EBM; (2) incorporating (as they apparently are) more holistic perspectives on the nature of evidence; (3) moving (as they are) toward the use of expert reviews and away from a focus on the skilled individual practitioner; (4) examining the literature on continuing education to devise more appropriate dissemination strategies, including, specifically, educational influentials; and (5) making specific attempts at conciliation and seeking common ground with other schools of thought.