Comparing like with like: some historical milestones in the evolution of methods to create unbiased comparison groups in therapeutic experiments

Clinical therapeutic trials

The term clinical trial implies an investigation of a therapeutic intervention in the pursuit of evidence of benefit, short or sustained, and observations on the possibility of toxicity related to the therapeutic intervention. It is possible that the first clinical trial took place in the court of the Babylonian King Nebuchadnezzar circa 600 BC, as recorded in Chapter 1 of the Book of Daniel, verse 3-20. However, it is in the last 500 years that there has been good written documentation at attempts to interpret therapeutic benefit from the use of treatments. Lind's demonstration on the usefulness of oranges and lemons in the treatment of scurvy in 1747, and the unethical experiment by Edward Jenner (1749-1823) on the inoculation in 1796, of an 8-year-old boy, with cow pox obtained from a milk maid, followed by an attempt to give the young boy smallpox by direct inoculation 18 days later, are striking examples of clinical trials. Human ethics, strict clinical observations, statistics, the governed scientific purity of therapeutic agents, and safety testing of therapeutics, devices, and physical interventions, have created the basis for the modern clinical trial.

Footwear-to-feet examination and analysis: Comparing worn footwear to persons and human remains

Footwear-to-feet examination and analysis refers to the process of examining and analyzing footwear in regard to criminal matters. This examination and analysis has been done to include or exclude an individual as the wearer of the footwear in question and to help associate or disassociate human remains to footwear. This review of the subject describes its investigative applications, the research that underpins the process, and considers its legal admissibility. The process of footwear-to-feet examination and analysis is scientifically valid and has been widely accepted. As a result, law enforcement and forensic scientists should be aware of the use of this process to assist in criminal investigations.

WHO guidance on COVID-19 vaccine trial designs in the context of authorized COVID-19 vaccines and expanding global access: Ethical considerations

While the degree of COVID-19 vaccine accessibility and uptake varies at both national and global levels, increasing vaccination coverage raises questions regarding the standard of prevention that ought to apply to different settings where COVID-19 vaccine trials are hosted. A WHO Expert Group has developed guidance on the ethical implications of conducting placebo-controlled trials in the context of expanding global COVID-19 vaccine coverage. The guidance also considers alternative trial designs to placebo controlled trials in the context of prototype vaccines, modified vaccines, and next generation vaccines.

Identifying compliant participants through data matching improved estimation of intervention efficacy: randomized trials with opt-in/opt-out strategies

OBJECTIVES

We propose a data-matching approach to estimate intervention efficacy for randomized controlled trials (RCTs) when there is noncompliance to the allocated treatment with induced selection bias.

STUDY DESIGN AND SETTING

We considered a large RCT to compare health care costs and hospital length of stay 12 months after randomization. Participants allocated to the intervention group were eligible to receive health-coaching and disease-management services. An opt-out approach was adopted for recruitment. Control-group participants received usual care but were allowed to opt-in to receive the intervention. Using "nearest-neighbor"-matched data, we identified compliant participants in both arms to estimate intervention efficacy. Results were compared with intention-to-treat (ITT), instrumental-variable-adjusted ITT, per-protocol (PP), and as-treated (AT) analyses.

RESULTS

The ITT estimated an intervention effect of a 1.5% reduction in cost, but 56.7% of intervention-group participants did not receive health coaching. The PP and AT found an increase in cost of 9.4% and 17.1%, respectively. The matching method estimated a 12.3% reduction in cost. After adjustment for baseline covariates, the intervention group had lower same-day admission cost (13.6%; 95% CI: 7.3%-20.0%; P < 0.001) and shorter hospital stay (11.2%; 95% CI: 2.6%-19.9%; P = 0.021).

CONCLUSION

Opt-in/opt-out strategies in RCTs misled intervention comparisons and the matching approach improved estimation of intervention efficacy.

Understanding and misunderstanding randomized controlled trials

Randomized Controlled Trials (RCTs) are increasingly popular in the social sciences, not only in medicine. We argue that the lay public, and sometimes researchers, put too much trust in RCTs over other methods of investigation. Contrary to frequent claims in the applied literature, randomization does not equalize everything other than the treatment in the treatment and control groups, it does not automatically deliver a precise estimate of the average treatment effect (ATE), and it does not relieve us of the need to think about (observed or unobserved) covariates. Finding out whether an estimate was generated by chance is more difficult than commonly believed. At best, an RCT yields an unbiased estimate, but this property is of limited practical value. Even then, estimates apply only to the sample selected for the trial, often no more than a convenience sample, and justification is required to extend the results to other groups, including any population to which the trial sample belongs, or to any individual, including an individual in the trial. Demanding 'external validity' is unhelpful because it expects too much of an RCT while undervaluing its potential contribution. RCTs do indeed require minimal assumptions and can operate with little prior knowledge. This is an advantage when persuading distrustful audiences, but it is a disadvantage for cumulative scientific progress, where prior knowledge should be built upon, not discarded. RCTs can play a role in building scientific knowledge and useful predictions but they can only do so as part of a cumulative program, combining with other methods, including conceptual and theoretical development, to discover not 'what works', but 'why things work'.

Evidence-Based Medicine: A Genealogy of the Dominant Science of Medical Education

Debates about how knowledge is made and valued in evidence-based medicine (EBM) have yet to understand what discursive, social, and historical conditions allowed the EBM approach to stabilize and proliferate across western medical education. This paper uses a genealogical approach to examine the epistemological tensions that emerged as a result of various problematizations of uncertainty in medical practice. I explain how the problematization of uncertainty in the literature and the contingency of specific social, political, economic, and historical relations allowed the EBM approach to become a programmatic and pedagogical focus of the Faculty of Medicine at McMaster University and beyond.

Lower Dislocation Rate Following Total Hip Arthroplasty via Direct Anterior Approach than via Posterior Approach: Five-Year-Average Follow-Up Results

Objective:

The aim of the study was to compare the dislocation rate between total hip arthroplasty (THA) via direct anterior approach (DAA) and via posterior approach (PA).

Methods:

We compared a consecutive series of 139 THAs via DAA with 177 THAs via PA. All study patients received ceramic-on-ceramic bearing surfaces and similar uncemented prostheses. Dislocation-free survival after THA was estimated using the Kaplan–Meier survival method and compared between groups using the log-rank test.

Results:

In the DAA group, none of 139 hips experienced dislocations in five-year-average follow-up. In the PA group, seven hips experienced dislocations among 177 hips (4 %). The dislocation was significantly less in the DAA group compared to the PA group (p = 0.033).

Conclusion:

The dislocation rate of THA via DAA was significantly less than that of THA via PA.

Patient preferences and performance bias in a weight loss trial with a usual care arm

OBJECTIVES

This qualitative study examines performance bias, i.e. unintended differences between groups, in the context of a weight loss trial in which a novel patient counseling program was compared to usual care in general practice.

METHODS

14/381 consecutive interviewees (6 intervention group, 8 control group) within the CAMWEL (Camden Weight Loss) effectiveness trial process study were asked about their engagement with various features of the research study and a thematic content analysis undertaken.

RESULTS

Decisions to participate were interwoven with decisions to change behavior, to the extent that for many participants the two were synonymous. The intervention group were satisfied with their allocation. The control group spoke of their disappointment at having been offered usual care when they had taken part in the trial to access new forms of help. Reactions to disappointment involved both movements toward and away from behavior change.

CONCLUSION

There is a prima facie case that reactions to disappointment may introduce bias, as they lead the randomized groups to differ in ways other than the intended experimental contrast.

PRACTICE IMPLICATIONS

In-depth qualitative studies nested within trials are needed to understand better the processes through which bias may be introduced.

Research participation effects: a skeleton in the methodological cupboard

Objective

There have been concerns about impacts of various aspects of taking part in research studies for a century. The concerns have not, however, been sufficiently well conceptualized to form traditions of study capable of defining and elaborating the nature of these problems. In this article we present a new way of thinking about a set of issues attracting long-standing attention.

Study Design and Setting

We briefly review existing concepts and empirical work on well-known biases in surveys and cohort studies and propose that they are connected.

Results

We offer the construct of “research participation effects” (RPE) as a vehicle for advancing multi-disciplinary understanding of biases. Empirical studies are needed to identify conditions in which RPE may be sufficiently large to warrant modifications of study design, analytic methods, or interpretation. We consider the value of adopting a more participant-centred view of the research process as a way of thinking about these issues, which may also have benefits in relation to research methodology more broadly.

Conclusion

Researchers may too readily overlook the extent to which research studies are unusual contexts, and that people may react in unexpected ways to what we invite them to do, introducing a range of biases.

When is it rational to participate in a clinical trial? A game theory approach incorporating trust, regret and guilt

BACKGROUND

Randomized controlled trials (RCTs) remain an indispensable form of human experimentation as a vehicle for discovery of new treatments. However, since their inception RCTs have raised ethical concerns. The ethical tension has revolved around "duties to individuals" vs. "societal value" of RCTs. By asking current patients "to sacrifice for the benefit of future patients" we risk subjugating our duties to patients' best interest to the utilitarian goal for the good of others. This tension creates a key dilemma: when is it rational, from the perspective of the trial patients and researchers (as societal representatives of future patients), to enroll in RCTs?

METHODS

We employed the trust version of the prisoner's dilemma since interaction between the patient and researcher in the setting of a clinical trial is inherently based on trust. We also took into account that the patient may have regretted his/her decision to participate in the trial, while a researcher may feel guilty because he/she abused the patient's trust.

RESULTS

We found that under typical circumstances of clinical research, most patients can be expected not to trust researchers, and most researchers can be expected to abuse the patients' trust. The most significant factor determining trust was the success of experimental or standard treatments, respectively. The more that a researcher believes the experimental treatment will be successful, the more incentive the researcher has to abuse trust. The analysis was sensitive to the assumptions about the utilities related to success and failure of therapies that are tested in RCTs. By varying all variables in the Monte Carlo analysis we found that, on average, the researcher can be expected to honor a patient's trust 41% of the time, while the patient is inclined to trust the researcher 69% of the time. Under assumptions of our model, enrollment into RCTs represents a rational strategy that can meet both patients' and researchers' interests simultaneously 19% of the time.

CONCLUSIONS

There is an inherent ethical dilemma in the conduct of RCTs. The factors that hamper full co-operation between patients and researchers in the conduct of RCTs can be best addressed by: a) having more reliable estimates on the probabilities that new vs. established treatments will be successful, b) improving transparency in the clinical trial system to ensure fulfillment of "the social contract" between patients and researchers.

RCTs in determining treatment indications for intracranial aneurysms: What can we learn from history?

BACKGROUND AND PURPOSE

Many important issues regarding the management of intracranial aneurysms remain controversial. We review the role played by randomized trials in the determination of the best management of intracranial aneurysms in the early era of surgical clipping.

METHODS

Landmark trials and cooperative studies are analyzed and results summarized in a narrative review.

RESULTS

The most convincing evidence in favour of surgical management of ruptured intracranial aneurysms came from early randomized trials conducted from the 1950s to 1970s. Large historical observational studies, performed between the 1970s and 2000, aiming to guide clinical practice, provided only statistical associations mixed with confounding variables. After the early RCTs, the next important gain in reliable knowledge occurred with completion of the ISAT trial, more than 25 years later.

CONCLUSION

The pioneering neurosurgeons of early trials can provide the inspiration necessary to make real progress in understanding the best clinical management of intracranial aneurysms.

Use of mifepristone to treat endometriosis: a review of clinical trials and trial-like studies conducted in China

China was the first country in the world that approved mifepristone (RU-486) for abortion. A total of 6 years after the report published in the Western world indicated that mifepristone may also be effective in treating endometriosis, the first paper on the same topic was published in China in 1997. Since then, over 160 studies on this topic have been published in China. We retrieved 104 papers on clinical trials and trial-like studies conducted in China evaluating the use of mifepristone to treat endometriosis that were published in the last 11 years. We found that the quality of these studies is well below an acceptable level, making it difficult to judge whether mifepristone is truly efficacious. There are intriguing signs that these studies, as a whole, have serious anomalies. The areas that are glaringly deficient are informed consent, choice of outcome measures, the evaluation of outcome measures, data analysis and randomization. The uniformly low quality is disquieting, given the large quantity of studies, the enormous amount of resource and energy put into these studies and, above all, the weighty issue of treatment efficacy that concerns each and every patient with endometriosis. Equally disquieting are the low-quality repetition, the absence of a critical, systematic review on the subject, the lack of suggestions for multicenter clinical trials and the seemingly unnecessary duplication of clinical trials without due informed consent. In view of this, it may be time to institute changes in attitude and practice, and to change education and training programs in the methodology of clinical trials in obstetrics and gynecology research in China.

Assessing the risk of bias in randomized controlled trials in the field of dentistry indexed in the Lilacs (Literatura Latino-Americana e do Caribe em Ciências da Saúde) database

CONTEXT AND OBJECTIVE

Well-conducted randomized controlled trials (RCTs) represent the highest level of evidence when the research question relates to the effect of therapeutic or preventive interventions. However, the degree of control over bias between RCTs presents great variability between studies. For this reason, with the increasing interest in and production of systematic reviews and meta-analyses, it has been necessary to develop methodology supported by empirical evidence, so as to encourage and enhance the production of valid RCTs with low risk of bias. The aim here was to conduct a methodological analysis within the field of dentistry, regarding the risk of bias in open-access RCTs available in the Lilacs (Literatura Latino-Americana e do Caribe em Ciências da Saúde) database.

DESIGN AND SETTING

This was a methodology study conducted at Universidade Federal de São Paulo (Unifesp) that assessed the risk of bias in RCTs, using the following dimensions: allocation sequence generation, allocation concealment, blinding, and data on incomplete outcomes.

RESULTS

Out of the 4,503 articles classified, only 10 studies (0.22%) were considered to be true RCTs and, of these, only a single study was classified as presenting low risk of bias. The items that the authors of these RCTs most frequently controlled for were blinding and data on incomplete outcomes.

CONCLUSION

The effective presence of bias seriously weakened the reliability of the results from the dental studies evaluated, such that they would be of little use for clinicians and administrators as support for decision-making processes.

["Expert approved international standards of evidence-based medicine": a proposal]

Section 35b of the German Code of Social Law Book V (SGB V) mentions "expert approved international standards of evidence-based medicine" (EbM). An idea which started as a reform movement in and for clinical medicine more than twenty years ago is here, from a socio-legal perspective, being utilised as a normative concept in terms of a standard-creating methodology, which is meant to form the basis for the "methods and criteria" of comparative benefit/risk assessment of medical drugs mentioned in the respective law. The present paper places the standards in a hierarchy between "description and analysis of benefits" at the bottom and, at the top, "fundamental concerns of EbM". Apart from a variety of standards, we propose seven basic principles and four fundamental concerns of EbM (pursuit of empirical truth, patient welfare, pragmatism, procedural justice). Each of the basic principles, standards, methods and criteria can be taken to substantiate the respective superordinate category (basic concerns etc.). The position and role of the standards will be explained using the example "search for and selection of literature".

Bias in experimental nursing research: Strategies to improve the quality and explanatory power of nursing science

In a guest editorial in this journal, Rahm Hallberg [Rahm Hallberg, I., 2006. Challenges for future nursing research: providing evidence for health-care practice. International Journal of Nursing Studies 43, 923-927] called for research which has greater explanatory power to determine the effectiveness of nursing interventions. In this paper we critique the suggestion made by the evidence-based nursing movement that randomisation per se is the principal route to better quality nursing research. In contrast, we evaluate the new CONSORT criteria for pragmatic RCTs, which assess the quality of strategies to reduce selection, performance, attrition and detection biases, allowing many different types of comparative studies to be covered by application of the checklist. We propose that randomisation alone is a necessary but insufficient strategy and that nursing researchers rise to Rahm Hallberg's challenge by adopting the extended criteria to assist in the critical appraisal, design and reporting of all experimental research in nursing.

Randomized trials published in some Chinese journals: how many are randomized?

Background

The approximately 1100 medical journals now active in China are publishing a rapidly increasing number of research reports, including many studies identified by their authors as randomized controlled trials. It has been noticed that these reports mostly present positive results, and their quality and authenticity have consequently been called into question. We investigated the adequacy of randomization of clinical trials published in recent years in China to determine how many of them met acceptable standards for allocating participants to treatment groups.

Methods

The China National Knowledge Infrastructure electronic database was searched for reports of randomized controlled trials on 20 common diseases published from January 1994 to June 2005. From this sample, a subset of trials that appeared to have used randomization methods was selected. Twenty-one investigators trained in the relevant knowledge, communication skills and quality control issues interviewed the original authors of these trials about the participant randomization methods and related quality-control features of their trials.

Results

From an initial sample of 37,313 articles identified in the China National Knowledge Infrastructure database, we found 3137 apparent randomized controlled trials. Of these, 1452 were studies of conventional medicine (published in 411 journals) and 1685 were studies of traditional Chinese medicine (published in 352 journals). Interviews with the authors of 2235 of these reports revealed that only 207 studies adhered to accepted methodology for randomization and could on those grounds be deemed authentic randomized controlled trials (6.8%, 95% confidence interval 5.9–7.7). There was no statistically significant difference in the rate of authenticity between randomized controlled trials of traditional interventions and those of conventional interventions. Randomized controlled trials conducted at hospitals affiliated to medical universities were more likely to be authentic than trials conducted at level 3 and level 2 hospitals (relative risk 1.58, 95% confidence interval 1.18–2.13, and relative risk 14.42, 95% confidence interval 9.40–22.10, respectively). The likelihood of authenticity was higher in level 3 hospitals than in level 2 hospitals (relative risk 9.32, 95% confidence interval 5.83–14.89). All randomized controlled trials of pre-market drug clinical trial were authentic by our criteria. Of the trials conducted at university-affiliated hospitals, 56.3% were authentic (95% confidence interval 32.0–81.0).

Conclusion

Most reports of randomized controlled trials published in some Chinese journals lacked an adequate description of randomization. Similarly, most so called 'randomized controlled trials' were not real randomized controlled trials owing toa lack of adequate understanding on the part of the authors of rigorous clinical trial design. All randomized controlled trials of pre-market drug clinical trial included in this research were authentic. Randomized controlled trials conducted by authors in high level hospitals, especially in hospitals affiliated to medical universities had a higher rate of authenticity. That so many non-randomized controlled trials were published as randomized controlled trials reflected the fact that peer review needs to be improved and a good practice guide for peer review including how to identify the authenticity of the study urgently needs to be developed.

The HAC Trial (Harmonic for Acute Cholecystitis) Study. Randomized, double-blind, controlled trial of Harmonic(H) versus Monopolar Diathermy (M) for laparoscopic cholecystectomy (LC) for acute cholecystitis (AC) in adults

BACKGROUND

In the developmental stage of laparoscopic cholecystectomy (LC) it was considered 'unsafe' or 'technically difficult' to perform laparoscopic cholecystectomy for acute cholecystitis (AC). With increasing experience in laparoscopic surgery, a number of centers have reported on the use of laparoscopic cholecystectomy for acute cholecystitis, suggesting that it is technically feasible but at the expense of a high conversion rate, which can be up to 35 per cent and common bile duct lesions.The HARMONIC SCALPEL (H) is the leading ultrasonic cutting and coagulating surgical device, offering surgeons important benefits including: minimal lateral thermal tissue damage, minimal charring and desiccation.Harmonic Scalpel technology reduces the need for ligatures with simultaneous cutting and coagulation: moreover there is not electricity to or through the patient Harmonic Scalpel has a greater precision near vital structures and it produces minimal smoke with improved visibility in the surgical field.In retrospective series LC performed with H was demonstrated feasible and effective with minimal operating time and blood loss: it was reported also a low conversion rate (3.9%).However there are not prospective randomized controlled trials showing the advantages of H compared to MD (the commonly used electrical scalpel) in LC.

METHODS/DESIGN

Aim of this RCT is to demonstrate that H can decrease the conversion rate compared to MD in LC for AC, without a significant increase of morbidity.The patients will be allocated in two groups: in the first group the patient will be submitted to early LC within 72 hours after the diagnosis with H while in the second group will be submitted to early LC within 72 hours with MD.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT00746850.

Efficacy of pneumococcal vaccination in adults: a meta-analysis

BACKGROUND

Clinical trials and meta-analyses have produced conflicting results of the efficacy of unconjugated pneumococcal polysaccharide vaccine in adults. We sought to evaluate the vaccine's efficacy on clinical outcomes as well as the methodologic quality of the trials.

METHODS

We searched several databases and all bibliographies of reviews and meta-analyses for clinical trials that compared pneumococcal polysaccharide vaccine with a control. We examined rates of pneumonia and death, taking the methodologic quality of the trials into consideration.

RESULTS

We included 22 trials involving 101 507 participants: 11 trials reported on presumptive pneumococcal pneumonia, 19 on all-cause pneumonia and 12 on all-cause mortality. The current 23-valent vaccine was used in 8 trials. The relative risk (RR) was 0.64 (95% confidence interval [CI] 0.43-0.96) for presumptive pneumococcal pneumonia and 0.73 (95% CI 0.56-0.94) for all-cause pneumonia. There was significant heterogeneity between the trials reporting on presumptive pneumonia (I(2) = 74%, p < 0.001) and between those reporting on all-cause pneumonia (I(2) = 90%, p < 0.001). The RR for all-cause mortality was 0.97 (95% CI 0.87-1.09), with moderate heterogeneity between trials (I(2) = 44%, p = 0.053). Trial quality, especially regarding double blinding, explained a substantial proportion of the heterogeneity in the trials reporting on presumptive pneumonia and all-cause pneumonia. There was little evidence of vaccine protection in trials of higher methodologic quality (RR 1.20, 95% CI 0.75-1.92, for presumptive pneumonia; and 1.19, 95% CI 0.95-1.49, for all-cause pneumonia in double-blind trials; p for heterogeneity > 0.05). The results for all-cause mortality in double-blind trials were similar to those in all trials combined. There was little evidence of vaccine protection among elderly patients or adults with chronic illness in analyses of all trials (RR 1.04, 95% CI 0.78-1.38, for presumptive pneumococcal pneumonia; 0.89, 95% CI 0.69-1.14, for all-cause pneumonia; and 1.00, 95% CI 0.87-1.14, for all-cause mortality).

INTERPRETATION

Pneumococcal vaccination does not appear to be effective in preventing pneumonia, even in populations for whom the vaccine is currently recommended.

Enriched enrollment: definition and effects of enrichment and dose in trials of pregabalin and gabapentin in neuropathic pain. A systematic review

AIMS

Enriched enrollment study designs have been suggested to be useful for proof of concept when only a proportion of the diseased population responds to a treatment intervention. We aim to investigate whether this really is the case in trials of pregabalin and gabapentin in neuropathic pain.

METHODS

We defined 'complete', 'partial' and 'non-enriched' enrollment, and examined pregabalin and gabapentin trials for the extent of enrichment and for effects of enrichment on efficacy and adverse event outcomes.

RESULTS

There were no studies using complete enriched enrollment; seven trials used partial enriched enrollment and 14 non-enriched enrollment. In pregabalin trials the maximum extent of enrichment was estimated at about 12%. Partial enriched enrollment did not change estimates of efficacy or harm. Over 150-600 mg maximum daily dose there was strong dose dependence for pregabalin.

CONCLUSIONS

A benefit of partial over non-enriched enrollment could not be demonstrated because the degree of enrichment was rather small, and possibly because enrichment produced little enhancement of treatment effect. Whether a greater degree of enrichment would result in important differences is unknown. Researchers reporting clinical trials with any enrichment must describe both process and extent of enrichment. As things stand, the effects of enriched enrollment remain unknown for neuropathic pain trials.

[Controlled clinical trials before Archie Cochrane]

The James Lind Library is an excellent source for documents on the history of the controlled clinical trial. It was named after the 18th century naval surgeon James Lind who performed a multi-arm controlled clinical trial on the treatment of scurvy. Iain Chalmers is a co-founder of the James Lind Library. In his presentation at the 10th anniversary of the GCC, he told us about a number of exemplary studies with special consideration of the evolution of methods to avoid allocation bias. He particularly stressed that over the centuries many individuals from many different countries have contributed to the standard of the randomised controlled trials of our days.

Multivariable modelling for meta-epidemiological assessment of the association between trial quality and treatment effects estimated in randomized clinical trials

Methodological deficiencies are known to affect the results of randomized trials. There are several components of trial quality, which, when inadequately attended to, may bias the treatment effect under study. The extent of this bias, so far only vaguely known, is currently being investigated by 'meta-epidemiological' re-analysis of data collected as part of systematic reviews. As inadequate quality components often co-occur, we maintain that the suspected biases must be evaluated simultaneously. Furthermore, the biases cannot safely be assumed to be homogeneous across systematic reviews. Therefore, a stable multivariable method that allows for heterogeneity is needed for assessing the 'bias coefficients'. We present two general statistical models for analysis of a study of 523 randomized trials from 48 meta-analyses in a random sample of Cochrane reviews: a logistic regression model uses the design of the trials as such to give estimates; a weighted regression model incorporates between-trial variation and thus gives wider confidence intervals, but is computationally lighter and can be used with trials of more general design. In both models, heterogeneity in the bias coefficients can be incorporated in two ways. A stratification approach pools the estimates from models estimated on subgroups of the data. We explore stratification by reviews and by broad trial types, the latter of which gives larger subgroups of the data, circumventing instabilities. A multilevel approach also avoids instabilities and addresses the more fundamental problem of interpretation of the pooled multivariable effect in the presence of heterogeneity.

The Kendrick-Eldering-(Frost) pertussis vaccine field trial

Additional material for this article is available from the James hind Library website [ http://www.jameslindlibrary.org ], where this article was previously published.

Entangled evidence: knowledge making in systematic reviews in healthcare

As the volume of biomedical information escalates and its uses diversify, systematic reviews and meta-analyses - the compilation, selection and statistical analysis of pooled results from similar studies - are becoming an increasingly accepted method in the evaluation of healthcare technologies and interventions. We thus observe a proliferation of laboratories conducting this type of research. How is knowledge constructed in systematic reviews and meta-analysis in healthcare? Drawing on ethnographic data collected during 18 months of fieldwork in a research centre devoted to the development of evidence-based clinical-practice guidelines and systematic reviews, the paper argues that knowledge construction in secondary research in healthcare is structured upon a parallel process of disentanglement and qualification of data. In disentanglement, knowledge practices attempt to extricate data from the milieus in which they are commonly found (databases, texts, other research centres, etc.). In qualification, the focus of activities is on endowing data with new qualities - such as precision, unbiasness or 'fairness'- through the use of templates, graphical platforms and techno-political debates. The accomplishment of these two processes is fundamental to establishing the persuasive power that meta-analyses appear to have in contemporary healthcare politics.

Updates from the British Association of Dermatologists 85th annual meeting, 5-8 July 2005, Glasgow, U.K

The conference highlighted the progress made in understanding recent biological, epidemiological and therapeutic advances in dermatology. Here we provide a synopsis of the main research and clinical findings presented at the meeting of the British Association of Dermatologists (BAD) held during 5-8 July 2005, in Glasgow, U.K., drawing attention to the most important advances and summaries. The BAD meeting was held at the Scottish Exhibition and Conference Centre, Glasgow (Fig. 1). The annual dinner was held in the wonderful setting of Stirling Castle, with Dr Robin Graham-Brown as host.

Bias Control in Trials of Bodywork: A Review of Methodological Issues

OBJECTIVE

To review and summarize the methodological challenges in clinical trials of bodywork or handson mind-body therapies such as Feldenkrais Method, Alexander Technique, Trager Work, Eutony, Body Awareness Therapy, Breath Therapy, and Rolfing, and to discuss ways these challenges can be addressed.

DESIGN

Review and commentary.

METHODS

Search of databases PubMed and EMBASE and screening of bibliographies. Published clinical studies were included if they used individual hands-on approaches and a focus on body awareness, and were not based on technical devices.

RESULTS

Of the 53 studies identified, 20 fulfilled inclusion criteria. No studies blinded subject to the treatment being given, but 5 used an alternative treatment and blinded participants to differential investigator expectations of efficacy. No study used a credible placebo intervention. No studies reported measures of patient expectations. Patient expectations have been measured in studies of other modalities but not of hands-on mind-body therapies. Options are presented for minimizing investigator and therapist bias and bias from differential patient expectations, and for maintaining some control for nonspecific treatment effects. Practical issues with recruitment and attrition resulting from volunteer bias are addressed.

CONCLUSIONS

Rigorous clinical trials of hands-on complementary and alternative therapy interventions are scarce, needed, and feasible. Difficulties with blinding, placebo, and recruitment can be systematically addressed by various methods that minimize the respective biases. The methods suggested here may enhance the rigor of further explanatory trials.

A review of the true methodological quality of nutritional support trials conducted in the critically ill: time for improvement

In this review we sought to appraise the true methodological quality of nutritional support studies conducted in critically ill patients and to compare these findings to the methodological quality of sepsis trials. An extensive literature search revealed 111 randomized controlled trials conducted in critically ill patients evaluating the impact of nutritional support interventions on clinically meaningful outcomes. Compared with sepsis trials, nutritional support studies were significantly less likely to use blinding (32 of 40 versus 35 of 111, P < 0.001) or present an intention-to-treat analysis (37 of 40 versus 64 of 111, P < 0.001). There was a trend toward the less frequent use of randomization methods that are known to maintain allocation concealment (12 of 40 versus 19 of 111, P = 0.10). Although nutritional support studies demonstrated a significant increase in the use of blinding after the publication of the CONSORT statement in 1996 (9 of 47 versus 26 of 64 post-CONSORT, P = 0.023), there were no improvements in other key areas. Previous publications have described the overall methodological quality of sepsis trials as "poor." Nutritional support studies were significantly worse than sepsis trials in all aspects of methodological quality, and there were few improvements noted over time. To detect important differences in clinically meaningful outcomes in critical care, the methodological quality of future studies must be improved.

Methodological issues arising from systematic reviews of the evidence of safety of vaccines

Adaptations to the recognized methods of systematic reviewing are required when addressing questions about safety, particularly about rare and/or long-term serious adverse events. Conducting a systematic review of vaccine safety requires the implementation of novel strategies for locating studies, the use of experimental instruments to assess the quality of non-randomized studies, and the employment of pooling methods for non-randomized data, where appropriate. Standardizing both the indexing of adverse event data on electronic libraries and their reporting would improve the potential of systematic reviews of vaccine to draw accurate conclusions about the safety of a vaccine.

Historiography taking issue: analyzing an experiment with heroin abusers

This article discusses the predicament of historians becoming part of the history they are investigating and illustrates the issue in a particular case. The case is that of the randomized controlled trial (RCT)-more specifically, its use for testing the effects of providing heroin to severe heroin abusers. I counter the established view of the RCT as a matter of timeless logic and argue that this research design was developed in the context of administrative knowledge making under twentieth-century economic liberalism of which it epitomizes some central values. I also argue that the applicability of the RCT depends on the degree to which its advocates can define the issue to be studied according to its inherent values. Next, I demonstrate how advocates of an RCT with heroin provision in the Netherlands steered the political discussion on heroin provision and how the values of economic liberalism also shaped the results of the Dutch maintenance experiment. In addition, I relate how my analysis of this experiment became part of political debates in the Netherlands. Contrary to my intentions, adversaries of heroin maintenance used my critique on the heroin RCT as an argument against heroin maintenance. Such risks are inherent to historiography and sociology of science aiming at practical relevance while challenging treasured scientific beliefs. I conclude that it still seems better to expose arguments on unjustified certainties than to suppress them for strategic reasons.

Randomized trials and public health interventions: time to end the scientific double standard

Randomized controlled trials have become the acceptedstandard for testing the efficacy and safety of clinical treatments but are rarely used in evaluating public health interventions. This article reviews the types of interventions and outcomes for which randomized trials are necessary to provide a scientifically rigorous evaluation. It also presents the results from two trials analyzed both experimentally and observationally to illustrate why observational studies can yield highly biased findings. This article provides a brief historical account of controlled human experiments, which reveals a surprising earlier tradition of rigorous experimental studies in the social sciences. After contrasting that tradition with the status quo, this article concludes with a plea to reject the double scientific standard of what constitutes acceptable evidence of efficacy for clinical versus public health interventions.

When can a clinical trial be called 'randomized'?

It is widely recognized that, in the context of the evaluation of medical interventions, randomized clinical trials constitute the gold standard. This is because randomization tends to balance both measured and unmeasured baseline characteristics, allows for masking, and provides a basis for inference. It is understandable, then, that investigators would wish to utilize this methodology whenever it is feasible to do so. Unfortunately, some studies are labeled as randomized when in fact they are not. These studies then receive more credibility, and influence medical practice, more than they deserve to. After reviewing the benefits of randomization, paying particular attention to the specific aspects of randomization that confer each benefit, we will explore the issue of what constitutes a randomized study.

Informing participants of allocation to placebo at trial closure: postal survey

OBJECTIVES

To assess whether and how investigators of placebo controlled randomised trials inform participants of their treatment allocation at trial closure and to assess barriers to feedback.

DESIGN

Postal survey with a semistructured questionnaire.

PARTICIPANTS

All investigators who published a placebo controlled randomised trial in 2000 in five leading medical journals, and a random sample of 120 trials listed in the national research register database.

MAIN OUTCOME MEASURES

Number of investigators who informed participants of their treatment allocation at trial closure, methods for delivering the information, and barriers to unmasking treatment.

RESULTS

45% of investigators informed either all or most participants of their treatment allocation, and 55% did not inform any participant or only informed those who asked. The main reasons for not informing participants were that the investigators never considered this option (40%) or to avoid biasing results at study follow up (24%).

CONCLUSION

Further research is required to examine sensitive ways to communicate treatment information to trial participants.