The importance of allocation concealment and patient blinding in osteoarthritis trials: a meta-epidemiologic study

OBJECTIVE

To evaluate the association of adequate allocation concealment and patient blinding with estimates of treatment benefits in osteoarthritis trials.

METHODS

We performed a meta-epidemiologic study of 16 meta-analyses with 175 trials that compared therapeutic interventions with placebo or nonintervention control in patients with hip or knee osteoarthritis. We calculated effect sizes from the differences in means of pain intensity between groups at the end of followup divided by the pooled SD and compared effect sizes between trials with and trials without adequate methodology.

RESULTS

Effect sizes tended to be less beneficial in 46 trials with adequate allocation concealment compared with 112 trials with inadequate or unclear concealment of allocation (difference -0.15; 95% confidence interval [95% CI] -0.31, 0.02). Selection bias associated with inadequate or unclear concealment of allocation was most pronounced in meta-analyses with large estimated treatment benefits (P for interaction < 0.001), meta-analyses with high between-trial heterogeneity (P = 0.009), and meta-analyses of complementary medicine (P = 0.019). Effect sizes tended to be less beneficial in 64 trials with adequate blinding of patients compared with 58 trials without (difference -0.15; 95% CI -0.39, 0.09), but differences were less consistent and disappeared after accounting for allocation concealment. Detection bias associated with a lack of adequate patient blinding was most pronounced for nonpharmacologic interventions (P for interaction < 0.001).

CONCLUSION

Results of osteoarthritis trials may be affected by selection and detection bias. Adequate concealment of allocation and attempts to blind patients will minimize these biases.

Matched-pair study showed higher quality of placebo-controlled trials in Western phytotherapy than conventional medicine

OBJECTIVES

Herbal medicine (phytotherapy) is widely used, but the evidence for its effectiveness is a matter of ongoing debate. We compared the quality and results of trials of Western phytotherapy and conventional medicine.

STUDY DESIGN AND SETTING

A random sample of placebo-controlled trials of Western phytotherapy was identified in a comprehensive literature search (19 electronic databases). Conventional medicine trials matched for condition and type of outcome were selected from the Cochrane Central Controlled Trials Register (issue 1, 2003). Data were extracted in duplicate. Trials described as double-blind, with adequate generation of allocation sequence and adequate concealment of allocation were assumed to be of higher methodological quality.

RESULTS

Eighty-nine herbal medicine and 89 matched conventional medicine trials were analyzed. Studies of Western herbalism were smaller, less likely to be published in English, and less likely to be indexed in MEDLINE than their counterparts from conventional medicine. Nineteen (21%) herbal and four (5%) conventional medicine trials were of higher quality. In both groups, smaller trials showed more beneficial treatment effects than larger trials.

CONCLUSIONS

Our findings challenge the widely held belief that the quality of the evidence on the effectiveness of herbal medicine is generally inferior to the evidence available for conventional medicine.