Meta-analysis of clinical trials as a scientific discipline. I: Control of bias and comparison with large co-operative trials

Randomised clinical trials in plastic surgery: Survey of output and quality of reporting

Randomised clinical trials (RCTs) are considered the best level of evidence when evaluating interventions. Report quality is often used as a surrogate measure of methodological quality, with poorly reported trials assumed to be poorly conducted. To address this problem the CONsolidated Standards of Reporting Trials (CONSORT) statement was published, encouraging authors to explicitly report certain items of information. The aim of this study was to survey RCTs published in the plastic surgery literature and determine quality of reporting. All RCTs published in Plastic & Reconstructive Surgery, British Journal of Plastic Surgery and Annals of Plastic Surgery from 1980 to 2004 were retrieved using a Medline search. Quality of reporting was assessed using a 17-item checklist derived from the CONSORT statement. One hundred and thirty three trials were eligible for assessment. 56 (42.1%) originated from European countries. Anaesthesia/analgesia was the most popular topic addressed and accounted for 23 (17.3%) of all studies. Quality of reporting analysis revealed wide variation between items. Sample size calculation was only reported in 17(12.8%) trials. Randomisation methodology, allocation concealment and blind investigator/assessment was reported in 39 (29.3%), 25 (18.8%) and 69 (51.9%) trials respectively. Study limitations were also infrequently reported and present in only 45 (33.8%) trials. This study indicates the annual output of plastic surgery RCTs is increasing and a variety of topics are covered. However, reporting of certain key items is inadequate. Awareness of the CONSORT statement and more attention to the quality of reporting may improve matters.

Ensayos clínicos aleatorizados (CONSORT)

It has been repeatedly shown that the information supplied in publications of clinical trials is frequently insufficient or inaccurate and that some methodologic problems are associated with exaggerated estimates of the effect of healthcare interventions. To improve the quality of reports of clinical trials, a group of scientists and editors developed the CONSORT statement (Consolidated Standards of Reporting Trials), a 22-item checklist (plus flow diagram), that can be used by authors, editors, reviewers, and readers. After publication in 1996, CONSORT was adopted by several journals and editorial groups. In 1999, a second version was drawn up, which was published in 2001. This article presents the Spanish translation of the two elements integrating the revised CONSORT, the flow diagram and the 22-item checklist, and provides a short comment on each of them. Previous publications of the CONSORT statement and other useful resources such as examples of what are considered good communications may be obtained from the CONSORT web site (http://www.consort-statement.org).

Language of publication restrictions in systematic reviews gave different results depending on whether the intervention was conventional or complementary

OBJECTIVE

To assess whether language of publication restrictions impact the estimates of an intervention's effectiveness, whether such impact is similar for conventional medicine and complementary medicine interventions, and whether the results are influenced by publication bias and statistical heterogeneity.

STUDY DESIGN AND SETTING

We set out to examine the extent to which including reports of randomized controlled trials (RCTs) in languages other than English (LOE) influences the results of systematic reviews, using a broad dataset of 42 language-inclusive systematic reviews, involving 662 RCTs, including both conventional medicine (CM) and complementary and alternative medicine (CAM) interventions.

RESULTS

For CM interventions, language-restricted systematic reviews, compared with language-inclusive ones, did not introduce biased results, in terms of estimates of intervention effectiveness (random effects ration of odds rations ROR=1.02; 95% CI=0.83-1.26). For CAM interventions, however, language-restricted systematic reviews resulted in a 63% smaller protective effect estimate than language-inclusive reviews (random effects ROR=1.63; 95% CI=1.03-2.60).

CONCLUSION

Language restrictions do not change the results of CM systematic reviews but do substantially alter the results of CAM systematic reviews. These findings are robust even after sensitivity analyses, and do not appear to be influenced by statistical heterogeneity and publication bias.

Sample size calculations in randomised trials: mandatory and mystical

Investigators should properly calculate sample sizes before the start of their randomised trials and adequately describe the details in their published report. In these a-priori calculations, determining the effect size to detect--eg, event rates in treatment and control groups--reflects inherently subjective clinical judgments. Furthermore, these judgments greatly affect sample size calculations. We question the branding of trials as unethical on the basis of an imprecise sample size calculation process. So-called underpowered trials might be acceptable if investigators use methodological rigor to eliminate bias, properly report to avoid misinterpretation, and always publish results to avert publication bias. Some shift of emphasis from a fixation on sample size to a focus on methodological quality would yield more trials with less bias. Unbiased trials with imprecise results trump no results at all. Clinicians and patients deserve guidance now.

Risk factors for pre-eclampsia at antenatal booking: systematic review of controlled studies

OBJECTIVE

To determine the risk of pre-eclampsia associated with factors that may be present at antenatal booking.

DESIGN

Systematic review of controlled studies published 1966-2002.

DATA SYNTHESIS

Unadjusted relative risks were calculated from published data.

RESULTS

Controlled cohort studies showed that the risk of pre-eclampsia is increased in women with a previous history of pre-eclampsia (relative risk 7.19, 95% confidence interval 5.85 to 8.83) and in those with antiphospholipids antibodies (9.72, 4.34 to 21.75), pre-existing diabetes (3.56, 2.54 to 4.99), multiple (twin) pregnancy (2.93, 2.04 to 4.21), nulliparity (2.91, 1.28 to 6.61), family history (2.90, 1.70 to 4.93), raised blood pressure (diastolic > or = 80 mm Hg) at booking (1.38, 1.01 to 1.87), raised body mass index before pregnancy (2.47, 1.66 to 3.67) or at booking (1.55, 1.28 to 1.88), or maternal age > or = 40 (1.96, 1.34 to 2.87, for multiparous women). Individual studies show that risk is also increased with an interval of 10 years or more since a previous pregnancy, autoimmune disease, renal disease, and chronic hypertension.

CONCLUSIONS

These factors and the underlying evidence base can be used to assess risk at booking so that a suitable surveillance routine to detect pre-eclampsia can be planned for the rest of the pregnancy.

Effects of the search technique on the measurement of the change in quality of randomized controlled trials over time in the field of brain injury

Background

To determine if the search technique that is used to sample randomized controlled trial (RCT) manuscripts from a field of medical science can influence the measurement of the change in quality over time in that field.

Methods

RCT manuscripts in the field of brain injury were identified using two readily-available search techniques: (1) a PubMed MEDLINE search, and (2) the Cochrane Injuries Group (CIG) trials registry. Seven criteria of quality were assessed in each manuscript and related to the year-of-publication of the RCT manuscripts by regression analysis.

Results

No change in the frequency of reporting of any individual quality criterion was found in the sample of RCT manuscripts identified by the PubMed MEDLINE search. In the RCT manuscripts of the CIG trials registry, three of the seven criteria showed significant or near-significant increases over time.

Conclusions

We demonstrated that measuring the change in quality over time of a sample of RCT manuscripts from the field of brain injury can be greatly affected by the search technique. This poorly recognized factor may make measurements of the change in RCT quality over time within a given field of medical science unreliable.

Comparison of risk stratification with pharmacologic and exercise stress myocardial perfusion imaging: a meta-analysis

BACKGROUND

Although pharmacologic stress myocardial perfusion imaging (MPI) and exercise stress MPI have comparable diagnostic accuracy, their comparative value for risk stratification of patients with known or suspected coronary disease is not known.

METHODS AND RESULTS

The data of 14,918 patients were combined from 24 studies evaluating prognosis in patients undergoing either pharmacologic stress or exercise stress MPI. Studies were included if a 2 x 2 table for hard cardiac events (cardiac death and myocardial infarction [MI]) could be constructed from the data available. Excluded were studies performed for post-MI, post-revascularization, or preoperative risk stratification. A weighted t test was used to compare the cardiac events, and a random effects model was used to calculate summary odds ratios. Summary odds ratios for hard cardiac events were similar for pharmacologic stress and exercise stress MPI. Summary receiver operating characteristic curves also showed no difference in discriminatory power between the stressors. The cardiac event rates were significantly higher with normal and abnormal test results with pharmacologic stress MPI than with exercise stress MPI (1.78% vs 0.65% [P < .001] for normal results and 9.98% vs 4.3% [P < .001] for abnormal results). Subgroup analysis revealed that both cardiac death and nonfatal MI were significantly higher with pharmacologic stress MPI. Patients undergoing pharmacologic stress MPI had a significantly higher prevalence of poor prognostic factors, and meta-regression revealed that exercise capacity was the single most important predictor of cardiac events.

CONCLUSIONS

This meta-analysis shows that exercise stress MPI and pharmacologic stress MPI are comparable in their ability to risk-stratify patients. However, patients undergoing pharmacologic stress studies are at a higher risk for subsequent cardiac events. This is true even for those with normal perfusion imaging results.

Systematic review of the literature on postpartum care: methodology and literature search results

BACKGROUND

The postpartum period is a time for multiple clinical interventions. To date, no critical review of these interventions exists. This systematic review examined evidence for the effectiveness of postpartum interventions that have been reported in the literature.

METHODS

MEDLINE, Cinahl, PsycINFO, and the Cochrane Library were searched for randomized controlled trials of interventions initiated from immediately after birth to 1 year in postnatal women that were conducted in North America, Europe, Australia, or New Zealand. The initial literature search was done in 1999, using postpartum content search terms, and was enhanced in 2003. In both years, bibliographic databases were searched from their inception. Studies were categorized into key topic areas. Data extraction forms were developed and completed for each study, and the quality of each study was systematically reviewed. Groups of studies in a topic area were reviewed together, and clinically relevant questions emanating from the studies were identified to determine whether the studies, alone or together, provided evidence to support the clinical intervention.

RESULTS

In the 1999 search, of 671 studies identified, 140 studies were randomized controlled trials that met the selection criteria: 41 studies related to breastfeeding, 33 to postpartum perineal pain management, and 63 to 11 other key topic areas (Papanicolaou test, rubella immunization, contraception, postpartum support, early discharge, postpartum depression and anxiety, postpartum medical disorders, smoking cessation, nutrition supplements other than breastfeeding, effects of pelvic floor exercise, and effects of early newborn contact). The results of the systematic review of each topic will be summarized in separate papers as they are completed.

CONCLUSIONS

This systematic search has identified key topic areas in postpartum care for which randomized controlled trials have been conducted. Our ultimate goal is to provide evidence-based guidelines on the use of routine postpartum interventions.

Standards for Clinical Trials in Male Sexual Dysfunction: Erectile Dysfunction and Rapid Ejaculation

INTRODUCTION

The introduction of safe and effective therapies for sexual dysfunctions depend upon appropriate clinical protocol design, study procedures, data collection and analysis.

AIM

To provide recommendations/guidelines concerning state-of-the-art knowledge for standards for clinical trials in sexual dysfunction in men, particularly in the areas of erectile dysfunction and rapid ejaculation.

METHODS

An International Consultation in collaboration with the major urology and sexual medicine associations assembled over 200 multidisciplinary experts from 60 countries into 17 committees. Committee members established specific objectives and scopes for various male and female sexual medicine topics. The recommendations concerning state-of-the-art knowledge in the respective sexual medicine topic represent the opinion of experts from five continents developed in a process over a 2-year period. Concerning the Standards for Clinical Trials in Male Sexual Dysfunction Committee, there were six experts from four countries.

MAIN OUTCOME MEASURE

Expert opinion was based on grading of evidence-based medical literature, widespread internal committee discussion, public presentation and debate.

RESULTS

Drug development requires a multiphased approach. Phase 1 studies investigate multiple-dose safety, tolerability and pharmacokinetic issues. Phase 2 programs explore dose ranging (lowest effective, maximally tolerated and toxic doses). Phase 3 trials provide the substantial evidence including drug-drug interaction data and studies in special populations. Clinical studies require validated outcome assessment instruments conducted in defined but representative patient populations. Daily diaries or per-event questionnaires are patient-reported outcomes that assist in retrospective questionnaire interpretation. A qualified biostatistician should calculate the sample power for the trial, type of statistical model and design employed, use of covariate or subgroup analyses, and calculation of effect sizes.

CONCLUSIONS

More research is needed in developing standards for use in the development of clinical trials and outcomes assessment researching either erectile dysfunction or rapid ejaculation.

The Persistence of Underpowered Studies in Psychological Research: Causes, Consequences, and Remedies

Underpowered studies persist in the psychological literature. This article examines reasons for their persistence and the effects on efforts to create a cumulative science. The "curse of multiplicities" plays a central role in the presentation. Most psychologists realize that testing multiple hypotheses in a single study affects the Type I error rate, but corresponding implications for power have largely been ignored. The presence of multiple hypothesis tests leads to 3 different conceptualizations of power. Implications of these 3 conceptualizations are discussed from the perspective of the individual researcher and from the perspective of developing a coherent literature. Supplementing significance tests with effect size measures and confidence intervals is shown to address some but not necessarily all problems associated with multiple testing.

Multivariate meta-analysis

Meta-analysis is now a standard statistical tool for assessing the overall strength and interesting features of a relationship, on the basis of multiple independent studies. There is, however, recent acknowledgement of the fact that in many applications responses are rarely uniquely determined. Hence there has been some change of focus from a single response to the analysis of multiple outcomes. In this paper we propose and evaluate three Bayesian multivariate meta-analysis models: two multivariate analogues of the traditional univariate random effects models which make different assumptions about the relationships between studies and estimates, and a multivariate random effects model which is a Bayesian adaptation of the mixed model approach. Our preferred method is then illustrated through an analysis of a new data set on parental smoking and two health outcomes (asthma and lower respiratory disease) in children.

Evidence-based medicine for orthopedic practice

Evidence-based medicine (EBM) is regarded as a new paradigm in medical practice, equal in enormity to the human genome project. However, there is still much confusion and misunderstanding about the concept and content of EBM. It is often limited to searching the literature and reading papers, serving cost cutters, and suppressing clinical freedom. Some believe that the use of clinical guidelines or the managed care system intimidates doctors' discretion during clinical practice and that EBM is a fashionable tendency of a group of medical academics armed with epidemiological and statistical jargon. Medical practice is a lifelong, continuous process of self-learning, and it requires clinicians to keep up to date on various developments. EBM is our practice for integrating individual clinical expertise with the best available evidence when making decisions about our care for each patient. EBM is one answer for making it possible to cover most of our activities as orthopedic surgeons, from the daily practice of patient care to writing and reading scientific papers.

Issues in the meta-analysis of cluster randomized trials

Meta-analyses involving the synthesis of evidence from cluster randomization trials are being increasingly reported. These analyses raise challenging methodologic issues beyond those raised by meta-analyses which include only individually randomized trials. In this paper we review and comment on a selected number of these issues, including problems of study heterogeneity, difficulties in estimating design effects from individual trials and the choice of statistical methods.

Meta-analysis: neither quick nor easy

BACKGROUND

Meta-analysis is often considered to be a simple way to summarize the existing literature. In this paper we describe how a meta-analysis resembles a conventional study, requiring a written protocol with design elements that parallel those of a record review.

METHODS

The paper provides a structure for creating a meta-analysis protocol. Some guidelines for measurement of the quality of papers are given. A brief overview of statistical considerations is included. Four papers are reviewed as examples. The examples generally followed the guidelines we specify in reporting the studies and results, but in some of the papers there was insufficient information on the meta-analysis process.

CONCLUSIONS

Meta-analysis can be a very useful method to summarize data across many studies, but it requires careful thought, planning and implementation.

Allocation concealment in randomised trials: defending against deciphering

Proper randomisation rests on adequate allocation concealment. An allocation concealment process keeps clinicians and participants unaware of upcoming assignments. Without it, even properly developed random allocation sequences can be subverted. Within this concealment process, the crucial unbiased nature of randomised controlled trials collides with their most vexing implementation problems. Proper allocation concealment frequently frustrates clinical inclinations, which annoys those who do the trials. Randomised controlled trials are anathema to clinicians. Many involved with trials will be tempted to decipher assignments, which subverts randomisation. For some implementing a trial, deciphering the allocation scheme might frequently become too great an intellectual challenge to resist. Whether their motives indicate innocent or pernicious intents, such tampering undermines the validity of a trial. Indeed, inadequate allocation concealment leads to exaggerated estimates of treatment effect, on average, but with scope for bias in either direction. Trial investigators will be crafty in any potential efforts to decipher the allocation sequence, so trial designers must be just as clever in their design efforts to prevent deciphering. Investigators must effectively immunise trials against selection and confounding biases with proper allocation concealment. Furthermore, investigators should report baseline comparisons on important prognostic variables. Hypothesis tests of baseline characteristics, however, are superfluous and could be harmful if they lead investigators to suppress reporting any baseline imbalances.

Effect of electrical stimulation on musculoskeletal systems; a meta-analysis of controlled clinical trials

This study was a meta-analysis to examine whether electrical stimulation has specific effects in the healing of musculoskeletal repair process and in the diminution of symptoms with bone and joint disorders. Using MEDLINE (1966-1999) and EMBASE (1985-1999) a search for articles was carried out with four medical subject headings. Data were extracted from all the accessed articles and additionally collected from appropriate journal lists. A total of 20 randomized controlled trials on bones was identified which assessed healing of fractures, bone graft, and other conditions; and 29 randomized controlled trials on soft tissues and joints were also found, dealing with healing of skin wounds or dermal ulcers, soft tissue injury, and other conditions. Using criteria through which the quality of studies was assessed, the content of the articles was reorganized into a tabular form. The majority of the identified articles reported positive findings, but all the trials showed methodological flaws to some extent. Because of heterogeneity of the studies and the various outcome measurements, pooling of only part of the data was performed. The combined results of 12 trials on bones and 16 trials on soft tissues, the cases in which major endpoints were mainly union or healing rate, revealed statistically significant effects. The studies in this review had some methodological limitations, and the selected pooled trials do not constitute acceptable proof that electrical stimulation has specific effects on health. However, one cannot ignore the statistically significant positive findings reported in the trials, from which extracted data were able to be combined.

Consensus evidence evaluation in resuscitation research: analysis of Type I and Type II errors

OBJECTIVE

This paper addresses the following statistical question: 'if genuine improvements in cardiopulmonary resuscitation (CPR) were discovered that doubled the probability of resuscitation success in a series of randomized clinical trials, would they be recognized and incorporated into consensus guidelines?'.

METHODS

Statistical powers for hypothetical individual clinical trials comparing experimental and control CPR were computed as a function of the study N when the true probabilities for immediate survival, 24 h survival, and discharge survival in the experimental group were twice those in the control group. Next, the binomial distributions describing the numbers of statistically significant studies in a series of equally powered trials of the same intervention were determined. These were compared with varying criteria for consensus among expert reviewers, expressed in terms of the number of 'positive' studies showing a statistically significant difference that reviewers would require before approving the experimental method.

RESULTS

False-negative evaluations (i.e. failures to approve a technique that actually doubled survival) were extremely common under a wide range of realistic assumptions and consensus criteria, especially when simulated long-term survival data were considered. Similar methods showed that false-positive evaluations would be extremely rare, provided that at least two of the clinical trials in a series showed a statistically significant benefit of the experimental method.

CONCLUSIONS

Optimization of evidence evaluation can and should be carried out to make better use of available data in creating resuscitation guidelines. One simple approach is the 'two and one quarter test': if at least two well-conducted studies in a series are significantly positive (P<0.05) comprising at least one-quarter of all studies in the series, a positive effect can be inferred with small Type I and Type II errors. In addition, greater reliance on modern, unbiased methods such as cumulative meta-analysis is needed to increase the sensitivity of evidence evaluation for detecting useful innovations in resuscitation.

Systematic review of population-based epidemiological studies of oro-facial pain

OBJECTIVES

To conduct a systematic review of epidemiological literature in order to determine the prevalence and associated risk factors of oro-facial pain.

DATA

Population based observational studies (cohorts, cross-sectional and case-control studies) of oro-facial pain, published in the English language, prior to 1999 were included.

SOURCES

Electronic databases (Medline, Embase, Cinahl, BIDS and Health CD) were searched. Reference lists of relevant articles were examined, and the journals "Pain" and "Community Dentistry and Oral Epidemiology" were handsearched for the years 1994-1998.

RESULTS

The results of the search strategy were screened for relevance. A standardised checklist was used to assess the methodological quality of each study by two reviewers before an attempt was made to summarise the results. The median quality score was 70% of the maximum attainable score. Due to methodological issues, it was not possible to pool the data on the prevalence of oro-facial pain. Age, gender and psychological factors were found to be associated with OFP, however there was not enough information on other factors such as local mechanical and co-morbidities to draw any reliable conclusions. None of the factors fully fulfilled criteria for causality.

CONCLUSIONS

There is a need for good quality epidemiological studies of oro-facial pain in the general population. To enable comprehensive examination of the aetiology of oro-facial pain, it is necessary to address a broad range of factors including demography and life-style, local mechanical factors, medical history and psychological factors. Future studies should recruit adequately sized samples for precise determination of the prevalence and detection of important associated factors. Data on potential confounders and effect modifiers should also be collected and adjusted for in the statistical analysis.

Trends in acute ischemic stroke trials through the 20th century

BACKGROUND AND PURPOSE

The advent of controlled clinical trials revolutionized clinical medicine over the course of the 20th century. The objective of this study was to quantitatively characterize developments in clinical trial methodology over time in the field of acute ischemic stroke.

METHODS

All controlled trials targeting acute ischemic stroke with a final report in English were identified through MEDLINE and international trial registries. Data regarding trial design, implementation, and results were extracted. A formal 100-point scale was used to rate trial quality.

RESULTS

A total of 178 controlled acute stroke trials were identified, encompassing 73 949 patients. Eighty-eight trials involved neuroprotective agents, 59 rheological/antithrombotic agents, 26 agents with both neuroprotective and rheological/antithrombotic effects, and 5 a nonpharmacological intervention. Only 3 trials met conventional criteria for a positive outcome. Between the 1950s and 1990s, the number of trials per decade increased from 3 to 99, and mean trial sample size increased from 38 (median, 26) to 661 (median, 113). During 1980-1999, median time window allowed for enrollment decreased per half decade from 48 to 12 hours. Reported pharmaceutical sponsorship increased substantially over time, from 38% before 1970 to 68% in the 1990s. Trial quality improved substantially from a median score of 12 in the 1950s to 72 in the 1990s.

CONCLUSIONS

Accelerating trends in acute stroke controlled trials include growth in number, sample size, and quality, and reduction in entry time window. These changes reflect an increased understanding of the pathophysiology of acute stroke, the imperative for treatment initiation within a critical time window, and more sophisticated trial design.

Influence of ankle support on joint range of motion before and after exercise: a meta-analysis

STUDY DESIGN

Meta-analysis.

OBJECTIVE

To evaluate the effects of different types of ankle support on ankle and foot joint range of motion before and after activity using meta-analysis procedures.

BACKGROUND

The effects of ankle support on joint range of motion before and after exercise has been extensively studied, but the results among studies are not consistent. Obtaining knowledge from synthesizing the available literature with a meta-analysis can provide a greater understanding of these effects.

METHODS AND MEASURES

A total of 253 cases from 19 studies were examined and included in this analysis. The treatment variables were ankle support with 3 levels (tape, lace-up, and semirigid) and time with 2 levels (before exercise and after exercise). Standardized effect sizes were computed for inversion, eversion, dorsiflexion, and plantar flexion range of motion to measure the difference between control and treatment groups at each point in time. Effect sizes were analyzed using a mixed-model factorial analysis of variance.

RESULTS

Before exercise, the semirigid condition (-2.97 +/- 0.63) demonstrated greater restriction compared with the tape (-2.33 +/- 0.38) and lace-up conditions (-2.18 +/- 0.86) for inversion range of motion. After exercise, the semirigid condition (-3.85 +/- 0.64) restricted inversion range of motion more than the tape (-1.07 +/- 0.20) and lace-up (-1.56 +/- 0.29) conditions. No differences were found between the mean effect sizes for the tape and lace-up conditions before and after exercise. With respect to eversion range of motion, the semirigid support (-2.69 +/- 0.43) provided greater restraint compared with the tape (-1.00 +/- 0.21) and lace-up (-1.40 +/- 0.47) conditions. The lace-up condition also displayed greater support compared with tape alone. For dorsiflexion range of motion, greatest overall support was provided by the tape condition (-0.94 +/- 0.06) compared with the lace-up condition (-0.51 +/- 0.06).

CONCLUSIONS

The greatest restriction of motion in the frontal plane was offered by the semirigid support condition, whereas taping offered the most support for limiting dorsiflexion range of motion. The results of this study may help clinicians make rational decisions concerning the selection of ankle appliances for preventing acute or chronic reinjury.

Corticosteroids for the resolution of malignant bowel obstruction in advanced gynaecological and gastrointestinal cancer

BACKGROUND

Gastrointestinal and ovarian cancers are common cancers. The incidence of associated malignant bowel obstruction in patients with advanced cancers of these types is not known, and the best management of these patients is controversial. Inappropriate management may result in uncontrolled (faeculant) vomiting, pain and distress. Management of the symptoms can include palliative surgery, nasogastric tube suction together with intravenous fluids, or pharmacological means, such as corticosteroids. There is uncertainty regarding both the efficacy and possible harmful effects of corticosteroids, and also the most effective type, dose/dosing regime, route and period of administration.

OBJECTIVES

To locate, appraise and summarise evidence from scientific studies on intestinal obstruction due to advanced gynaecological and gastrointestinal cancer, in order to assess the efficacy of corticosteroids.

SEARCH STRATEGY

A comprehensive list of all studies was provided by an extensive search of the electronic databases, relevant journals, reference lists, the grey literature, contact with investigators and other search strategies outlined in the methods.

SELECTION CRITERIA

As the review concentrates on the 'best evidence' available of the role of corticosteroids in malignant bowel obstruction due to advanced gynaecological and gastrointestinal cancer the inclusion criteria were kept fairly broad so as to include all studies relevant to the question

DATA COLLECTION AND ANALYSIS

Data extraction forms were used to collect data from the studies included in the review. The data was checked by a secondary searcher to reduce error. A qualitative analysis was performed of the dichotomous data of resolution of obstruction and death at one month, obtained from the randomised controlled trials of corticosteroids versus placebo. Both fixed and random effect models were used. Number needed to treat (NNT) was derived from the odds ratio. Kaplan-Meier survival curves from individual patient data were also analysed. Studies of lower methodological quality were assessed in a qualitative manner.

MAIN RESULTS

Three unpublished, randomised, placebo, double blind controlled trials and seven published (prospective and retrospective) trials were considered eligible. Using only the randomised trials, there is a trend, which is not statistically significant, for the resolution of bowel obstruction using corticosteroids. There is no statistically significant difference in mortality at one month, nor in the Kaplan-Meier curves, which describe the survival of patients on corticosteroids or placebo. Number needed to treat is 6 (3, infinity) ie six patients need to be treated with corticosteroids to resolve one episode of bowel obstruction. The results are robust to fixed and random effects models and to 'best' and 'worst case' scenarios on the missing data from patients. The morbidity associated with corticosteroids appears to be very low, though the quality of the data limits this conclusion. No other outcomes were available from the published data or from the authors.

REVIEWER'S CONCLUSIONS

There is a trend for evidence that corticosteroids of dose range 6-16 mg dexamethasone given intravenously may bring about the resolution of bowel obstruction. Equally, the incidence of side effects in all the included studies is extremely low. Corticosteroids do not seem to affect the length of survival of these patients.

Surgery for the resolution of symptoms in malignant bowel obstruction in advanced gynaecological and gastrointestinal cancer

BACKGROUND

Intestinal obstruction commonly occurs in progressive advanced gynaecological and gastrointestinal cancers. Management of these patients is difficult due to the patients deteriorating mobility and function (performance status), the lack of further chemotherapeutic options and the high mortality and morbidity associated with palliative surgery. There are marked variations in clinical practice concerning surgery in these patients between different countries, gynaecological oncology units, and general hospitals as well as referral patterns from oncologists under whom these patients are often admitted under. There is therefore a need for all the present information to be collated, analysed (with appropriate palliative care outcomes) to establish if surgery is of benefit and what further research is needed.

OBJECTIVES

The objective was to locate, appraise and summarise evidence from scientific studies on intestinal obstruction due to advanced gynaecological and gastrointestinal cancer, in order to assess the efficacy of surgery.

SEARCH STRATEGY

A comprehensive list of studies was provided by an extensive search of electronic databases, relevant journals, bibliographic databases, conference proceedings, reference lists, the grey literature, personal contact and the world wide web.

SELECTION CRITERIA

As the review concentrates on the 'best evidence' available of the role of surgery in malignant bowel obstruction in advanced gynaecological and gastrointestinal cancer the inclusion criteria were kept broad (included both prospective and retrospective studies) so as to include all studies relevant to the question.

DATA COLLECTION AND ANALYSIS

Data extraction forms were used to collect data from the studies included in the review. Two researchers extracted the data independently to reduce error. Due to the methodological quality of the studies, only a qualitative assessment was possible.

MAIN RESULTS

The role of surgery in malignant bowel obstruction remains controversial, and no firm conclusions from the many retrospective case series can be made. Control of symptoms varies from 42% to over 80%, though it is often unclear how symptoms were measured and whether the tools used to collect symptom scores are validated. There is a large range in the rates of re-obstruction, from 10-50%, though time to re-obstruction was often not included. There is a wide range of postoperative morbidity and mortality, although again the definition of both these surgical outcomes varied between many of the papers.

REVIEWER'S CONCLUSIONS

The role of surgery in malignant bowel obstruction needs careful evaluation, using validated outcome measures of symptom control and quality of life scores. Further information would include re-obstruction rates together with the morbidity associated with the various surgical procedures. Currently, bowel obstruction is managed empirically, and there are marked variations in clinical practice by different units. There needs to be a greater standardisation of management so that comparisons between different series can be made.

Moderate-term, low-dose corticosteroids for rheumatoid arthritis

OBJECTIVES

To perform a systematic review of low-dose corticosteroid efficacy in the moderate term for the treatment of rheumatoid arthritis (RA).

SEARCH STRATEGY

We conducted a search in MEDLINE from 1966 to 1998, using the keywords "corticosteroids" and "rheumatoid arthritis". We also handsearched all issues of Arthritis and Rheumatism and the Scandinavian Journal of Rheumatology from their dates of first publication to 1994. Furthermore, we examined all Arthritis and Rheumatism abstracts over the 15 year period preceding 1994. References of all identified studies were searched for relevant trials. Authors of unpublished manuscripts were contacted.

SELECTION CRITERIA

Studies were selected by two independent reviewers (LC, KS) using a set of predetermined criteria. Specifically, we required that trials be randomized or cross-over and report at least one of the following outcome measures in a quantitative manner: joint tenderness, joint swelling, grip strength, or erythrocyte sedimentation rate (ESR). We also required that trials be of at least three months duration and use prednisone (or a comparable corticosteroid preparation) at a mean dosage of less than or equal to 15 mg/day. We included studies that used either placebo or active drug controls (i.e., comparative studies).

DATA COLLECTION AND ANALYSIS

We compared the effectiveness of prednisone to placebo and/or active controls using a fixed effects model for continuous data. A chi square test for homogeneity was performed, and where heterogeneity existed a random effects model was used. We reported results for all available outcomes recommended by the Outcome Measures for Rheumatology Trials (OMERACT) group. These included the number of tender and swollen joints, pain, functional status and ESR. Grip strength was also evaluated. Standardized mean differences (SMD) were used for outcomes assessing the same concept with different scales (eg. swollen joint counts).

MAIN RESULTS

Very few studies directly assessed the effectiveness of corticosteroids for RA treatment and many were of poor methodologic quality. Only seven of 34 studies identified by our search met criteria for inclusion. Our results indicated that corticosteroids were significantly more effective than placebo controls for four of six outcomes assessed [standardized mean difference for tender joints = -0.37 (95%CI: -0. 59, -0.14), swollen joints = -0.41 (-0.67, -0.16), pain = -0.43 (-0. 74, -0.12), and functional status = -0.57 (-0.92, -0.22)]. The results for grip strength and ESR were not significant [GS = +0.30 (-0.19, +0.80), weighted mean difference (WMD) for ESR = -7.03 (-18. 06, +4.01)]. The single trial that compared prednisone to aspirin indicated no statistically significant difference between these groups for joint tenderness (0.10 (-0.35, +0.55) and for ESR [0.00 (-11.09, +11.09]. Overall, the four outcomes assessed in the single trial that compared prednisone to chloroquine suggested that the effectiveness of these two agents is similar [SMD for joint tenderness = +0.23 (-0.30, +0.75), swollen joints = +0.43 (-0.11, +0. 96), functional status = -0.27 (-0.80, +0.26), and WMD for ESR = -16. 00 (-30.58, -1.42)].

REVIEWER'S CONCLUSIONS

Based on the limited data available, moderate-term prednisone treatment of RA appears to be superior to placebo and comparable to treatment with aspirin or chloroquine in improving several common rheumatoid arthritis disease activity measures.

Systematic review of surgery in malignant bowel obstruction in advanced gynecological and gastrointestinal cancer. The Systematic Review Steering Committee

OBJECTIVE

The objective was to locate, appraise, and summarize evidence from scientific studies on intestinal obstruction due to advanced gynecological and gastrointestinal cancer in order to assess the efficacy of surgery.

MATERIALS AND METHODS

DATA SOURCES

A comprehensive list of studies was provided by an extensive search of electronic databases, relevant journals, bibliographic databases, conference proceedings, reference lists, the gray literature, personal contact, and the worldwide web.

DATA SYNTHESIS

Two researchers extracted the data independently. Due to the methodological quality of the studies, only a qualitative assessment was possible.

RESULTS

The role of surgery in malignant bowel obstruction remains controversial, and no firm conclusions from the many retrospective case series can be made. Control of symptoms varies from 42% to over 80%, although it is often unclear how symptoms were measured and whether the symptom scores used are validated. There is a large range in the rates of reobstruction, from 10 to 50%, although time to reobstruction was often not included. There is a wide range of postoperative morbidity and mortality, although again the definition of both of these surgical outcomes varied among many of the papers.

CONCLUSION

The role of surgery in malignant bowel obstruction needs careful evaluation, using validated outcome measures on symptom control and quality of life scores. Further information would include reobstruction rates together with the morbidity associated with the various surgical procedures. Currently, bowel obstruction is managed empirically, and there are marked variations in clinical practice by different units. There needs to be a greater standardization of management so that comparisons between different series can be made.

[Assessment of quality of obstetric hospital care: use of corticoid in preterm labor]

This study aimed to assess quality of obstetric care for preterm labor patients, using referents, indicators, and standards derived from scientific evidence, focusing on antenatal corticotherapy. Available meta-analyses and randomized controlled trials were examined to establish referents, defining indicators and estimating process and outcome standards for the present study. Data from hospital discharge summaries of seven public maternity hospitals in Rio de Janeiro were analyzed. The standard of process used was 100%. It was not possible to estimate outcome standards, since the necessary adjustment for gestational age was not feasible. Utilization of antenatal corticotherapy in the present study was very low, about 4% and 2%, considering patients up to 33 weeks and 6 days and 36 weeks and 6 days, respectively. Failure to use antenatal corticotherapy when formally indicated deserves attention by health planners and managers, considering: a) the ease in incorporating such a technology, in contrast to the adequate incorporation of special/intensive neonatal care; b) benefits and costs associated with this technology compared to those of delivering neonatal care to premature babies.

A combined analysis of double-blind studies with risperidone vs. placebo and other antipsychotic agents: factors associated with extrapyramidal symptoms

Combined data from double-blind risperidone studies were used to analyse the severity of extrapyramidal symptoms (EPS) associated with treatment in patients with chronic schizophrenia. Factors associated with maximum EPS severity were increasing risperidone dose (< or = 8 mg/day was similar to placebo), lower baseline EPS scores, and longer duration of psychotic symptoms, particularly in older patients. EPS severity was significantly greater in patients receiving haloperidol or other antipsychotics than in those receiving risperidone (4 to 8 mg/day) or placebo. Antiparkinsonian medications were required by significantly fewer patients treated with risperidone (4 to 8 mg/day) than by patients treated with haloperidol or other antipsychotics. Combined efficacy data showed that 4 to 8 mg/day was also the most efficacious dose range; there was no increase in efficacy with doses over 4 mg/day. Based on these data and post-marketing experience, 4 mg/day is an appropriate initial target dose for most patients with schizophrenia. Higher doses may be appropriate for patients with chronic illness, and lower doses may be appropriate for patients with a first psychotic episode or for elderly patients.

Colon and rectal anastomoses do not require routine drainage: a systematic review and meta-analysis

OBJECTIVE

Many surgeons continue to place a prophylactic drain in the pelvis after completion of a colorectal anastomosis, despite considerable evidence that this practice may not be useful. The authors conducted a systematic review and meta-analysis of randomized controlled trials to determine if placement of a drain after a colonic or rectal anastomosis can reduce the rate of complications.

METHODS

A search of the Medline database of English-language articles published from 1987 to 1997 was conducted using the terms "colon," "rectum," "postoperative complications," "surgical anastomosis," and "drainage." A manual search was also conducted. Four randomized controlled trials, including a total of 414 patients, were identified that compared the routine use of drainage of colonic and/or rectal anastomoses to no drainage. Two reviewers assessed the trials independently. Trial quality was critically appraised using a previously published scale, and data on mortality, clinical and radiologic anastomotic leakage rate, wound infection rate, and major complication rate were extracted.

RESULTS

The overall quality of the studies was poor. Use of a drain did not significantly affect the rate of any of the outcomes examined, although the power of this analysis to exclude any difference was low. Comparison of pooled results revealed an odds ratio for clinical leak of 1.5 favoring the control (no drain) group. Of the 20 observed leaks among all four studies that occurred in a patient with a drain in place, in only one case (5%) did pus or enteric content actually appear in the effluent of the existing drain.

CONCLUSIONS

Any significant benefit of routine drainage of colon and rectal anastomoses in reducing the rate of anastomotic leakage or other surgical complications can be excluded with more confidence based on pooled data than by the individual trials alone. Additional well-designed randomized controlled trials would further reinforce this conclusion.

Serendipity in diagnostic imaging: magnetic resonance imaging of the breast

BACKGROUND

Magnetic resonance imaging (MRI) of the breast has been proposed as a noninvasive diagnostic test for evaluation of suspicious ("index") lesions noted on mammography and/or clinical breast examination (CBE). However, women may have incidental ("serendipitous") lesions detected by MRI that are not found on mammography or CBE. To understand better whether or not biopsy procedures should be performed to evaluate serendipitous lesions, we estimated the breast cancer risk for women with this type of lesion.

METHODS

A decision analysis model was used to estimate the positive predictive value (i.e., the chance that a woman with a serendipitous lesion has cancer) of MRI for serendipitous lesions in women who had an abnormal mammogram and/or CBE suspicious for cancer (where a biopsy procedure is recommended). We restricted the analysis to data from women whose index lesions were noncancerous and used meta-analysis of published medical literature to determine the likelihood ratios (measures of how test results change the probability of having cancer) for MRI and the combination of CBE and mammography. The positive predictive value of MRI was calculated using the U.S. population prevalence of cancer (derived from registry data) and the likelihood ratios of the diagnostic tests.

RESULTS

Under a wide variety of assumptions, the positive predictive value of MRI was extremely low for serendipitous lesions. For instance, assuming sensitivity and specificity values for MRI of 95.6% and 68.6%, respectively, approximately four of 1000 55- to 59-year-old women with serendipitous lesions would be expected to have cancer (positive predictive value = 0.44%, 95% confidence interval = 0.24%-0.67%).

CONCLUSION

In women with a suspicious lesion discovered by mammography and/or CBE that is found to be benign, serendipitous breast lesions detected by MRI are extremely unlikely to represent invasive breast cancer. Immediate biopsy of such serendipitous lesions may, therefore, not be required.

Quinine for nocturnal leg cramps: a meta-analysis including unpublished data

OBJECTIVE

With respect to the use of quinine for the treatment of nocturnal leg cramps, to determine whether the findings of a previously performed meta-analysis of published data are altered with the addition of unpublished data, and whether publication bias is present in this area.

DESIGN

A meta-analysis of eight (four published and four unpublished) randomized, double-blind, placebo-controlled trials, seven of which had a crossover design.

SETTING

Randomized trials that were available as of July 1997.

SUBJECTS

Ambulatory patients (659) who suffered from regular nocturnal leg cramps.

MAIN RESULTS

When individual patient data from all crossover studies were pooled, persons had 3.60 (95% confidence interval [CI] 2.15, 5.05) fewer cramps in a 4-week period when taking quinine compared with placebo. This compared with an estimate of 8.83 fewer cramps (95% CI 4.16, 13.49) from pooling published studies alone. The corresponding relative risk reductions were 21% (95% CI 12%, 30%) and 43% (95% CI 21%, 65%), respectively. Compared with placebo, the use of quinine was associated with an increased incidence of side effects, particularly tinnitus. Publication bias is present in the reporting of the efficacy of quinine for this indication, as almost all published studies reported larger estimates of its efficacy than did unpublished studies.

CONCLUSIONS

This study confirms that quinine is efficacious in the prevention of nocturnal leg cramps. However, its benefit may not be as large as reported from the pooling of published studies alone. Given the side effect profile of quinine, nonpharmacologic therapy (e.g., regular passive stretching of the affected muscle) is the best first-line treatment. For persons who find this ineffective and whose quality of life is significantly affected, a trial of quinine is warranted. Prescribing physicians must closely monitor the risks and benefits in individual patients. Publication bias is present in this area even though there is controversy about the role of quinine in the treatment of leg cramps. To minimize the possibility of this bias, persons performing medication-related meta-analyses should seek high-quality unpublished data from drug regulatory agencies and pharmaceutical companies.

Consensus among experts and research synthesis. A comparison of methods

A comparison of two assessment methods, consensus among experts and research synthesis of the scientific literature, was performed using a surgical procedure, carotid endarterectomy (CE), as an example. These two methods have been widely advocated as being scientifically valid. While the comparison revealed a number of areas of general agreement, important differences between the two methods emerged. For example, 30-day mortality for asymptomatic patients was considered an effective outcome (ranked first) by the synthesis, but only "equivocal" (ranked third) of six major indicators reported by the consensus method. The synthesis results are also consistent with other literature reviews as well as with recent large-scale randomized trial results. A number of factors that could account for differences between the two methods were examined. Overall, use of consensus panels may be appropriate early in the development of an intervention where the evidence is sparse, while quantitative research synthesis is preferable when a number of high-quality studies have been performed.

Comparison of intrauterine insemination with timed intercourse in superovulated cycles with gonadotropins: a meta-analysis

OBJECTIVE

To compare timed intercourse and IUI with the husband's sperm in patients with unexplained infertility who are undergoing superovulation with gonadotropins.

DESIGN

Meta-analysis. All published reports of randomized, prospective studies with an English-language abstract extracted from MEDLINE were analyzed. A crossover search was done from the papers obtained.

SETTING

Academic center.

PATIENT(S)

Couples with unexplained infertility.

INTERVENTION(S)

Meta-analysis of studies evaluating patients superovulated with gonadotropins and randomized for timed intercourse or IUI.

MAIN OUTCOME MEASURE(S)

Pregnancy rates (PRs) were obtained. The common odds ratio (OR) and 95% confidence intervals (95% CI) were calculated.

RESULT(S)

There were 49 pregnancies in 431 cycles of timed intercourse (11.37%), whereas there were 110 pregnancies in 549 cycles of IUI (20.04%). The PRs for IUI were significantly increased compared with those for timed intercourse in superovulation cycles (common OR = 1.84; 95% CI = 1.30-2.62).

CONCLUSION(S)

On the basis of the meta-analysis of 980 cycles in randomized and prospective studies, a patient's chances of becoming pregnant are greater with IUI with her husband's sperm than with timed intercourse in cycles superovulated with gonadotropins.

Overcoming the limitations of current meta-analysis of randomised controlled trials

For a meta-analysis to give definitive information, it should meet at least the minimum standards that would be expected of a well-designed, adequately powered, and carefully conducted randomised controlled trial. These minimum standards include both qualitative characteristics--a prospective protocol, comparable definitions of key outcomes, quality control of data, and inclusion of all patients from all trials in the final analysis--and quantitative standards--an assessment of whether the total sample is large enough to provide reliable results and the use of appropriate statistical monitoring guidelines to indicate when the results of the accumulating data of a meta-analysis are conclusive. We believe that rigorous meta-analyses undertaken according to these principles will lead to more reliable evidence about the efficacy and safety of interventions than either retrospective meta-analysis or individual trials.

Strengths and limitations of meta-analysis: larger studies may be more reliable

Meta-analysis of randomized controlled trials combines information from independent studies that address a similar question to provide more reliable estimates of treatment effects. At the present time, the methodology and usefulness of meta-analysis is under scrutiny. In the first part of this paper, we summarize the limitations of meta-analysis and make suggestions for improvements. In the second part, we illustrate strengths and limitations using examples of meta-analyses and subsequent large trials that address the same question. We develop the hypothesis that the size of the meta-analysis may be a useful measure of reliability. Small meta-analyses (i.e., those with less than 200 outcome events) may only be useful for summarizing the available information and generating hypotheses for future research. The results of small meta-analyses should be regarded with caution, even if the p value shows extreme statistical significance. Larger meta-analyses (i.e., those with several hundred events) are likely to be more reliable and may be clinically useful. Well-conducted meta-analyses of large trials using individual patient data may provide the best estimates of treatment effects in the cohort overall and in clinically important subgroups.

Do authors of review articles use systematic methods to identify, assess and synthesize information?

PURPOSE

Review articles are an important source of summary information for practising clinicians to assist them in remaining current with the rapidly expanding medical literature. Consequently it is essential that these be of the highest quality. In this study we evaluate, according to published criteria, the methodological quality of review articles (R) including meta-analysis (MA) appearing in a major cancer journal, Journal of Clinical Oncology (JCO), 1983-1995.

METHODS

A hand-search of JCO was performed, from the first issue January 1983 through December 1995, to identify R, defined as publications that describe and comment on studies relevant to a specific topic or clinical intervention. Only those dealing with aspects of treatment of human cancer were considered further. Methodological quality was first assessed using 8 criteria proposed by Mulrow, rated independently by two medical oncologists as: specified, unclear or not specified. MA, including studies of dose intensity, were further analyzed according to 23 more detailed criteria proposed by Sacks et al. and rated as adequate, partial or no/unknown compliance.

RESULTS

Of 176 review articles, 122 dealt with aspects of treatment of cancer. Compliance with four of Mulrow's eight criteria was generally good, in that 99% clearly stated a purpose, all attempted qualitative synthesis of data. 95% presented a summary and 76% considered future directions. However, in the 106 qualitative reviews (QR), authors rarely gave information on methods of data identification (11.3%), data selection (10.4%) and assessment of validity (8.4%). Structured abstracts seemed to improve the focus and clarity of QR and there was a minor improvement in deficient areas in the later time cohort (1990-1995). Based on 'adequate' compliance with each of the 23 criteria identified by Sacks et al., six dose intensity studies scored 7-12, seven literature data MA scored 10-15 and three individual patient data MA scored 16-18. The highest scores were in the sections relating to prospective design, combinability and statistical analysis. Factors relating to control of bias, sensitivity analysis and application of results were addressed less consistently.

CONCLUSIONS

With the exception of MA, the majority of authors contributing reviews to a major cancer journal, JCO, did not use systematic methods to identify, assess and synthesize information. Initiatives such as the Cochrane Collaboration Cancer Network can support and educate clinicians who wish to perform systematic reviews, but quality of reviews would also improve if author, editors and readers systematically applied any of the sets of criteria now available in the literature.

[Methods of investigation in clinical cardiology. VIII. Meta-analysis and systematic reviews in cardiology]

Over the last 20 years, the development of meta-analysis has been aimed at obtaining objective synthesis of the available results on specific research questions. The main achievements of meta-analysis include the application of techniques to perform systematic literature searches and to obtain unbiased selection of studies, data extraction and pooled estimates of effect. This paper discusses the methodologic steps to follow when conducting a meta-analysis, with emphasis on study selections, data collection and statistical methods to combine the results from individual studies. We also present a set of guided questions as an aid to critically evaluate the conclusions of published meta-analyses. The application of meta-analytic techniques to cardiology is illustrated using a meta-analysis of the randomized controlled trials of angioplasty versus bypass surgery in the management of patients with ischemic heart disease.

Meta-analysis and endocrinology

It is likely that more studies that use meta-analysis will be published in the endocrinologic literature. Major strengths of meta-analysis are the systematic ascertainment of research on a given topic and the explicit delineation of reasons for accepting or rejecting studies as a basis for drawing conclusions. The tendency of meta-analysis to focus on a single estimate of effect and to ignore heterogeneity are problems both with the conduct of meta-analysis and the way in which it is interpreted. Meta-analysis cannot overcome bias in the original studies. It is difficult to perform a good meta-analysis and easy to perform a bad one. The critical reader should not be overawed by the results of a meta-analysis. Reading a meta-analysis should not substitute for careful reading of the primary studies on which the meta-analysis is based. Meta-analysis should not be used to stifle the conduct of original research.

Meta-analysis and prospective meta-analysis in childhood leukemia clinical research

In this paper, we consider the role of meta-analysis and 'prospective meta-analysis' studies in childhood acute lymphocytic leukemia (ALL). In this issue, Valsecchi and Masera [1] give a thoughtful discourse, generally favorable to this approach. This article presents the opposite point of view. The aims of our article are to present the implications in clinical, rather than biostatistical terms, and to provide an extensive literature review of the subject of meta-analysis. We conclude that treatment assessments, resulting from meta-analysis of closed studies (retrospective) should be met with healthy skepticism. Trials requiring international resources should be true intergroup trials with a single coordinating center, rather than prospective meta-analysis, unless it is a question grafted onto each group's own research agenda. For example, each group might ask its own systemic control question, but a CNS protection question is asked collectively.

Low-dose corticosteroids in rheumatoid arthritis. A meta-analysis of their moderate-term effectiveness

OBJECTIVE

To perform a systematic literature review and meta-analysis of the effectiveness of low-dose corticosteroids in the treatment of rheumatoid arthritis (RA).

METHODS

After identifying all relevant studies meeting preselected inclusion criteria, we performed 2 meta-analyses. First, we compared the effectiveness of prednisone to placebo and active drug controls (aspirin, chloroquine, or deflazacort) using standard meta-analysis methods for continuous data. Then, to compare the relative effectiveness of prednisone to second-line agents, we used methods similar to prior meta-analyses of second-line agents for RA treatment. Outcomes assessed were the number of tender and swollen joints, grip strength, and the erythrocyte sedimentation rate (ESR).

RESULTS

Very few studies directly assessed the effectiveness of corticosteroids for RA treatment, and many were of poor methodologic quality. Only 9 of 34 studies identified by our search met criteria for inclusion. The results of our standard meta-analysis indicated that corticosteroids appeared to be more effective than either placebo or active drug controls in improving most conventional outcome measures (effect size 0.90 for the number of tender joints, 1.05 for the number of swollen joints, and 1.20 for the ESR). In our second comparative meta-analysis, corticosteroids were nearly equivalent to second-line agents previously examined in meta-analyses (combined effect size 0.82).

CONCLUSION

Based on the limited data available, during moderate-term treatment periods averaging slightly over 7 months, corticosteroids appeared to be as effective or more effective than alternative therapies in improving several common RA disease activity measures.

How to systematically review the medical literature

In contrast to traditional narrative reviews, systematic reviews are true hypothesis-driven research. Meta-analysis is a form of systematic review in which studies are selected and combined by use of a predefined protocol to reduce bias and subjectivity. A sensitivity analysis shows how results vary through the use of different assumptions, tests, and criteria. The most valid synthesis of information occurs when published and unpublished materials are subjected to the same rigorous evaluation and when results are calculated with and without unpublished sources of data. A good systematic review captures the reader's attention through a skillful blend of numbers and narrative and qualifies for publication as original research in a peer-reviewed journal. Otolaryngologists have published systematic reviews of varying quality since 1990. This article should help improve the quality and validity of future efforts.

Blinding and exclusions after allocation in randomised controlled trials: survey of published parallel group trials in obstetrics and gynaecology

OBJECTIVE

To assess the methodological quality of approaches to blind ing and to handling of exclusions as reported in randomised trials from one medical specialty.

DESIGN

Survey of published, parallel group randomised controlled trials.

DATA SOURCES

A random sample of 110 reports in which allocation was described as randomised from 1990 and 1991 volumes of four journals of obstetrics and gynaecology.

MAIN OUTCOME MEASURES

The adequacy of the descriptions of double blinding and exclusions after randomisation.

RESULTS

Through 31 trials reported being double blind, about twice as many could have been. Of the 31 trials only eight (26%) provided information on the protection of the allocation schedule and only five (16%) provided some written assurance of successful implementation of double blinding. Of 38 trials in which the authors provided sufficient information for readers to infer that no exclusions after randomisation had occurred, six (16%) reported adequate allocation concealment and none stated that an intention to treat analysis had been performed. That compared with 14 (27%) and six (12%), respectively, for the 52 trials that reported exclusions.

CONCLUSIONS

Investigators could have double blinded more often. When they did double blind, they reported poorly and rarely evaluated it. Paradoxically, trials that reported exclusions seemed generally of a higher methodological standard than those that had no apparent exclusions. Exclusions from analysis may have been made in some of the trials in which no exclusions were reported. Editors and readers of reports of randomised trials should understand that flawed reporting of exclusions may often provide a misleading impression of the quality of the trial.