Meta-analysis and the synthesis of evidence

Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations

Background

Systematic reviews and meta-analyses of observational studies are frequently performed, but no widely accepted guidance is available at present. We performed a systematic scoping review of published methodological recommendations on how to systematically review and meta-analyse observational studies.

Methods

We searched online databases and websites and contacted experts in the field to locate potentially eligible articles. We included articles that provided any type of recommendation on how to conduct systematic reviews and meta-analyses of observational studies. We extracted and summarised recommendations on pre-defined key items: protocol development, research question, search strategy, study eligibility, data extraction, dealing with different study designs, risk of bias assessment, publication bias, heterogeneity, statistical analysis. We summarised recommendations by key item, identifying areas of agreement and disagreement as well as areas where recommendations were missing or scarce.

Results

The searches identified 2461 articles of which 93 were eligible. Many recommendations for reviews and meta-analyses of observational studies were transferred from guidance developed for reviews and meta-analyses of RCTs. Although there was substantial agreement in some methodological areas there was also considerable disagreement on how evidence synthesis of observational studies should be conducted. Conflicting recommendations were seen on topics such as the inclusion of different study designs in systematic reviews and meta-analyses, the use of quality scales to assess the risk of bias, and the choice of model (e.g. fixed vs. random effects) for meta-analysis.

Conclusion

There is a need for sound methodological guidance on how to conduct systematic reviews and meta-analyses of observational studies, which critically considers areas in which there are conflicting recommendations.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0495-9) contains supplementary material, which is available to authorized users.

Evidence-Based Medicine and the Challenge of Low Back Pain: Where Are We Now?

Low back pain has long been described as a challenge for both primary care physicians and specialists. Management of low back pain has also been criticized as frequently arbitrary, inappropriate, or ineffective. Contributing factors have been an inadequate evidence base and a need for more rigorous appraisals of the available literature. Evidence-based medicine, an approach to clinical problem solving, is predicated on the premise that high-quality health care will result from practices consistent with the best evidence. In contrast to the traditional medical paradigm that placed a heavy reliance on expert opinion, authority, and unsystematic clinical observations, evidence-based medicine emphasizes the need for rigorous critical appraisals of the scientific literature to inform medical decision making. Evidence-based medicine places strong weight on the requirement for valid studies, particularly randomized controlled trials, to appropriately evaluate the effectiveness of health care interventions. Because of the rapidly increasing volume of medical literature, however, most clinicians are unable to keep up-to-date with all the new data. Two types of preprocessed evidence that can aid busy clinicians in medical decision making are systematic reviews and evidence-based clinical practice guidelines. Like primary studies, systematic reviews and clinical practice guidelines must adhere to high methodologic standards to reduce error and bias. As in other areas of medicine, the approach to the management of low back pain has been positively affected by the availability of more clinical trials and better use of critical appraisal techniques to evaluate and apply research findings. In addition to more rigorous primary studies, an increasing number of high-quality systematic reviews and evidence-based clinical practice guidelines for low back pain are also available. Although some research gaps and methodologic shortcomings persist, the richer evidence base has greatly improved our understanding of what does and does not work for low back pain. Despite these advances, the best available evidence often does not inform everyday clinical decisions for low back pain. Nonetheless, there is widespread agreement that adherence to evidence-based practice will help improve low back pain patient outcomes and reduce arbitrary variations in care. This article reviews basic principles of evidence-based medicine, discusses evidence-based medicine in the context of low back pain management, and summarizes some useful evidence-based medicine resources.

A chain of evidence with mixed comparisons: models for multi-parameter synthesis and consistency of evidence

Multi-parameter evidence synthesis is a generalization of meta-analysis in which several parameters are estimated jointly. Here this approach is applied to a common form of data structure in which mixed pairwise comparisons are made between treatments in a 'chain of evidence' structure. The data set investigated here, which first appeared in the confidence profile method literature, features three types of data relating to thrombolytic treatment following acute myocardial infarction. There is information on reperfusion (coronary patency) following treatment, information on survival in reperfused and non-reperfused patients (conditional survival), and independent information on the effect of treatment on survival without specifying reperfusion state (overall survival). The objective of this study is to explore models for combining these three types of evidence within a single model, and some of the evidence consistency issues that arise. Bayesian Markov chain Monte Carlo methods are used to fit two models. The first, proposed in the confidence profile literature, assumes there are no differences between studies in baseline effects (equal study effects). The second model assumes fixed treatment effects but allows for study differences in baseline reperfusion and conditional survival rates by assuming random study effects. The equal study effects model fits the data poorly, and cross-validation shows that the overall survival evidence is not consistent with the reperfusion and conditional survival evidence under this model. The evidence appears to be consistent within the more loosely structured random study effects model, but results are sensitive to prior assumptions about the between-study variance. The inconsistency between the three evidence types relates to overall survival, rather than to the effect of treatment on survival. Multi-parameter evidence synthesis, with suitable checks for evidence consistency, can be used to help determine whether chains of evidence 'add up' epidemiologically within an overall model, and to construct parameter distributions for an entire model simultaneously, consistent with all the available data.

Meta-analysis of method comparison studies

Methods for the meta-analysis of results from randomized controlled trials are well established. However, there are currently no methods for the meta-analysis of method comparison studies. Here the combination of results from studies comparing two methods of measurement on the same unit of observation is required. We compare standard methods for the pooling of k samples from the same Normal population to those for pooling parameter estimates, in order to estimate the pooled mean difference and 95 per cent limits of agreement. Methods for investigating heterogeneity across studies and for calculating random effects estimates are proposed. We postulate that for published studies either the estimated mean or variance of the difference between measurements will tend to be smaller than for unpublished studies and investigate the evidence for the existence of such publication bias. The methods are illustrated with an example evaluating the accuracy of temperature measured at the axilla compared to the rectum in children.