Making the best use of available evidence: the case of new generation antidepressants: A response to: Are all antidepressants equal?

Twenty years of network meta-analysis: Continuing controversies and recent developments

Network meta-analysis (NMA) is an extension of pairwise meta-analysis (PMA) which combines evidence from trials on multiple treatments in connected networks. NMA delivers internally consistent estimates of relative treatment efficacy, needed for rational decision making. Over its first 20 years NMA's use has grown exponentially, with applications in both health technology assessment (HTA), primarily re-imbursement decisions and clinical guideline development, and clinical research publications. This has been a period of transition in meta-analysis, first from its roots in educational and social psychology, where large heterogeneous datasets could be explored to find effect modifiers, to smaller pairwise meta-analyses in clinical medicine on average with less than six studies. This has been followed by narrowly-focused estimation of the effects of specific treatments at specific doses in specific populations in sparse networks, where direct comparisons are unavailable or informed by only one or two studies. NMA is a powerful and well-established technique but, in spite of the exponential increase in applications, doubts about the reliability and validity of NMA persist. Here we outline the continuing controversies, and review some recent developments. We suggest that heterogeneity should be minimized, as it poses a threat to the reliability of NMA which has not been fully appreciated, perhaps because it has not been seen as a problem in PMA. More research is needed on the extent of heterogeneity and inconsistency in datasets used for decision making, on formal methods for making recommendations based on NMA, and on the further development of multi-level network meta-regression.

Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool

The ever increasing number of alternative treatment options and the plethora of clinical trials have put systematic reviews and meta-analysis under a new perspective by emphasizing the need to make inferences about competing treatments for the same condition. The statistical component in reviews that compare multiple interventions, network meta-analysis, is the next generation evidence synthesis toolkit which, when properly applied, can serve decision-making better than the established pairwise meta-analysis. The criticism and enthusiasm for network meta-analysis echo those that greeted the advent of simple meta-analysis. The main criticism is associated with the difficulty in evaluating the assumption underlying the statistical synthesis of direct and indirect evidence. In the present article, the assumption of the network meta-analysis are presented using various formulations, the statistical and nonstatistical methodological considerations are elucidated, and the progress achieved in this field is summarized. Throughout, focus is put on highlighting the analogy between the concerns and difficulties that the scientific community had some time ago when advancing from individual trials to their quantitative synthesis via meta-analysis and those currently expressed about the transition from head-to-head meta-analyses to network meta-analysis. Copyright © 2012 John Wiley & Sons, Ltd.

Adjusting for publication biases across similar interventions performed well when compared with gold standard data

OBJECTIVE

To extend, apply, and evaluate a regression-based approach to adjusting meta-analysis for publication and related biases. The approach uses related meta-analyses to improve estimation by borrowing strength on the degree of bias.

STUDY DESIGN AND SETTING

The proposed adjustment approach is described. Adjustments are applied both independently and by borrowing strength across journal-extracted data on the effectiveness of 12 antidepressant drugs from placebo-controlled trials. The methods are also applied to Food and Drug Administration (FDA) data obtained on the same 12 drugs. Results are compared, viewing the FDA observed data as gold standard.

RESULTS

Estimates adjusted for publication biases made independently for each drug were very uncertain using both the journal and FDA data. Adjusted estimates were much more precise when borrowing strength across meta-analyses. Reassuringly, adjustments in this way made to the journal data agreed closely with the observed estimates from the FDA data, while the adjusted FDA results changed only minimally from those observed from the FDA data.

CONCLUSION

The method worked well in the case study considered and therefore further evaluation is encouraged. It is suggested that this approach may be especially useful when adjusting several meta-analyses on similar interventions and outcomes, particularly when there are small numbers of studies.