GetReal in network meta-analysis: a review of the methodology

Multivariate and network meta-analysis of multiple outcomes and multiple treatments: rationale, concepts, and examples

Organisations such as the National Institute for Health and Care Excellence require the synthesis of evidence from existing studies to inform their decisions—for example, about the best available treatments with respect to multiple efficacy and safety outcomes. However, relevant studies may not provide direct evidence about all the treatments or outcomes of interest. Multivariate and network meta-analysis methods provide a framework to address this, using correlated or indirect evidence from such studies alongside any direct evidence. In this article, the authors describe the key concepts and assumptions of these methods, outline how correlated and indirect evidence arises, and illustrate the contribution of such evidence in real clinical examples involving multiple outcomes and multiple treatments

Efficacy of home-based non-pharmacological interventions for treating depression: a systematic review and network meta-analysis of randomised controlled trials

OBJECTIVES

To systematically review and compare the efficacy of all available home-based non-pharmacological treatments of depression.

DESIGN

Systematic review and network meta-analysis of randomised controlled trials.

DATA SOURCES

Medline, Scopus and Cumulative Index to Nursing and Allied Health Literature (CINAHL) databases were searched since inceptions to 7 August 2016.

ELIGIBILITY CRITERIA

Randomised controlled trials comparing the efficacy of home-based non-pharmacological interventions with usual care of patients with depression were included in the review.

MAIN OUTCOMES

Depression symptom scores and disease remission rates at the end of treatment.

RESULTS

Seventeen studies were included in the review. Home-based non-pharmacological interventions were categorised as (1) home-based psychological intervention, (2) home-based exercise intervention, (3) combined home-based psychological intervention with exercise intervention and (4) complementary medicine. Complementary medicine approaches were excluded from the meta-analysis due to heterogeneity. The standardised mean differences of post-treatment depression symptom scores between usual care groups and home-based psychological intervention, home-based exercise intervention and combined home-based psychological intervention with exercise intervention were âˆ'0.57 (95% CI âˆ'0.84 to âˆ'0.31), âˆ'1.03 (95% CI âˆ'2.89 to 0.82) and âˆ'0.78 (95% CI âˆ'1.09 to âˆ'0.47), respectively. These results suggest that only home-based psychological intervention and combined home-based psychological intervention with exercise intervention could significantly decrease depression scores. Compared with usual care groups, the disease remission rate was also significantly higher for home-based psychological intervention (pooled risk ratio=1.53; 95% CI 1.19 to 1.98) and combined home-based psychological intervention with exercise intervention (pooled risk ratio=3.47; 95% CI 2.11 to 5.70). Of all the studied interventions, combined home-based psychological intervention with exercise intervention had the highest probability of resulting in disease remission.

CONCLUSION

Our study confirms the efficacy of home-based psychological intervention and combined home-based psychological intervention with exercise intervention in the treatment of depression. Combined home-based psychological intervention and exercise intervention was the best treatment and should be considered for inclusion in clinical guidelines for managing depression.

An investigation of the impact of using different methods for network meta-analysis: a protocol for an empirical evaluation

BACKGROUND

Network meta-analysis, a method to synthesise evidence from multiple treatments, has increased in popularity in the past decade. Two broad approaches are available to synthesise data across networks, namely, arm- and contrast-synthesis models, with a range of models that can be fitted within each. There has been recent debate about the validity of the arm-synthesis models, but to date, there has been limited empirical evaluation comparing results using the methods applied to a large number of networks. We aim to address this gap through the re-analysis of a large cohort of published networks of interventions using a range of network meta-analysis methods.

METHODS

We will include a subset of networks from a database of network meta-analyses of randomised trials that have been identified and curated from the published literature. The subset of networks will include those where the primary outcome is binary, the number of events and participants are reported for each direct comparison, and there is no evidence of inconsistency in the network. We will re-analyse the networks using three contrast-synthesis methods and two arm-synthesis methods. We will compare the estimated treatment effects, their standard errors, treatment hierarchy based on the surface under the cumulative ranking (SUCRA) curve, the SUCRA value, and the between-trial heterogeneity variance across the network meta-analysis methods. We will investigate whether differences in the results are affected by network characteristics and baseline risk.

DISCUSSION

The results of this study will inform whether, in practice, the choice of network meta-analysis method matters, and if it does, in what situations differences in the results between methods might arise. The results from this research might also inform future simulation studies.

Systematic review with network meta-analysis: comparative efficacy and tolerability of different intravenous iron formulations for the treatment of iron deficiency anaemia in patients with inflammatory bowel disease

BACKGROUND

Iron deficiency anaemia (IDA) is a common complication of inflammatory bowel disease (IBD) associated with reduced quality of life and increased hospitalisation rates. While the best way of treating IDA in IBD patients is not clearly established, current European guidelines recommend intravenous iron therapy in IBD patients with severe anaemia or intolerance to oral iron compounds.

AIM

To compare the efficacy and tolerability of different intravenous iron formulations used to treat IDA in IBD patients in a systematic review and Bayesian network meta-analysis (NMA), PROSPERO registration number: 42016046565.

METHODS

In June 2016, we systematically searched for studies analysing efficacy and safety of intravenous iron for IDA therapy in IBD. Primary outcome was therapy response, defined as Hb normalisation or increase ≥2 g/dL.

RESULTS

Five randomised, controlled trials (n = 1143 patients) were included in a network meta-analysis. Only ferric carboxymaltose was significantly more effective than oral iron [OR=1.9, 95% CrI: (1.1;3.2)]. Rank probabilities showed ferric carboxymaltose to be most effective, followed by iron sucrose, iron isomaltose and oral iron. Pooled data from the systematic review (n = 1746 patients) revealed adverse event rates of 12.0%, 15.3%, 12.0%, 17.0% for ferric carboxymaltose, iron sucrose, iron dextran and iron isomaltose respectively. One drug-related serious adverse event (SAE) each was reported for ferric carboxymaltose and iron isomaltoside, and one possibly drug-related SAE for iron sucrose.

CONCLUSIONS

Ferric carboxymaltose was the most effective intravenous iron formulation, followed by iron sucrose. In addition, ferric carboxymaltose tended to be better tolerated. Thus, nanocolloidal IV iron products exhibit differing therapeutic and safety characteristics and are not interchangeable.

Network meta-analysis of antibiotic prophylaxis for prevention of surgical-site infection after groin hernia surgery

BACKGROUND

First-generation cephalosporins (such as cefazolin) are recommended as antibiotic prophylaxis in groin hernia repair, but other broad-spectrum antibiotics have also been prescribed in clinical practice. This was a systematic review and network meta-analysis to compare the efficacy of different antibiotic classes for prevention of surgical-site infection (SSI) after hernia repair.

METHODS

RCTs were identified that compared efficacy of antibiotic prophylaxis on SSI after inguinal or femoral hernia repair from PubMed and Scopus databases up to March 2016. Data were extracted independently by two reviewers. Network meta-analysis was applied to assess treatment efficacy. The probability of being the best antibiotic prophylaxis was estimated using surface under the cumulative ranking curve (SUCRA) analysis.

RESULTS

Fifteen RCTs (5159 patients) met the inclusion criteria. Interventions were first-generation (7 RCTs, 1237 patients) and second-generation (2 RCTs, 532) cephalosporins, β-lactam/β-lactamase inhibitors (6 RCTs, 619) and fluoroquinolones (2 RCTs, 581), with placebo as the most common comparator (14 RCTs, 2190). A network meta-analysis showed that β-lactam/β-lactamase inhibitors and first-generation cephalosporins were significantly superior to placebo, with a pooled risk ratio of 0·44 (95 per cent c.i. 0·25 to 0·75) and 0·62 (0·42 to 0·92) respectively. However, none of the antibiotic classes was significantly different from the others. SUCRA results indicated that β-lactam/β-lactamase inhibitors and first-generation cephalosporins were ranked first and second respectively for best prophylaxis.

CONCLUSION

β-Lactam/β-lactamase inhibitors followed by first-generation cephalosporins ranked as the most effective SSI prophylaxis for adult patients undergoing groin hernia repair.

Network meta-analysis: a technique to gather evidence from direct and indirect comparisons

Systematic reviews and pairwise meta-analyses of randomized controlled trials, at the intersection of clinical medicine, epidemiology and statistics, are positioned at the top of evidence-based practice hierarchy. These are important tools to base drugs approval, clinical protocols and guidelines formulation and for decision-making. However, this traditional technique only partially yield information that clinicians, patients and policy-makers need to make informed decisions, since it usually compares only two interventions at the time. In the market, regardless the clinical condition under evaluation, usually many interventions are available and few of them have been studied in head-to-head studies. This scenario precludes conclusions to be drawn from comparisons of all interventions profile (e.g. efficacy and safety). The recent development and introduction of a new technique – usually referred as network meta-analysis, indirect meta-analysis, multiple or mixed treatment comparisons – has allowed the estimation of metrics for all possible comparisons in the same model, simultaneously gathering direct and indirect evidence. Over the last years this statistical tool has matured as technique with models available for all types of raw data, producing different pooled effect measures, using both Frequentist and Bayesian frameworks, with different software packages. However, the conduction, report and interpretation of network meta-analysis still poses multiple challenges that should be carefully considered, especially because this technique inherits all assumptions from pairwise meta-analysis but with increased complexity. Thus, we aim to provide a basic explanation of network meta-analysis conduction, highlighting its risks and benefits for evidence-based practice, including information on statistical methods evolution, assumptions and steps for performing the analysis.

What Is the Best Treatment of Locally Advanced Nasopharyngeal Carcinoma? An Individual Patient Data Network Meta-Analysis

Purpose The role of adjuvant chemotherapy (AC) or induction chemotherapy (IC) in the treatment of locally advanced nasopharyngeal carcinoma is controversial. The individual patient data from the Meta-Analysis of Chemotherapy in Nasopharynx Carcinoma database were used to compare all available treatments. Methods All randomized trials of radiotherapy (RT) with or without chemotherapy in nonmetastatic nasopharyngeal carcinoma were considered. Overall, 20 trials and 5,144 patients were included. Treatments were grouped into seven categories: RT alone (RT), IC followed by RT (IC-RT), RT followed by AC (RT-AC), IC followed by RT followed by AC (IC-RT-AC), concomitant chemoradiotherapy (CRT), IC followed by CRT (IC-CRT), and CRT followed by AC (CRT-AC). P-score was used to rank the treatments. Fixed- and random-effects frequentist network meta-analysis models were applied. Results The three treatments with the highest probability of benefit on overall survival (OS) were CRT-AC, followed by CRT and IC-CRT, with respective hazard ratios (HRs [95% CIs]) compared with RT alone of 0.65 (0.56 to 0.75), 0.77 (0.64 to 0.92), and 0.81 (0.63 to 1.04). HRs (95% CIs) of CRT-AC compared with CRT for OS, progression-free survival (PFS), locoregional control, and distant control (DC) were, respectively, 0.85 (0.68 to 1.05), 0.81 (0.66 to 0.98), 0.70 (0.48 to 1.02), and 0.87 (0.61 to 1.25). IC-CRT ranked second for PFS and the best for DC. CRT never ranked first. HRs of CRT compared with IC-CRT for OS, PFS, locoregional control, and DC were, respectively, 0.95 (0.72 to 1.25), 1.13 (0.88 to 1.46), 1.05 (0.70 to 1.59), and 1.55 (0.94 to 2.56). Regimens with more chemotherapy were associated with increased risk of acute toxicity. Conclusion The addition of AC to CRT achieved the highest survival benefit and consistent improvement for all end points. The addition of IC to CRT achieved the highest effect on DC.

Combining randomized and non-randomized evidence in network meta-analysis

Non-randomized studies aim to reveal whether or not interventions are effective in real-life clinical practice, and there is a growing interest in including such evidence in the decision-making process. We evaluate existing methodologies and present new approaches to using non-randomized evidence in a network meta-analysis of randomized controlled trials (RCTs) when the aim is to assess relative treatment effects. We first discuss how to assess compatibility between the two types of evidence. We then present and compare an array of alternative methods that allow the inclusion of non-randomized studies in a network meta-analysis of RCTs: the naïve data synthesis, the design-adjusted synthesis, the use of non-randomized evidence as prior information and the use of three-level hierarchical models. We apply some of the methods in two previously published clinical examples comparing percutaneous interventions for the treatment of coronary in-stent restenosis and antipsychotics in patients with schizophrenia. We discuss in depth the advantages and limitations of each method, and we conclude that the inclusion of real-world evidence from non-randomized studies has the potential to corroborate findings from RCTs, increase precision and enhance the decision-making process. Copyright © 2017 John Wiley & Sons, Ltd.

GetReal in mathematical modelling: a review of studies predicting drug effectiveness in the real world

The performance of a drug in a clinical trial setting often does not reflect its effect in daily clinical practice. In this third of three reviews, we examine the applications that have been used in the literature to predict real‐world effectiveness from randomized controlled trial efficacy data. We searched MEDLINE, EMBASE from inception to March 2014, the Cochrane Methodology Register, and websites of key journals and organisations and reference lists. We extracted data on the type of model and predictions, data sources, validation and sensitivity analyses, disease area and software. We identified 12 articles in which four approaches were used: multi‐state models, discrete event simulation models, physiology‐based models and survival and generalized linear models. Studies predicted outcomes over longer time periods in different patient populations, including patients with lower levels of adherence or persistence to treatment or examined doses not tested in trials. Eight studies included individual patient data. Seven examined cardiovascular and metabolic diseases and three neurological conditions. Most studies included sensitivity analyses, but external validation was performed in only three studies. We conclude that mathematical modelling to predict real‐world effectiveness of drug interventions is not widely used at present and not well validated. © 2016 The Authors Research Synthesis Methods Published by John Wiley & Sons Ltd.

GetReal: from efficacy in clinical trials to relative effectiveness in the real world

The GetReal consortium ("incorporating real-life data into drug development") addresses the efficacy-effectiveness gap that opens between the data from well-controlled randomized trials in selected patient groups submitted to regulators and the real-world evidence on effectiveness and safety of drugs required by decision makers. Workpackage 4 of GetReal develops evidence synthesis and modelling approaches to generate the real-world evidence. In this commentary, we discuss how questions change when moving from the well-controlled randomized trial setting to real-life medical practice, the evidence required to answer these questions, the populations to which estimates will be applicable to and the methods and data sources used to produce these estimates. We then introduce the methodological reviews written by GetReal authors and published in Research Synthesis Methods on network meta-analysis, individual patient data meta-analysis and mathematical modelling to predict drug effectiveness. The critical reviews of key methods are a good starting point for the ambitious programme of work GetReal has embarked on. The different strands of work under way in GetReal have great potential to contribute to making clinical trials research as relevant as it can be to patients, caregivers and policy makers. Copyright © 2016 John Wiley & Sons, Ltd.

Bayesian network meta-analysis for cluster randomized trials with binary outcomes

Network meta-analysis is becoming a common approach to combine direct and indirect comparisons of several treatment arms. In recent research, there have been various developments and extensions of the standard methodology. Simultaneously, cluster randomized trials are experiencing an increased popularity, especially in the field of health services research, where, for example, medical practices are the units of randomization but the outcome is measured at the patient level. Combination of the results of cluster randomized trials is challenging. In this tutorial, we examine and compare different approaches for the incorporation of cluster randomized trials in a (network) meta-analysis. Furthermore, we provide practical insight on the implementation of the models. In simulation studies, it is shown that some of the examined approaches lead to unsatisfying results. However, there are alternatives which are suitable to combine cluster randomized trials in a network meta-analysis as they are unbiased and reach accurate coverage rates. In conclusion, the methodology can be extended in such a way that an adequate inclusion of the results obtained in cluster randomized trials becomes feasible. Copyright © 2016 John Wiley & Sons, Ltd.

Living cumulative network meta-analysis to reduce waste in research: A paradigmatic shift for systematic reviews?

In a recent research article in BMC Medicine, Créquit and colleagues demonstrate how published systematic reviews in lung cancer provide a fragmented, out-of-date picture of the evidence for all treatments. The results and conclusions drawn from this study, based on cumulative network meta-analyses (NMA) of evidence from randomized clinical trials over time, are quite compelling. The inherent waste of research resulting from incomplete evidence synthesis has wide-reaching implications for a range of target groups including developers of systematic reviews and guidelines and their end-users, health care professionals and patients at the point of care. Building on emerging concepts for living systematic reviews and NMA, the authors propose "living cumulative NMA" as a potential solution and paradigmatic shift. Here we describe how recent innovations within authoring, dissemination, and updating of systematic reviews and trustworthy guidelines may greatly facilitate the production of living NMA. Some additional challenges need to be solved for NMA in general, and for living cumulative NMA in particular, before a paradigmatic shift for systematic reviews can become reality.Please see related research article: https://bmcmedicine.biomedcentral.com/articles/10.1186/s12916-016-0555-0.