Synthesis of survival and disease progression outcomes for health technology assessment of cancer therapies

Twenty years of network meta-analysis: Continuing controversies and recent developments

Network meta-analysis (NMA) is an extension of pairwise meta-analysis (PMA) which combines evidence from trials on multiple treatments in connected networks. NMA delivers internally consistent estimates of relative treatment efficacy, needed for rational decision making. Over its first 20 years NMA's use has grown exponentially, with applications in both health technology assessment (HTA), primarily re-imbursement decisions and clinical guideline development, and clinical research publications. This has been a period of transition in meta-analysis, first from its roots in educational and social psychology, where large heterogeneous datasets could be explored to find effect modifiers, to smaller pairwise meta-analyses in clinical medicine on average with less than six studies. This has been followed by narrowly-focused estimation of the effects of specific treatments at specific doses in specific populations in sparse networks, where direct comparisons are unavailable or informed by only one or two studies. NMA is a powerful and well-established technique but, in spite of the exponential increase in applications, doubts about the reliability and validity of NMA persist. Here we outline the continuing controversies, and review some recent developments. We suggest that heterogeneity should be minimized, as it poses a threat to the reliability of NMA which has not been fully appreciated, perhaps because it has not been seen as a problem in PMA. More research is needed on the extent of heterogeneity and inconsistency in datasets used for decision making, on formal methods for making recommendations based on NMA, and on the further development of multi-level network meta-regression.

A non-parametric approach for jointly combining evidence on progression free and overall survival time in network meta-analysis

Randomised controlled trials of cancer treatments typically report progression free survival (PFS) and overall survival (OS) outcomes. Existing methods to synthesise evidence on PFS and OS either rely on the proportional hazards assumption or make parametric assumptions which may not capture the diverse survival curve shapes across studies and treatments. Furthermore, PFS and OS are not independent: OS is the sum of PFS and post-progression survival (PPS). Our aim was to develop a non-parametric approach for jointly synthesising evidence from published Kaplan-Meier survival curves of PFS and OS without assuming proportional hazards. Restricted mean survival times (RMST) are estimated by the area under the survival curves (AUCs) up to a restricted follow-up time. The correlation between AUCs due to the constraint that OS>PFS is estimated using bootstrap re-sampling. Network meta-analysis models are given for RMST for PFS and PPS and ensure that OS=PFS + PPS. Both additive and multiplicative network meta-analysis models are presented to obtain relative treatment effects as either differences or ratios of RMST. The methods are illustrated with a network meta-analysis of treatments for Stage IIIA-N2 Non-Small Cell Lung Cancer. The approach has implications for health economic models of cancer treatments which require estimates of the mean time spent in the PFS and PPS health-states. The methods can be applied to a single time-to-event outcome, and so have wide applicability in any field where time-to-event outcomes are reported, the proportional hazards assumption is in doubt, and survival curve shapes differ across studies and interventions. This article is protected by copyright. All rights reserved.

The impact of childhood malnutrition on mortality from pneumonia: a systematic review and network meta-analysis

INTRODUCTION

Childhood malnutrition is widespread in low-income and middle-income countries (LMICs) and increases the frequency and severity of infections such as pneumonia. We aimed to identify studies investigating pneumonia deaths in malnourished children and estimate mortality risk by malnutrition severity.

METHODS

We conducted a systematic review of MEDLINE, EMBASE and Global Health databases to identify relevant studies. We used a network meta-analysis to derive ORs of death from pneumonia for moderately and severely underweight children using low weight-for-age, the most reported measure of malnutrition. We compared meta-estimates of studies conducted before and after 2000 to assess changes in mortality risk over time. We estimated the prevalence of underweight hospitalised children from hospital-based cohort studies and calculated the population attributable fraction of in-hospital pneumonia deaths from being underweight using our results.

RESULTS

Our network meta-analysis included 33 544 underweight children from 23 studies. The estimated OR of death from pneumonia was 2.0 (95% CI 1.6 to 2.6) and 4.6 (95% CI 3.7 to 5.9) for children moderately and severely underweight, respectively. The OR of death from pneumonia for those severely underweight was 5.3 (95% CI 3.9 to 7.4) pre-2000 and remained high post-2000 at 4.1 (95% CI 3.0 to 6.0). Prevalence of underweight children hospitalised with pneumonia varied (median 40.2%, range 19.6-66.3) but was high across many LMIC settings. We estimated a median 18.3% (range 10.8-34.6) and 40.9% (range 14.7-69.9) of in-hospital pneumonia deaths were attributable to being moderately and severely underweight, respectively.

CONCLUSIONS

The risk of death from childhood pneumonia dramatically increases with malnutrition severity. This risk has remained high in recent years with an estimated over half of in-hospital pneumonia deaths attributable to child malnutrition. Prevention and treatment of all child malnutrition must be prioritised to maintain progress on reducing pneumonia deaths.

Classifying information-sharing methods

BACKGROUND

Sparse relative effectiveness evidence is a frequent problem in Health Technology Assessment (HTA). Where evidence directly pertaining to the decision problem is sparse, it may be feasible to expand the evidence-base to include studies that relate to the decision problem only indirectly: for instance, when there is no evidence on a comparator, evidence on other treatments of the same molecular class could be used; similarly, a decision on children may borrow-strength from evidence on adults. Usually, in HTA, such indirect evidence is either included by ignoring any differences ('lumping') or not included at all ('splitting'). However, a range of more sophisticated methods exists, primarily in the biostatistics literature. The objective of this study is to identify and classify the breadth of the available information-sharing methods.

METHODS

Forwards and backwards citation-mining techniques were used on a set of seminal papers on the topic of information-sharing. Papers were included if they specified (network) meta-analytic methods for combining information from distinct populations, interventions, outcomes or study-designs.

RESULTS

Overall, 89 papers were included. A plethora of evidence synthesis methods have been used for information-sharing. Most papers (n=79) described methods that shared information on relative treatment effects. Amongst these, there was a strong emphasis on methods for information-sharing across multiple outcomes (n=42) and treatments (n=25), with fewer papers focusing on study-designs (n=23) or populations (n=8). We categorise and discuss the methods under four 'core' relationships of information-sharing: functional, exchangeability-based, prior-based and multivariate relationships, and explain the assumptions made within each of these core approaches.

CONCLUSIONS

This study highlights the range of information-sharing methods available. These methods often impose more moderate assumptions than lumping or splitting. Hence, the degree of information-sharing that they impose could potentially be considered more appropriate. Our identification of four 'core' methods of information-sharing allows for an improved understanding of the assumptions underpinning the different methods. Further research is required to understand how the methods differ in terms of the strength of sharing they impose and the implications of this for health care decisions.

Shared parameter model for competing risks and different data summaries in meta-analysis: Implications for common and rare outcomes

This paper considers the problem in aggregate data meta-analysis of studies reporting multiple competing binary outcomes and of studies using different summary formats for those outcomes. For example, some may report numbers of patients with at least one of each outcome while others may report the total number of such outcomes. We develop a shared parameter model on hazard ratio scale accounting for different data summaries and competing risks. We adapt theoretical arguments from the literature to demonstrate that the models are equivalent if events are rare. We use constructed data examples and a simulation study to find an event rate threshold of approximately 0.2 above which competing risks and different data summaries may bias results if no adjustments are made. Below this threshold, simpler models may be sufficient. We recommend analysts to consider the absolute event rates and only use a simple model ignoring data types and competing risks if all of underlying events are rare (below our threshold of approximately 0.2). If one or more of the absolute event rates approaches or exceeds our informal threshold, it may be necessary to account for data types and competing risks through a shared parameter model in order to avoid biased estimates.

Bevacizumab in combination with different platinum-based doublets in the first-line treatment for advanced nonsquamous non-small-cell lung cancer: A network meta-analysis

Platinum-based doublet chemotherapy with or without bevacizumab is the standard treatment for untreated advanced nonsquamous non-small-cell lung cancer (NS-NSCLC). However, adding bevacizumab to chemotherapies other than paclitaxel-carboplatin is, though widely applied clinically, largely unjustified due to the lack of head-to-head data. We performed a Bayesian network meta-analysis (NMA) to address this important issue. Data of 8,548 patients from 18 randomized controlled trials (RCTs) receiving six treatments, including taxane-platinum (Taxane-Pt), gemcitabine-platinum (Gem-Pt), pemetrexed-platinum (Pem-Pt), taxane-platinum + bevacizumab (Taxane-Pt + B), gemcitabine-platinum + bevacizumab (Gem-Pt + B) and pemetrexed-platinum + bevacizumab (Pem-Pt + B), were incorporated into the analyses. Direct and indirect evidence of overall survival (OS) and progression-free survival (PFS) were synthesized at the hazard ratio (HR) scale and evidence of objective response rate (ORR) and serious adverse events (SAE) were synthesized at the odds ratio (OR) scale. Taxane-Pt + B showed significant advantages in OS (HR = 0.79, p < 0.001), PFS (HR = 0.54, p < 0.001) and ORR (OR = 2.7, p < 0.001) over Taxane-Pt with comparable tolerability (OR = 3.1, p = 0.08). Gem-Pt + B showed no OS benefit compared to any other treatment. No significant differences were detected between Pem-Pt + B and Pem-Pt in four outcomes. In terms of the benefit-risk ratio, Pem-Pt and Taxane-Pt + B were ranked the first and second, respectively. In conclusion, in the first-line treatment for advanced NS-NSCLC, Taxane-Pt and Gem-Pt are the most and least preferable regimens to be used with bevacizumab, respectively. Adding bevacizumab to Pem-Pt remains unjustified because it fails to improve efficacy or tolerability. In terms of the benefit-risk ratio, Pem-Pt and Taxane-Pt + B are the best and second-best treatment for this population.

Acupuncture for chronic pain and depression in primary care: a programme of research

Background

There has been an increase in the utilisation of acupuncture in recent years, yet the evidence base is insufficiently well established to be certain about its clinical effectiveness and cost-effectiveness. Addressing the questions related to the evidence base will reduce uncertainty and help policy- and decision-makers with regard to whether or not wider access is appropriate and provides value for money.

Aim

Our aim was to establish the most reliable evidence on the clinical effectiveness and cost-effectiveness of acupuncture for chronic pain by drawing on relevant evidence, including recent high-quality trials, and to develop fresh evidence on acupuncture for depression. To extend the evidence base we synthesised the results of published trials using robust systematic review methodology and conducted a randomised controlled trial (RCT) of acupuncture for depression.

Methods and results

We synthesised the evidence from high-quality trials of acupuncture for chronic pain, consisting of musculoskeletal pain related to the neck and low back, osteoarthritis of the knee, and headache and migraine, involving nearly 18,000 patients. In an individual patient data (IPD) pairwise meta-analysis, acupuncture was significantly better than both sham acupuncture (p < 0.001) and usual care (p < 0.001) for all conditions. Using network meta-analyses, we compared acupuncture with other physical therapies for osteoarthritis of the knee. In both an analysis of all available evidence and an analysis of a subset of better-quality trials, using aggregate-level data, we found acupuncture to be one of the more effective therapies. We developed new Bayesian methods for analysing multiple individual patient-level data sets to evaluate heterogeneous continuous outcomes. An accompanying cost-effectiveness analysis found transcutaneous electrical nerve stimulation (TENS) to be cost-effective for osteoarthritis at a threshold of £20,000 per quality-adjusted life-year when all trials were synthesised. When the analysis was restricted to trials of higher quality with adequate allocation concealment, acupuncture was cost-effective. In a RCT of acupuncture or counselling compared with usual care for depression, in which half the patients were also experiencing comorbid pain, we found acupuncture and counselling to be clinically effective and acupuncture to be cost-effective. For patients in whom acupuncture is inappropriate or unavailable, counselling is cost-effective.

Conclusion

We have provided the most robust evidence from high-quality trials on acupuncture for chronic pain. The synthesis of high-quality IPD found that acupuncture was more effective than both usual care and sham acupuncture. Acupuncture is one of the more clinically effective physical therapies for osteoarthritis and is also cost-effective if only high-quality trials are analysed. When all trials are analysed, TENS is cost-effective. Promising clinical and economic evidence on acupuncture for depression needs to be extended to other contexts and settings. For the conditions we have investigated, the drawing together of evidence on acupuncture from this programme of research has substantially reduced levels of uncertainty. We have identified directions for further research. Our research also provides a valuable basis for considering the potential role of acupuncture as a referral option in health care and enabling providers and policy-makers to make decisions based on robust sources of evidence.

Trial registration

Current Controlled Trials ISRCTN63787732.

Funding

The National Institute for Health Research Programme Grants for Applied Research programme.

GetReal in network meta-analysis: a review of the methodology

Pairwise meta-analysis is an established statistical tool for synthesizing evidence from multiple trials, but it is informative only about the relative efficacy of two specific interventions. The usefulness of pairwise meta-analysis is thus limited in real-life medical practice, where many competing interventions may be available for a certain condition and studies informing some of the pairwise comparisons may be lacking. This commonly encountered scenario has led to the development of network meta-analysis (NMA). In the last decade, several applications, methodological developments, and empirical studies in NMA have been published, and the area is thriving as its relevance to public health is increasingly recognized. This article presents a review of the relevant literature on NMA methodology aiming to pinpoint the developments that have appeared in the field. Copyright © 2016 John Wiley & Sons, Ltd.

Network meta-analysis of (individual patient) time to event data alongside (aggregate) count data

BACKGROUND

Network meta-analysis methods extend the standard pair-wise framework to allow simultaneous comparison of multiple interventions in a single statistical model. Despite published work on network meta-analysis mainly focussing on the synthesis of aggregate data, methods have been developed that allow the use of individual patient-level data specifically when outcomes are dichotomous or continuous. This paper focuses on the synthesis of individual patient-level and summary time to event data, motivated by a real data example looking at the effectiveness of high compression treatments on the healing of venous leg ulcers.

METHODS

This paper introduces a novel network meta-analysis modelling approach that allows individual patient-level (time to event with censoring) and summary-level data (event count for a given follow-up time) to be synthesised jointly by assuming an underlying, common, distribution of time to healing. Alternative model assumptions were tested within the motivating example. Model fit and adequacy measures were used to compare and select models.

RESULTS

Due to the availability of individual patient-level data in our example we were able to use a Weibull distribution to describe time to healing; otherwise, we would have been limited to specifying a uniparametric distribution. Absolute effectiveness estimates were more sensitive than relative effectiveness estimates to a range of alternative specifications for the model.

CONCLUSIONS

The synthesis of time to event data considering individual patient-level data provides modelling flexibility, and can be particularly important when absolute effectiveness estimates, and not just relative effect estimates, are of interest.

Evidence synthesis for decision making 6: embedding evidence synthesis in probabilistic cost-effectiveness analysis

When multiple parameters are estimated from the same synthesis model, it is likely that correlations will be induced between them. Network meta-analysis (mixed treatment comparisons) is one example where such correlations occur, along with meta-regression and syntheses involving multiple related outcomes. These correlations may affect the uncertainty in incremental net benefit when treatment options are compared in a probabilistic decision model, and it is therefore essential that methods are adopted that propagate the joint parameter uncertainty, including correlation structure, through the cost-effectiveness model. This tutorial paper sets out 4 generic approaches to evidence synthesis that are compatible with probabilistic cost-effectiveness analysis. The first is evidence synthesis by Bayesian posterior estimation and posterior sampling where other parameters of the cost-effectiveness model can be incorporated into the same software platform. Bayesian Markov chain Monte Carlo simulation methods with WinBUGS software are the most popular choice for this option. A second possibility is to conduct evidence synthesis by Bayesian posterior estimation and then export the posterior samples to another package where other parameters are generated and the cost-effectiveness model is evaluated. Frequentist methods of parameter estimation followed by forward Monte Carlo simulation from the maximum likelihood estimates and their variance-covariance matrix represent’a third approach. A fourth option is bootstrap resampling—a frequentist simulation approach to parameter uncertainty. This tutorial paper also provides guidance on how to identify situations in which no correlations exist and therefore simpler approaches can be adopted. Software suitable for transferring data between different packages, and software that provides a user-friendly interface for integrated software platforms, offering investigators a flexible way of examining alternative scenarios, are reviewed.

Evidence synthesis for decision making 5: the baseline natural history model

Most cost-effectiveness analyses consist of a baseline model that represents the absolute natural history under a standard treatment in a comparator set and a model for relative treatment effects. We review synthesis issues that arise on the construction of the baseline natural history model. We cover both the absolute response to treatment on the outcome measures on which comparative effectiveness is defined and the other elements of the natural history model, usually “downstream” of the shorter-term effects reported in trials. We recommend that the same framework be used to model the absolute effects of a “standard treatment” or placebo comparator as that used for synthesis of relative treatment effects and that the baseline model is constructed independently from the model for relative treatment effects, to ensure that the latter are not affected by assumptions made about the baseline. However, simultaneous modeling of baseline and treatment effects could have some advantages when evidence is very sparse or when other research or study designs give strong reasons for believing in a particular baseline model. The predictive distribution, rather than the fixed effect or random effects mean, should be used to represent the baseline to reflect the observed variation in baseline rates. Joint modeling of multiple baseline outcomes based on data from trials or combinations of trial and observational data is recommended where possible, as this is likely to make better use of available evidence, produce more robust results, and ensure that the model is internally coherent.

Evidence synthesis for decision making 7: a reviewer's checklist

This checklist is for the review of evidence syntheses for treatment efficacy used in decision making based on either efficacy or cost-effectiveness. It is intended to be used for pairwise meta-analysis, indirect comparisons, and network meta-analysis, without distinction. It does not generate a quality rating and is not prescriptive. Instead, it focuses on a series of questions aimed at revealing the assumptions that the authors of the synthesis are expecting readers to accept, the adequacy of the arguments authors advance in support of their position, and the need for further analyses or sensitivity analyses. The checklist is intended primarily for those who review evidence syntheses, including indirect comparisons and network meta-analyses, in the context of decision making but will also be of value to those submitting syntheses for review, whether to decision-making bodies or journals. The checklist has 4 main headings: A) definition of the decision problem, B) methods of analysis and presentation of results, C) issues specific to network synthesis, and D) embedding the synthesis in a probabilistic cost-effectiveness model. The headings and implicit advice follow directly from the other tutorials in this series. A simple table is provided that could serve as a pro forma checklist.

Meta-regression models to address heterogeneity and inconsistency in network meta-analysis of survival outcomes

BACKGROUND

Recently, network meta-analysis of survival data with a multidimensional treatment effect was introduced. With these models the hazard ratio is not assumed to be constant over time, thereby reducing the possibility of violating transitivity in indirect comparisons. However, bias is still present if there are systematic differences in treatment effect modifiers across comparisons.

METHODS

In this paper we present multidimensional network meta-analysis models for time-to-event data that are extended with covariates to explain heterogeneity and adjust for confounding bias in the synthesis of evidence networks of randomized controlled trials. The impact of a covariate on the treatment effect can be assumed to be treatment specific or constant for all treatments compared.

RESULTS

An illustrative example analysis is presented for a network of randomized controlled trials evaluating different interventions for advanced melanoma. Incorporating a covariate related to the study date resulted in different estimates for the hazard ratios over time than an analysis without this covariate, indicating the importance of adjusting for changes in contextual factors over time.

CONCLUSION

Adding treatment-by-covariate interactions to multidimensional network meta-analysis models for published survival curves can be worthwhile to explain systematic differences across comparisons, thereby reducing inconsistencies and bias. An additional advantage is that heterogeneity in treatment effects can be explored.