Simulation and data‐generation for random‐effects network meta‐analysis of binary outcome

A comparison of methods for enriching network meta-analyses in the absence of individual patient data

During drug development, a biomarker is sometimes identified as separating a patient population into those with more and those with less benefit from evaluated treatments. Consequently, later studies might be targeted, while earlier ones are performed in mixed patient populations. This poses a challenge in evidence synthesis, especially if only aggregated data are available. Starting from this scenario, we investigate three commonly used network meta-analytic estimation methods, the naive estimation approach, the stand-alone analysis, and the network meta-regression. Additionally, we adapt and modify two methods which are used in evidence synthesis to combine randomized controlled trials with observational studies, the enrichment-through-weighting approach and the informative prior estimation. We evaluate all five methods in a simulation study with 32 scenarios using bias, RMSE, coverage, precision, and power. Additionally, we re-visit a clinical data set to exemplify and discuss the application. In the simulation study, none of the methods was observed to be clearly favorable over all investigated scenarios. However, the stand-alone analysis and the naive estimation performed comparably or worse than the other methods in all evaluated performance measures and simulation scenarios and are therefore not recommended. While substantial between-trial heterogeneity is challenging for all estimation approaches, the performance of the network meta-regression, the enriching-through weighting approach and the informative prior approach was dependent on the simulation scenario and the performance measure of interest. Furthermore, as these estimation methods are drawing slightly different assumptions, some of which require the presence of additional information for estimation, we recommend sensitivity-analyses wherever possible. This article is protected by copyright. All rights reserved.

Degree irregularity and rank probability bias in network meta-analysis

Network meta-analysis (NMA) is a statistical technique for the comparison of treatment options. Outcomes of Bayesian NMA include estimates of treatment effects, and the probabilities that each treatment is ranked best, second best and so on. How exactly network topology affects the accuracy and precision of these outcomes is not fully understood. Here we carry out a simulation study and find that disparity in the number of trials involving different treatments leads to a systematic bias in estimated rank probabilities. This bias is associated with an increased variation in the precision of treatment effect estimates. Using ideas from the theory of complex networks, we define a measure of "degree irregularity" to quantify asymmetry in the number of studies involving each treatment. Our simulations indicate that more regular networks have more precise treatment effect estimates and smaller bias of rank probabilities. Conversely, these topological effects are not observed for the accuracy of treatment effect estimates. This reinforces the importance of taking into account multiple measures, rather than making decisions based on a single metric. We also find that degree regularity is a better indicator for the accuracy and precision of parameter estimates in NMA than both the total number of studies in a network and the disparity in the number of trials per comparison. These results have implications for planning future trials. We demonstrate that choosing trials which reduce the network's irregularity can improve the precision and accuracy of parameter estimates from NMA.

A comparison of Bayesian and frequentist methods in random-effects network meta-analysis of binary data

The performance of statistical methods is often evaluated by means of simulation studies. In case of network meta-analysis of binary data, however, simulations are not currently available for many practically relevant settings. We perform a simulation study for sparse networks of trials under between-trial heterogeneity and including multi-arm trials. Results of the evaluation of two popular frequentist methods and a Bayesian approach using two different prior specifications are presented. Methods are evaluated using coverage, width of intervals, bias, and root mean squared error (RMSE). In addition, deviations from the theoretical surface under the cumulative rankings (SUCRAs) or P-scores of the treatments are evaluated. Under low heterogeneity and when a large number of trials informs the contrasts, all methods perform well with respect to the evaluated performance measures. Coverage is observed to be generally higher for the Bayesian than the frequentist methods. The width of credible intervals is larger than those of confidence intervals and is increasing when using a flatter prior for between-trial heterogeneity. Bias was generally small, but increased with heterogeneity, especially in netmeta. In some scenarios, the direction of bias differed between frequentist and Bayesian methods. The RMSE was comparable between methods but larger in indirectly than in directly estimated treatment effects. The deviation of the SUCRAs or P-scores from their theoretical values was mostly comparable over the methods but differed depending on the heterogeneity and the geometry of the investigated network. Multivariate meta-regression or Bayesian estimation using a half-normal prior scaled to 0.5 seems to be promising with respect to the evaluated performance measures in network meta-analysis of sparse networks.

Integrated evaluation of targeted and non-targeted therapies in a network meta-analysis

Individualized therapies for patients with biomarkers are moving more and more into the focus of research interest when developing new treatments. Hereby, the term individualized (or targeted) therapy denotes a treatment specifically developed for biomarker-positive patients. A network meta-analysis model for a binary endpoint combining the evidence for a targeted therapy from individual patient data with the evidence for a non-targeted therapy from aggregate data is presented and investigated. The biomarker status of the patients is either available at patient-level in individual patient data or at study-level in aggregate data. Both types of biomarker information have to be included. The evidence synthesis model follows a Bayesian approach and applies a meta-regression to the studies with aggregate data. In a simulation study, we address three treatment arms, one of them investigating a targeted therapy. The bias and the root-mean-square error of the treatment effect estimate for the subgroup of biomarker-positive patients based on studies with aggregate data are investigated. Thereby, the meta-regression approach is compared to approaches applying alternative solutions. The regression approach has a surprisingly small bias even in the presence of few studies. By contrast, the root-mean-square error is relatively greater. An illustrative example is provided demonstrating implementation of the presented network meta-analysis model in a clinical setting.