A method for the meta-analysis of mutually exclusive binary outcomes

Alcohol cessation and breast cancer risk stratified by hormone receptor status

Because alcohol consumption is an established cause of female breast cancer, understanding whether cessation affects risk is of public health importance. In a recent meta-analysis, compared with continuing consumption, the relative risk (RR) for cessation was 0.95 (95% confidence interval [CI] 0.88-1.01). Because intake of alcohol is more consistently associated with estrogen receptor positive (ER+) than negative (ER-) subtypes, we conducted a meta-analysis of alcohol cessation for ER-specific breast cancer risk using data from three cohort studies and one population-based case-control study (ER + n = 3,793; ER- n = 627) with information reported on cessation and ER status. Compared with continuing consumption, cessation was associated with lower risk of ER+ (RR = 0.88, 95%CI, 0.79-0.98) but not ER- (RR = 1.23, 95%CI, 0.98-1.55) breast cancer. These results suggest that, compared with continuing consumption, alcohol cessation may reduce ER + but not ER- breast cancer risk. However, research that considers duration of cessation is warranted.

Bayesian estimation and prediction for network meta-analysis with contrast-based approach

Network meta-analysis is gaining prominence in clinical epidemiology and health technology assessments that enable comprehensive assessment of comparative effectiveness for multiple available treatments. In network meta-analysis, Bayesian methods have been one of the standard approaches for the arm-based approach and are widely applied in practical data analyses. Also, for most cases in these applications, proper noninformative priors are adopted, which does not incorporate subjective prior knowledge into the analyses, and reference Bayesian analyses are major choices. In this article, we provide generic Bayesian analysis methods for the contrast-based approach of network meta-analysis, where the generic Bayesian methods can treat proper and improper prior distributions. The proposed methods enable direct sampling from the posterior and posterior predictive distributions without formal iterative computations such as Markov chain Monte Carlo, and technical convergence checks are not required. In addition, representative noninformative priors that can be treated in the proposed framework involving the Jeffreys prior are provided. We also provide an easy-to-handle R statistical package, BANMA, to implement these Bayesian analyses via simple commands. The proposed Bayesian methods are illustrated using various noninformative priors through applications to two real network meta-analyses.

Improved methods to construct prediction intervals for network meta-analysis

Network meta-analysis has played an important role in evidence-based medicine for assessing the comparative effectiveness of multiple available treatments. The prediction interval has been one of the standard outputs in recent network meta-analysis as an effective measure that enables simultaneous assessment of uncertainties in treatment effects and heterogeneity among studies. To construct the prediction interval, a large-sample approximating method based on the t-distribution has generally been applied in practice; however, recent studies have shown that similar t-approximation methods for conventional pairwise meta-analyses can substantially underestimate the uncertainty under realistic situations. In this article, we performed simulation studies to assess the validity of the current standard method for network meta-analysis, and we show that its validity can also be violated under realistic situations. To address the invalidity issue, we developed two new methods to construct more accurate prediction intervals through bootstrap and Kenward-Roger-type adjustment. In simulation experiments, the two proposed methods exhibited better coverage performance and generally provided wider prediction intervals than the ordinary t-approximation. We also developed an R package, PINMA (https://cran.r-project.org/web/packages/PINMA/), to perform the proposed methods using simple commands. We illustrate the effectiveness of the proposed methods through applications to two real network meta-analyses.

Kenward-Roger-type corrections for inference methods of network meta-analysis and meta-regression

Network meta-analysis has been an essential methodology of systematic reviews for comparative effectiveness research. The restricted maximum likelihood (REML) method is one of the current standard inference methods for multivariate, contrast-based meta-analysis models, but recent studies have revealed the resultant confidence intervals of average treatment effect parameters in random-effects models can seriously underestimate statistical errors; that is, the actual coverage probability of a true parameter cannot retain the nominal level (e.g., 95%). In this article, we provided improved inference methods for the network meta-analysis and meta-regression models using higher-order asymptotic approximations based on the approach of Kenward and Roger (Biometrics 1997;53:983-997). We provided two corrected covariance matrix estimators for the REML estimator and improved approximations for its sample distribution using a t-distribution with adequate degrees of freedom. All of the proposed procedures can be implemented using only simple matrix calculations. In simulation studies under various settings, the REML-based Wald-type confidence intervals seriously underestimated the statistical errors, especially in cases of small numbers of trials meta-analyzed. By contrast, the proposed Kenward-Roger-type inference methods consistently showed accurate coverage properties under all the settings considered in our experiments. We also illustrated the effectiveness of the proposed methods through applications to two real network meta-analysis datasets.

Multivariate meta-analysis of multiple outcomes: characteristics and predictors of borrowing of strength from Cochrane reviews

OBJECTIVES

Multivariate meta-analysis allows the joint synthesis of multiple outcomes accounting for their correlation. This enables borrowing of strength (BoS) across outcomes, which may lead to greater efficiency and even different conclusions compared to separate univariate meta-analyses. However, multivariate meta-analysis is complex to apply, so guidance is needed to flag (in advance of analysis) when the approach is most useful.

STUDY DESIGN AND SETTING

We use 43 Cochrane intervention reviews to empirically investigate the characteristics of meta-analysis datasets that are associated with a larger BoS statistic (from 0 to 100%) when applying a bivariate meta-analysis of binary outcomes.

RESULTS

Four characteristics were identified as strongly associated with BoS: the total number of studies, the number of studies with the outcome of interest, the percentage of studies missing the outcome of interest, and the largest absolute within-study correlation. Using these characteristics, we then develop a model for predicting BoS in a new dataset, which is shown to have good performance (an adjusted R² of 50%). Applied examples are used to illustrate the use of the BoS prediction model.

CONCLUSIONS

Cochrane reviewers mainly use univariate meta-analysis methods, but the identified characteristics associated with BoS and our subsequent prediction model for BoS help to flag when a multivariate meta-analysis may also be beneficial in Cochrane reviews with multiple binary outcomes. Extension to non-Cochrane reviews and other outcome types is still required.

Shared parameter model for competing risks and different data summaries in meta-analysis: Implications for common and rare outcomes

This paper considers the problem in aggregate data meta-analysis of studies reporting multiple competing binary outcomes and of studies using different summary formats for those outcomes. For example, some may report numbers of patients with at least one of each outcome while others may report the total number of such outcomes. We develop a shared parameter model on hazard ratio scale accounting for different data summaries and competing risks. We adapt theoretical arguments from the literature to demonstrate that the models are equivalent if events are rare. We use constructed data examples and a simulation study to find an event rate threshold of approximately 0.2 above which competing risks and different data summaries may bias results if no adjustments are made. Below this threshold, simpler models may be sufficient. We recommend analysts to consider the absolute event rates and only use a simple model ignoring data types and competing risks if all of underlying events are rare (below our threshold of approximately 0.2). If one or more of the absolute event rates approaches or exceeds our informal threshold, it may be necessary to account for data types and competing risks through a shared parameter model in order to avoid biased estimates.

AHRQ EPC Series on Improving Translation of Evidence: Web-Based Interactive Presentation of Systematic Review Reports

Systematic reviews are used by a diverse range of users to address an ever-expanding set of questions and needs. It is unlikely that a single static report will efficiently satisfy the different needs of diverse users.

METHODS

An open-source Web-based interactive report presentation of a systematic review was developed to allow users to generate their own "reports" from the information produced by the review. Data from a broad-scope systematic review were used with network meta-analysis conducted on nonsurgical treatments of urinary incontinence (UI) in women. Stakeholders informed and piloted the tool and assessed its usefulness.

RESULTS

The final tool allows users to obtain descriptive and analytic results for a network of treatment categories and various outcomes (cure, improvement, satisfaction, quality of life, adverse events) across several subgroups (all women, older women, or those with stress or urgency UI), along with study-level information, and overall conclusions. The stakeholders were satisfied with the functionality of the tool and proposed a number of improvements regarding presentation (for example, present information on numbers of trials in figures), analyses (for example, allow on-the-fly subgroup analyses, explore trade-offs between several outcomes), and information sharing (for example, provide ability to import/export data from/to other software).

CONCLUSION

A prototype tool to present customized analyses from broad-scope systematic reviews is presented. Further improvements are suggested to develop a scalable tool to make systematic reviews useful to increasingly diverse user groups.

Empirical comparison of univariate and multivariate meta-analyses in Cochrane Pregnancy and Childbirth reviews with multiple binary outcomes

BACKGROUND

Multivariate meta-analysis (MVMA) jointly synthesizes effects for multiple correlated outcomes. The MVMA model is potentially more difficult and time-consuming to apply than univariate models, so if its use makes little difference to parameter estimates, it could be argued that it is redundant.

METHODS

We assessed the applicability and impact of MVMA in Cochrane Pregnancy and Childbirth (CPCB) systematic reviews. We applied MVMA to CPCB reviews published between 2011 and 2013 with two or more binary outcomes with at least three studies and compared findings with results of univariate meta-analyses. Univariate random effects meta-analysis models were fitted using restricted maximum likelihood estimation (REML).

RESULTS

Eighty CPCB reviews were published. MVMA could not be applied in 70 of these reviews. MVMA was not feasible in three of the remaining 10 reviews because the appropriate models failed to converge. Estimates from MVMA agreed with those of univariate analyses in most of the other seven reviews. Statistical significance changed in two reviews: In one, this was due to a very small change in P value; in the other, the MVMA result for one outcome suggested that previous univariate results may be vulnerable to small-study effects and that the certainty of clinical conclusions needs consideration.

CONCLUSIONS

MVMA methods can be applied only in a minority of reviews of interventions in pregnancy and childbirth and can be difficult to apply because of missing correlations or lack of convergence. Nevertheless, clinical and/or statistical conclusions from MVMA may occasionally differ from those from univariate analyses.

Multivariate Methods for Meta-Analysis of Genetic Association Studies

Multivariate meta-analysis of genetic association studies and genome-wide association studies has received a remarkable attention as it improves the precision of the analysis. Here, we review, summarize and present in a unified framework methods for multivariate meta-analysis of genetic association studies and genome-wide association studies. Starting with the statistical methods used for robust analysis and genetic model selection, we present in brief univariate methods for meta-analysis and we then scrutinize multivariate methodologies. Multivariate models of meta-analysis for a single gene-disease association studies, including models for haplotype association studies, multiple linked polymorphisms and multiple outcomes are discussed. The popular Mendelian randomization approach and special cases of meta-analysis addressing issues such as the assumption of the mode of inheritance, deviation from Hardy-Weinberg Equilibrium and gene-environment interactions are also presented. All available methods are enriched with practical applications and methodologies that could be developed in the future are discussed. Links for all available software implementing multivariate meta-analysis methods are also provided.

Bartlett-type corrections and bootstrap adjustments of likelihood-based inference methods for network meta-analysis

In network meta-analyses that synthesize direct and indirect comparison evidence concerning multiple treatments, multivariate random effects models have been routinely used for addressing between-studies heterogeneities. Although their standard inference methods depend on large sample approximations (eg, restricted maximum likelihood estimation) for the number of trials synthesized, the numbers of trials are often moderate or small. In these situations, standard estimators cannot be expected to behave in accordance with asymptotic theory; in particular, confidence intervals cannot be assumed to exhibit their nominal coverage probabilities (also, the type I error probabilities of the corresponding tests cannot be retained). The invalidity issue may seriously influence the overall conclusions of network meta-analyses. In this article, we develop several improved inference methods for network meta-analyses to resolve these problems. We first introduce 2 efficient likelihood-based inference methods, the likelihood ratio test-based and efficient score test-based methods, in a general framework of network meta-analysis. Then, to improve the small-sample inferences, we developed improved higher-order asymptotic methods using Bartlett-type corrections and bootstrap adjustment methods. The proposed methods adopt Monte Carlo approaches using parametric bootstraps to effectively circumvent complicated analytical calculations of case-by-case analyses and to permit flexible application to various statistical models network meta-analyses. These methods can also be straightforwardly applied to multivariate meta-regression analyses and to tests for the evaluation of inconsistency. In numerical evaluations via simulations, the proposed methods generally performed well compared with the ordinary restricted maximum likelihood-based inference method. Applications to 2 network meta-analysis datasets are provided.

Quantifying indirect evidence in network meta-analysis

Network meta-analysis enables comprehensive synthesis of evidence concerning multiple treatments and their simultaneous comparisons based on both direct and indirect evidence. A fundamental pre-requisite of network meta-analysis is the consistency of evidence that is obtained from different sources, particularly whether direct and indirect evidence are in accordance with each other or not, and how they may influence the overall estimates. We have developed an efficient method to quantify indirect evidence, as well as a testing procedure to evaluate their inconsistency using Lindsay's composite likelihood method. We also show that this estimator has complete information for the indirect evidence. Using this method, we can assess the degree of consistency between direct and indirect evidence and their contribution rates to the overall estimate. Sensitivity analyses can be also conducted with this method to assess the influences of potentially inconsistent treatment contrasts on the overall results. These methods can provide useful information for overall comparative results that might be biased from specific inconsistent treatment contrasts. We also provide some fundamental requirements for valid inference on these methods concerning consistency restrictions on multi-arm trials. In addition, the efficiency of the developed method is demonstrated based on simulation studies. Applications to a network meta-analysis of 12 new-generation antidepressants are presented. Copyright © 2016 John Wiley & Sons, Ltd.

Accounting for correlation in network meta-analysis with multi-arm trials

Multi-arm trials (trials with more than two arms) are particularly valuable forms of evidence for network meta-analysis (NMA). Trial results are available either as arm-level summaries, where effect measures are reported for each arm, or as contrast-level summaries, where the differences in effect between arms compare with the control arm chosen for the trial. We show that likelihood-based inference in both contrast-level and arm-level formats is identical if there are only two-arm trials, but that if there are multi-arm trials, results from the contrast-level format will be incorrect unless correlations are accounted for in the likelihood. We review Bayesian and frequentist software for NMA with multi-arm trials that can account for this correlation and give an illustrative example of the difference in estimates that can be introduced if the correlations are not incorporated. We discuss methods of imputing correlations when they cannot be derived from the reported results and urge trialists to report the standard error for the control arm even if only contrast-level summaries are reported. Copyright © 2012 John Wiley & Sons, Ltd.

Indirect comparisons of ranibizumab and dexamethasone in macular oedema secondary to retinal vein occlusion

Background

Two treatments, ranibizumab and dexamethasone implant, for visual impairment due to macular oedema (ME) secondary to retinal vein occlusion (RVO) have recently been studied in clinical trials. There have been no head to head comparisons of the two treatments, and improvement measured as gain in Best Corrected Visual Acuity (BCVA) was reported using different outcomes thresholds between trials. To overcome these limitations, and inform an economic model, we developed a combination of a multinomial model and an indirect Bayesian comparison model for multinomial outcomes.

Methods

Outcomes of change from baseline in BCVA for dexamethasone compatible with those available for ranibizumab, reported by 4 randomised controlled trials, were estimated by fitting a multinomial distribution model to the probability of a patient achieving outcomes in a range of changes from baseline in BCVA (numbers of letters) at month 1. A Bayesian indirect comparison multinomial model was then developed to compare treatments in the Branch RVO (BRVO) and Central RVO (CRVO) populations.

Results

The multinomial model had excellent fit to the observed results. With the Bayesian indirect comparison, the probabilities of achieving ≥20 letters, with 95% credible intervals, at month 1 in patients with BRVO were 0.191 (0.130, 0.261) with ranibizumab and 0.093 (0.027, 0.213) with dexamethasone. In patients with CRVO, probabilities were 0.133 (0.082, 0.195) (ranibizumab) and 0.063 (0.016, 0.153) (dexamethasone). Probabilities of a gain in ≥10 letters in BRVO patients were 0.500 (0.365, 0.650) v 0.459 (0.248, 0.724) and in CRVO patients 0.459 (0.332, 0.602) v 0.498 (0.263, 0.791) for ranibizumab and dexamethasone treatments respectively. The comparisons also favoured ranibizumab at month 6 although changes to therapies after month 3 may have introduced bias.

Conclusion

The newly developed combination of multinomial and indirect Bayesian comparison models indicated a trend for ranibizumab association with a greater percentage of ME patients achieving visual gains than dexamethasone at months 1 and 6 in a common clinical context, although results were not classically significant. The method was a useful tool for comparisons of probability distributions between clinical trials that reported events on different categorical scales and estimates can be used to inform economic models.

Multivariate meta-analysis using individual participant data

When combining results across related studies, a multivariate meta-analysis allows the joint synthesis of correlated effect estimates from multiple outcomes. Joint synthesis can improve efficiency over separate univariate syntheses, may reduce selective outcome reporting biases, and enables joint inferences across the outcomes. A common issue is that within-study correlations needed to fit the multivariate model are unknown from published reports. However, provision of individual participant data (IPD) allows them to be calculated directly. Here, we illustrate how to use IPD to estimate within-study correlations, using a joint linear regression for multiple continuous outcomes and bootstrapping methods for binary, survival and mixed outcomes. In a meta-analysis of 10 hypertension trials, we then show how these methods enable multivariate meta-analysis to address novel clinical questions about continuous, survival and binary outcomes; treatment–covariate interactions; adjusted risk/prognostic factor effects; longitudinal data; prognostic and multiparameter models; and multiple treatment comparisons. Both frequentist and Bayesian approaches are applied, with example software code provided to derive within-study correlations and to fit the models.

Addressing multimorbidity in evidence integration and synthesis

To minimize bias, clinical practice guidelines (CPG) for managing patients with multiple conditions should be informed by well-planned syntheses of the totality of the relevant evidence by means of systematic reviews and meta-analyses. However, deficiencies along the entire evidentiary pathway hinder the development of evidence-based CPGs. Published reports of trials and observational studies often do not provide usable data on treatment effect heterogeneity, perhaps because their design, analysis and presentation is seldom geared towards informing on how multimorbidity modifies the effect of treatments. Systematic reviews and meta-analyses inherit all the limitations of their building blocks and introduce additional of their own, including selection biases at the level of the included studies, ecological biases, and analytical challenges. To generate recommendations to help negotiate some of the challenges in synthesizing the primary literature, so that the results of the evidence synthesis is applicable to the care of those with multiple conditions. Informal group process. We have built upon established general guidance, and provide additional recommendations specific to systematic reviews that could improve the CPGs for multimorbid patients. We suggest that following the additional recommendations is good practice, but acknowledge that not all proposed recommendations are of equal importance, validity and feasibility, and that further work is needed to test and refine the recommendations.

A multivariate model for the meta-analysis of study level survival data at multiple times

Motivated by our meta-analytic dataset involving survival rates after treatment for critical leg ischemia, we develop and apply a new multivariate model for the meta-analysis of study level survival data at multiple times. Our data set involves 50 studies that provide mortality rates at up to seven time points, which we model simultaneously, and we compare the results to those obtained from standard methodologies. Our method uses exact binomial within-study distributions and enforces the constraints that both the study specific and the overall mortality rates must not decrease over time. We directly model the probabilities of mortality at each time point, which are the quantities of primary clinical interest. We also present I² statistics that quantify the impact of the between-study heterogeneity, which is very considerable in our data set. © 2014 The Authors. Research Synthesis Methods published by John Wiley & Sons, Ltd.

An empirical comparison of univariate and multivariate meta-analyses for categorical outcomes

Treatment effects for multiple outcomes can be meta-analyzed separately or jointly, but no systematic empirical comparison of the two approaches exists. From the Cochrane Library of Systematic Reviews, we identified 45 reviews, including 1473 trials and 258,675 patients, that contained two or three univariate meta-analyses of categorical outcomes for the same interventions that could also be analyzed jointly. Eligible were meta-analyses with at least seven trials reporting all outcomes for which the cross-classification tables were exactly recoverable (e.g., outcomes were mutually exclusive, or one was a subset of the other). This ensured known correlation structures. Outcomes in 40 reviews had an is-subset-of relationship, and those in 5 were mutually exclusive. We analyzed these data with univariate and multivariate models based on discrete and approximate likelihoods. Discrete models were fit in the Bayesian framework using slightly informative priors. The summary effects for each outcome were similar with univariate and multivariate meta-analyses (both using the approximate and discrete likelihoods); however, the multivariate model with the discrete likelihood gave smaller between-study variance estimates, and narrower predictive intervals for new studies. When differences in the summary treatment effects were examined, the multivariate models gave similar summary estimates but considerably longer (shorter) uncertainty intervals because of positive (negative) correlation between outcome treatment effects. It is unclear whether any of the examined reviews would change their overall conclusions based on multivariate versus univariate meta-analyses, because extra-analytical and context-specific considerations contribute to conclusions and, secondarily, because numerical differences were often modest.

Estimating within-study covariances in multivariate meta-analysis with multiple outcomes

Multivariate meta-analysis allows the joint synthesis of effect estimates based on multiple outcomes from multiple studies, accounting for the potential correlations among them. However, standard methods for multivariate meta-analysis for multiple outcomes are restricted to problems where the within-study correlation is known or where individual participant data are available. This paper proposes an approach to approximating the within-study covariances based on information about likely correlations between underlying outcomes. We developed methods for both continuous and dichotomous data and for combinations of the two types. An application to a meta-analysis of treatments for stroke illustrates the use of the approximated covariance in multivariate meta-analysis with correlated outcomes.

Meta-analysis of two-arm studies: Modeling the intervention effect from survival probabilities

Pooling the hazard ratios is not always feasible in meta-analyses of two-arm survival studies, because the measure of the intervention effect is not systematically reported. An alternative approach proposed by Moodie et al. is to use the survival probabilities of the included studies, all collected at a single point in time: the intervention effect is then summarised as the pooled ratio of the logarithm of survival probabilities (which is an estimator of the hazard ratios when hazards are proportional). In this article, we propose a generalization of this method. By using survival probabilities at several points in time, this generalization allows a flexible modeling of the intervention over time. The method is applicable to partially proportional hazards models, with the advantage of not requiring the specification of the baseline survival. As in Moodie et al.'s method, the study-level factors modifying the survival functions can be ignored as long as they do not modify the intervention effect. The procedures of estimation are presented for fixed and random effects models. Two illustrative examples are presented.

Meta-analysis inside and outside particle physics: two traditions that should converge?

The use of meta-analysis in medicine and epidemiology really took off in the 1970s. However, in high-energy physics, the Particle Data Group has been carrying out meta-analyses of measurements of particle masses and other properties since 1957. Curiously, there has been virtually no interaction between those working inside and outside particle physics. In this paper, we use statistical models to study two major differences in practice. The first is the usefulness of systematic errors, which physicists are now beginning to quote in addition to statistical errors. The second is whether it is better to treat heterogeneity by scaling up errors as do the Particle Data Group or by adding a random effect as does the rest of the community. Besides fitting models, we derive and use an exact test of the error-scaling hypothesis. We also discuss the other methodological differences between the two streams of meta-analysis. Our conclusion is that systematic errors are not currently very useful and that the conventional random effects model, as routinely used in meta-analysis, has a useful role to play in particle physics. The moral we draw for statisticians is that we should be more willing to explore 'grassroots' areas of statistical application, so that good statistical practice can flow both from and back to the statistical mainstream. Copyright © 2012 John Wiley & Sons, Ltd.

Meta-analysis of effect sizes reported at multiple time points: A multivariate approach

Background

Many comparative studies report results at multiple time points. Such data are correlated because they pertain to the same patients, but are typically meta-analyzed as separate quantitative syntheses at each time point, ignoring the correlations between time points.

Purpose

To develop a meta-analytic approach that estimates treatment effects at successive time points and takes account of the stochastic dependencies of those effects.

Methods

We present both fixed and random effects methods for multivariate meta-analysis of effect sizes reported at multiple time points. We provide formulas for calculating the covariance (and correlations) of the effect sizes at successive time points for four common metrics (log odds ratio, log risk ratio, risk difference, and arcsine difference) based on data reported in the primary studies. We work through an example of a meta-analysis of 17 randomized trials of radiotherapy and chemotherapy versus radiotherapy alone for the postoperative treatment of patients with malignant gliomas, where in each trial survival is assessed at 6, 12, 18, and 24 months post randomization. We also provide software code for the main analyses described in the article.

Results

We discuss the estimation of fixed and random effects models and explore five options for the structure of the covariance matrix of the random effects. In the example, we compare separate (univariate) meta-analyses at each of the four time points with joint analyses across all four time points using the proposed methods. Although results of univariate and multivariate analyses are generally similar in the example, there are small differences in the magnitude of the effect sizes and the corresponding standard errors. We also discuss conditional multivariate analyses where one compares treatment effects at later time points given observed data at earlier time points.

Limitations

Simulation and empirical studies are needed to clarify the gains of multivariate analyses compared with separate meta-analyses under a variety of conditions.

Conclusions

Data reported at multiple time points are multivariate in nature and are efficiently analyzed using multivariate methods. The latter are an attractive alternative or complement to performing separate meta-analyses.

On the covariance of two correlated log-odds ratios

In many applications two correlated estimates of an effect size need to be considered simultaneously to be combined or compared. Apparently, there is a need for calculating their covariance, which however requires access to the individual data that may not be available to a researcher performing the analysis. We present a simple and efficient method for calculating the covariance of two correlated log-odds ratios. The method is very simple, is based on the well-known large sample approximations, can be applied using only data that are available in the published reports and more importantly, is very general, because it is shown to encompass several previously derived estimates (multiple outcomes, multiple treatments, dose-response models, mutually exclusive outcomes, genetic association studies) as special cases. By encompassing the previous approaches in a unified framework, the method allows easily deriving estimates for the covariance concerning problems that were not easy to be obtained otherwise. We show that the method can be used to derive the covariance of log-odds ratios from matched and unmatched case-control studies that use the same cases, a situation that has been addressed in the past only using individual data. Future applications of the method are discussed.

Multivariate meta-analysis: potential and promise

The multivariate random effects model is a generalization of the standard univariate model. Multivariate meta-analysis is becoming more commonly used and the techniques and related computer software, although continually under development, are now in place. In order to raise awareness of the multivariate methods, and discuss their advantages and disadvantages, we organized a one day ‘Multivariate meta-analysis’ event at the Royal Statistical Society. In addition to disseminating the most recent developments, we also received an abundance of comments, concerns, insights, critiques and encouragement. This article provides a balanced account of the day's discourse. By giving others the opportunity to respond to our assessment, we hope to ensure that the various view points and opinions are aired before multivariate meta-analysis simply becomes another widely used de facto method without any proper consideration of it by the medical statistics community. We describe the areas of application that multivariate meta-analysis has found, the methods available, the difficulties typically encountered and the arguments for and against the multivariate methods, using four representative but contrasting examples. We conclude that the multivariate methods can be useful, and in particular can provide estimates with better statistical properties, but also that these benefits come at the price of making more assumptions which do not result in better inference in every case. Although there is evidence that multivariate meta-analysis has considerable potential, it must be even more carefully applied than its univariate counterpart in practice. Copyright © 2011 John Wiley & Sons, Ltd.

Network meta-analysis with competing risk outcomes

BACKGROUND

Cost-effectiveness analysis often requires information on the effectiveness of interventions on multiple outcomes, and commonly these take the form of competing risks. Nevertheless, methods for synthesis of randomized controlled trials with competing risk outcomes are limited.

OBJECTIVE

The aim of this study was to develop and illustrate flexible evidence synthesis methods for trials reporting competing risk results, which allow for studies with different follow-up times, and that take account of the statistical dependencies between outcomes, regardless of the number of outcomes and treatments.

METHODS

We propose a competing risk meta-analysis based on hazards, rather than probabilities, estimated in a Bayesian Markov chain Monte Carlo (MCMC) framework using WinBUGS software. Our approach builds on existing work on mixed treatment comparison (network) meta-analysis, which can be applied to any number of treatments, and any number of competing outcomes, and to data sets with varying follow-up times. We show how a fixed effect model can be estimated, and two random treatment effect models with alternative structures for between-trial variation. We suggest methods for choosing between these alternative models.

RESULTS

We illustrate the methods by applying them to a data set involving 17 trials comparing nine antipsychotic treatments for schizophrenia including placebo, on three competing outcomes: relapse, discontinuation because of intolerable side effects, and discontinuation for other reasons.

CONCLUSIONS

Bayesian MCMC provides a flexible framework for synthesis of competing risk outcomes with multiple treatments, particularly suitable for embedding within probabilistic cost-effectiveness analysis.

Extending DerSimonian and Laird's methodology to perform multivariate random effects meta-analyses

Multivariate meta-analysis is increasingly used in medical statistics. In the univariate setting, the non-iterative method proposed by DerSimonian and Laird is a simple and now standard way of performing random effects meta-analyses. We propose a natural and easily implemented multivariate extension of this procedure which is accessible to applied researchers and provides a much less computationally intensive alternative to existing methods. In a simulation study, the proposed procedure performs similarly in almost all ways to the more established iterative restricted maximum likelihood approach. The method is applied to some real data sets and an extension to multivariate meta-regression is described.

Why add anything to nothing? The arcsine difference as a measure of treatment effect in meta-analysis with zero cells

For clinical trials with binary endpoints there are a variety of effect measures, for example risk difference, risk ratio and odds ratio (OR). The choice of metric is not always straightforward and should reflect the clinical question. Additional issues arise if the event of interest is rare. In systematic reviews, trials with zero events in both arms are encountered and often excluded from the meta-analysis.The arcsine difference (AS) is a measure which is rarely considered in the medical literature. It appears to have considerable promise, because it handles zeros naturally, and its asymptotic variance does not depend on the event probability.This paper investigates the pros and cons of using the AS as a measure of intervention effect. We give a pictorial representation of its meaning and explore its properties in relation to other measures. Based on analytical calculation of the variance of the arcsine transformation, a more conservative variance estimate for the rare event setting is proposed. Motivated by a published meta-analysis in cardiac surgery, we examine the statistical properties of the various metrics in the rare event setting.We find the variance estimate of the AS to be more stable than that of the log-OR, even if events are rare. However, parameter estimation is biased if the groups are markedly unbalanced. Though, from a theoretical viewpoint, the AS is a natural choice, its practical use is likely to continue to be limited by its less direct interpretation.