Multivariate meta-analysis: a robust approach based on the theory of U-statistic

Multistep estimators of the between-study covariance matrix under the multivariate random-effects model for meta-analysis

A wide variety of methods are available to estimate the between-study variance under the univariate random-effects model for meta-analysis. Some, but not all, of these estimators have been extended so that they can be used in the multivariate setting. We begin by extending the univariate generalised method of moments, which immediately provides a wider class of multivariate methods than was previously available. However, our main proposal is to use this new type of estimator to derive multivariate multistep estimators of the between-study covariance matrix. We then use the connection between the univariate multistep and Paule-Mandel estimators to motivate taking the limit, where the number of steps tends toward infinity. We illustrate our methodology using two contrasting examples and investigate its properties in a simulation study. We conclude that the proposed methodology is a fully viable alternative to existing estimation methods, is well suited to sensitivity analyses that explore the use of alternative estimators, and should be used instead of the existing DerSimonian and Laird-type moments based estimator in application areas where data are expected to be heterogeneous. However, multistep estimators do not seem to outperform the existing estimators when the data are more homogeneous. Advantages of the new multivariate multistep estimator include its semi-parametric nature and that it is computationally feasible in high dimensions. Our proposed estimation methods are also applicable for multivariate random-effects meta-regression, where study-level covariates are included in the model.

Random-effects meta-analysis of combined outcomes based on reconstructions of individual patient data

Meta-analyses of clinical trials typically focus on one outcome at a time. However, treatment decision-making depends on an overall assessment of outcomes balancing benefit in various domains and potential risks. This calls for meta-analysis methods for combined outcomes that encompass information from different domains. When individual patient data (IPD) are available from all studies, combined outcomes can be calculated for each individual and standard meta-analysis methods would apply. However, IPD are usually difficult to obtain. We propose a method to estimate the overall treatment effect for combined outcomes based on first reconstructing pseudo IPD from available summary statistics and then pooling estimates from multiple reconstructed datasets. We focus on combined outcomes constructed from two continuous original outcomes. The reconstruction step requires the specification of the joint distribution of these two original outcomes, including the correlation which is often unknown. For outcomes that are combined in a linear fashion, misspecifications of this correlation affect efficiency, but not consistency, of the resulting treatment effect estimator. For other combined outcomes, an accurate estimate of the correlation is necessary to ensure the consistency of treatment effect estimates. To this end, we propose several ways to estimate this correlation under different data availability scenarios. We evaluate the performance of the proposed methods through simulation studies and apply these to two examples: (a) a meta-analysis of dipeptidyl peptidase-4 inhibitors vs control on treating type 2 diabetes; and (b) a meta-analysis of positive airway pressure therapy vs control on lowering blood pressure among patients with obstructive sleep apnea.

Skew-normal random-effects model for meta-analysis of diagnostic test accuracy (DTA) studies

Hierarchical models are recommended for meta-analyzing diagnostic test accuracy (DTA) studies. The bivariate random-effects model is currently widely used to synthesize a pair of test sensitivity and specificity using logit transformation across studies. This model assumes a bivariate normal distribution for the random-effects. However, this assumption is restrictive and can be violated. When the assumption fails, inferences could be misleading. In this paper, we extended the current bivariate random-effects model by assuming a flexible bivariate skew-normal distribution for the random-effects in order to robustly model logit sensitivities and logit specificities. The marginal distribution of the proposed model is analytically derived so that parameter estimation can be performed using standard likelihood methods. The method of weighted-average is adopted to estimate the overall logit-transformed sensitivity and specificity. An extensive simulation study is carried out to investigate the performance of the proposed model compared to other standard models. Overall, the proposed model performs better in terms of confidence interval width of the average logit-transformed sensitivity and specificity compared to the standard bivariate linear mixed model and bivariate generalized linear mixed model. Simulations have also shown that the proposed model performed better than the well-established bivariate linear mixed model in terms of bias and comparable with regards to the root mean squared error (RMSE) of the between-study (co)variances. The proposed method is also illustrated using a published meta-analysis data.

The role of secondary outcomes in multivariate meta-analysis

Univariate meta‐analysis concerns a single outcome of interest measured across a number of independent studies. However, many research studies will have also measured secondary outcomes. Multivariate meta‐analysis allows us to take these secondary outcomes into account and can also include studies where the primary outcome is missing. We define the efficiency E as the variance of the overall estimate from a multivariate meta‐analysis relative to the variance of the overall estimate from a univariate meta‐analysis. The extra information gained from a multivariate meta‐analysis of n studies is then similar to the extra information gained if a univariate meta‐analysis of the primary effect had a further n(1−E)/E studies. The variance contribution of a study's secondary outcomes (its borrowing of strength) can be thought of as a contrast between the variance matrix of the outcomes in that study and the set of variance matrices of all the studies in the meta‐analysis. In the bivariate case this is given a simple graphical interpretation as the borrowing‐of‐strength plot. We discuss how these findings can also be used in the context of random‐effects meta‐analysis. Our discussion is motivated by a published meta‐analysis of 10 antihypertension clinical trials.

Borrowing of strength and study weights in multivariate and network meta-analysis

Multivariate and network meta-analysis have the potential for the estimated mean of one effect to borrow strength from the data on other effects of interest. The extent of this borrowing of strength is usually assessed informally. We present new mathematical definitions of 'borrowing of strength'. Our main proposal is based on a decomposition of the score statistic, which we show can be interpreted as comparing the precision of estimates from the multivariate and univariate models. Our definition of borrowing of strength therefore emulates the usual informal assessment. We also derive a method for calculating study weights, which we embed into the same framework as our borrowing of strength statistics, so that percentage study weights can accompany the results from multivariate and network meta-analyses as they do in conventional univariate meta-analyses. Our proposals are illustrated using three meta-analyses involving correlated effects for multiple outcomes, multiple risk factor associations and multiple treatments (network meta-analysis).

Multivariate meta-analysis with an increasing number of parameters

Meta-analysis can average estimates of multiple parameters, such as a treatment's effect on multiple outcomes, across studies. Univariate meta-analysis (UVMA) considers each parameter individually, while multivariate meta-analysis (MVMA) considers the parameters jointly and accounts for the correlation between their estimates. The performance of MVMA and UVMA has been extensively compared in scenarios with two parameters. Our objective is to compare the performance of MVMA and UVMA as the number of parameters, p, increases. Specifically, we show that (i) for fixed-effect (FE) meta-analysis, the benefit from using MVMA can substantially increase as p increases; (ii) for random effects (RE) meta-analysis, the benefit from MVMA can increase as p increases, but the potential improvement is modest in the presence of high between-study variability and the actual improvement is further reduced by the need to estimate an increasingly large between study covariance matrix; and (iii) when there is little to no between-study variability, the loss of efficiency due to choosing RE MVMA over FE MVMA increases as p increases. We demonstrate these three features through theory, simulation, and a meta-analysis of risk factors for non-Hodgkin lymphoma.

The choice of prior distribution for a covariance matrix in multivariate meta-analysis: a simulation study

Bayesian meta-analysis is an increasingly important component of clinical research, with multivariate meta-analysis a promising tool for studies with multiple endpoints. Model assumptions, including the choice of priors, are crucial aspects of multivariate Bayesian meta-analysis (MBMA) models. In a given model, two different prior distributions can lead to different inferences about a particular parameter. A simulation study was performed in which the impact of families of prior distributions for the covariance matrix of a multivariate normal random effects MBMA model was analyzed. Inferences about effect sizes were not particularly sensitive to prior choice, but the related covariance estimates were. A few families of prior distributions with small relative biases, tight mean squared errors, and close to nominal coverage for the effect size estimates were identified. Our results demonstrate the need for sensitivity analysis and suggest some guidelines for choosing prior distributions in this class of problems. The MBMA models proposed here are illustrated in a small meta-analysis example from the periodontal field and a medium meta-analysis from the study of stroke. Copyright © 2015 John Wiley & Sons, Ltd.

Multivariate Meta-Analysis of Genetic Association Studies: A Simulation Study

In a meta-analysis with multiple end points of interests that are correlated between or within studies, multivariate approach to meta-analysis has a potential to produce more precise estimates of effects by exploiting the correlation structure between end points. However, under random-effects assumption the multivariate estimation is more complex (as it involves estimation of more parameters simultaneously) than univariate estimation, and sometimes can produce unrealistic parameter estimates. Usefulness of multivariate approach to meta-analysis of the effects of a genetic variant on two or more correlated traits is not well understood in the area of genetic association studies. In such studies, genetic variants are expected to roughly maintain Hardy-Weinberg equilibrium within studies, and also their effects on complex traits are generally very small to modest and could be heterogeneous across studies for genuine reasons. We carried out extensive simulation to explore the comparative performance of multivariate approach with most commonly used univariate inverse-variance weighted approach under random-effects assumption in various realistic meta-analytic scenarios of genetic association studies of correlated end points. We evaluated the performance with respect to relative mean bias percentage, and root mean square error (RMSE) of the estimate and coverage probability of corresponding 95% confidence interval of the effect for each end point. Our simulation results suggest that multivariate approach performs similarly or better than univariate method when correlations between end points within or between studies are at least moderate and between-study variation is similar or larger than average within-study variation for meta-analyses of 10 or more genetic studies. Multivariate approach produces estimates with smaller bias and RMSE especially for the end point that has randomly or informatively missing summary data in some individual studies, when the missing data in the endpoint are imputed with null effects and quite large variance.

Bayesian Inference for Multivariate Meta-regression with a Partially Observed Within-Study Sample Covariance Matrix

Multivariate meta-regression models are commonly used in settings where the response variable is naturally multi-dimensional. Such settings are common in cardiovascular and diabetes studies where the goal is to study cholesterol levels once a certain medication is given. In this setting, the natural multivariate endpoint is Low Density Lipoprotein Cholesterol (LDL-C), High Density Lipoprotein Cholesterol (HDL-C), and Triglycerides (TG) (LDL-C, HDL-C, TG). In this paper, we examine study level (aggregate) multivariate meta-data from 26 Merck sponsored double-blind, randomized, active or placebo-controlled clinical trials on adult patients with primary hypercholesterolemia. Our goal is to develop a methodology for carrying out Bayesian inference for multivariate meta-regression models with study level data when the within-study sample covariance matrix S for the multivariate response data is partially observed. Specifically, the proposed methodology is based on postulating a multivariate random effects regression model with an unknown within-study covariance matrix Σ in which we treat the within-study sample correlations as missing data, the standard deviations of the within-study sample covariance matrix S are assumed observed, and given Σ, S follows a Wishart distribution. Thus, we treat the off-diagonal elements of S as missing data, and these missing elements are sampled from the appropriate full conditional distribution in a Markov chain Monte Carlo (MCMC) sampling scheme via a novel transformation based on partial correlations. We further propose several structures (models) for Σ, which allow for borrowing strength across different treatment arms and trials. The proposed methodology is assessed using simulated as well as real data, and the results are shown to be quite promising.

Multivariate meta-analysis using individual participant data

When combining results across related studies, a multivariate meta-analysis allows the joint synthesis of correlated effect estimates from multiple outcomes. Joint synthesis can improve efficiency over separate univariate syntheses, may reduce selective outcome reporting biases, and enables joint inferences across the outcomes. A common issue is that within-study correlations needed to fit the multivariate model are unknown from published reports. However, provision of individual participant data (IPD) allows them to be calculated directly. Here, we illustrate how to use IPD to estimate within-study correlations, using a joint linear regression for multiple continuous outcomes and bootstrapping methods for binary, survival and mixed outcomes. In a meta-analysis of 10 hypertension trials, we then show how these methods enable multivariate meta-analysis to address novel clinical questions about continuous, survival and binary outcomes; treatment–covariate interactions; adjusted risk/prognostic factor effects; longitudinal data; prognostic and multiparameter models; and multiple treatment comparisons. Both frequentist and Bayesian approaches are applied, with example software code provided to derive within-study correlations and to fit the models.

A refined method for multivariate meta-analysis and meta-regression

Making inferences about the average treatment effect using the random effects model for meta-analysis is problematic in the common situation where there is a small number of studies. This is because estimates of the between-study variance are not precise enough to accurately apply the conventional methods for testing and deriving a confidence interval for the average effect. We have found that a refined method for univariate meta-analysis, which applies a scaling factor to the estimated effects’ standard error, provides more accurate inference. We explain how to extend this method to the multivariate scenario and show that our proposal for refined multivariate meta-analysis and meta-regression can provide more accurate inferences than the more conventional approach. We explain how our proposed approach can be implemented using standard output from multivariate meta-analysis software packages and apply our methodology to two real examples. © 2013 The Authors. Statistics in Medicine published by John Wiley & Sons, Ltd.

Estimating within-study covariances in multivariate meta-analysis with multiple outcomes

Multivariate meta-analysis allows the joint synthesis of effect estimates based on multiple outcomes from multiple studies, accounting for the potential correlations among them. However, standard methods for multivariate meta-analysis for multiple outcomes are restricted to problems where the within-study correlation is known or where individual participant data are available. This paper proposes an approach to approximating the within-study covariances based on information about likely correlations between underlying outcomes. We developed methods for both continuous and dichotomous data and for combinations of the two types. An application to a meta-analysis of treatments for stroke illustrates the use of the approximated covariance in multivariate meta-analysis with correlated outcomes.

A matrix-based method of moments for fitting the multivariate random effects model for meta-analysis and meta-regression

Multivariate meta-analysis is becoming more commonly used. Methods for fitting the multivariate random effects model include maximum likelihood, restricted maximum likelihood, Bayesian estimation and multivariate generalisations of the standard univariate method of moments. Here, we provide a new multivariate method of moments for estimating the between-study covariance matrix with the properties that (1) it allows for either complete or incomplete outcomes and (2) it allows for covariates through meta-regression. Further, for complete data, it is invariant to linear transformations. Our method reduces to the usual univariate method of moments, proposed by DerSimonian and Laird, in a single dimension. We illustrate our method and compare it with some of the alternatives using a simulation study and a real example.

Bayesian multivariate meta-analysis with multiple outcomes

There has been a recent growth in developments of multivariate meta-analysis. We extend the methodology of Bayesian multivariate meta-analysis to the situation when there are more than two outcomes of interest, which is underexplored in the current literature. Our objective is to meta-analyse summary data from multiple outcomes simultaneously, accounting for potential dependencies among the data. One common issue is that studies do not all report all of the outcomes of interests, and we take an approach relying on marginal modelling of only the reported data. We employ a separation prior for the between-study variance-covariance matrix, which offers an improvement on the conventional inverse-Wishart prior, showing robustness in estimation and flexibility in incorporating prior information. Particular challenges arise when the number of outcomes is large relative to the number of studies because the number of parameters in the variance-covariance matrix can become substantial and there can be very little information with which to estimate between-study correlation coefficients. We explore assumptions that reduce the number of parameters in this matrix, including assumptions of homogenous variances, homogenous correlations for certain outcomes and positive correlation coefficients. We illustrate the methods with an example data set from the Cochrane Database of Systematic Reviews.

A method of moments estimator for random effect multivariate meta-analysis

Meta-analysis is a powerful approach to combine evidence from multiple studies to make inference about one or more parameters of interest, such as regression coefficients. The validity of the fixed effect model meta-analysis depends on the underlying assumption that all studies in the meta-analysis share the same effect size. In the presence of heterogeneity, the fixed effect model incorrectly ignores the between-study variance and may yield false positive results. The random effect model takes into account both within-study and between-study variances. It is more conservative than the fixed effect model and should be favored in the presence of heterogeneity. In this paper, we develop a noniterative method of moments estimator for the between-study covariance matrix in the random effect model multivariate meta-analysis. To our knowledge, it is the first such method of moments estimator in the matrix form. We show that our estimator is a multivariate extension of DerSimonian and Laird's univariate method of moments estimator, and it is invariant to linear transformations. In the simulation study, our method performs well when compared to existing random effect model multivariate meta-analysis approaches. We also apply our method in the analysis of a real data example.