Importance of Homogeneous Effect Modification for Causal Interpretation of Meta-analyses

Data Integration in Causal Inference

Integrating data from multiple heterogeneous sources has become increasingly popular to achieve a large sample size and diverse study population. This paper reviews development in causal inference methods that combines multiple datasets collected by potentially different designs from potentially heterogeneous populations. We summarize recent advances on combining randomized clinical trial with external information from observational studies or historical controls, combining samples when no single sample has all relevant variables with application to two-sample Mendelian randomization, distributed data setting under privacy concerns for comparative effectiveness and safety research using real-world data, Bayesian causal inference, and causal discovery methods.

Partial Identification of the Average Causal Effect in Multiple Study Populations: The Challenge of Combining Mendelian Randomization Studies

BACKGROUND

Researchers often use random-effects or fixed-effects meta-analysis to combine findings from multiple study populations. However, the causal interpretation of these models is not always clear, and they do not easily translate to settings where bounds, rather than point estimates, are computed.

METHODS

If bounds on an average causal effect of interest in a well-defined population are computed in multiple study populations under specified identifiability assumptions, then under those assumptions the average causal effect would lie within all study-specific bounds and thus the intersection of the study-specific bounds. We demonstrate this by pooling bounds on the average causal effect of prenatal alcohol exposure on attention deficit-hyperactivity disorder symptoms, computed in two European cohorts and under multiple sets of assumptions in Mendelian randomization (MR) analyses.

RESULTS

For all assumption sets considered, pooled bounds were wide and did not identify the direction of effect. The narrowest pooled bound computed implied the risk difference was between -4 and 34 percentage points.

CONCLUSIONS

All pooled bounds computed in our application covered the null, illustrating how strongly point estimates from prior MR studies of this effect rely on within-study homogeneity assumptions. We discuss how the interpretation of both pooled bounds and point estimation in MR is complicated by possible heterogeneity of effects across populations.

Meta-regression methods to characterize evidence strength using meaningful-effect percentages conditional on study characteristics

Meta-regression analyses usually focus on estimating and testing differences in average effect sizes between individual levels of each meta-regression covariate in turn. These metrics are useful but have limitations: they consider each covariate individually, rather than in combination, and they characterize only the mean of a potentially heterogeneous distribution of effects. We propose additional metrics that address both limitations. Given a chosen threshold representing a meaningfully strong effect size, these metrics address the questions: "For a given joint level of the covariates, what percentage of the population effects are meaningfully strong?" and "For any two joint levels of the covariates, what is the difference between these percentages of meaningfully strong effects?" We provide semiparametric methods for estimation and inference and assess their performance in a simulation study. We apply the proposed methods to meta-regression analyses on memory consolidation and on dietary behavior interventions, illustrating how the methods can provide more information than standard reporting alone. To facilitate implementing the methods in practice, we provide reporting guidelines and simple R code.