Systematically missing confounders in individual participant data meta-analysis of observational cohort studies

Retrieval barriers in individual participant data reviews with network meta-analysis

OBJECTIVES

Individual participant data (IPD) from randomised controlled trials (RCTs) can be used in network meta-analysis (NMA) to underpin patient care and are the best analyses to support the development of guidelines about the use of healthcare interventions for a specific condition. However, barriers to IPD retrieval pose a major threat. The aim of this study was to present barriers we encountered during retrieval of IPD from RCTs in two published systematic reviews with IPD-NMA.

METHODS

We evaluated retrieval of IPD from RCTs for IPD-NMA in Alzheimer's dementia and type 1 diabetes. We requested IPD from authors, industry sponsors and data repositories, and recorded IPD retrieval, reasons for IPD unavailability, and retrieval challenges.

RESULTS

In total, we identified 108 RCTs: 78 industry sponsored, 11 publicly sponsored and 19 with no funding information. After failing to obtain IPD from any trial authors, we requested it from industry sponsors. Seven of the 17 industry sponsors shared IPD for 12 950 participants (59%) through proprietary-specific data sharing platforms from 26 RCTs (33%). We found that lack of RCT identifiers (eg, National Clinical Trial number) and unclear data ownership were major challenges in IPD retrieval. Incomplete information in retrieved datasets was another important problem that led to exclusion of RCTs from the NMA. There were also practical challenges in obtaining IPD from or analysing it within platforms, and additional costs were incurred in accessing IPD this way.

CONCLUSIONS

We found no clear evidence of retrieval bias (where IPD availability was linked to trial findings) in either IPD-NMA, but because retrieval bias could impact NMA findings, subsequent decision-making and guideline development, this should be considered when assessing risk of bias in IPD syntheses.

Developing prediction models when there are systematically missing predictors in individual patient data meta-analysis

Clinical prediction models are widely used in modern clinical practice. Such models are often developed using individual patient data (IPD) from a single study, but often there are IPD available from multiple studies. This allows using meta-analytical methods for developing prediction models, increasing power and precision. Different studies, however, often measure different sets of predictors, which may result to systematically missing predictors, that is, when not all studies collect all predictors of interest. This situation poses challenges in model development. We hereby describe various approaches that can be used to develop prediction models for continuous outcomes in such situations. We compare four approaches: a "restrict predictors" approach, where the model is developed using only predictors measured in all studies; a multiple imputation approach that ignores study-level clustering; a multiple imputation approach that accounts for study-level clustering; and a new approach that develops a prediction model in each study separately using all predictors reported, and then synthesizes all predictions in a multi-study ensemble. We explore in simulations the performance of all approaches under various scenarios. We find that imputation methods and our new method outperform the restrict predictors approach. In several scenarios, our method outperformed imputation methods, especially for few studies, when predictor effects were small, and in case of large heterogeneity. We use a real dataset of 12 trials in psychotherapies for depression to illustrate all methods in practice, and we provide code in R.

Efficient integration of aggregate data and individual participant data in one-way mixed models

Often both aggregate data (AD) studies and individual participant data (IPD) studies are available for specific treatments. Combining these two sources of data could improve the overall meta-analytic estimates of treatment effects. Moreover, often for some studies with AD, the associated IPD maybe available, albeit at some extra effort or cost to the analyst. We propose a method for combining treatment effects across trials when the response is from the exponential family of distribution and hence a generalized linear model structure can be used. We consider the case when treatment effects are fixed and common across studies. Using the proposed combination method, we study the relative efficiency of analyzing all IPD studies vs combining various percentages of AD and IPD studies. For many different models, design constraints under which the AD estimators are the IPD estimators, and hence fully efficient, are known. For such models, we advocate a selection procedure that chooses AD studies over IPD studies in a manner that force least departure from design constraints and hence ensures an efficient combined AD and IPD estimator.

A Bayesian multivariate meta-analysis of prevalence data

When conducting a meta-analysis involving prevalence data for an outcome with several subtypes, each of them is typically analyzed separately using a univariate meta-analysis model. Recently, multivariate meta-analysis models have been shown to correspond to a decrease in bias and variance for multiple correlated outcomes compared with univariate meta-analysis, when some studies only report a subset of the outcomes. In this article, we propose a novel Bayesian multivariate random effects model to account for the natural constraint that the prevalence of any given subtype cannot be larger than that of the overall prevalence. Extensive simulation studies show that this new model can reduce bias and variance when estimating subtype prevalences in the presence of missing data, compared with standard univariate and multivariate random effects models. The data from a rapid review on occupation and lower urinary tract symptoms by the Prevention of Lower Urinary Tract Symptoms Research Consortium are analyzed as a case study to estimate the prevalence of urinary incontinence and several incontinence subtypes among women in suspected high risk work environments.

The Prospective Studies of Atherosclerosis (Proof-ATHERO) Consortium: Design and Rationale

Atherosclerosis - the pathophysiological mechanism shared by most cardiovascular diseases - can be directly or indirectly assessed by a variety of clinical tests including measurement of carotid intima-media thickness, carotid plaque, -ankle-brachial index, pulse wave velocity, and coronary -artery calcium. The Prospective Studies of Atherosclerosis -(Proof-ATHERO) consortium (https://clinicalepi.i-med.ac.at/research/proof-athero/) collates de-identified individual-participant data of studies with information on atherosclerosis measures, risk factors for cardiovascular disease, and incidence of cardiovascular diseases. It currently comprises 74 studies that involve 106,846 participants from 25 countries and over 40 cities. In summary, 21 studies recruited participants from the general population (n = 67,784), 16 from high-risk populations (n = 22,677), and 37 as part of clinical trials (n = 16,385). Baseline years of contributing studies range from April 1980 to July 2014; the latest follow-up was until June 2019. Mean age at baseline was 59 years (standard deviation: 10) and 50% were female. Over a total of 830,619 person-years of follow-up, 17,270 incident cardiovascular events (including coronary heart disease and stroke) and 13,270 deaths were recorded, corresponding to cumulative incidences of 2.1% and 1.6% per annum, respectively. The consortium is coordinated by the Clinical Epidemiology Team at the Medical University of Innsbruck, Austria. Contributing studies undergo a detailed data cleaning and harmonisation procedure before being incorporated in the Proof-ATHERO central database. Statistical analyses are being conducted according to pre-defined analysis plans and use established methods for individual-participant data meta-analysis. Capitalising on its large sample size, the multi-institutional collaborative Proof-ATHERO consortium aims to better characterise, understand, and predict the development of atherosclerosis and its clinical consequences.

Evidence synthesis in prognosis research

Over the past few years, evidence synthesis has become essential to investigate and improve the generalizability of medical research findings. This strategy often involves a meta-analysis to formally summarize quantities of interest, such as relative treatment effect estimates. The use of meta-analysis methods is, however, less straightforward in prognosis research because substantial variation exists in research objectives, analysis methods and the level of reported evidence. We present a gentle overview of statistical methods that can be used to summarize data of prognostic factor and prognostic model studies. We discuss how aggregate data, individual participant data, or a combination thereof can be combined through meta-analysis methods. Recent examples are provided throughout to illustrate the various methods.

Combining Multiple Observational Data Sources to Estimate Causal Effects

The era of big data has witnessed an increasing availability of multiple data sources for statistical analyses. We consider estimation of causal effects combining big main data with unmeasured confounders and smaller validation data with supplementary information on these confounders. Under the unconfoundedness assumption with completely observed confounders, the smaller validation data allow for constructing consistent estimators for causal effects, but the big main data can only give error-prone estimators in general. However, by leveraging the information in the big main data in a principled way, we can improve the estimation efficiencies yet preserve the consistencies of the initial estimators based solely on the validation data. Our framework applies to asymptotically normal estimators, including the commonly used regression imputation, weighting, and matching estimators, and does not require a correct specification of the model relating the unmeasured confounders to the observed variables. We also propose appropriate bootstrap procedures, which makes our method straightforward to implement using software routines for existing estimators. Supplementary materials for this article are available online.

The role of secondary outcomes in multivariate meta-analysis

Univariate meta‐analysis concerns a single outcome of interest measured across a number of independent studies. However, many research studies will have also measured secondary outcomes. Multivariate meta‐analysis allows us to take these secondary outcomes into account and can also include studies where the primary outcome is missing. We define the efficiency E as the variance of the overall estimate from a multivariate meta‐analysis relative to the variance of the overall estimate from a univariate meta‐analysis. The extra information gained from a multivariate meta‐analysis of n studies is then similar to the extra information gained if a univariate meta‐analysis of the primary effect had a further n(1−E)/E studies. The variance contribution of a study's secondary outcomes (its borrowing of strength) can be thought of as a contrast between the variance matrix of the outcomes in that study and the set of variance matrices of all the studies in the meta‐analysis. In the bivariate case this is given a simple graphical interpretation as the borrowing‐of‐strength plot. We discuss how these findings can also be used in the context of random‐effects meta‐analysis. Our discussion is motivated by a published meta‐analysis of 10 antihypertension clinical trials.

Borrowing of strength and study weights in multivariate and network meta-analysis

Multivariate and network meta-analysis have the potential for the estimated mean of one effect to borrow strength from the data on other effects of interest. The extent of this borrowing of strength is usually assessed informally. We present new mathematical definitions of 'borrowing of strength'. Our main proposal is based on a decomposition of the score statistic, which we show can be interpreted as comparing the precision of estimates from the multivariate and univariate models. Our definition of borrowing of strength therefore emulates the usual informal assessment. We also derive a method for calculating study weights, which we embed into the same framework as our borrowing of strength statistics, so that percentage study weights can accompany the results from multivariate and network meta-analyses as they do in conventional univariate meta-analyses. Our proposals are illustrated using three meta-analyses involving correlated effects for multiple outcomes, multiple risk factor associations and multiple treatments (network meta-analysis).

Uncovering exposures responsible for birth season - disease effects: a global study

Objective

Birth month and climate impact lifetime disease risk, while the underlying exposures remain largely elusive. We seek to uncover distal risk factors underlying these relationships by probing the relationship between global exposure variance and disease risk variance by birth season.

Material and Methods

This study utilizes electronic health record data from 6 sites representing 10.5 million individuals in 3 countries (United States, South Korea, and Taiwan). We obtained birth month–disease risk curves from each site in a case-control manner. Next, we correlated each birth month–disease risk curve with each exposure. A meta-analysis was then performed of correlations across sites. This allowed us to identify the most significant birth month–exposure relationships supported by all 6 sites while adjusting for multiplicity. We also successfully distinguish relative age effects (a cultural effect) from environmental exposures.

Results

Attention deficit hyperactivity disorder was the only identified relative age association. Our methods identified several culprit exposures that correspond well with the literature in the field. These include a link between first-trimester exposure to carbon monoxide and increased risk of depressive disorder (R = 0.725, confidence interval [95% CI], 0.529-0.847), first-trimester exposure to fine air particulates and increased risk of atrial fibrillation (R = 0.564, 95% CI, 0.363-0.715), and decreased exposure to sunlight during the third trimester and increased risk of type 2 diabetes mellitus (R = −0.816, 95% CI, −0.5767, −0.929).

Conclusion

A global study of birth month–disease relationships reveals distal risk factors involved in causal biological pathways that underlie them.

A comparison of existing methods for multiple imputation in individual participant data meta-analysis

Multiple imputation is a popular method for addressing missing data, but its implementation is difficult when data have a multilevel structure and one or more variables are systematically missing. This systematic missing data pattern may commonly occur in meta-analysis of individual participant data, where some variables are never observed in some studies, but are present in other hierarchical data settings. In these cases, valid imputation must account for both relationships between variables and correlation within studies. Proposed methods for multilevel imputation include specifying a full joint model and multiple imputation with chained equations (MICE). While MICE is attractive for its ease of implementation, there is little existing work describing conditions under which this is a valid alternative to specifying the full joint model. We present results showing that for multilevel normal models, MICE is rarely exactly equivalent to joint model imputation. Through a simulation study and an example using data from a traumatic brain injury study, we found that in spite of theoretical differences, MICE imputations often produce results similar to those obtained using the joint model. We also assess the influence of prior distributions in MICE imputation methods and find that when missingness is high, prior choices in MICE models tend to affect estimation of across-study variability more than compatibility of conditional likelihoods. Copyright © 2017 John Wiley & Sons, Ltd.

Multivariate meta-analysis with an increasing number of parameters

Meta-analysis can average estimates of multiple parameters, such as a treatment's effect on multiple outcomes, across studies. Univariate meta-analysis (UVMA) considers each parameter individually, while multivariate meta-analysis (MVMA) considers the parameters jointly and accounts for the correlation between their estimates. The performance of MVMA and UVMA has been extensively compared in scenarios with two parameters. Our objective is to compare the performance of MVMA and UVMA as the number of parameters, p, increases. Specifically, we show that (i) for fixed-effect (FE) meta-analysis, the benefit from using MVMA can substantially increase as p increases; (ii) for random effects (RE) meta-analysis, the benefit from MVMA can increase as p increases, but the potential improvement is modest in the presence of high between-study variability and the actual improvement is further reduced by the need to estimate an increasingly large between study covariance matrix; and (iii) when there is little to no between-study variability, the loss of efficiency due to choosing RE MVMA over FE MVMA increases as p increases. We demonstrate these three features through theory, simulation, and a meta-analysis of risk factors for non-Hodgkin lymphoma.

A graphical model approach to systematically missing data in meta-analysis of observational studies

When studies in meta-analysis include different sets of confounders, simple analyses can cause a bias (omitting confounders that are missing in certain studies) or precision loss (omitting studies with incomplete confounders, i.e. a complete-case meta-analysis). To overcome these types of issues, a previous study proposed modelling the high correlation between partially and fully adjusted regression coefficient estimates in a bivariate meta-analysis. When multiple differently adjusted regression coefficient estimates are available, we propose exploiting such correlations in a graphical model. Compared with a previously suggested bivariate meta-analysis method, such a graphical model approach is likely to reduce the number of parameters in complex missing data settings by omitting the direct relationships between some of the estimates. We propose a structure-learning rule whose justification relies on the missingness pattern being monotone. This rule was tested using epidemiological data from a multi-centre survey. In the analysis of risk factors for early retirement, the method showed a smaller difference from a complete data odds ratio and greater precision than a commonly used complete-case meta-analysis. Three real-world applications with monotone missing patterns are provided, namely, the association between (1) the fibrinogen level and coronary heart disease, (2) the intima media thickness and vascular risk and (3) allergic asthma and depressive episodes. The proposed method allows for the inclusion of published summary data, which makes it particularly suitable for applications involving both microdata and summary data. Copyright © 2016 John Wiley & Sons, Ltd.

Propensity Score-Based Approaches to Confounding by Indication in Individual Patient Data Meta-Analysis: Non-Standardized Treatment for Multidrug Resistant Tuberculosis

Background

In the absence of randomized clinical trials, meta-analysis of individual patient data (IPD) from observational studies may provide the most accurate effect estimates for an intervention. However, confounding by indication remains an important concern that can be addressed by incorporating individual patient covariates in different ways. We compared different analytic approaches to account for confounding in IPD from patients treated for multi-drug resistant tuberculosis (MDR-TB).

Methods

Two antibiotic classes were evaluated, fluoroquinolones—considered the cornerstone of effective MDR-TB treatment—and macrolides, which are known to be safe, yet are ineffective in vitro. The primary outcome was treatment success against treatment failure, relapse or death. Effect estimates were obtained using multivariable and propensity-score based approaches.

Results

Fluoroquinolone antibiotics were used in 28 included studies, within which 6,612 patients received a fluoroquinolone and 723 patients did not. Macrolides were used in 15 included studies, within which 459 patients received this class of antibiotics and 3,670 did not. Both standard multivariable regression and propensity score-based methods resulted in similar effect estimates for early and late generation fluoroquinolones, while macrolide antibiotics use was associated with reduced treatment success.

Conclusions

In this individual patient data meta-analysis, standard multivariable and propensity-score based methods of adjusting for individual patient covariates for observational studies yielded produced similar effect estimates. Even when adjustment is made for potential confounding, interpretation of adjusted estimates must still consider the potential for residual bias.

Synthesis of clinical prediction models under different sets of covariates with one individual patient data

Background

Recently, increased development of clinical prediction models has been reported in the medical literature. However, evidence synthesis methodologies for these prediction models have not been sufficiently studied, especially for practical situations such as a meta-analyses where only aggregated summaries of important predictors are available. Also, in general, the covariate sets involved in the prediction models are not common across studies. As in ordinary model misspecification problems, dropping relevant covariates would raise potentially serious biases to the prediction models, and consequently to the synthesized results.

Methods

We developed synthesizing methods for logistic clinical prediction models with possibly different sets of covariates. In order to aggregate the regression coefficient estimates from different prediction models, we adopted a generalized least squares approach with non-linear terms (a sort of generalization of multivariate meta-analysis). Firstly, we evaluated omitted variable biases in this approach. Then, under an assumption of homogeneity of studies, we developed bias-corrected estimating procedures for regression coefficients of the synthesized prediction models.

Results

Numerical evaluations with simulations showed that our approach resulted in smaller biases and more precise estimates compared with conventional methods, which use only studies with common covariates or which utilize a mean imputation method for omitted coefficients. These methods were also applied to a series of Japanese epidemiologic studies on the incidence of a stroke.

Conclusions

Our proposed methods adequately correct the biases due to different sets of covariates between studies, and would provide precise estimates compared with the conventional approach. If the assumption of homogeneity within studies is plausible, this methodology would be useful for incorporating prior published information into the construction of new prediction models.

Multivariate meta-analysis of prognostic factor studies with multiple cut-points and/or methods of measurement

A prognostic factor is any measure that is associated with the risk of future health outcomes in those with existing disease. Often, the prognostic ability of a factor is evaluated in multiple studies. However, meta-analysis is difficult because primary studies often use different methods of measurement and/or different cut-points to dichotomise continuous factors into 'high' and 'low' groups; selective reporting is also common. We illustrate how multivariate random effects meta-analysis models can accommodate multiple prognostic effect estimates from the same study, relating to multiple cut-points and/or methods of measurement. The models account for within-study and between-study correlations, which utilises more information and reduces the impact of unreported cut-points and/or measurement methods in some studies. The applicability of the approach is improved with individual participant data and by assuming a functional relationship between prognostic effect and cut-point to reduce the number of unknown parameters. The models provide important inferential results for each cut-point and method of measurement, including the summary prognostic effect, the between-study variance and a 95% prediction interval for the prognostic effect in new populations. Two applications are presented. The first reveals that, in a multivariate meta-analysis using published results, the Apgar score is prognostic of neonatal mortality but effect sizes are smaller at most cut-points than previously thought. In the second, a multivariate meta-analysis of two methods of measurement provides weak evidence that microvessel density is prognostic of mortality in lung cancer, even when individual participant data are available so that a continuous prognostic trend is examined (rather than cut-points).

Imputation of systematically missing predictors in an individual participant data meta-analysis: a generalized approach using MICE

Individual participant data meta-analyses (IPD-MA) are increasingly used for developing and validating multivariable (diagnostic or prognostic) risk prediction models. Unfortunately, some predictors or even outcomes may not have been measured in each study and are thus systematically missing in some individual studies of the IPD-MA. As a consequence, it is no longer possible to evaluate between-study heterogeneity and to estimate study-specific predictor effects, or to include all individual studies, which severely hampers the development and validation of prediction models. Here, we describe a novel approach for imputing systematically missing data and adopt a generalized linear mixed model to allow for between-study heterogeneity. This approach can be viewed as an extension of Resche-Rigon's method (Stat Med 2013), relaxing their assumptions regarding variance components and allowing imputation of linear and nonlinear predictors. We illustrate our approach using a case study with IPD-MA of 13 studies to develop and validate a diagnostic prediction model for the presence of deep venous thrombosis. We compare the results after applying four methods for dealing with systematically missing predictors in one or more individual studies: complete case analysis where studies with systematically missing predictors are removed, traditional multiple imputation ignoring heterogeneity across studies, stratified multiple imputation accounting for heterogeneity in predictor prevalence, and multilevel multiple imputation (MLMI) fully accounting for between-study heterogeneity. We conclude that MLMI may substantially improve the estimation of between-study heterogeneity parameters and allow for imputation of systematically missing predictors in IPD-MA aimed at the development and validation of prediction models.

Multivariate meta-analysis using individual participant data

When combining results across related studies, a multivariate meta-analysis allows the joint synthesis of correlated effect estimates from multiple outcomes. Joint synthesis can improve efficiency over separate univariate syntheses, may reduce selective outcome reporting biases, and enables joint inferences across the outcomes. A common issue is that within-study correlations needed to fit the multivariate model are unknown from published reports. However, provision of individual participant data (IPD) allows them to be calculated directly. Here, we illustrate how to use IPD to estimate within-study correlations, using a joint linear regression for multiple continuous outcomes and bootstrapping methods for binary, survival and mixed outcomes. In a meta-analysis of 10 hypertension trials, we then show how these methods enable multivariate meta-analysis to address novel clinical questions about continuous, survival and binary outcomes; treatment–covariate interactions; adjusted risk/prognostic factor effects; longitudinal data; prognostic and multiparameter models; and multiple treatment comparisons. Both frequentist and Bayesian approaches are applied, with example software code provided to derive within-study correlations and to fit the models.

Multiple imputation for handling systematically missing confounders in meta-analysis of individual participant data

A variable is 'systematically missing' if it is missing for all individuals within particular studies in an individual participant data meta-analysis. When a systematically missing variable is a potential confounder in observational epidemiology, standard methods either fail to adjust the exposure-disease association for the potential confounder or exclude studies where it is missing. We propose a new approach to adjust for systematically missing confounders based on multiple imputation by chained equations. Systematically missing data are imputed via multilevel regression models that allow for heterogeneity between studies. A simulation study compares various choices of imputation model. An illustration is given using data from eight studies estimating the association between carotid intima media thickness and subsequent risk of cardiovascular events. Results are compared with standard methods and also with an extension of a published method that exploits the relationship between fully adjusted and partially adjusted estimated effects through a multivariate random effects meta-analysis model. We conclude that multiple imputation provides a practicable approach that can handle arbitrary patterns of systematic missingness. Bias is reduced by including sufficient between-study random effects in the imputation model.

Combining multiple imputation and meta-analysis with individual participant data

Multiple imputation is a strategy for the analysis of incomplete data such that the impact of the missingness on the power and bias of estimates is mitigated. When data from multiple studies are collated, we can propose both within-study and multilevel imputation models to impute missing data on covariates. It is not clear how to choose between imputation models or how to combine imputation and inverse-variance weighted meta-analysis methods. This is especially important as often different studies measure data on different variables, meaning that we may need to impute data on a variable which is systematically missing in a particular study. In this paper, we consider a simulation analysis of sporadically missing data in a single covariate with a linear analysis model and discuss how the results would be applicable to the case of systematically missing data. We find in this context that ensuring the congeniality of the imputation and analysis models is important to give correct standard errors and confidence intervals. For example, if the analysis model allows between-study heterogeneity of a parameter, then we should incorporate this heterogeneity into the imputation model to maintain the congeniality of the two models. In an inverse-variance weighted meta-analysis, we should impute missing data and apply Rubin's rules at the study level prior to meta-analysis, rather than meta-analyzing each of the multiple imputations and then combining the meta-analysis estimates using Rubin's rules. We illustrate the results using data from the Emerging Risk Factors Collaboration.

Individual participant data meta-analyses should not ignore clustering

Objectives

Individual participant data (IPD) meta-analyses often analyze their IPD as if coming from a single study. We compare this approach with analyses that rather account for clustering of patients within studies.

Study Design and Setting

Comparison of effect estimates from logistic regression models in real and simulated examples.

Results

The estimated prognostic effect of age in patients with traumatic brain injury is similar, regardless of whether clustering is accounted for. However, a family history of thrombophilia is found to be a diagnostic marker of deep vein thrombosis [odds ratio, 1.30; 95% confidence interval (CI): 1.00, 1.70; P = 0.05] when clustering is accounted for but not when it is ignored (odds ratio, 1.06; 95% CI: 0.83, 1.37; P = 0.64). Similarly, the treatment effect of nicotine gum on smoking cessation is severely attenuated when clustering is ignored (odds ratio, 1.40; 95% CI: 1.02, 1.92) rather than accounted for (odds ratio, 1.80; 95% CI: 1.29, 2.52). Simulations show models accounting for clustering perform consistently well, but downwardly biased effect estimates and low coverage can occur when ignoring clustering.

Conclusion

Researchers must routinely account for clustering in IPD meta-analyses; otherwise, misleading effect estimates and conclusions may arise.

Wide clinic-level variation in adherence to oral diabetes medications in the VA

BACKGROUND

While there has been extensive research into patient-specific predictors of medication adherence and patient-specific interventions to improve adherence, there has been little examination of variation in clinic-level medication adherence.

OBJECTIVE

We examined the clinic-level variation of oral hypoglycemic agent (OHA) medication adherence among patients with diabetes treated in the Department of Veterans Affairs (VA) primary care clinics. We hypothesized that there would be systematic variation in clinic-level adherence measures, and that adherence within organizationally-affiliated clinics, such as those sharing local management and support, would be more highly correlated than adherence between unaffiliated clinics.

DESIGN

Retrospective cohort study.

SETTING

VA hospital and VA community-based primary care clinics in the contiguous 48 states.

PATIENTS

444,418 patients with diabetes treated with OHAs and seen in 158 hospital-based clinics and 401 affiliated community primary care clinics during fiscal years 2006 and 2007.

MAIN MEASURES

Refill-based medication adherence to OHA.

KEY RESULTS

Adjusting for patient characteristics, the proportion of patients adherent to OHAs ranged from 57 % to 81 % across clinics. Adherence between organizationally affiliated clinics was high (Pearson Correlation = 0.82), and adherence between unaffiliated clinics was low (Pearson Correlation = 0.04).

CONCLUSION

The proportion of patients adherent to OHAs varied widely across VA primary care clinics. Clinic-level adherence was highly correlated to other clinics in the same organizational unit. Further research should identify which factors common to affiliated clinics influence medication adherence.

Individual participant data meta-analysis for a binary outcome: one-stage or two-stage?

BACKGROUND

A fundamental aspect of epidemiological studies concerns the estimation of factor-outcome associations to identify risk factors, prognostic factors and potential causal factors. Because reliable estimates for these associations are important, there is a growing interest in methods for combining the results from multiple studies in individual participant data meta-analyses (IPD-MA). When there is substantial heterogeneity across studies, various random-effects meta-analysis models are possible that employ a one-stage or two-stage method. These are generally thought to produce similar results, but empirical comparisons are few.

OBJECTIVE

We describe and compare several one- and two-stage random-effects IPD-MA methods for estimating factor-outcome associations from multiple risk-factor or predictor finding studies with a binary outcome. One-stage methods use the IPD of each study and meta-analyse using the exact binomial distribution, whereas two-stage methods reduce evidence to the aggregated level (e.g. odds ratios) and then meta-analyse assuming approximate normality. We compare the methods in an empirical dataset for unadjusted and adjusted risk-factor estimates.

RESULTS

Though often similar, on occasion the one-stage and two-stage methods provide different parameter estimates and different conclusions. For example, the effect of erythema and its statistical significance was different for a one-stage (OR = 1.35, [Formula: see text]) and univariate two-stage (OR = 1.55, [Formula: see text]). Estimation issues can also arise: two-stage models suffer unstable estimates when zero cell counts occur and one-stage models do not always converge.

CONCLUSION

When planning an IPD-MA, the choice and implementation (e.g. univariate or multivariate) of a one-stage or two-stage method should be prespecified in the protocol as occasionally they lead to different conclusions about which factors are associated with outcome. Though both approaches can suffer from estimation challenges, we recommend employing the one-stage method, as it uses a more exact statistical approach and accounts for parameter correlation.

Meta-analytic approaches to determine gender differences in the age-incidence characteristics of schizophrenia and related psychoses

A recent systematic review and meta-analysis of the incidence and prevalence of schizophrenia and other psychoses in England investigated the variation in the rates of psychotic disorders. However, some of the questions of interest, and the data collected to answer these, could not be adequately addressed using established meta-analysis techniques. We developed a novel statistical method, which makes combined use of fractional polynomials and meta-regression. This was used to quantify the evidence of gender differences and a secondary peak onset in women, where the outcome of interest is the incidence of schizophrenia. Statistically significant and epidemiologically important effects were obtained using our methods. Our analysis is based on data from four studies that provide 50 incidence rates, stratified by age and gender. We describe several variations of our method, in particular those that might be used where more data is available, and provide guidance for assessing the model fit.

Two-stage meta-analysis of survival data from individual participants using percentile ratios

Methods for individual participant data meta-analysis of survival outcomes commonly focus on the hazard ratio as a measure of treatment effect. Recently, Siannis et al. (2010, Statistics in Medicine 29:3030-3045) proposed the use of percentile ratios as an alternative to hazard ratios. We describe a novel two-stage method for the meta-analysis of percentile ratios that avoids distributional assumptions at the study level.

Melanocortin-1 receptor, skin cancer and phenotypic characteristics (M-SKIP) project: study design and methods for pooling results of genetic epidemiological studies

Background

For complex diseases like cancer, pooled-analysis of individual data represents a powerful tool to investigate the joint contribution of genetic, phenotypic and environmental factors to the development of a disease. Pooled-analysis of epidemiological studies has many advantages over meta-analysis, and preliminary results may be obtained faster and with lower costs than with prospective consortia.

Design and methods

Based on our experience with the study design of the Melanocortin-1 receptor (MC1R) gene, SKin cancer and Phenotypic characteristics (M-SKIP) project, we describe the most important steps in planning and conducting a pooled-analysis of genetic epidemiological studies. We then present the statistical analysis plan that we are going to apply, giving particular attention to methods of analysis recently proposed to account for between-study heterogeneity and to explore the joint contribution of genetic, phenotypic and environmental factors in the development of a disease. Within the M-SKIP project, data on 10,959 skin cancer cases and 14,785 controls from 31 international investigators were checked for quality and recoded for standardization. We first proposed to fit the aggregated data with random-effects logistic regression models. However, for the M-SKIP project, a two-stage analysis will be preferred to overcome the problem regarding the availability of different study covariates. The joint contribution of MC1R variants and phenotypic characteristics to skin cancer development will be studied via logic regression modeling.

Discussion

Methodological guidelines to correctly design and conduct pooled-analyses are needed to facilitate application of such methods, thus providing a better summary of the actual findings on specific fields.

Meta-analysis methodology for combining treatment effects from Cox proportional hazard models with different covariate adjustments

In Cox proportional hazard models with censored survival data, estimates of treatment effects with some important covariates omitted will be biased toward zero (Gail et al., Biometrika 71: 431-444). This can be especially problematic in meta-analyses that combine estimates of parameters from studies where different covariate adjustments are made. Presently, few constructive solutions have been provided to address this issue. In this paper, we review the existing meta-analysis methodologies for aggregated patient data (APD) and propose two meta-regression models (meta-ANOVA model and meta-polynomial model) with indicators of different covariates in Cox proportional hazard models to adjust the heterogeneity of treatment effects due to omitted covariates across studies. Both parametric and nonparametric estimators for the pooled treatment effect and the heterogeneity variance are presented and compared. We illustrate the advantages of our proposed analytic procedures over the existing methodologies by simulation studies and real data analysis. The existing methodologies yield large estimation bias in the presence of an "incomparability" issue, whereas our proposed models can adjust the bias and thus provide an accurate estimation.

Multivariate meta-analysis: potential and promise

The multivariate random effects model is a generalization of the standard univariate model. Multivariate meta-analysis is becoming more commonly used and the techniques and related computer software, although continually under development, are now in place. In order to raise awareness of the multivariate methods, and discuss their advantages and disadvantages, we organized a one day ‘Multivariate meta-analysis’ event at the Royal Statistical Society. In addition to disseminating the most recent developments, we also received an abundance of comments, concerns, insights, critiques and encouragement. This article provides a balanced account of the day's discourse. By giving others the opportunity to respond to our assessment, we hope to ensure that the various view points and opinions are aired before multivariate meta-analysis simply becomes another widely used de facto method without any proper consideration of it by the medical statistics community. We describe the areas of application that multivariate meta-analysis has found, the methods available, the difficulties typically encountered and the arguments for and against the multivariate methods, using four representative but contrasting examples. We conclude that the multivariate methods can be useful, and in particular can provide estimates with better statistical properties, but also that these benefits come at the price of making more assumptions which do not result in better inference in every case. Although there is evidence that multivariate meta-analysis has considerable potential, it must be even more carefully applied than its univariate counterpart in practice. Copyright © 2011 John Wiley & Sons, Ltd.

Statistical methods for the time-to-event analysis of individual participant data from multiple epidemiological studies

Background Meta-analysis of individual participant time-to-event data from multiple prospective epidemiological studies enables detailed investigation of exposure–risk relationships, but involves a number of analytical challenges.

Methods This article describes statistical approaches adopted in the Emerging Risk Factors Collaboration, in which primary data from more than 1 million participants in more than 100 prospective studies have been collated to enable detailed analyses of various risk markers in relation to incident cardiovascular disease outcomes.

Results Analyses have been principally based on Cox proportional hazards regression models stratified by sex, undertaken in each study separately. Estimates of exposure–risk relationships, initially unadjusted and then adjusted for several confounders, have been combined over studies using meta-analysis. Methods for assessing the shape of exposure–risk associations and the proportional hazards assumption have been developed. Estimates of interactions have also been combined using meta-analysis, keeping separate within- and between-study information. Regression dilution bias caused by measurement error and within-person variation in exposures and confounders has been addressed through the analysis of repeat measurements to estimate corrected regression coefficients. These methods are exemplified by analysis of plasma fibrinogen and risk of coronary heart disease, and Stata code is made available.

Conclusion Increasing numbers of meta-analyses of individual participant data from observational data are being conducted to enhance the statistical power and detail of epidemiological studies. The statistical methods developed here can be used to address the needs of such analyses.