Effect Estimation in Point-Exposure Studies with Binary Outcomes and High-Dimensional Covariate Data - A Comparison of Targeted Maximum Likelihood Estimation and Inverse Probability of Treatment Weighting

Assessment of Glucose Lowering Medications' Effectiveness for Cardiovascular Clinical Risk Management of Real-World Patients with Type 2 Diabetes: Targeted Maximum Likelihood Estimation under Model Misspecification and Missing Outcomes

The results from many cardiovascular (CV) outcome trials suggest that glucose lowering medications (GLMs) are effective for the CV clinical risk management of type 2 diabetes (T2D) patients. The aim of this study is to compare the effectiveness of two GLMs (SGLT2i and GLP-1RA) for the CV clinical risk management of T2D patients in a real-world setting, by simultaneously reducing glycated hemoglobin, body weight, and systolic blood pressure. Data from the real-world Italian multicenter retrospective study Dapagliflozin Real World evideNce in Type 2 Diabetes (DARWINT 2D) are analyzed. Different statistical approaches are compared to deal with the real-world-associated issues, which can arise from model misspecification, nonrandomized treatment assignment, and a high percentage of missingness in the outcome, and can potentially bias the marginal treatment effect (MTE) estimate and thus have an influence on the clinical risk management of patients. We compare the logistic regression (LR), propensity score (PS)-based methods, and the targeted maximum likelihood estimator (TMLE), which allows for the use of machine learning (ML) models. Furthermore, a simulation study is performed, resembling the structure of the conditional dependencies among the main variables in DARWIN-T2D. LR and PS methods do not underline any difference in the effectiveness regarding the attainment of combined CV risk factor goals between the two treatments. TMLE suggests instead that dapagliflozin is significantly more effective than GLP-1RA for the CV risk management of T2D patients. The results from the simulation study suggest that TMLE has the lowest bias and SE for the estimate of the MTE.

Review and comparison of treatment effect estimators using propensity and prognostic scores

In finding effects of a binary treatment, practitioners use mostly either propensity score matching (PSM) or inverse probability weighting (IPW). However, many new treatment effect estimators are available now using propensity score and "prognostic score", and some of these estimators are much better than PSM and IPW in several aspects. In this paper, we review those recent treatment effect estimators to show how they are related to one another, and why they are better than PSM and IPW. We compare 26 estimators in total through extensive simulation and empirical studies. Based on these, we recommend recent treatment effect estimators using "overlap weight", and "targeted MLE" using statistical/machine learning, as well as a simple regression imputation/adjustment estimator using linear prognostic score models.

Evaluating the robustness of targeted maximum likelihood estimators via realistic simulations in nutrition intervention trials

Several recently developed methods have the potential to harness machine learning in the pursuit of target quantities inspired by causal inference, including inverse weighting, doubly robust estimating equations and substitution estimators like targeted maximum likelihood estimation. There are even more recent augmentations of these procedures that can increase robustness, by adding a layer of cross-validation (cross-validated targeted maximum likelihood estimation and double machine learning, as applied to substitution and estimating equation approaches, respectively). While these methods have been evaluated individually on simulated and experimental data sets, a comprehensive analysis of their performance across real data based simulations have yet to be conducted. In this work, we benchmark multiple widely used methods for estimation of the average treatment effect using ten different nutrition intervention studies data. A nonparametric regression method, undersmoothed highly adaptive lasso, is used to generate the simulated distribution which preserves important features from the observed data and reproduces a set of true target parameters. For each simulated data, we apply the methods above to estimate the average treatment effects as well as their standard errors and resulting confidence intervals. Based on the analytic results, a general recommendation is put forth for use of the cross-validated variants of both substitution and estimating equation estimators. We conclude that the additional layer of cross-validation helps in avoiding unintentional over-fitting of nuisance parameter functionals and leads to more robust inferences.

The Direct Effect of Body Mass Index on Cardiovascular Outcomes among Participants Without Central Obesity by Targeted Maximum Likelihood Estimation

BACKGROUND

Body mass index (BMI) is the most commonly used index to categorize a person as obese or non-obese, which is subject to important limitations.

OBJECTIVE

To evaluate the direct effect of BMI on cardiovascular outcomes among participants without central obesity.

METHODS

This analysis included 14,983 males and females aged 45-75 years from the Atherosclerosis Risk in Communities Study (ARIC). BMI was measured as general obesity, and waist circumference (WC), waist-to-hip ratio (WHR) and hip circumference as central obesity. Targeted maximum likelihood estimation (TMLE) was used to estimate the total effects (TEs) and the controlled direct effects (CDEs). The proportion of TE that would be eliminated if all participants were non-obese regarding central obesity was computed using the proportion eliminated (PE) index. P <0.05 was considered statistically significant. Analyses were performed in the TMLE R package.

RESULTS

The risk of cardiovascular outcomes attributed to BMI was significantly reversed by eliminating WHR obesity (p<0.001). The proportion eliminated of BMI effects was more tangible for non-obese participants regarding WC (PE=127%; 95%CI (126,128)) and WHR (PE=97%; 95%CI (96,98)) for coronary heart disease (CHD), and WHR (PE=92%; 95%CI (91,94)) for stroke, respectively. With respect to sex, the proportion eliminated of BMI effects was more tangible for non-obese participants regarding WHR (PE=428%; 95%CI (408,439)) for CHD in males, and WC (PE=99%; 95%CI (89,111)) for stroke in females, respectively.

CONCLUSION

These results indicate different potential effects of eliminating central obesity on the association between BMI and cardiovascular outcomes for males and females. (Arq Bras Cardiol. 2021; 116(5):879-886).

Effect modification of general and central obesity by sex and age on cardiovascular outcomes: Targeted maximum likelihood estimation in the atherosclerosis risk in communities study

BACKGROUND AND AIMS

To elucidate the effect modification of general and central obesity by sex and age on the risk of cardiovascular events.

METHODS

The analysis included 14,983 males and females aged 45-75 years from the Atherosclerosis Risk in Communities study. Obesity was defined with body mass index (BMI), waist-to-hip ratio (WHR) and body shape index (BSI) which categorized the participants as obese and non-obese. Targeted maximum likelihood estimation was used to estimate the risk ratio (RR) with the tmle package in R software.

RESULTS

After adjustment, the strongest effect of BMI on CHD was in females (RR (95%CI): 1.26 (1.11, 1.42)) and in age>54 (RR (95%CI): 1.16 (1.06, 1.27)) and for HF it was in age>54 (RR (95%CI): 1.18 (1.10, 1.26)) and in females (RR (95%CI): 1.17 (1.08, 1.28)). Regarding central obesity, WHR (RR (95%CI): 1.19 (1.05, 1.34)) had the strongest effects on CHD for males and BSI (RR (95%CI): 1.140 (1.02, 1.26)) for age ≤ 54, and for HF the WHR (RR (95%CI): 1.22 (1.10, 1.36)) and BSI (RR (95%CI): 1.18 (1.07, 1.30)) had the strongest effects for age≤54, respectively.

CONCLUSION

Among males and age≤54, WHR index was associated with a higher risk of CHD and HF while BMI was so for females and age>54.

Visualization tool of variable selection in bias-variance tradeoff for inverse probability weights

PURPOSE

Inversed probability weighted (IPW) estimators are commonly used to adjust for time-fixed or time-varying confounders. However, in high-dimensional settings, including all identified confounders may result in unstable weights leading to higher variance. We aimed to develop a visualization tool demonstrating the impact of each confounder on the bias and variance of IPW estimates, as well as the propensity score overlap.

METHODS

A SAS macro was developed for this visualization tool and we demonstrate how this tool can be used to identify potentially problematic confounders of the association of statin use after myocardial infarction on one-year mortality in a plasmode simulation study using a cohort of 39,792 patients from the UK (1998-2012).

RESULTS

Through the tool's output, we can identify problematic confounders (two instrumental variables) and important confounders by comparing the estimated psuedo MSE with that from the fully adjusted model and propensity score overlap plot.

CONCLUSION

Our results suggest that the analytic impact of all confounders should be considered carefully when fitting IPW estimators.

Building a framework for the evaluation of knowledge translation for the Canadian Network for Observational Drug Effect Studies

Purpose

The Canadian Network for Observational Drug Effect Studies (CNODES), a network of pharmacoepidemiologists and other researchers from seven provincial sites, provides evidence on the benefits and risks of drugs used by Canadians. The Knowledge Translation Team, one of CNODES' four main teams, evaluates the impact of its efforts using an iterative and emergent approach. This article shares key lessons from early evaluation phases, including identifying stakeholders and their evaluation needs, choosing evaluation theories and approaches, and developing evaluation questions, designs, and methods appropriate for the CNODES context.

Methods

Stakeholder analysis was conducted using documentary analysis to determine key contextual factors and research evidence needs of decision maker partners and other stakeholders. Selected theories and frameworks from the evaluation and knowledge translation literature informed decisions about evaluation design and implementation. A developmental approach to evaluation was deemed appropriate due to the innovative, complex, and ever‐changing context.

Results

A theory of change, logic model, and potential evaluation questions were developed, informed by the stakeholder analysis. Early indicators of program impact (citation metrics, alternative metrics) have been documented; efforts to collect data on additional indicators are ongoing.

Conclusion

A flexible, iterative, and emergent evaluation approach allows the Knowledge Translation Team to apply lessons learned from completed projects to ongoing research projects, adapt its approaches based on stakeholder needs, document successes, and be accountable to funders/stakeholders. This evaluation approach may be useful for other international pharmacoepidemiology research networks planning and implementing evaluations of similarly complex, multistakeholder initiatives that are subject to constant change.

Modernizing the Bradford Hill criteria for assessing causal relationships in observational data

Perhaps no other topic in risk analysis is more difficult, more controversial, or more important to risk management policy analysts and decision-makers than how to draw valid, correctly qualified causal conclusions from observational data. Statistical methods can readily quantify associations between observed variables using measures such as relative risk (RR) ratios, odds ratios (OR), slope coefficients for exposure or treatment variables in regression models, and quantities derived from these measures. Textbooks of epidemiology explain how to calculate population attributable fractions, attributable risks, burden-of-disease estimates, and probabilities of causation from relative risk (RR) ratios. Despite their suggestive names, these association-based measures have no necessary connection to causation if the associations on which they are based arise from bias, confounding, p-hacking, coincident historical trends, or other noncausal sources. But policy analysts and decision makers need something more: trustworthy predictions - and, later, evaluations - of the changes in outcomes caused by changes in policy variables. This concept of manipulative causation differs from the more familiar concepts of associational and attributive causation most widely used in epidemiology. Drawing on modern literature on causal discovery and inference principles and algorithms for drawing limited but useful causal conclusions from observational data, we propose seven criteria for assessing consistency of data with a manipulative causal exposure-response relationship - mutual information, directed dependence, internal and external consistency, coherent causal explanation of biological plausibility, causal mediation confirmation, and refutation of non-causal explanations - and discuss to what extent it is now possible to automate discovery of manipulative causal dependencies and quantification of causal effects from observational data. We compare our proposed principles for causal discovery and inference to the traditional Bradford Hill considerations from 1965. Understanding how old and new principles are related can clarify and enrich both.

Estimating long-term treatment effects in observational data: A comparison of the performance of different methods under real-world uncertainty

In the presence of time-dependent confounding, there are several methods available to estimate treatment effects. With correctly specified models and appropriate structural assumptions, any of these methods could provide consistent effect estimates, but with real-world data, all models will be misspecified and it is difficult to know if assumptions are violated. In this paper, we investigate five methods: inverse probability weighting of marginal structural models, history-adjusted marginal structural models, sequential conditional mean models, g-computation formula, and g-estimation of structural nested models. This work is motivated by an investigation of the effects of treatments in cystic fibrosis using the UK Cystic Fibrosis Registry data focussing on two outcomes: lung function (continuous outcome) and annual number of days receiving intravenous antibiotics (count outcome). We identified five features of this data that may affect the performance of the methods: misspecification of the causal null, long-term treatment effects, effect modification by time-varying covariates, misspecification of the direction of causal pathways, and censoring. In simulation studies, under ideal settings, all five methods provide consistent estimates of the treatment effect with little difference between methods. However, all methods performed poorly under some settings, highlighting the importance of using appropriate methods based on the data available. Furthermore, with the count outcome, the issue of non-collapsibility makes comparison between methods delivering marginal and conditional effects difficult. In many situations, we would recommend using more than one of the available methods for analysis, as if the effect estimates are very different, this would indicate potential issues with the analyses.

Collaborative targeted learning using regression shrinkage

Causal inference practitioners are routinely presented with the challenge of model selection and, in particular, reducing the size of the covariate set with the goal of improving estimation efficiency. Collaborative targeted minimum loss-based estimation (CTMLE) is a general framework for constructing doubly robust semiparametric causal estimators that data-adaptively limit model complexity in the propensity score to optimize a preferred loss function. This stepwise complexity reduction is based on a loss function placed on a strategically updated model for the outcome variable through which the error is assessed using cross-validation. We demonstrate how the existing stepwise variable selection CTMLE can be generalized using regression shrinkage of the propensity score. We present 2 new algorithms that involve stepwise selection of the penalization parameter(s) in the regression shrinkage. Simulation studies demonstrate that, under a misspecified outcome model, mean squared error and bias can be reduced by a CTMLE procedure that separately penalizes individual covariates in the propensity score. We demonstrate these approaches in an example using electronic medical data with sparse indicator covariates to evaluate the relative safety of 2 similarly indicated asthma therapies for pregnant women with moderate asthma.