Time-dependent propensity score and collider-stratification bias: an example of beta2-agonist use and the risk of coronary heart disease

Propensity Score Methods in Health Technology Assessment: Principles, Extended Applications, and Recent Advances

Randomized clinical trials (RCT) are accepted as the gold-standard approaches to measure effects of intervention or treatment on outcomes. They are also the designs of choice for health technology assessment (HTA). Randomization ensures comparability, in both measured and unmeasured pretreatment characteristics, of individuals assigned to treatment and control or comparator. However, even adequately powered RCTs are not always feasible for several reasons such as cost, time, practical and ethical constraints, and limited generalizability. RCTs rely on data collected on selected, homogeneous population under highly controlled conditions; hence, they provide evidence on efficacy of interventions rather than on effectiveness. Alternatively, observational studies can provide evidence on the relative effectiveness or safety of a health technology compared to one or more alternatives when provided under the setting of routine health care practice. In observational studies, however, treatment assignment is a non-random process based on an individual’s baseline characteristics; hence, treatment groups may not be comparable in their pretreatment characteristics. As a result, direct comparison of outcomes between treatment groups might lead to biased estimate of the treatment effect. Propensity score approaches have been used to achieve balance or comparability of treatment groups in terms of their measured pretreatment covariates thereby controlling for confounding bias in estimating treatment effects. Despite the popularity of propensity scores methods and recent important methodological advances, misunderstandings on their applications and limitations are all too common. In this article, we present a review of the propensity scores methods, extended applications, recent advances, and their strengths and limitations.

Insomnia symptoms as a cause of type 2 diabetes Incidence: a 20 year cohort study

BACKGROUND

Insomnia symptoms are associated with type 2 diabetes incidence but are also associated with a range of potential time-varying covariates which may confound and/or mediate associations. We aimed to assess whether cumulative exposure to insomnia symptoms has a causal effect on type 2 diabetes incidence.

METHODS

A prospective cohort study in the West of Scotland, following respondents for 20 years from age 36. 996 respondents were free of diabetes at baseline and had valid data from up to four follow-up visits. Type 2 diabetes was assessed at the final visit by self-report, taking diabetic medication, or blood-test (HbA_1c ≥ 6.5% or 48 mmol/mol). Effects of cumulative insomnia exposure on type 2 diabetes incidence were estimated with traditional regression and marginal structural models, adjusting for time-dependent confounding (smoking, diet, physical inactivity, obesity, heavy drinking, psychiatric distress) as well as for gender and baseline occupational class.

RESULTS

Traditional regression yielded an odds ratio (OR) of 1.34 (95% CI: 1.06-1.70) for type 2 diabetes incidence for each additional survey wave in which insomnia was reported. Marginal structural models adjusted for prior covariates (assuming concurrently measured covariates were potential mediators), reduced this OR to 1.20 (95% CI: 0.98-1.46), and when concurrent covariates were also included (viewing them as potential confounders) this dropped further to 1.08 (95% CI: 0.85-1.37).

CONCLUSIONS

The association between cumulative experience of insomnia and type 2 diabetes incidence appeared confounded. Evidence for a residual causal effect depended on assumptions as to whether concurrently measured covariates were confounders or mediators.

The IMI PROTECT project: purpose, organizational structure, and procedures

The Pharmacoepidemiological Research on Outcomes of Therapeutics by a European ConsorTium (PROTECT) initiative was a collaborative European project that sought to address limitations of current methods in the field of pharmacoepidemiology and pharmacovigilance. Initiated in 2009 and ending in 2015, PROTECT was part of the Innovative Medicines Initiative, a joint undertaking by the European Union and pharmaceutical industry. Thirty-five partners including academics, regulators, small and medium enterprises, and European Federation of Pharmaceuticals Industries and Associations companies contributed to PROTECT. Two work packages within PROTECT implemented research examining the extent to which differences in the study design, methodology, and choice of data source can contribute to producing discrepant results from observational studies on drug safety. To evaluate the effect of these differences, the project applied different designs and analytic methodology for six drug-adverse event pairs across several electronic healthcare databases and registries. This papers introduces the organizational structure and procedures of PROTECT, including how drug-adverse event and data sources were selected, study design and analyses documents were developed, and results managed centrally.

Multi-centre, multi-database studies with common protocols: lessons learnt from the IMI PROTECT project

PURPOSE

To assess the impact of a variety of methodological parameters on the association between six drug classes and five key adverse events in multiple databases.

METHODS

The selection of Drug-Adverse Event pairs was based on public health impact, regulatory relevance, and the possibility to study a broad range of methodological issues. Common protocols and data analytical specifications were jointly developed and independently and blindly executed in different databases in Europe with replications in the same and different databases.

RESULTS

The association between antibiotics and acute liver injury, benzodiazepines and hip fracture, antidepressants and hip fracture, inhaled long-acting beta2-agonists and acute myocardial infarction was consistent in direction across multiple designs, databases and methods to control for confounding. Some variation in magnitude of the associations was observed depending on design, exposure and outcome definitions, but none of the differences were statistically significant. The association between anti-epileptics and suicidality was inconsistent across the UK CPRD, Danish National registries and the French PGRx system. Calcium channel blockers were not associated with the risk of cancer in the UK CPRD, and this was consistent across different classes of calcium channel blockers, cumulative durations of use up to >10 years and different types of cancer.

CONCLUSIONS

A network for observational drug effect studies allowing the execution of common protocols in multiple databases was created. Increased consistency of findings across multiple designs and databases in different countries will increase confidence in findings from observational drug research and benefit/risk assessment of medicines.

Methodological comparison of marginal structural model, time-varying Cox regression, and propensity score methods: the example of antidepressant use and the risk of hip fracture

BACKGROUND

Observational studies including time-varying treatments are prone to confounding. We compared time-varying Cox regression analysis, propensity score (PS) methods, and marginal structural models (MSMs) in a study of antidepressant [selective serotonin reuptake inhibitors (SSRIs)] use and the risk of hip fracture.

METHODS

A cohort of patients with a first prescription for antidepressants (SSRI or tricyclic antidepressants) was extracted from the Dutch Mondriaan and Spanish Base de datos para la Investigación Farmacoepidemiológica en Atención Primaria (BIFAP) general practice databases for the period 2001-2009. The net (total) effect of SSRI versus no SSRI on the risk of hip fracture was estimated using time-varying Cox regression, stratification and covariate adjustment using the PS, and MSM. In MSM, censoring was accounted for by inverse probability of censoring weights.

RESULTS

The crude hazard ratio (HR) of SSRI use versus no SSRI use on hip fracture was 1.75 (95%CI: 1.12, 2.72) in Mondriaan and 2.09 (1.89, 2.32) in BIFAP. After confounding adjustment using time-varying Cox regression, stratification, and covariate adjustment using the PS, HRs increased in Mondriaan [2.59 (1.63, 4.12), 2.64 (1.63, 4.25), and 2.82 (1.63, 4.25), respectively] and decreased in BIFAP [1.56 (1.40, 1.73), 1.54 (1.39, 1.71), and 1.61 (1.45, 1.78), respectively]. MSMs with stabilized weights yielded HR 2.15 (1.30, 3.55) in Mondriaan and 1.63 (1.28, 2.07) in BIFAP when accounting for censoring and 2.13 (1.32, 3.45) in Mondriaan and 1.66 (1.30, 2.12) in BIFAP without accounting for censoring.

CONCLUSIONS

In this empirical study, differences between the different methods to control for time-dependent confounding were small. The observed differences in treatment effect estimates between the databases are likely attributable to different confounding information in the datasets, illustrating that adequate information on (time-varying) confounding is crucial to prevent bias.

Prevalent new-user cohort designs for comparative drug effect studies by time-conditional propensity scores

PURPOSE

Studies of the real-world comparative effectiveness of drugs conducted using computerized healthcare databases typically involve an incident new-user cohort design for head-to-head comparisons between two medications, using exclusively treatment-naïve patients. However, the desired contrast often involves one new drug compared with an older drug, of which many users of the new drug may have switched from, seriously restricting the scope of incident new-user studies.

METHODS

We introduce prevalent new-user cohort designs for head-to-head comparative drug effect studies, where incident new users are scarce. We define time-based and prescription-based exposure sets to compute time-conditional propensity scores of initiating the newer drug and to identify matched subjects receiving the comparator drug. We illustrate this approach using data from the UK's Clinical Practice Research Datalink to evaluate whether the newer glucagon-like peptide-1 receptor agonists (GLP-1 analogs) used to treat type 2 diabetes increase the risk of heart failure, in comparison with the older similarly indicated sulfonylureas.

RESULTS

Of the 170 031 users of antidiabetic agents from 2000 onwards, 79 682 used sulfonylureas (first use 2000), while 6196 used GLP-1 analogs (first use 2007), 75% of which had previously used a sulfonylurea. After matching each GLP-1 analog user to a sulfonylurea user on the time-conditional propensity scores from prescription-based exposure sets, the hazard ratio of heart failure with GLP-1 use was 0.73 (95%CI: 0.57-0.93).

CONCLUSION

The proposed prevalent new-user cohort design for comparative drug effects studies allows the use of all or most patients exposed to the newer drug, thus permitting a more comprehensive assessment of a new drug's safety. Copyright © 2016 John Wiley & Sons, Ltd.

The Obesity Paradox and Heart Failure: A Systematic Review of a Decade of Evidence

There is scientific consensus that obesity increases the risk of cardiovascular diseases, including heart failure. However, among persons who already have heart failure, outcomes seem to be better in obese persons as compared with lean persons: this has been termed the obesity paradox, the mechanisms of which remain unclear. This study systematically reviewed the evidence of the relationship between heart failure mortality (and survival) and weight status. Search of the PubMed/MEDLINE and EMBASE databases was done according to the PRISMA protocol. The initial search identified 9879 potentially relevant papers, out of which ten studies met the inclusion criteria. One study was a randomized clinical trial and 9 were observational cohort studies: 6 prospective and 3 retrospective studies. All studies used the BMI, WC, or TSF as measure of body fatness and NYHA Classification of Heart Failure and had single outcomes, death, as study endpoint. All studies included in review were longitudinal studies. All ten studies reported improved outcomes for obese heart failure patients as compared with their normal weight counterparts; worse prognosis was demonstrated for extreme obesity (BMI > 40 kg/m(2)). The findings of this review will be of significance in informing the practice of asking obese persons with heart failure to lose weight. However, any such recommendation on weight loss must be consequent upon more conclusive evidence on the mechanisms of the obesity paradox in heart failure and exclusion of collider bias.

The Rotterdam Study: 2016 objectives and design update

The Rotterdam Study is a prospective cohort study ongoing since 1990 in the city of Rotterdam in The Netherlands. The study targets cardiovascular, endocrine, hepatic, neurological, ophthalmic, psychiatric, dermatological, otolaryngological, locomotor, and respiratory diseases. As of 2008, 14,926 subjects aged 45 years or over comprise the Rotterdam Study cohort. The findings of the Rotterdam Study have been presented in over 1200 research articles and reports (see www.erasmus-epidemiology.nl/rotterdamstudy ). This article gives the rationale of the study and its design. It also presents a summary of the major findings and an update of the objectives and methods.

Performance of time-dependent propensity scores: a pharmacoepidemiology case study

PURPOSE

Pharmacoepidemiologic studies of acute effects of episodic exposures often must control for many time-dependent confounders. Marginal structural models permit this and provide unbiased estimates when confounders are on the causal pathway. However, if causal pathway confounding is minimal, analyses with time-dependent propensity scores, calculated for time periods defined by individual drug prescriptions, may have better efficiency. We justify time-dependent propensity scores and compare the performance of these methods in a case study from a previous investigation of the risk of medication toxicity death in current users of propoxyphene and hydrocodone, with both substantial time-dependent confounding and a large number of covariates.

METHODS

The cohort included Tennessee Medicaid enrollees who filled a qualifying study opioid prescription between 1992 and 2007. We identified 22 time-dependent covariates that accounted for most of the confounding in the original study. We compared analyses with all covariates in the regression model with those based on time-dependent propensity scores and those from marginal structural models.

RESULTS

We identified 489,008 persons with 1,771,295 propoxyphene and 4,088,754 hydrocodone prescriptions. The unadjusted hazard ratio (propoxyphene : hydrocodone) was 0.70 (95%CI, 0.46-1.07). Estimates from inclusion of all covariates in the model, time-dependent propensity score analysis with inverse probability of treatment weighting, and marginal structural models were 1.63 (1.04-2.57), 1.65 (1.01-2.72), and 1.64 (0.83-3.27), respectively. Findings varied little with use of alternative propensity score methods, time origin, or techniques for marginal structural model estimation.

CONCLUSIONS

Time-dependent propensity scores may be useful for pharmacoepidemiologic studies with time-varying exposures when causal pathway confounding is limited.

Reporting of covariate selection and balance assessment in propensity score analysis is suboptimal: a systematic review

OBJECTIVES

To assess the current practice of propensity score (PS) analysis in the medical literature, particularly the assessment and reporting of balance on confounders.

STUDY DESIGN AND SETTING

A PubMed search identified studies using PS methods from December 2011 through May 2012. For each article included in the review, information was extracted on important aspects of the PS such as the type of PS method used, variable selection for PS model, and assessment of balance.

RESULTS

Among 296 articles that were included in the review, variable selection for PS model was explicitly reported in 102 studies (34.4%). Covariate balance was checked and reported in 177 studies (59.8%). P-values were the most commonly used statistical tools to report balance (125 of 177, 70.6%). The standardized difference and graphical displays were reported in 45 (25.4%) and 11 (6.2%) articles, respectively. Matching on the PS was the most commonly used approach to control for confounding (68.9%), followed by PS adjustment (20.9%), PS stratification (13.9%), and inverse probability of treatment weighting (IPTW, 7.1%). Balance was more often checked in articles using PS matching and IPTW, 70.6% and 71.4%, respectively.

CONCLUSION

The execution and reporting of covariate selection and assessment of balance is far from optimal. Recommendations on reporting of PS analysis are provided to allow better appraisal of the validity of PS-based studies.