Estimating long-term treatment effects in observational data: A comparison of the performance of different methods under real-world uncertainty

Forced Expiratory Volume in 1 Second Variability Predicts Lung Transplant or Mortality in People with Cystic Fibrosis in the United States

Rationale: Declines in percent predicted forced expiratory volume in 1 second (ppFEV1) are an important marker of clinical progression of cystic fibrosis (CF). Objectives: We examined ppFEV1 variability in relation to a combined outcome of lung transplant or death. Methods: We estimated the association between ppFEV1 variability and the combined outcome of lung transplant or death. We included children aged 8 years and older with CF and two prior years of ppFEV1 data before baseline between 2005 and 2021. We defined ppFEV1 increased variability as any relative increase or decrease of at least 10% in ppFEV1 from a 2-year averaged baseline. A marginal structural Cox proportional hazards model was used. We examined a cumulative measure of ppFEV1 variability, defined as the cumulative proportion of visits with ppFEV1 variability at each visit. Kaplan-Meier survival curves were generated on the basis of quartiles of the cumulative distribution of ppFEV1 variability. Results: We included 9,706 patients with CF in our cohort. The median age at cohort entry was 8.3 (interquartile range, 8.2-8.4) years; 50% of patients were female; 94% were White; and the median baseline ppFEV1 was 94.4 (interquartile range, 81.6-106.1). The unadjusted hazard ratio for increased ppFEV1 variability on lung transplant/mortality was 4.13 (95% confidence interval, 3.48-4.90), and the weighted hazard ratio was 1.49 (95% confidence interval, 1.19-1.86). Survival curves stratified by quartile of cumulative variability demonstrated an increased hazard of lung transplant/mortality as the proportion of cumulative ppFEV1 variability increased. Conclusions: We found a strong association between ppFEV1 variability and lung transplant or mortality in a cohort of people with CF in the United States.

Methods for structural sexism and population health research: Introducing a novel analytic framework to capture life-course and intersectional effects

Accumulating evidence links structural sexism to gendered health inequities, yet methodological challenges have precluded comprehensive examinations into life-course and/or intersectional effects. To help address this gap, we introduce an analytic framework that uses sequential conditional mean models (SCMMs) to jointly account for longitudinal exposure trajectories and moderation by multiple dimensions of social identity/position, which we then apply to study how early life-course exposure to U.S. state-level structural sexism shapes mental health outcomes within and between gender groups. Data came from the Growing Up Today Study, a cohort of 16,875 children aged 9-14 years in 1996 who we followed through 2016. Using a composite index of relevant public policies and societal conditions (e.g., abortion bans, wage gaps), we assigned each U.S. state a year-specific structural sexism score and calculated participants' cumulative exposure by averaging the scores associated with states they had lived in during the study period, weighted according to duration of time spent in each. We then fit a series of SCMMs to estimate overall and group-specific associations between cumulative exposure from baseline through a given study wave and subsequent depressive symptomology; we also fit models using simplified (i.e., non-cumulative) exposure variables for comparison purposes. Analyses revealed that cumulative exposure to structural sexism: (1) was associated with significantly increased odds of experiencing depressive symptoms by the subsequent wave; (2) disproportionately impacted multiply marginalized groups (e.g., sexual minority girls/women); and (3) was more strongly associated with depressive symptomology compared to static or point-in-time exposure operationalizations (e.g., exposure in a single year). Substantively, these findings suggest that long-term exposure to structural sexism may contribute to the inequitable social patterning of mental distress among young people living in the U.S. More broadly, the proposed analytic framework represents a promising approach to examining the complex links between structural sexism and health across the life course and for diverse social groups.

Pancreatic Cancer Multidisciplinary Clinic is Associated with Improved Treatment and Elimination of Socioeconomic Disparities

OBJECTIVE

To identify the association between multidisciplinary clinic (MDC) management and disparities in treatment for patients with pancreatic cancer.

BACKGROUND

Socioeconomic status (SES) predicts treatment and survival for pancreatic cancer. Multidisciplinary clinics (MDCs) may improve surgical management for these patients.

METHODS

This is a retrospective cohort study (2010-2018) of all pancreatic cancer patients within a large, regional hospital system with a high-volume pancreatic cancer MDC. The primary outcome was receipt of treatment (surgery, chemotherapy, radiation, clinical trial participation, and palliative care); the secondary outcomes were overall survival and MDC management. Multiple logistic regressions were used for binary outcomes. Survival was analyzed using Kaplan-Meier survival analysis, Cox proportional hazards, and inverse probability of treatment weighting (IPTW).

RESULTS

Of the 4141 patients studied, 1420 (34.3%) were managed by the MDC. MDC management was more likely for patients who were younger age, married, and privately insured, while less likely for low SES patients (all p < 0.05). MDC patients were more likely to receive all treatments, including neoadjuvant chemotherapy (OR 3.33, 95% CI 2.82-3.93), surgery (OR 1.39, 95% CI 1.15-1.68), palliative care (OR 1.21, 95% CI 1.05-1.38), and clinical trial participation (OR 3.76, 95% CI 2.86-4.93). Low SES patients were less likely to undergo surgery outside of the MDC (OR 0.47, 95% CI 0.31-0.73) but there was no difference within the MDC (OR 1.10, 95% CI 0.68-1.77). Across multiple survival analyses, low SES predicted inferior survival outside of the MDC, but there was no association among MDC patients.

CONCLUSION

Multidisciplinary team-based care increases rates of treatment and eliminates socioeconomic disparities for pancreatic cancer patients.

Addressing missing data in the estimation of time-varying treatments in comparative effectiveness research

Comparative effectiveness research is often concerned with evaluating treatment strategies sustained over time, that is, time-varying treatments. Inverse probability weighting (IPW) is often used to address the time-varying confounding by re-weighting the sample according to the probability of treatment receipt at each time point. IPW can also be used to address any missing data by re-weighting individuals according to the probability of observing the data. The combination of these two distinct sets of weights may lead to inefficient estimates of treatment effects due to potentially highly variable total weights. Alternatively, multiple imputation (MI) can be used to address the missing data by replacing each missing observation with a set of plausible values drawn from the posterior predictive distribution of the missing data given the observed data. Recent studies have compared IPW and MI for addressing the missing data in the evaluation of time-varying treatments, but they focused on missing confounders and monotone missing data patterns. This article assesses the relative advantages of MI and IPW to address missing data in both outcomes and confounders measured over time, and across monotone and non-monotone missing data settings. Through a comprehensive simulation study, we find that MI consistently provided low bias and more precise estimates compared to IPW across a wide range of scenarios. We illustrate the implications of method choice in an evaluation of biologic drugs for patients with severe rheumatoid arthritis, using the US National Databank for Rheumatic Diseases, in which 25% of participants had missing health outcomes or time-varying confounders.

Emulated trial investigating effects of multiple treatments: estimating combined effects of mucoactive nebulisers in cystic fibrosis using registry data

INTRODUCTION

People with cystic fibrosis (CF) are often on multiple long-term treatments, including mucoactive nebulisers. In the UK, the most common mucoactive nebuliser is dornase alfa (DNase). A common therapeutic approach for people already on DNase is to add hypertonic saline (HS). The effects of DNase and HS used alone have been studied in randomised trials, but their effects in combination have not. This study investigates whether, for people already prescribed DNase, adding HS has additional benefit for lung function or use of intravenous antibiotics.

METHODS

Using UK CF Registry data from 2007 to 2018, we emulated a target trial. We included people aged 6 years and over who were prescribed DNase without HS for 2 years. We investigated the effects of combinations of DNase and HS over 5 years of follow-up. Inverse-probability-of-treatment weighting was used to control confounding. The period predated triple combination CF transmembrane conductance regulator modulators in routine care.

RESULTS

4498 individuals were included. At baseline, average age and forced expiratory volume in 1 s (FEV1%) predicted were 21.1 years and 69.7 respectively. During first year of follow-up, 3799 individuals were prescribed DNase alone; 426 added HS; 57 switched to HS alone and 216 were prescribed neither. We found no evidence that adding HS improved FEV1% at 1-5 years, or use of intravenous antibiotics at 1-4 years, compared with DNase alone.

CONCLUSION

For individuals with CF prescribed DNase, we found no evidence that adding HS had an effect on FEV1% or prescription of intravenous antibiotics. Our study illustrates the emulated target trial approach using CF Registry data.

Causal inference in survival analysis using longitudinal observational data: Sequential trials and marginal structural models

Longitudinal observational data on patients can be used to investigate causal effects of time-varying treatments on time-to-event outcomes. Several methods have been developed for estimating such effects by controlling for the time-dependent confounding that typically occurs. The most commonly used is marginal structural models (MSM) estimated using inverse probability of treatment weights (IPTW) (MSM-IPTW). An alternative, the sequential trials approach, is increasingly popular, and involves creating a sequence of "trials" from new time origins and comparing treatment initiators and non-initiators. Individuals are censored when they deviate from their treatment assignment at the start of each "trial" (initiator or noninitiator), which is accounted for using inverse probability of censoring weights. The analysis uses data combined across trials. We show that the sequential trials approach can estimate the parameters of a particular MSM. The causal estimand that we focus on is the marginal risk difference between the sustained treatment strategies of "always treat" vs "never treat." We compare how the sequential trials approach and MSM-IPTW estimate this estimand, and discuss their assumptions and how data are used differently. The performance of the two approaches is compared in a simulation study. The sequential trials approach, which tends to involve less extreme weights than MSM-IPTW, results in greater efficiency for estimating the marginal risk difference at most follow-up times, but this can, in certain scenarios, be reversed at later time points and relies on modelling assumptions. We apply the methods to longitudinal observational data from the UK Cystic Fibrosis Registry to estimate the effect of dornase alfa on survival.

The effect of blood cadmium levels on hypertension in male firefighters in a metropolitan city

Background

This study investigated the effect of dispatch frequency on blood cadmium levels and the effect of blood cadmium levels on hypertension in male firefighters in a metropolitan city.

Methods

We conducted a retrospective longitudinal study of male firefighters who completed the regular health checkups, including a health examination survey and blood cadmium measurements. We followed them for 3 years. To investigate the effect of dispatch frequency on blood cadmium levels and the effect of blood cadmium levels on hypertension, we estimated the short-term (model 1) and long-term (model 2) effects of exposure and hypothesized a reversed causal pathway model (model 3) for sensitivity analysis. Sequential conditional mean models were fitted using generalized estimating equations, and the odds ratios (ORs) and the respective 95% confidence intervals (CIs) were calculated for hypertension for log-transformed (base 2) blood cadmium levels and quartiles.

Results

Using the lowest category of dispatch frequency as a reference, we observed that the highest category showed an increase in blood cadmium levels of 1.879 (95% CI: 0.673, 3.086) μg/dL and 0.708 (95% CI: 0.023, 1.394) μg/dL in models 2 and 3, respectively. In addition, we observed that doubling the blood cadmium level significantly increased the odds of hypertension in model 1 (OR: 1.772; 95% CI: 1.046, 3.003) and model 3 (OR: 4.288; 95% CI: 1.110, 16.554). Using the lowest quartile of blood cadmium levels as a reference, the highest quartile showed increased odds of hypertension in model 1 (OR: 2.968; 95% CI: 1.121, 7.861) and model 3 (OR: 33.468; 95% CI: 1.881, 595.500).

Conclusions

We found that dispatch frequency may affect blood cadmium levels in male firefighters, and high blood cadmium levels may influence hypertension in a dose-response manner.

Evaluating a Cox marginal structural model to assess the comparative effectiveness of inhaled corticosteroids versus no inhaled corticosteroid treatment in chronic obstructive pulmonary disease

PURPOSE

To evaluate the potential of a Cox marginal structural model (MSM) to estimate the time-varying causal inference of a known clinical trial association where the effectiveness of inhaled corticosteroid- (ICS-) versus non-ICS-containing treatments has been compared in patients with chronic obstructive pulmonary disease (COPD).

METHODS

This retrospective study from 2006 to 2016 used linked data from Clinical Practice Research Datalink-GOLD, Hospital Episode Statistics and Office for National Statistics mortality. A Cox MSM, incorporating a new-user design, was deemed capable of replicating a clinical trial-like pathway. Repeated outcomes for exacerbation events and stabilised weights were used to include time-varying and fixed covariate exposures.

RESULTS

Of 45,958 patients, 55% were male; 52% had moderate COPD. ICS-treated patients had a higher incidence of comorbid asthma than non-ICS-treated patients. Adjusted hazard risk ratios for any exacerbation event: ICS/long-acting β₂-agonist (LABA) versus long-acting muscarinic antagonist (LAMA), 1.07 (95% confidence interval 1.04-1.10); ICS/LABA versus LABA/LAMA, 1.05 (1.00-1.10); ICS/LABA/LAMA versus LAMA, 1.04 (1.01-1.06); ICS/LABA/LAMA versus LABA/LAMA 1.02 (0.97-1.07).

CONCLUSIONS

The Cox MSM was not able to fully demonstrate results consistent with the previously established benefits of ICS-containing treatments seen in clinical trials. Future studies should continue to investigate causal inference methods and their capability to estimate the long-term outcomes of treatment in COPD.

Replicating Randomized Trial Results with Observational Data Using the Parametric g-Formula: An Application to Intravenous Iron Treatment in Hemodialysis Patients

BACKGROUND

Reproducibility of clinical and epidemiologic research is important to generalize findings and has increasingly been scrutinized. A recently published randomized trial, PIVOTAL, evaluated high vs low intravenous iron dosing strategies to manage anemia in hemodialysis patients in the UK. Our objective was to assess the reproducibility of the PIVOTAL trial findings using data from a well-established cohort study, the Dialysis Outcomes and Practice Patterns Study (DOPPS).

METHODS

To overcome the absence of randomization in the DOPPS, we applied the parametric g-formula, an extension of standardization to longitudinal data. We estimated the effect of a proactive high-dose vs reactive low-dose iron supplementation strategy on all-cause mortality (primary outcome), hemoglobin, two measures of iron concentration (ferritin and TSAT), and erythropoiesis-stimulating agent dose over 12 months of follow-up in 6325 DOPPS patients.

RESULTS

Comparing high- vs low-iron dose strategies, the 1-year mortality risk difference was 0.020 (95% CI: 0.008, 0.031) and risk ratio was 1.20 (95% CI: 1.07, 1.33), compared with null 1-year findings in the PIVOTAL trial. Differences in secondary outcomes were directionally consistent but of lesser magnitude than in the PIVOTAL trial.

CONCLUSION

Our findings are somewhat consistent with the recent PIVOTAL trial, with discrepancies potentially attributable to model misspecification and differences between the two study populations. In addition to the importance of our results to nephrologists and hence hemodialysis patients, our analysis illustrates the utility of the parametric g-formula for generalizing results and comparing complex and dynamic treatment strategies using observational data.

Cognitively stimulating activities and risk of probable dementia or cognitive impairment in the English Longitudinal Study of Ageing

Objectives

To examine the association between cognitive stimulating activities (CSA) in later life (internet/email use, employment, volunteering, evening classes, social club membership and newspaper reading) and risk of cognitive impairment or dementia using marginal structural models to account for time-varying confounding affected by prior exposure.

Methods

Data were used from the English Longitudinal Study of Ageing waves 1 (2002) to 7 (2014), a nationally representative sample of adults in England aged ≥50. Self-reported participation in CSAs were measured as binary exposures from waves 2 (2004) to 6 (2012), with final sample sizes between n = 3937 and n = 2530 for different CSAs. Baseline exposure and covariates were used to create inverse probability of treatment and censoring weights (IPTCW). IPTCW repeated measures Poisson and linear regression were used to estimate each CSAs effect on risk of probable cognitive impairment or dementia at wave 7 (defined as a score of ≤11/27 on a modified telephone interview for cognitive status (TICS-27)). Results were compared to standard regression adjustment.

Results

Internet use at any wave (Risk ratios between 0.62 and 0.69) and volunteering in waves 3 to 6 (RRs between 0.516 and 0.633) were associated with reduced risk of cognitive impairment in IPTCW models. Standard estimates were similar for both internet use and volunteering.

Newspaper reading (RR 95% Confidence interval 0.74–0.99) and social club membership (RR 95% CI 0.54–0.86) at wave 6 were significantly associated with risk of cognitive impairment in standard models, but not in the IPTCW models (RR 95% CI 0.82–1.11 and 0.60–1.08 respectively). Employment and evening classes were not associated with cognitive impairment in either model.

Conclusions

We found that volunteering and internet use were associated with reduced risk of cognitive impairment. Associations between newspaper reading or social club membership and cognitive impairment may be due to time-varying confounding affected by prior exposure.

•

Confounding affected by past exposure is a problem in studies of cognitive function.

•

We addressed this using inverse probability weighted marginal structural models.

•

Volunteering and internet use were protective against cognitive impairment.

•

Other cognitively stimulating activities were protective with standard regression.

•

But these associations were non-significant in the marginal structural models.

Fitting Marginal Structural and G-Estimation Models Under Complex Treatment Patterns: Investigating the Association Between De Novo Vitamin D Supplement Use After Breast Cancer Diagnosis and All-Cause Mortality Using Linked Pharmacy Claim and Registry Data

Studies have shown that accounting for time-varying confounding through time-dependent Cox proportional hazards models may provide biased estimates of the causal effect of treatment when the confounder is also a mediator. We explore 2 alternative approaches to addressing this problem while examining the association between vitamin D supplementation initiated after breast cancer diagnosis and all-cause mortality. Women aged 50-80 years were identified in the National Cancer Registry Ireland (n = 5,417) between 2001 and 2011. Vitamin D use was identified from linked prescription data (n = 2,570). We sought to account for the time-varying nature of vitamin D use and time-varying confounding by bisphosphonate use using 1) marginal structural models (MSMs) and 2) G-estimation of structural nested accelerated failure-time models (SNAFTMs). Using standard adjusted Cox proportional hazards models, we found a reduction in all-cause mortality in de novo vitamin D users compared with nonusers (hazard ratio (HR) = 0.84, 95% confidence interval (CI): 0.73, 0.99). Additional adjustment for vitamin D and bisphosphonate use in the previous month reduced the hazard ratio (HR = 0.45, 95% CI: 0.33, 0.63). Results derived from MSMs (HR = 0.44, 95% CI: 0.32, 0.61) and SNAFTMs (HR = 0.45, 95% CI: 0.34, 0.52) were similar. Utilizing MSMs and SNAFTMs to account for time-varying bisphosphonate use did not alter conclusions in this example.

Dynamic Prediction of Survival in Cystic Fibrosis: A Landmarking Analysis Using UK Patient Registry Data

BACKGROUND

Cystic fibrosis (CF) is an inherited, chronic, progressive condition affecting around 10,000 individuals in the United Kingdom and over 70,000 worldwide. Survival in CF has improved considerably over recent decades, and it is important to provide up-to-date information on patient prognosis.

METHODS

The UK Cystic Fibrosis Registry is a secure centralized database, which collects annual data on almost all CF patients in the United Kingdom. Data from 43,592 annual records from 2005 to 2015 on 6181 individuals were used to develop a dynamic survival prediction model that provides personalized estimates of survival probabilities given a patient's current health status using 16 predictors. We developed the model using the landmarking approach, giving predicted survival curves up to 10 years from 18 to 50 years of age. We compared several models using cross-validation.

RESULTS

The final model has good discrimination (C-indexes: 0.873, 0.843, and 0.804 for 2-, 5-, and 10-year survival prediction) and low prediction error (Brier scores: 0.036, 0.076, and 0.133). It identifies individuals at low and high risk of short- and long-term mortality based on their current status. For patients 20 years of age during 2013-2015, for example, over 80% had a greater than 95% probability of 2-year survival and 40% were predicted to survive 10 years or more.

CONCLUSIONS

Dynamic personalized prediction models can guide treatment decisions and provide personalized information for patients. Our application illustrates the utility of the landmarking approach for making the best use of longitudinal and survival data and shows how models can be defined and compared in terms of predictive performance.

On the use of flexible excess hazard regression models for describing long-term breast cancer survival: a case-study using population-based cancer registry data

BACKGROUND

Breast cancer prognosis has dramatically improved over 40 years. There is, however, no proof of population 'cure'. This research aimed to examine the pattern of long-term excess mortality due to breast cancer and evaluate its determinants in the context of cancer registry data.

METHODS

We used data from the Geneva Cancer Registry to identify women younger than 75 years diagnosed with invasive, localised and operated breast cancer between 1995 and 2002. Flexible modelling of excess mortality hazard, including time-dependent (TD) regression parameters, was used to estimate mortality related to breast cancer. We derived a single "final" model using a backward selection procedure and evaluated its stability through sensitivity analyses using a bootstrap technique.

RESULTS

We analysed data from 1574 breast cancer women including 351 deaths (22.3%). The model building strategy retained age at diagnosis (TD), tumour size and grade (TD), chemotherapy and hormonal treatment (TD) as prognostic factors, while the sensitivity analysis on bootstrap samples identified nodes involvement and hormone receptors (TD) as additional long-term prognostic factors but did not identify chemotherapy and hormonal treatment as important prognostic factors.

CONCLUSIONS

Two main issues were observed when describing the determinants of long-term survival. First, the modelling strategy presented a lack of robustness, probably due to the limited number of events observed in our study. The second was the misspecification of the model, probably due to confounding by indication. Our results highlight the need for more detailed data and the use of causal inference methods.

Estimating long-term treatment effects in observational data: A comparison of the performance of different methods under real-world uncertainty

In the presence of time-dependent confounding, there are several methods available to estimate treatment effects. With correctly specified models and appropriate structural assumptions, any of these methods could provide consistent effect estimates, but with real-world data, all models will be misspecified and it is difficult to know if assumptions are violated. In this paper, we investigate five methods: inverse probability weighting of marginal structural models, history-adjusted marginal structural models, sequential conditional mean models, g-computation formula, and g-estimation of structural nested models. This work is motivated by an investigation of the effects of treatments in cystic fibrosis using the UK Cystic Fibrosis Registry data focussing on two outcomes: lung function (continuous outcome) and annual number of days receiving intravenous antibiotics (count outcome). We identified five features of this data that may affect the performance of the methods: misspecification of the causal null, long-term treatment effects, effect modification by time-varying covariates, misspecification of the direction of causal pathways, and censoring. In simulation studies, under ideal settings, all five methods provide consistent estimates of the treatment effect with little difference between methods. However, all methods performed poorly under some settings, highlighting the importance of using appropriate methods based on the data available. Furthermore, with the count outcome, the issue of non-collapsibility makes comparison between methods delivering marginal and conditional effects difficult. In many situations, we would recommend using more than one of the available methods for analysis, as if the effect estimates are very different, this would indicate potential issues with the analyses.