Core concepts in pharmacoepidemiology: Violations of the positivity assumption in the causal analysis of observational data: Consequences and statistical approaches

Designing target trials using electronic health records: A case study of second-line disease-modifying anti-rheumatic drugs and cardiovascular disease outcomes in patients with rheumatoid arthritis

BACKGROUND

Emulation of the "target trial" (TT), a hypothetical pragmatic randomized controlled trial (RCT), using observational data can be used to mitigate issues commonly encountered in comparative effectiveness research (CER) when randomized trials are not logistically, ethically, or financially feasible. However, cardiovascular (CV) health research has been slow to adopt TT emulation. Here, we demonstrate the design and analysis of a TT emulation using electronic health records to study the comparative effectiveness of the addition of a disease-modifying anti-rheumatic drug (DMARD) to a regimen of methotrexate on CV events among rheumatoid arthritis (RA) patients.

METHODS

We used data from an electronic medical records-based cohort of RA patients from Northwestern Medicine to emulate the TT. Follow-up began 3 months after initial prescription of MTX (2000-2020) and included all available follow-up through June 30, 2020. Weighted pooled logistic regression was used to estimate differences in CVD risk and survival. Cloning was used to handle immortal time bias and weights to improve baseline and time-varying covariate imbalance.

RESULTS

We identified 659 eligible people with RA with average follow-up of 46 months and 31 MACE events. The month 24 adjusted risk difference for MACE comparing initiation vs non-initiation of a DMARD was -1.47% (95% confidence interval [CI]: -4.74, 1.95%), and the marginal hazard ratio (HR) was 0.72 (95% CI: 0.71, 1.23). In analyses subject to immortal time bias, the HR was 0.62 (95% CI: 0.29-1.44).

CONCLUSION

In this sample, we did not observe evidence of differences in risk of MACE, a finding that is compatible with previously published meta-analyses of RCTs. Thoughtful application of the TT framework provides opportunities to conduct CER in observational data. Benchmarking results of observational analyses to previously published RCTs can lend credibility to interpretation.

An AI assistant to help review and improve causal reasoning in epidemiological documents

Drawing sound causal inferences from observational data is often challenging for both authors and reviewers. This paper discusses the design and application of an Artificial Intelligence Causal Research Assistant (AIA) that seeks to help authors improve causal inferences and conclusions drawn from epidemiological data in health risk assessments. The AIA-assisted review process provides structured reviews and recommendations for improving the causal reasoning, analyses and interpretations made in scientific papers based on epidemiological data. Causal analysis methodologies range from earlier Bradford-Hill considerations to current causal directed acyclic graph (DAG) and related models. AIA seeks to make these methods more accessible and useful to researchers. AIA uses an external script (a "Causal AI Booster" (CAB) program based on classical AI concepts of slot-filling in frames organized into task hierarchies to complete goals) to guide Large Language Models (LLMs), such as OpenAI's ChatGPT or Google's LaMDA (Bard), to systematically review manuscripts and create both (a) recommendations for what to do to improve analyses and reporting; and (b) explanations and support for the recommendations. Review tables and summaries are completed systematically by the LLM in order. For example, recommendations for how to state and caveat causal conclusions in the Abstract and Discussion sections reflect previous analyses of the Study Design and Data Analysis sections. This work illustrates how current AI can contribute to reviewing and providing constructive feedback on research documents. We believe that such AI-assisted review shows promise for enhancing the quality of causal reasoning and exposition in epidemiological studies. It suggests the potential for effective human-AI collaboration in scientific authoring and review processes.

Locoregional delivery of IL-13Rα2-targeting CAR-T cells in recurrent high-grade glioma: a phase 1 trial

Chimeric antigen receptor T cell (CAR-T) therapy is an emerging strategy to improve treatment outcomes for recurrent high-grade glioma, a cancer that responds poorly to current therapies. Here we report a completed phase I trial evaluating IL-13Rα2-targeted CAR-T cells in 65 patients with recurrent high-grade glioma, the majority being recurrent glioblastoma (rGBM). Primary objectives were safety and feasibility, maximum tolerated dose/maximum feasible dose and a recommended phase 2 dose plan. Secondary objectives included overall survival, disease response, cytokine dynamics and tumor immune contexture biomarkers. This trial evolved to evaluate three routes of locoregional T cell administration (intratumoral (ICT), intraventricular (ICV) and dual ICT/ICV) and two manufacturing platforms, culminating in arm 5, which utilized dual ICT/ICV delivery and an optimized manufacturing process. Locoregional CAR-T cell administration was feasible and well tolerated, and as there were no dose-limiting toxicities across all arms, a maximum tolerated dose was not determined. Probable treatment-related grade 3+ toxicities were one grade 3 encephalopathy and one grade 3 ataxia. A clinical maximum feasible dose of 200 × 106 CAR-T cells per infusion cycle was achieved for arm 5; however, other arms either did not test or achieve this dose due to manufacturing feasibility. A recommended phase 2 dose will be refined in future studies based on data from this trial. Stable disease or better was achieved in 50% (29/58) of patients, with two partial responses, one complete response and a second complete response after additional CAR-T cycles off protocol. For rGBM, median overall survival for all patients was 7.7 months and for arm 5 was 10.2 months. Central nervous system increases in inflammatory cytokines, including IFNγ, CXCL9 and CXCL10, were associated with CAR-T cell administration and bioactivity. Pretreatment intratumoral CD3 T cell levels were positively associated with survival. These findings demonstrate that locoregional IL-13Rα2-targeted CAR-T therapy is safe with promising clinical activity in a subset of patients. ClinicalTrials.gov Identifier: NCT02208362 .

Propensity score stratified MAP prior and posterior inference for incorporating information across multiple potentially heterogeneous data sources

Incorporation of external information is becoming increasingly common when designing clinical trials. Availability of multiple sources of information has inspired the development of methodologies that account for potential heterogeneity not only between the prospective trial and the pooled external data sources but also between the different external data sources themselves. Our approach proposes an intuitive way of handling such a scenario for the continuous outcomes setting by using propensity score-based stratification and then utilizing robust meta-analytic predictive priors for each stratum to incorporate the prior data to distinguish among different external data sources in each stratum. Through extensive simulations, our approach proves to be more efficient and less biased than the currently available methods. A real case study using clinical trials that study schizophrenia from multiple different sources is also included.

Heterogeneous treatment effect estimation for observational data using model-based forests

The estimation of heterogeneous treatment effects has attracted considerable interest in many disciplines, most prominently in medicine and economics. Contemporary research has so far primarily focused on continuous and binary responses where heterogeneous treatment effects are traditionally estimated by a linear model, which allows the estimation of constant or heterogeneous effects even under certain model misspecifications. More complex models for survival, count, or ordinal outcomes require stricter assumptions to reliably estimate the treatment effect. Most importantly, the noncollapsibility issue necessitates the joint estimation of treatment and prognostic effects. Model-based forests allow simultaneous estimation of covariate-dependent treatment and prognostic effects, but only for randomized trials. In this paper, we propose modifications to model-based forests to address the confounding issue in observational data. In particular, we evaluate an orthogonalization strategy originally proposed by Robinson (1988, Econometrica) in the context of model-based forests targeting heterogeneous treatment effect estimation in generalized linear models and transformation models. We found that this strategy reduces confounding effects in a simulated study with various outcome distributions. We demonstrate the practical aspects of heterogeneous treatment effect estimation for survival and ordinal outcomes by an assessment of the potentially heterogeneous effect of Riluzole on the progress of Amyotrophic Lateral Sclerosis.

Leveraging baseline covariates to analyze small cluster-randomized trials with a rare binary outcome

Cluster-randomized trials (CRTs) involve randomizing entire groups of participants-called clusters-to treatment arms but are often comprised of a limited or fixed number of available clusters. While covariate adjustment can account for chance imbalances between treatment arms and increase statistical efficiency in individually randomized trials, analytical methods for individual-level covariate adjustment in small CRTs have received little attention to date. In this paper, we systematically investigate, through extensive simulations, the operating characteristics of propensity score weighting and multivariable regression as two individual-level covariate adjustment strategies for estimating the participant-average causal effect in small CRTs with a rare binary outcome and identify scenarios where each adjustment strategy has a relative efficiency advantage over the other to make practical recommendations. We also examine the finite-sample performance of the bias-corrected sandwich variance estimators associated with propensity score weighting and multivariable regression for quantifying the uncertainty in estimating the participant-average treatment effect. To illustrate the methods for individual-level covariate adjustment, we reanalyze a recent CRT testing a sedation protocol in 31 pediatric intensive care units.

Effects of intimate partner violence and homophobic bullying on ART adherence among young Thai men who have sex with men: a causal mediation analysis

Background

Adherence to antiretroviral therapy is crucial in determining health outcomes and secondary HIV transmission for people living with HIV/AIDS. Young men who have sex with men (YMSM) living with HIV are often challenged by the prevailing experiences of psychosocial stressors, such as intimate partner violence and homophobic bullying, which may negatively affect their HIV care engagement.

Methods

This study is the first to utilize a prospective cohort design (N= 185) involving YMSM living with HIV in Thailand. We examined the effects of intimate partner violence and homophobic bullying on ART adherence. We also tested the mediating effect of depression on the relationship between intimate partner violence and homophobic bullying on ART adherence.

Results

We found that intimate partner violence (AOR: 2.58, 95% CI: 1.13, 5.42) and homophobic bullying (AOR: 2.40, 95% CI: 1.26, 4.48) were associated with subsequent ART nonadherence. Moreover, depression partially mediated 17.4% (95% CI: 0.75%, 56%) of the effect of homophobic bullying on ART nonadherence.

Conclusions

The results suggest that tailored interventions to optimize ART adherence should address the impacts of intimate partner violence and homophobic bullying for HIV+ YMSM. The screening and subsequent treatment of depression alone may not be sufficient to address the effects of intimate partner violence, homophobic bullying, and possibly other MSM-specific psychosocial stressors on ART adherence.

Current practices in studies applying the target trial emulation framework: a protocol for a systematic review

INTRODUCTION

Observational studies represent an alternative to estimate real-world causal effects in the absence of available randomised controlled trials (RCTs). Target trial emulation is a framework for the application of RCT design principles to emulate a hypothetical open-label RCT (the hypothetical target trial) using existing observational data as the primary data source as opposed to the prospective recruitment and measurement of randomised units. The aim of this systematic review is to investigate the practices of studies applying the target trial emulation framework to evaluate the effectiveness of interventions.

METHODS AND ANALYSIS

We will systematically search in Medline (via Ovid), Embase (via Ovid, entries from medRxiv are included), PsycINFO (via Ovid), SCOPUS, Web of Science, Cochrane Library, the ISRCTN registry and ClinicalTrials.gov for all study reports and protocols which used the trial emulation framework (without time restriction). We will extract information concerning study design, data source, analysis, results, interpretation and dissemination. Two reviewers will perform study selection, data extraction and quality assessment. Disagreements between reviewers will be resolved by a third reviewer. A narrative approach will be used to synthesise and report qualitative and quantitative data. Reporting of the review will be informed by Preferred Reporting Items for Systematic Review and Meta-Analysis guidance (PRISMA).

ETHICS AND DISSEMINATION

Ethical approval is not required as it is a protocol for a systematic review. Findings will be disseminated through peer-reviewed publications and conference presentations.

matchRanges: generating null hypothesis genomic ranges via covariate-matched sampling

MOTIVATION

Deriving biological insights from genomic data commonly requires comparing attributes of selected genomic loci to a null set of loci. The selection of this null set is non-trivial, as it requires careful consideration of potential covariates, a problem that is exacerbated by the non-uniform distribution of genomic features including genes, enhancers, and transcription factor binding sites. Propensity score-based covariate matching methods allow the selection of null sets from a pool of possible items while controlling for multiple covariates; however, existing packages do not operate on genomic data classes and can be slow for large data sets making them difficult to integrate into genomic workflows.

RESULTS

To address this, we developed matchRanges, a propensity score-based covariate matching method for the efficient and convenient generation of matched null ranges from a set of background ranges within the Bioconductor framework.

AVAILABILITY AND IMPLEMENTATION

Package: https://bioconductor.org/packages/nullranges, Code: https://github.com/nullranges, Documentation: https://nullranges.github.io/nullranges.

Effectiveness research in oncology with electronic health record data: A retrospective cohort study emulating the PALOMA-2 trial

PURPOSE

Oncology electronic health record (EHR) databases have increased in quality and availability over the past decade, yet it remains unclear whether these clinical practice data can be used to conduct reliable comparative effectiveness studies. We sought to emulate a clinical trial with EHR data in the advanced breast cancer population and compare our results against the trial.

METHODS

This cohort study used EHR data from US oncology practices. All elements of the study were defined to mimic the PALOMA-2 trial as closely as possible. Patients with hormone-positive, HER-2 negative metastatic breast cancer with no prior treatment for metastatic disease were included. Patients initiating palbociclib and letrozole on the same day following the earliest record of metastasis were compared to those initiating letrozole only. The primary associational measure was the conditional hazard ratio for time-to-next treatment (TTNT). TTNT is well-measured in our data source and amenable for calibration against the randomized study results of the PALOMA-2 trial. We used multiple imputation for several patient characteristics with missing values.

RESULTS

There were 3836 study-eligible women with advanced breast cancer. The hazard ratio for TTNT in the observational study (HR: 0.62; 95% CI: 0.56-0.68) was closely aligned with that of the randomized trial (HR: 0.64; 95% CI: 0.52-0.78).

CONCLUSIONS

Under our assumptions on missing data and comparability of the two study populations, results from our non-randomized study closely matched that of the randomized trial. Further studies are needed to determine whether EHR data can yield reliable conclusions on treatment effects in oncology.

Learning end-to-end patient representations through self-supervised covariate balancing for causal treatment effect estimation

A causal effect can be defined as a comparison of outcomes that result from two or more alternative actions, with only one of the action-outcome pairs actually being observed. In healthcare, the gold standard for causal effect measurements is randomized controlled trials (RCTs), in which a target population is explicitly defined and each study sample is randomly assigned to either the treatment or control cohorts. The great potential to derive actionable insights from causal relationships has led to a growing body of machine-learning research applying causal effect estimators to observational data in the fields of healthcare, education, and economics. The primary difference between causal effect studies utilizing observational data and RCTs is that for observational data, the study occurs after the treatment, and therefore we do not have control over the treatment assignment mechanism. This can lead to massive differences in covariate distributions between control and treatment samples, making a comparison of causal effects confounded and unreliable. Classical approaches have sought to solve this problem piecemeal, first by predicting treatment assignment and then treatment effect separately. Recent work extended part of these approaches to a new family of representation-learning algorithms, showing that the upper bound of the expected treatment effect estimation error is determined by two factors: the outcome generalization-error of the representation and the distance between the treated and control distributions induced by the representation. To achieve minimal dissimilarity in learning such distributions, in this work we propose a specific auto-balancing, self-supervised objective. Experiments on real and benchmark datasets revealed that our approach consistently produced less biased estimates than previously published state-of-the-art methods. We demonstrate that the reduction in error can be directly attributed to the ability to learn representations that explicitly reduce such dissimilarity; further, in case of violations of the positivity assumption (frequent in observational data), we show our approach performs significantly better than the previous state of the art. Thus, by learning representations that induce similar distributions of the treated and control cohorts, we present evidence to support the error bound dissimilarity hypothesis as well as providing a new state-of-the-art model for causal effect estimation.

Addressing positivity violations in causal effect estimation using Gaussian process priors

In observational studies, causal inference relies on several key identifying assumptions. One identifiability condition is the positivity assumption, which requires the probability of treatment be bounded away from 0 and 1. That is, for every covariate combination, it should be possible to observe both treated and control subjects the covariate distributions should overlap between treatment arms. If the positivity assumption is violated, population-level causal inference necessarily involves some extrapolation. Ideally, a greater amount of uncertainty about the causal effect estimate should be reflected in such situations. With that goal in mind, we construct a Gaussian process model for estimating treatment effects in the presence of practical violations of positivity. Advantages of our method include minimal distributional assumptions, a cohesive model for estimating treatment effects, and more uncertainty associated with areas in the covariate space where there is less overlap. We assess the performance of our approach with respect to bias and efficiency using simulation studies. The method is then applied to a study of critically ill female patients to examine the effect of undergoing right heart catheterization.

Oncology Drug Effectiveness from Electronic Health Record Data Calibrated Against RCT Evidence: The PARSIFAL Trial Emulation

Background

The use of electronic health records (EHR) data to assess drug effectiveness in clinical oncology practice is of great interest to regulators, clinicians, and payers. However, the utility of EHR data in clinical effectiveness studies may be limited by missing data, unmeasured confounding, and imperfect outcome surveillance. This study sought to emulate and compare the results of a randomized controlled trial investigating the efficacy of palbociclib with fulvestrant vs letrozole in advanced breast cancer.

Methods

This was a cohort study using longitudinal EHR data derived from outpatient oncology practices in the United States. Eligibility criteria from the PARSIFAL trial were emulated as closely as possible. Patients were included if they had hormone-positive, human epidermal growth factor receptor - 2 (HER-2) negative metastatic breast cancer and had no record of prior treatment for metastatic disease. Patients initiating first-line treatment with palbociclib and fulvestrant following their first record of metastasis were compared to those initiating palbociclib and letrozole on the same day. Treatments were ascertained by oncology medication ordering records in the data source. The primary outcome was death as recorded in the oncologists' EHR systems.

Results

There were 1886 eligible women in the study cohort. Although the 3-year survival was meaningfully lower in clinical practice (59%) compared to the randomized trial (78%), the relative effect size was a hazard ratio (HR) of 1.07 (95% CI: 0.86-1.35), similar to the randomized trial (HR = 1.00; 95% CI: 0.68-1.48).

Conclusion

Despite common challenges encountered in EHR-based studies, it is possible to achieve similar conclusions to emulated randomized trials with the application of analytic approaches that address missing data, confounding, and selection bias. This is a promising finding in light of other emulations and ongoing efforts to improve data from clinical practice and causal analytics.

Assignment-Control Plots: A Visual Companion for Causal Inference Study Design

An important step for any causal inference study design is understanding the distribution of the subjects in terms of measured baseline covariates. However, not all baseline variation is equally important. We propose a set of visualizations that reduce the space of measured covariates into two components of baseline variation important to the design of an observational causal inference study: a propensity score summarizing baseline variation associated with treatment assignment, and prognostic score summarizing baseline variation associated with the untreated potential outcome. These assignment-control plots and variations thereof visualize study design trade-offs and illustrate core methodological concepts in causal inference. As a practical demonstration, we apply assignment-control plots to a hypothetical study of cardiothoracic surgery. To demonstrate how these plots can be used to illustrate nuanced concepts, we use them to visualize unmeasured confounding and to consider the relationship between propensity scores and instrumental variables. While the family of visualization tools for studies of causality is relatively sparse, simple visual tools can be an asset to education, application, and methods development.