Emulation of a randomized controlled trial in ulcerative colitis with US and French claims data: Infliximab with thiopurines compared to infliximab monotherapy

Use of sensitivity analyses to assess uncontrolled confounding from unmeasured variables in observational, active comparator pharmacoepidemiologic studies: a systematic review

Understanding the potential for, and direction and magnitude of uncontrolled confounding is critical for generating informative real-world evidence. Many sensitivity analyses are available to assess robustness of study results to residual confounding, but it is unclear how researchers are using these methods. We conducted a systematic review of published active-comparator cohort studies of drugs or biologics to summarize use of sensitivity analyses aimed at assessing uncontrolled confounding from an unmeasured variable. We reviewed articles in 5 medical and 7 epidemiologic journals published between January 1, 2017, and June 30, 2022. We identified 158 active-comparator cohort studies: 76 from medical and 82 from epidemiologic journals. Residual, unmeasured, or uncontrolled confounding was noted as a potential concern in 93% of studies, but only 84 (53%) implemented at least 1 sensitivity analysis to assess uncontrolled confounding from an unmeasured variable. The most common analyses were E-values among medical journal articles (21%) and restriction on measured variables among epidemiologic journal articles (22%). Researchers must rigorously consider the role of residual confounding in their analyses and the best sensitivity analyses for assessing this potential bias. This article is part of a Special Collection on Pharmacoepidemiology.

Medication use evaluation of tocilizumab implementation in COVID-19 treatment guidelines: A causal inference approach

PURPOSE

Introduction of new medications to health-system formularies is often not accompanied by assessments of their clinical impact on the local patient population. The growing availability of electronic health record (EHR) data and advancements in pharmacoepidemiology methods offer institutions the opportunity to monitor the medication implementation process and assess clinical effectiveness in the local clinical context. In this study, we applied novel causal inference methods to evaluate the effects of a formulary policy introducing tocilizumab therapy for critically ill patients with coronavirus disease 2019 (COVID-19).

METHODS

We conducted a medication use evaluation utilizing EHR data from patients admitted to a large medical center during the 6 months before and after implementation of a formulary policy endorsing the use of tocilizumab for treatment of COVID-19. The impact of tocilizumab on 28-day all-cause mortality was assessed using a difference-in-differences analysis, with ineligible patients serving as a nonequivalent control group, and a matched analysis guided by a target trial emulation framework. Safety endpoints assessed included the incidence of secondary infections and liver enzyme elevations. Our findings were benchmarked against clinical trials, an observational study, and a meta-analysis.

RESULTS

Following guideline modification, tocilizumab was administered to 69% of eligible patients. This implementation was associated with a 3.1% absolute risk reduction in 28-day mortality (odds ratio, 0.86; number needed to treat to prevent one death, 32) attributable to the inclusion of tocilizumab in the guidelines and an additional 8.6% absolute risk reduction (odds ratio, 0.65; number needed to treat to prevent one death, 12) linked to its administration. These findings were consistent with estimates from published literature, although the effect estimates from the difference-in-differences analysis exhibited imprecision.

CONCLUSION

Evaluating formulary management decisions through novel causal inference approaches offers valuable estimates of clinical effectiveness and the potential to optimize the impact of new medications on population outcomes.

Pragmatic Trial Design to Compare Real-world Effectiveness of Different Treatments for Inflammatory Bowel Diseases: The PRACTICE-IBD European Consensus

BACKGROUND AND AIMS

Pragmatic studies designed to test interventions in everyday clinical settings can successfully complement the evidence from registration and explanatory clinical trials. The European consensus project PRACTICE-IBD was developed to identify essential criteria and address key methodological issues needed to design valid, comparative, pragmatic studies in inflammatory bowel diseases [BDs].

METHODS

Statements were issued by a panel of 11 European experts in IBD management and trial methodology, on four main topics: [I] study design; [II] eligibility, recruitment and organisation, flexibility; [III] outcomes; [IV] analysis. The consensus process followed a modified Delphi approach, involving two rounds of assessment and rating of the level of agreement [1 to 9; cut-off ≥7 for approval] with the statements by 18 additional European experts in IBD.

RESULTS

At the first voting round, 25 out of the 26 statements reached a mean score ≥7. Following the discussion that preceded the second round of voting, it was decided to eliminate two statements and to split one into two. At the second voting round, 25 final statements were approved: seven for study design; six for eligibility, recruitment and organisation, flexibility; eight for outcomes; and four for analysis.

CONCLUSIONS

Pragmatic, randomised, clinical trials can address important questions in IBD clinical practice, and may provide complementary, high-level evidence, as long as they follow a methodologically rigorous approach. These 25 statements intend to offer practical guidance in the design of high-quality, pragmatic, clinical trials that can aid decision making in choosing a management strategy for IBDs.

External control arm analysis: an evaluation of propensity score approaches, G-computation, and doubly debiased machine learning

BACKGROUND

An external control arm is a cohort of control patients that are collected from data external to a single-arm trial. To provide an unbiased estimation of efficacy, the clinical profiles of patients from single and external arms should be aligned, typically using propensity score approaches. There are alternative approaches to infer efficacy based on comparisons between outcomes of single-arm patients and machine-learning predictions of control patient outcomes. These methods include G-computation and Doubly Debiased Machine Learning (DDML) and their evaluation for External Control Arms (ECA) analysis is insufficient.

METHODS

We consider both numerical simulations and a trial replication procedure to evaluate the different statistical approaches: propensity score matching, Inverse Probability of Treatment Weighting (IPTW), G-computation, and DDML. The replication study relies on five type 2 diabetes randomized clinical trials granted by the Yale University Open Data Access (YODA) project. From the pool of five trials, observational experiments are artificially built by replacing a control arm from one trial by an arm originating from another trial and containing similarly-treated patients.

RESULTS

Among the different statistical approaches, numerical simulations show that DDML has the smallest bias followed by G-computation. In terms of mean squared error, G-computation usually minimizes mean squared error. Compared to other methods, DDML has varying Mean Squared Error performances that improves with increasing sample sizes. For hypothesis testing, all methods control type I error and DDML is the most conservative. G-computation is the best method in terms of statistical power, and DDML has comparable power at [Formula: see text] but inferior ones for smaller sample sizes. The replication procedure also indicates that G-computation minimizes mean squared error whereas DDML has intermediate performances in between G-computation and propensity score approaches. The confidence intervals of G-computation are the narrowest whereas confidence intervals obtained with DDML are the widest for small sample sizes, which confirms its conservative nature.

CONCLUSIONS

For external control arm analyses, methods based on outcome prediction models can reduce estimation error and increase statistical power compared to propensity score approaches.

Decreased risk of treatment failure with vedolizumab and thiopurines combined compared with vedolizumab monotherapy in Crohn's disease

OBJECTIVE

While infliximab combined to thiopurines is more effective than infliximab monotherapy in patients with Crohn's disease (CD) and UC, the impact of adding thiopurines to vedolizumab remains controversial. We emulated two target trials comparing the effectiveness of combination therapy versus vedolizumab monotherapy in CD and UC.

DESIGN

Based on two US and the French nationwide healthcare databases, patients with CD and UC who initiated vedolizumab were identified. The study methodology, including confounding adjustment and outcome definitions, were previously validated in successful emulations of the SONIC and SUCCESS trials. Risk ratios for treatment failure based on hospitalisation or surgery related to disease activity, treatment switch, or prolonged corticosteroids use, were estimated after 1:1 propensity score (PS) matching.

RESULTS

Among a total of 10 299 vedolizumab users, 804 CD and 1088 UC pairs of combination therapy versus vedolizumab monotherapy users were PS matched. Treatment failure occurred at week 26 in 236 (29.3%) and 376 (34.3%) patients with CD and at week 16 in 236 (21.7%) and 263 (24.2%) patients with UC initiating combination therapy and vedolizumab monotherapy, respectively. The risk of treatment failure was decreased with combination therapy compared with vedolizumab monotherapy in CD (RR 0.85, 95% CI: 0.74 to 0.98) and to a lesser extent in UC (RR 0.90, 95% CI: 0.77 to 1.05). Findings were consistent across databases.

CONCLUSION

Using validated methodologies, combination therapy with vedolizumab and thiopurines was associated with lower treatment failure compared with vedolizumab monotherapy in CD but not UC across the USA and France.