Empirical use of causal inference methods to evaluate survival differences in a real-world registry vs those found in randomized clinical trials

Real-world data and evidence in pain research: a qualitative systematic review of methods in current practice

The use of routinely collected health data (real-world data, RWD) to generate real-world evidence (RWE) for research purposes is a growing field. Computerized search methods, large electronic databases, and the development of novel statistical methods allow for valid analysis of data outside its primary clinical purpose. Here, we systematically reviewed the methodology used for RWE studies in pain research. We searched 3 databases (PubMed, EMBASE, and Web of Science) for studies using retrospective data sources comparing multiple groups or treatments. The protocol was registered under the DOI:10.17605/OSF.IO/KGVRM. A total of 65 studies were included. Of those, only 4 compared pharmacological interventions, whereas 49 investigated differences in surgical procedures, with the remaining studying alternative or psychological interventions or epidemiological factors. Most 39 studies reported significant results in their primary comparison, and an additional 12 reported comparable effectiveness. Fifty-eight studies used propensity scores to account for group differences, 38 of them using 1:1 case:control matching. Only 17 of 65 studies provided sensitivity analyses to show robustness of their findings, and only 4 studies provided links to publicly accessible protocols. RWE is a relevant construct that can provide evidence complementary to randomized controlled trials (RCTs), especially in scenarios where RCTs are difficult to conduct. The high proportion of studies reporting significant differences between groups or comparable effectiveness could imply a relevant degree of publication bias. RWD provides a potentially important resource to expand high-quality evidence beyond clinical trials, but rigorous quality standards need to be set to maximize the validity of RWE studies.

Causal analyses with target trial emulation for real-world evidence removed large self-inflicted biases: systematic bias assessment of ovarian cancer treatment effectiveness

BACKGROUND AND OBJECTIVES

Drawing causal conclusions from real-world data (RWD) poses methodological challenges and risk of bias. We aimed to systematically assess the type and impact of potential biases that may occur when analyzing RWD using the case of progressive ovarian cancer.

METHODS

We retrospectively compared overall survival with and without second-line chemotherapy (LOT2) using electronic medical records. Potential biases were determined using directed acyclic graphs. We followed a stepwise analytic approach ranging from crude analysis and multivariable-adjusted Cox model up to a full causal analysis using a marginal structural Cox model with replicates emulating a reference randomized controlled trial (RCT). To assess biases, we compared effect estimates (hazard ratios [HRs]) of each approach to the HR of the reference trial.

RESULTS

The reference trial showed an HR for second line vs. delayed therapy of 1.01 (95% confidence interval [95% CI]: 0.82-1.25). The corresponding HRs from the RWD analysis ranged from 0.51 for simple baseline adjustments to 1.41 (95% CI: 1.22-1.64) accounting for immortal time bias with time-varying covariates. Causal trial emulation yielded an HR of 1.12 (95% CI: 0.96-1.28).

CONCLUSION

Our study, using ovarian cancer as an example, shows the importance of a thorough causal design and analysis if one is expecting RWD to emulate clinical trial results.

Trial emulation and survival analysis for disease incidence registers: A case study on the causal effect of pre-emptive kidney transplantation

When drawing causal inference from observed data, failure time outcomes present additional challenges of censoring often combined with other missing data patterns. In this article, we follow incident cases of end-stage renal disease to examine the effect on all-cause mortality of starting treatment with transplant, so-called pre-emptive kidney transplantation, vs starting with dialysis possibly followed by delayed transplantation. The question is relatively simple: which start-off treatment is expected to bring the best survival for a target population? To address it, we emulate a target trial drawing on the long term Swedish Renal Registry, where a growing common set of baseline covariates was measured nationwide. Several lessons are learned which pertain to long term disease registers more generally. With characteristics of cases and versions of treatment evolving over time, informative censoring is already introduced in unadjusted Kaplan-Meier curves. This leads to misrepresented survival chances in observed treatment groups. The resulting biased treatment association may be aggravated upon implementing IPW for treatment. Aware of additional challenges, we further recall how similar studies to date have selected patients into treatment groups based on events occurring post treatment initiation. Our study reveals the dramatic impact of resulting immortal time bias combined with other typical features of long-term incident disease registers, including missing covariates during the early phases of the register. We discuss feasible ways of accommodating these features when targeting relevant estimands, and demonstrate how more than one causal question can be answered relying on the no unmeasured baseline confounders assumption.