Target Trial Emulation for Transparent and Robust Estimation of Treatment Effects for Health Technology Assessment Using Real-World Data: Opportunities and Challenges

Overall survival of patients with CLL treated with ibrutinib in the first line compared to second-line ibrutinib after chemotherapy/chemoimmunotherapy

OBJECTIVE

To evaluate the overall survival (OS) of patients with chronic lymphocytic leukemia (CLL) receiving either ibrutinib monotherapy as a first-line (1L) treatment or chemotherapy/chemoimmunotherapy-based (CT/CIT) regimens in 1L followed by ibrutinib in the second line (1L CT/CIT-2L ibrutinib) after disease progression by emulating a randomized trial comparing both treatment sequences.

METHODS

Patient-level data from the RESONATE-2 trial (NCT01722487) and real-world PHEDRA databases were analyzed. Three scenarios were considered using the following data sources: (1) RESONATE-2, (2) combined RESONATE-2/PHEDRA, (3) combined RESONATE-2/PHEDRA for 1L ibrutinib and PHEDRA for 1L CT/CIT-2L ibrutinib. Propensity score-based weights and inverse probability of censoring weighting were used to adjust for baseline (Scenarios 2 and 3) and time-dependent confounding (all scenarios), and to address potential biases. A weighted Cox proportional hazards model was used to estimate the OS hazard ratio (HR) and 95% confidence interval (CI) for 1L ibrutinib versus 1L CT/CIT-2L ibrutinib.

RESULTS

Results from Scenario 1 showed a significantly lower risk of death with 1L ibrutinib compared with 1L chlorambucil followed by 2L ibrutinib (HR 0.35 [95% CI 0.20-0.62]). Results from Scenarios 2 and 3 demonstrated a reduced risk of death with 1L ibrutinib compared with 1L CT/CIT-2L ibrutinib (HR 0.35 [0.21-0.61] and 0.64 [0.39-1.04], respectively).

CONCLUSION

The analyses consistently showed a reduced risk of death when ibrutinib was used as a 1L treatment in CLL compared with delaying its use until 2L after CT/CIT regimens, which suggests that initiating ibrutinib in 1L is advantageous for improving survival outcomes.

Challenges and Opportunities in Interdisciplinary Research and Real-World Data for Treatment Sequences in Health Technology Assessments

With an ever-increasing number of treatment options, the assessment of treatment sequences has become crucial in health technology assessment (HTA). This review systematically explores the multifaceted challenges inherent in evaluating sequences, delving into their interplay and nuances that go beyond economic model structures. We synthesised a 'roadmap' of literature from key methodological studies, highlighting the evolution of recent advances and emerging research themes. These insights were compared against HTA guidelines to identify potential avenues for future research. Our findings reveal a spectrum of challenges in sequence evaluation, encompassing selecting appropriate decision-analytic modelling approaches and comparators, deriving appropriate clinical effectiveness evidence in the face of data scarcity, scrutinising effectiveness assumptions and statistical adjustments, considering treatment displacement, and optimising model computations. Integrating methodologies from diverse disciplines-statistics, epidemiology, causal inference, operational research and computer science-has demonstrated promise in addressing these challenges. An updated review of application studies is warranted to provide detailed insights into the extent and manner in which these methodologies have been implemented. Data scarcity on the effectiveness of treatment sequences emerged as a dominant concern, especially because treatment sequences are rarely compared in clinical trials. Real-world data (RWD) provide an alternative means for capturing evidence on effectiveness and future research should prioritise harnessing causal inference methods, particularly Target Trial Emulation, to evaluate treatment sequence effectiveness using RWD. This approach is also adaptable for analysing trials harbouring sequencing information and adjusting indirect comparisons when collating evidence from heterogeneous sources. Such investigative efforts could lend support to reviews of HTA recommendations and contribute to synthesising external control arms involving treatment sequences.

Acceptability of Using Real-World Data to Estimate Relative Treatment Effects in Health Technology Assessments: Barriers and Future Steps

OBJECTIVES

Evidence about the comparative effects of new treatments is typically collected in randomized controlled trials (RCTs). In some instances, RCTs are not possible, or their value is limited by an inability to capture treatment effects over the longer term or in all relevant population subgroups. In these cases, nonrandomized studies (NRS) using real-world data (RWD) are increasingly used to complement trial evidence on treatment effects for health technology assessment (HTA). However, there have been concerns over a lack of acceptability of this evidence by HTA agencies. This article aims to identify the barriers to the acceptance of NRS and steps that may facilitate increases in the acceptability of NRS in the future.

METHODS

Opinions of the authorship team based on their experience in real-world evidence research in academic, HTA, and industry settings, supported by a critical assessment of existing studies.

RESULTS

Barriers were identified that are applicable to key stakeholder groups, including HTA agencies (eg, the lack of comprehensive methodological guidelines for using RWD), evidence generators (eg, avoidable deviations from best practices), and external stakeholders (eg, data controllers providing timely access to high-quality RWD). Future steps that may facilitate future acceptability of NRS include improvements in the quality, integration, and accessibility of RWD, wider use of demonstration projects to highlight the value and applicability of nonrandomized designs, living, and more detailed HTA guidelines, and improvements in HTA infrastructure relating to RWD.

CONCLUSION

NRS can represent a crucial source of evidence on treatment effects for use in HTA when RCT evidence is limited.

Target trial emulation: Do antimicrobials or gastrointestinal nutraceuticals prescribed at first presentation for acute diarrhoea cause a better clinical outcome in dogs under primary veterinary care in the UK?

Target trial emulation applies design principles from randomised controlled trials to the analysis of observational data for causal inference and is increasingly used within human epidemiology. Veterinary electronic clinical records represent a potentially valuable source of information to estimate real-world causal effects for companion animal species. This study employed the target trial framework to evaluate the usefulness on veterinary observational data. Acute diarrhoea in dogs was used as a clinical exemplar. Inclusion required dogs aged ≥ 3 months and < 10 years, presenting for veterinary primary care with acute diarrhoea during 2019. Treatment strategies were: 1. antimicrobial prescription compared to no antimicrobial prescription and 2. gastrointestinal nutraceutical prescription compared to no gastrointestinal nutraceutical prescription. The primary outcome was clinical resolution (defined as no revisit with ongoing diarrhoea within 30 days from the date of first presentation). Informed from a directed acyclic graph, data on the following covariates were collected: age, breed, bodyweight, insurance status, comorbidities, vomiting, reduced appetite, haematochezia, pyrexia, duration, additional treatment prescription and veterinary group. Inverse probability of treatment weighting was used to balance covariates between the treatment groups for each of the two target trials. The risk difference (RD) of 0.4% (95% CI -4.5% to 5.3%) was non-significant for clinical resolution in dogs treated with antimicrobials compared with dogs not treated with antimicrobials. The risk difference (RD) of 0.3% (95% CI -4.5% to 5.0%) was non-significant for clinical resolution in dogs treated with gastrointestinal nutraceuticals compared with dogs not treated with gastrointestinal nutraceuticals. This study successfully applied the target trial framework to veterinary observational data. The findings show that antimicrobial or gastrointestinal prescription at first presentation of acute diarrhoea in dogs causes no difference in clinical resolution. The findings support the recommendation for veterinary professionals to limit antimicrobial use for acute diarrhoea in dogs.

Reporting of Observational Studies Explicitly Aiming to Emulate Randomized Trials: A Systematic Review

Importance

Observational (nonexperimental) studies that aim to emulate a randomized trial (ie, the target trial) are increasingly informing medical and policy decision-making, but it is unclear how these studies are reported in the literature. Consistent reporting is essential for quality appraisal, evidence synthesis, and translation of evidence to policy and practice.

Objective

To assess the reporting of observational studies that explicitly aimed to emulate a target trial.

Evidence Review

We searched Medline, Embase, PsycINFO, and Web of Science for observational studies published between March 2012 and October 2022 that explicitly aimed to emulate a target trial of a health or medical intervention. Two reviewers double-screened and -extracted data on study characteristics, key predefined components of the target trial protocol and its emulation (eligibility criteria, treatment strategies, treatment assignment, outcome[s], follow-up, causal contrast[s], and analysis plan), and other items related to the target trial emulation.

Findings

A total of 200 studies that explicitly aimed to emulate a target trial were included. These studies included 26 subfields of medicine, and 168 (84%) were published from January 2020 to October 2022. The aim to emulate a target trial was explicit in 70 study titles (35%). Forty-three studies (22%) reported use of a published reporting guideline (eg, Strengthening the Reporting of Observational Studies in Epidemiology). Eighty-five studies (43%) did not describe all key items of how the target trial was emulated and 113 (57%) did not describe the protocol of the target trial and its emulation.

Conclusion and Relevance

In this systematic review of 200 studies that explicitly aimed to emulate a target trial, reporting of how the target trial was emulated was inconsistent. A reporting guideline for studies explicitly aiming to emulate a target trial may improve the reporting of the target trial protocols and other aspects of these emulation attempts.

Emulating Target Trials With Real-World Data to Inform Health Technology Assessment: Findings and Lessons From an Application to Emergency Surgery

OBJECTIVES

International health technology assessment (HTA) agencies recommend that real-world data (RWD) are used in some circumstances to add to the evidence base about the effectiveness and cost-effectiveness of health interventions. The target trial framework applies the design principles of randomized-controlled trials to RWD and can help alleviate inevitable concerns about bias and design flaws with nonrandomized studies. This article aimed to tackle the lack of guidance and exemplar applications on how this methodology can be applied to RWD to inform HTA decision making.

METHODS

We use Hospital Episode Statistics data from England on emergency hospital admissions from 2010 to 2019 to evaluate the cost-effectiveness of emergency surgery for 2 acute gastrointestinal conditions. We draw on the case study to describe the main challenges in applying the target trial framework alongside RWD and provide recommendations for how these can be addressed in practice.

RESULTS

The 4 main challenges when applying the target trial framework to RWD are (1) defining the study population, (2) defining the treatment strategies, (3) establishing time zero (baseline), and (4) adjusting for unmeasured confounding. The recommendations for how to address these challenges, mainly around the incorporation of expert judgment and use of appropriate methods for handling unmeasured confounding, are illustrated within the case study.

CONCLUSIONS

The recommendations outlined in this study could help future studies seeking to inform HTA decision processes. These recommendations can complement checklists for economic evaluations and design tools for estimating treatment effectiveness in nonrandomized studies.

R WE ready for reimbursement? A round up of developments in real-world evidence relating to health technology assessment: part 7

This month we focus on papers that provide insights into the potential for target trial emulation to be used in health technology assessment, the role of real-world evidence (RWE) in informing additional elements of value and the importance of RWE to European joint clinical assessments.