The implementation of target trial emulation for causal inference: a scoping review

Methodological expectations for demonstration of health product effectiveness by observational studies

The issue of assessing the effectiveness of health technologies (drugs, devices, etc.) through observational studies is becoming increasingly important as registration and market access agencies consider them in their evaluation process. In this context, observational studies must be able to provide real demonstrations of a level of reliability comparable to those produced by the conventional randomized controlled trial (RCT) approach. The objective of the roundtable was to establish the acceptability criteria for an observational study (non-randomized, non-interventional study) to be able to provide these demonstrations, and possibly serve as a confirmatory study for registration and market access authorities, the construction of therapeutic strategies or the development of recommendations. In order to do this, the study must be a real confirmatory study respecting the hypothetical-deductive approach and guaranteeing the absence of HARKing and p-hacking by attesting to the establishment of a protocol and a statistical analysis plan, recorded before any inferential analysis. It must also be part of a formalized approach to causal inference and demonstrate that it correctly identifies the causal estimand sought. The study should ensure that there is no residual confusion bias by taking into account all confounding factors affecting the comparison, which should be determined by a formal approach (such as a graphical causality approach, DAGs). Residual confusion bias diagnoses by forgery and nullification analysis should be non-existent. The study shall be at low risk of bias, in particular selection bias, among others by using a target test emulation design. Overall type I error risk should be strictly controlled. The absence of selective publication of results and selection bias should be ensured.

Conducting observational analyses with the target trial emulation approach: a methodological systematic review

OBJECTIVES

Target trial emulation is an approach that is increasingly used to improve transparency in observational studies and help mitigate biases. For studies declaring that they emulated a target trial, we aimed to evaluate the specification of the target trial, examine its consistency with the observational emulation and assess the risk of bias in the observational analysis.

DESIGN

Methodological systematic review reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement.

DATA SOURCES

The database MEDLINE (Medical Literature Analysis and Retrieval System Online) was interrogated for all studies published from 1 January 2021 to 3 July 2022. We performed an additional manual search of 20 general medical and specialised journals that spanned the same period.

ELIGIBILITY CRITERIA

All studies that declared emulating a hypothetical or real randomised trial were eligible.

DATA EXTRACTION AND SYNTHESIS

Two independent reviewers performed the whole systematic review process (screening and selection of studies, data extraction and risk of bias assessment). The main outcomes were the definition of the key protocol components of the target trial and its emulation, consistency between the target trial and its emulation and risk of bias according to the ROBINS-I (Risk Of Bias In Non-randomised Studies - of Interventions) tool.

RESULTS

Among the selected sample of 100 studies, 24 (24%) did not specify the target trial. Only 40 studies (40%) provided detailed information on all components of the target trial protocol. Eligibility criteria, intervention strategies and outcomes were consistent between the target trial and its emulation in 35 studies (46% of those specifying the target trial). Overall, 28 studies (28%) exhibited serious risk of bias and 41 (41%) had misalignments in the timing of eligibility assessment, treatment assignment and the start of follow-up (time-zero). As compared with studies that did not specify the target trial, those that did specify the trial less frequently seemed to have both time-zero issues (39% vs 52%) and serious risk of bias (26% vs 33%).

CONCLUSIONS

One-quarter of studies declaring that they emulated a target trial did not specify the trial. Target trials and their emulations were particularly inconsistent for studies emulating a real randomised trial. Risk of methodological issues seemed lower in observational analyses that specified versus did not specify the target trial.

Association of moderate-to-vigorous physical activity with reduction of acute exacerbation in COPD patients using a dual ultra-long-acting bronchodilators

Inhaler therapy and physical activity (PA) are important methods of chronic obstructive pulmonary disease (COPD) management. This study aimed to investigate the additional benefit of moderate-to-vigorous PA (MVPA) in patients with COPD using a long-acting beta-agonists (LABA)/long-acting muscarinic antagonist (LAMA) combination. We emulated a target trial to estimate the benefit of MVPA in patients with COPD using a dual ultra-long-acting bronchodilators. We enrolled patients aged ≥ 40 who were diagnosed with COPD between 2014 and 2018, initiated a LABA/LAMA combination, and had not undergone regular MVPA. The main exposure was the initiation of MVPA, defined as vigorous aerobic exercise > 20 min per day on ≥ 3 days/week or moderate aerobic exercise > 30 min per day on ≥ 5 days/week. The main outcomes were the future usage of inhaled corticosteroids (ICS) and severe exacerbation. We identified 1,526 patients who initiated MVPA and 4,516 who did not. The median follow-up period was 3.0 years. The hazard ratio (HR) for future ICS usage in the MVPA initiation group was 0.83 (95% confidence intervals (CI): 0.72, 0.97) compared to the control group. The HR for severe exacerbation in the MVPA initiation group was 0.81 (95% CI: 0.68, 0.96) compared to the control group. Subgroup analyses by age, sex, body mass index, residence area, smoking and drinking status showed consistent benefits in these outcomes. Initiation of MVPA may offer an additional benefit for even COPD patients who use a dual ultra-long-acting bronchodilators.

Adapting Health Technology Assessment agency standards for surrogate outcomes in early stage cancer trials: what needs to happen?

INTRODUCTION

An avalanche of early stage cancer clinical trials is coming. The majority of these solely use surrogate outcomes that have not been validated against a target outcome of interest (e.g. overall survival). Current HTA guidance on surrogate outcome validation are not methodologically or practically conducive to this scenario.

AREAS COVERED

We provide a high-level overview of methods, approaches, and conceptual thinking for making better use of limited evidence within early stage cancer HTA submissions. We outline regulatory and HTA issues and emphasize how evidence transitions from one to another, what major gaps currently exist, and how these may be bridged. We summarize current methodologies and practices, their pros and cons. We outline how complementary measurements strengthen evaluations and address fallacies and biases of conventional statistical methods for surrogate outcomes validation. The value of real-world data to support some of the necessary validity components is discussed. Lastly, we address the importance of the patient voice for better understanding which surrogate outcomes may appropriately inform HTA.

EXPERT OPINION

Conventional surrogate outcome validation represents a fraught and sub-optimal framework for HTA purposes, particularly for early stage cancer. Tools for optimizing use of limited evidence exist. Education of stakeholders is highly needed.

Introduction to target trial emulation in rehabilitation: a systematic approach to emulate a randomized controlled trial using observational data

Rehabilitation providers and policymakers need valid evidence to make informed decisions about the healthcare needs of the population. Whenever possible, these decisions should be informed by randomized controlled trials (RCTs). However, there are circumstances when evidence needs to be generated rapidly, or when RCTs are not ethical or feasible. These situations apply to studying the effects of complex interventions, including rehabilitation as defined by Cochrane Rehabilitation. Therefore, we explore using the target trial emulation framework by Hernán and colleagues to obtain valid estimates of the causal effects of rehabilitation when RCTs cannot be conducted. Target trial emulation is a framework guiding the design and analysis of non-randomized comparative effectiveness studies using observational data, by emulating a hypothetical RCT. In the context of rehabilitation, we outline steps for applying the target trial emulation framework using real world data, highlighting methodological considerations, limitations, potential mitigating strategies, and causal inference and counterfactual theory as foundational principles to estimating causal effects. Overall, we aim to strengthen methodological approaches used to estimate causal effects of rehabilitation when RCTs cannot be conducted.

Development of the TrAnsparent ReportinG of observational studies Emulating a Target trial (TARGET) guideline

BACKGROUND

Observational studies are increasingly used to inform health decision-making when randomised trials are not feasible, ethical or timely. The target trial approach provides a framework to help minimise common biases in observational studies that aim to estimate the causal effect of interventions. Incomplete reporting of studies using the target trial framework limits the ability for clinicians, researchers, patients and other decision-makers to appraise, synthesise and interpret findings to inform clinical and public health practice and policy. This paper describes the methods that we will use to develop the TrAnsparent ReportinG of observational studies Emulating a Target trial (TARGET) reporting guideline.

METHODS/DESIGN

The TARGET reporting guideline will be developed in five stages following recommended guidance. The first stage will identify target trial reporting practices by systematically reviewing published studies that explicitly emulated a target trial. The second stage will identify and refine items to be considered for inclusion in the TARGET guideline by consulting content experts using sequential online surveys. The third stage will prioritise and consolidate key items to be included in the TARGET guideline at an in-person consensus meeting of TARGET investigators. The fourth stage will produce and pilot-test both the TARGET guideline and explanation and elaboration document with relevant stakeholders. The fifth stage will disseminate the TARGET guideline and resources via journals, conferences and courses.

ETHICS AND DISSEMINATION

Ethical approval for the survey has been attained (HC220536). The TARGET guideline will be disseminated widely in partnership with stakeholders to maximise adoption and improve reporting of these studies.