Why estimands are needed to define treatment effects in clinical trials

BACKGROUND

The estimand for a clinical trial is a precise definition of the treatment effect to be estimated. Traditionally, estimates of treatment effects are based on either an ITT analysis or a per-protocol analysis. However, there are important clinical questions which are not addressed by either of these analyses. For example, consider a trial where patients take a rescue medication. The ITT analysis includes data after use of rescue, while the per-protocol analysis excludes these patients altogether. Neither of these analyses addresses the important question of what the treatment effect would have been if patients did not take rescue medication.

MAIN TEXT

Trial estimands provide a broader perspective compared to the limitations of ITT and per-protocol analysis. Trial treatment effects depend on how events occurring after treatment initiation such as use of alternative medication or discontinuation of the intervention are included in the definition. These events can be accounted for in different ways, depending on the clinical question of interest.

CONCLUSION

The estimand framework is an important step forward in improving the clarity and transparency of clinical trials. The centrality of estimands to clinical trials is currently not reflected in methods recommended by the Cochrane group or the CONSORT statement, the current standard for reporting clinical trials in medical journals. We encourage revisions to these guidelines.

Marking 2-Years of New Thinking in Clinical Trials: The Estimand Journey

The ICH E9(R1) addendum on Estimands and Sensitivity Analyses in Clinical Trials has introduced a new estimand framework for the design, conduct, analysis, and interpretation of clinical trials. We share Pharmaceutical Industry experiences of implementing the estimand framework in the first two years since the final guidance became available with key lessons learned and highlight what else needs to be done to continue the journey in embedding the estimand framework in clinical trials. Emerging best practices and points to consider on strategies for implementing a new estimand thinking process are provided. Whilst much of the focus of implementing ICH E9(R1) to date has been on defining estimands, we highlight some of the important aspects relating to the choice of statistical analysis methods and sensitivity analyses to ensure estimands can be estimated robustly with minimal bias. In particular, we discuss the implications if complete follow-up is not possible when the treatment policy strategy is being used to handle intercurrent events. ICH E9(R1) was introduced just before the start of the COVID-19 pandemic, but a positive outcome from the pandemic has been an acceleration in the adoption of the estimand framework, including differentiating intercurrent events related or not related to the pandemic. In summary, much has been learned on the estimand journey and continued sharing of case studies will help to further advance the understanding and increase awareness across all clinical researchers of the estimand framework.

Per protocol analyses produced larger treatment effect sizes than intention to treat: a meta-epidemiological study

OBJECTIVE

To undertake meta-analysis and compare treatment effects estimated by the intention-to-treat (ITT) method and per-protocol (PP) method in randomized controlled trials (RCTs). PP excludes trial participants who are non-adherent to trial protocol in terms of eligibility, interventions, or outcome assessment.

STUDY DESIGN AND SETTING

Five high impact journals were searched for all RCTs published between July 2017 to June 2019. Primary outcome was a pooled estimate that quantified the difference between the treatment effects estimated by the two methods. Results are presented as ratio of odds ratios (ROR). Meta-regression was used to explore the association between level of trial protocol non-adherence and treatment effect. Sensitivity analyses compared results with varying within-study correlations and across various study characteristics.

RESULTS

Random-effects meta-analysis (N = 156) showed that PP estimates were on average 2% greater compared to the ITT estimates (ROR: 1.02, 95% CI: 1.00-1.04, P = 0.03). The divergence further increased with higher degree of protocol non-adherence. Sensitivity analyses reassured consistent results with various within-study correlations and across various study characteristics.

CONCLUSION

There was evidence of larger treatment effect with PP compared to ITT analysis. PP analysis should not be used to assess the impact of protocol non-adherence in RCTs. Instead, in addition to ITT, investigators should consider randomization based casual method such as Complier Average Causal Effect (CACE).

Per-protocol analysis of the ZINC trial for HIV disease among alcohol users

BACKGROUND

The Zinc for INflammation and Chronic disease in HIV (ZINC) trial randomized person who live with HIV (PLWH) who engage in heavy drinking to either daily zinc supplementation or placebo. The primary outcome was change in the Veterans Aging Cohort Study (VACS) index, a predictor of mortality, between baseline and 18 months. Because adherence and follow-up were suboptimal, the intention-to-treat analysis, which was not statistically significant, may have underestimated the effect of the zinc supplementation.

OBJECTIVE

We estimated the per-protocol effect of zinc versus placebo in the ZINC trial (i.e., the effect that would have been observed if all participants had had high adherence and none was lost to follow-up).

METHODS

Adherence was measured as the self-reported percentage of pills taken in the previous 6 weeks and assessed at all post-baseline visits. We used inverse probability weighting to estimate and compare the change in the VACS index at 18 months in the zinc and placebo groups, had all the trial participants had high adherence (i.e., cumulative adherence ≥80% at 18 months). To examine trends by level of adherence, we rerun the analyses using thresholds for high adherence of 70% and 90% of average self-reported pill coverage.

RESULTS

The estimated (95% confidence interval) change in the VACS index was - 2.16 (- 8.07, 3.59) and 5.84 (0.73, 11.80) under high adherence and no loss to follow-up in the zinc and placebo groups, respectively. The per-protocol effect estimate of the mean difference in the change between the zinc and placebo groups was - 8.01 (- 16.42, 0.01), somewhat larger than the intention-to-treat effect difference in change (- 4.68 (- 9.62, 0.25)), but it was still not statistically significant. The mean difference in the change between individuals in the zinc and placebo groups was - 4.07 (- 11.5, 2.75) and -12.34 (- 20.14, -4.14) for high adherence defined as 70% and 90% of pill coverage, respectively.

CONCLUSIONS

Overall, high adherence to zinc was associated with a lower VACS score, but confidence intervals were wide and crossed 0. Further studies with a larger sample size are needed to quantify the benefits of zinc supplementation in this population.

TRIAL REGISTRATION

ClinicalTrials.gov NCT01934803 . Registered on August 30, 2013.

The use of randomisation-based efficacy estimators in non-inferiority trials

BACKGROUND

In a non-inferiority (NI) trial, analysis based on the intention-to-treat (ITT) principle is anti-conservative, so current guidelines recommend analysing on a per-protocol (PP) population in addition. However, PP analysis relies on the often implausible assumption of no confounders. Randomisation-based efficacy estimators (RBEEs) allow for treatment non-adherence while maintaining a comparison of randomised groups. Fischer et al. have developed an approach for estimating RBEEs in randomised trials with two active treatments, a common feature of NI trials. The aim of this paper was to demonstrate the use of RBEEs in NI trials using this approach, and to appraise the feasibility of these estimators as the primary analysis in NI trials.

METHODS

Two NI trials were used. One comparing two different dosing regimens for the maintenance of remission in people with ulcerative colitis (CODA), and the other comparing an orally administered treatment to an intravenously administered treatment in preventing skeletal-related events in patients with bone metastases from breast cancer (ZICE). Variables that predicted adherence in each of the trial arms, and were also independent of outcome, were sought in each of the studies. Structural mean models (SMMs) were fitted that conditioned on these variables, and the point estimates and confidence intervals compared to that found in the corresponding ITT and PP analyses.

RESULTS

In the CODA study, no variables were found that differentially predicted treatment adherence while remaining independent of outcome. The SMM, using standard methodology, moved the point estimate closer to 0 (no difference between arms) compared to the ITT and PP analyses, but the confidence interval was still within the NI margin, indicating that the conclusions drawn would remain the same. In the ZICE study, cognitive functioning as measured by the corresponding domain of the QLQ-C30, and use of chemotherapy at baseline were both differentially associated with adherence while remaining independent of outcome. However, while the SMM again moved the point estimate closer to 0, the confidence interval was wide, overlapping with any NI margin that could be justified.

CONCLUSION

Deriving RBEEs in NI trials with two active treatments can provide a randomisation-respecting estimate of treatment efficacy that accounts for treatment adherence, is straightforward to implement, but requires thorough planning during the design stage of the study to ensure that strong baseline predictors of treatment are captured. Extension of the approach to handle nonlinear outcome variables is also required.

TRIAL REGISTRATION

The CODA study: ClinicalTrials.gov, identifier: NCT00708656 . Registered on 8 April 2008. The ZICE study trial: ClinicalTrials.gov, identifier: NCT00326820 . Registered on 16 May 2006.

Beyond intention-to-treat: The effect of brief counseling for tobacco cessation in secondary analyses of a cluster randomized controlled trial in Swedish dental clinics

In experimental studies the assigned intervention measures the received intervention if full protocol adherence is achieved, but this is rarely the case in public health. The objective of this study was to estimate the effect of a brief counseling intervention delivered in Swedish dental clinics on tobacco use cessation, taking non-adherence into account. We conducted three secondary analyses. In a per-protocol analysis the experimental counseling delivered as intended was contrasted to usual care (control). In an as-treated analysis individuals were compared according to the counseling components actually received, disregarding randomization. In an instrumental variable analysis the effect of the intervention among those who would always be treated as assigned was estimated. Logistic regression was used to examine the association between tobacco cessation outcomes (seven-day abstinence, three-month abstinence, half-reduction, quit attempts) and the defined exposure to the intervention. Protocol adherence in the intervention group was 73.4%. The per-protocol analysis closely replicated the results of the intention-to-treat analysis, showing a statistically significant effect of the brief counseling on the reduction in tobacco consumption OR = 1.81, 95% CI [1.06, 3.07], but no significant effect for other outcomes. In the as-treated analysis, receiving more counseling components compared with no tobacco counseling increased the likelihood of half-reduction. The instrumental variable yielded biased results. We conclude that despite application problems, conducting per-protocol, as-treated and instrumental variable analyses in randomized trials where experimental conditions are not strictly standardized strengthens and puts in context the inference based on intention-to-treat analysis.