A tutorial on principal stratification-based sensitivity analysis: application to smoking cessation studies

Baseline Characteristics and Maintenance Therapy Choice on Symptom Control, Reliever Use, Exacerbation Risk in Moderate-Severe Asthma: A Clinical Modelling and Simulation Study

INTRODUCTION

Although some factors associated with asthma symptom deterioration and risk of exacerbation have been identified, these are not yet fully characterised. We conducted a clinical modelling and simulation study to understand baseline factors affecting symptom control, reliever use and exacerbation risk in patients with moderate-severe asthma during follow-up on regularly dosed inhaled corticosteroid (ICS) monotherapy, or ICS/long-acting beta2-agonist (LABA) combination therapy.

METHODS

Individual patient data from randomised clinical trials (undertaken between 2001 and 2019) were used to model the time course of symptoms (n = 7593), patterns of reliever medication use (n = 3768) and time-to-first exacerbation (n = 6763), considering patient-specific and extrinsic factors, including treatment. Model validation used standard graphical and statistical criteria. Change in symptom control scores (Asthma Control Questionnaire 5 [ACQ-5]), reduction in reliever use and annualised exacerbation rate were then simulated in patient cohorts with different baseline characteristics and treatment settings.

RESULTS

Being a smoker, having higher baseline ACQ-5 and body mass index affected symptom control scores, reliever use and exacerbation risk (p < 0.01). In addition, low forced expiratory volume in 1 s percent predicted, female sex, season and previous exacerbations were found to contribute to a further increase in exacerbation risk (p < 0.01), whereas long asthma history was associated with more frequent reliever use (p < 0.01). These effects were independent from the underlying maintenance therapy. In different scenarios, fluticasone furoate (FF)/vilanterol was associated with greater reductions in reliever use and exacerbation rates compared with FF or fluticasone propionate (FP) alone or budesonide/formoterol, independently from other factors (p < 0.01).

CONCLUSIONS

This study provided further insight into the effects of individual baseline characteristics on treatment response and highlighted significant differences in the performance of ICS/LABA combination therapy on symptom control, reliever use and exacerbation risk. These factors should be incorporated into clinical practice as the basis for tailored management of patients with moderate-severe asthma.

A Simulation Study of the Effect of Clinical Characteristics and Treatment Choice on Reliever Medication Use, Symptom Control and Exacerbation Risk in Moderate-Severe Asthma

INTRODUCTION

The relationship between immediate symptom control, reliever medication use and exacerbation risk on treatment response and factors that modify it have not been assessed in an integrated manner. Here we apply simulation scenarios to evaluate the effect of individual baseline characteristics on treatment response in patients with moderate-severe asthma on regular maintenance dosing monotherapy with fluticasone propionate (FP) or combination therapy with fluticasone propionate/salmeterol (FP/SAL) or budesonide/formoterol (BUD/FOR).

METHODS

Reduction in reliever medication use (puffs/24 h), change in symptom control scores (ACQ-5), and annualised exacerbation rate over 12 months were simulated in a cohort of patients with different baseline characteristics (e.g. time since diagnosis, asthma control questionnaire (ACQ-5) symptom score, smoking status, body mass index (BMI) and sex) using drug-disease models derived from large phase III/IV clinical studies.

RESULTS

Simulation scenarios show that being a smoker, having higher baseline ACQ-5 and BMI, and long asthma history is associated with increased reliever medication use (p < 0.01). This increase correlates with a higher exacerbation risk and higher ACQ-5 scores over the course of treatment, irrespective of the underlying maintenance therapy. Switching non-responders to ICS monotherapy to combination therapy after 3 months resulted in immediate reduction in reliever medication use (i.e. 1.3 vs. 1.0 puffs/24 h for FP/SAL and BUD/FOR, respectively). In addition, switching patients with ACQ-5 > 1.5 at baseline to FP/SAL resulted in 34% less exacerbations than those receiving regular dosing BUD/FOR (p < 0.01).

CONCLUSIONS

We have identified baseline characteristics of patients with moderate to severe asthma that are associated with greater reliever medication use, poor symptom control and higher exacerbation risk. Moreover, the effects of different inhaled corticosteroid (ICS)/long-acting beta agonist (LABA) combinations vary significantly when considering long-term treatment performance. These factors should be considered in clinical practice as a basis for personalised management of patients with moderate-severe asthma symptoms.

Using principal stratification in analysis of clinical trials

The ICH E9(R1) addendum (2019) proposed principal stratification (PS) as one of five strategies for dealing with intercurrent events. Therefore, understanding the strengths, limitations, and assumptions of PS is important for the broad community of clinical trialists. Many approaches have been developed under the general framework of PS in different areas of research, including experimental and observational studies. These diverse applications have utilized a diverse set of tools and assumptions. Thus, need exists to present these approaches in a unifying manner. The goal of this tutorial is threefold. First, we provide a coherent and unifying description of PS. Second, we emphasize that estimation of effects within PS relies on strong assumptions and we thoroughly examine the consequences of these assumptions to understand in which situations certain assumptions are reasonable. Finally, we provide an overview of a variety of key methods for PS analysis and use a real clinical trial example to illustrate them. Examples of code for implementation of some of these approaches are given in Supplemental Materials.

Incorporating partial adherence into the principal stratification analysis framework

Participants in pragmatic clinical trials often partially adhere to treatment. However, to simplify the analysis, most studies dichotomize adherence (supposing that subjects received either full or no treatment), which can introduce biases in the results. For example, the popular approach of principal stratification is based on the concept that the population can be separated into strata based on how they will react to treatment assignment, but this framework does not include strata in which a partially adhering participant would belong. We expanded the principal stratification framework to allow partial adherers to have their own principal stratum and treatment level. The expanded approach is feasible in pragmatic settings. We have designed a Monte Carlo posterior sampling method to obtain the relevant parameter estimates. Simulations were completed under a range of settings where participants partially adhered to treatment, including a hypothetical setting from a published simulation trial on the topic of partial adherence. The inference method is additionally applied to data from a real randomized clinical trial that features partial adherence. Comparison of the simulation results indicated that our method is superior in most cases to the biased estimators obtained through standard principal stratification. Simulation results further suggest that our proposed method may lead to increased accuracy of inference in settings where study participants only partially adhere to assigned treatment.

Latent Class Survival Models Linked by Principal Stratification to Investigate Heterogenous Survival Subgroups Among Individuals With Early-Stage Kidney Cancer

Rates of kidney cancer have been increasing, with small incidental tumors experiencing the fastest growth rates. Much of the increase could be due to increased use of CT scans, MRIs, and ultrasounds for unrelated conditions. Many tumors might never have been detected or become symptomatic in the past. This suggests that many patients might benefit from less aggressive therapy, such as active surveillance by which tumors are surgically removed only if they become sufficiently large. However, it has been difficult for clinicians to identify subgroups of patients for whom treatment might be especially beneficial or harmful. In this work, we use a principal stratification framework to estimate the proportion and characteristics of individuals who have large or small hazard rates of death in two treatment arms. This allows us to assess who might be helped or harmed by aggressive treatment. We also use Weibull mixture models. This work differs from much previous work in that the survival classes upon which principal stratification is based are latent variables. That is, survival class is not an observed variable. We apply this work using Surveillance Epidemiology and End Results-Medicare claims data. Clinicians can use our methods for investigating treatments with heterogeneous effects.

Simple Estimation of Patient-Oriented Effects From Randomized Trials: An Open and Shut CACE

In randomized controlled trials, the intention-to-treat estimator provides an unbiased estimate of the causal effect of treatment assignment on the outcome. However, patients often want to know what the effect would be if they were to take the treatment as prescribed (the patient-oriented effect), and several researchers have suggested that the more relevant causal effect for this question is the complier average causal effect (CACE), also referred to as the local average treatment effect. Sophisticated approaches to estimating the CACE include Bayesian and frequentist methods for principal stratification, inverse-probability-of-treatment-weighted estimators, and instrumental-variable (IV) analysis. All of these approaches exploit information about adherence to assigned treatment to improve upon the intention-to-treat estimator, but they are rarely used in practice, probably because of their complexity. The IV principal stratification estimator is simple to implement but has had limited use in practice, possibly due to lack of familiarity. Here, we show that the IV principal stratification estimator is a modified per-protocol estimator that should be obtainable from any randomized controlled trial, and we provide a closed form for its robust variance (and its uncertainty). Finally, we illustrate sensitivity analyses we conducted to assess inference in light of potential violations of the exclusion restriction assumption.

Beyond intention to treat: what is the right question?

BACKGROUND

Most methodologists recommend intention-to-treat (ITT) analysis in order to minimize bias. Although ITT analysis provides an unbiased estimate for the effect of treatment assignment on the outcome, the estimate is biased for the actual effect of receiving treatment (active treatment) compared to some comparison group (control). Other common analyses include measuring effects in (1) participants who follow their assigned treatment (Per Protocol), (2) participants according to treatment received (As Treated), and (3) those who would comply with recommended treatment (Complier Average Causal Effect (CACE) as estimated by Principal Stratification or Instrumental Variable Analyses). As each of these analyses compares different study subpopulations, they address different research questions.

PURPOSE

For each type of analysis, we review and explain (1) the terminology being used, (2) the main underlying concepts, (3) the questions that are answered and whether the method provides valid causal estimates, and (4) the situations when the analysis should be conducted.

METHODS

We first review the major concepts in relation to four nuances of the clinical question, 'Does treatment improve health?' After reviewing these concepts, we compare the results of the different analyses using data from two published randomized controlled trials (RCTs). Each analysis has particular underlying assumptions and all require dichotomizing adherence into Yes or No. We apply sensitivity analyses so that intermediate adherence is considered (1) as adherence and (2) as non-adherence.

RESULTS

The ITT approach provides an unbiased estimate for how active treatment will improve (1) health in the population if a policy or program is enacted or (2) health of patients if a clinician changes treatment practice. The CACE approach generally provides an unbiased estimate of the effect of active treatment on health of patients who would follow the clinician's advice to take active treatment. Unfortunately, there is no current analysis for clinicians and patients who want to know whether active treatment will improve the patient's health if taken, which is different from the effect in patients who would follow the clinician's advice to take active treatment. Sensitivity analysis for the CACE using two published data sets suggests that the underlying assumptions appeared to be violated.

LIMITATIONS

There are several methods within each analytical approach we describe. Our analyses are based on a subset of these approaches.

CONCLUSIONS

Although adherence-based analyses may provide meaningful information, the analytical method should match the clinical question, and investigators should clearly outline why they believe assumptions hold and should provide empirical tests of the assumptions where possible.

Response to Pearl's comments on principal stratification

Dr. Pearl invites researchers to justify their use of principal stratification. This comment explains how the use of principal stratification simplified a complex mediational problem encountered when evaluating a smoking cessation intervention's effect on reducing smoking withdrawal symptoms.