Reference-based sensitivity analysis for time-to-event data

Handling Partially Observed Trial Data After Treatment Withdrawal: Introducing Retrieved Dropout Reference-Base Centred Multiple Imputation

The ICH E9(R1) Addendum (International Council for Harmonization 2019) suggests treatment-policy as one of several strategies for addressing intercurrent events such as treatment withdrawal when defining an estimand. This strategy requires the monitoring of patients and collection of primary outcome data following termination of randomised treatment. However, when patients withdraw from a study early before completion this creates true missing data complicating the analysis. One possible way forward uses multiple imputation to replace the missing data based on a model for outcome on- and off-treatment prior to study withdrawal, often referred to as retrieved dropout multiple imputation. This article introduces a novel approach to parameterising this imputation model so that those parameters which may be difficult to estimate have mildly informative Bayesian priors applied during the imputation stage. A core reference-based model is combined with a retrieved dropout compliance model, using both on- and off-treatment data, to form an extended model for the purposes of imputation. This alleviates the problem of specifying a complex set of analysis rules to accommodate situations where parameters which influence the estimated value are not estimable, or are poorly estimated leading to unrealistically large standard errors in the resulting analysis. We refer to this new approach as retrieved dropout reference-base centred multiple imputation.

[Special issue PRO] A demonstration of estimands and sensitivity analyses for time-to-deterioration of patient reported outcomes

In oncology trials, health-related quality of life (HRQoL), specifically patient-reported symptom burden and functional status, can support the interpretation of survival endpoints, such as progression-free survival. However, applying time-to-event endpoints to patient-reported outcomes (PRO) data is challenging. For example, in time-to-deterioration analyses clinical events such as disease progression are common in many settings and are often handled through censoring the patient at the time of occurrence; however, disease progression and HRQoL are often related leading to informative censoring. Special consideration to the definition of events and intercurrent events (ICEs) is necessary. In this work, we demonstrate time-to-deterioration of PRO estimands and sensitivity analyses to answer research questions using composite, hypothetical, and treatment policy strategies applied to a single endpoint of disease-related symptoms. Multiple imputation methods under both the missing-at-random and missing-not-at-random assumptions are used as sensitivity analyses of primary estimands. Hazard ratios ranged from 0.52 to 0.66 over all the estimands and sensitivity analyses modeling a robust treatment effect favoring the treatment in time to disease symptom deterioration or death. Differences in the estimands include how people who experience disease progression or discontinue the randomized treatment due to AEs are accounted for in the analysis. We use the estimand framework to define interpretable and principled approaches for different time-to-deterioration research questions and provide practical recommendations. Reporting the proportions of patient events and patient censoring by reason helps understand the mechanisms that drive the results, allowing for optimal interpretation.

An illness-death multistate model to implement delta adjustment and reference-based imputation with time-to-event endpoints

With a treatment policy strategy, therapies are evaluated regardless of the disturbance caused by intercurrent events (ICEs). Implementing this estimand is challenging if subjects are not followed up after the ICE. This circumstance can be dealt with using delta adjustment (DA) or reference-based (RB) imputation. In the survival field, DA and RB imputation have been researched so far using multiple imputation (MI). Here, we present a fully analytical solution. We use the illness-death multistate model with the following transitions: (a) from the initial state to the event of interest, (b) from the initial state to the ICE, and (c) from the ICE to the event. We estimate the intensity function of transitions (a) and (b) using flexible parametric survival models. Transition (c) is assumed unobserved but identifiable using DA or RB imputation assumptions. Various rules have been considered: no ICE effect, DA under proportional hazards (PH) or additive hazards (AH), jump to reference (J2R), and (either PH or AH) copy increment from reference. We obtain the marginal survival curve of interest by calculating, via numerical integration, the probability of transitioning from the initial state to the event of interest regardless of having passed or not by the ICE state. We use the delta method to obtain standard errors (SEs). Finally, we quantify the performance of the proposed estimator through simulations and compare it against MI. Our analytical solution is more efficient than MI and avoids SE misestimation-a known phenomenon associated with Rubin's variance equation.

Imputation of Missing Data for Time-to-Event Endpoints Using Retrieved Dropouts

We have explored several statistical approaches to impute missing time-to-event data that arise from outcome trials with relatively long follow-up periods. Aligning with the primary estimand, such analyses evaluate the robustness of results by imposing an assumption different from censoring at random (CAR). Although there have been debates over which assumption and which method is more appropriate to be applied to the imputation, we propose to use the collection of retrieved dropouts as the basis of missing data imputation. As retrieved dropouts share a similar disposition, such as treatment discontinuation, with subjects who have missing data, they can reasonably be assumed to characterize the distribution of time-to-event among subjects with missing data. In terms of computational intensity and robustness to violation of underlying distributional assumption, we have compared parametric approaches via MCMC or MLE multivariate sampling procedures to a non-parametric bootstrap approach with respect to baseline hazard function. Each of these approaches follows a process of multiple imputation ("proper imputations"), analysis of complete datasets, and final combination. The type-I error, and power rates are examined under a wide range of scenarios to inform the performance characteristics. A subset of a real unblinded phase III CVOT is used to demonstrate the application of the proposed approaches, compared to the Cox proportional hazards model and jump-to-reference multiple imputation.

Handling intercurrent events and missing data in non-inferiority trials using the estimand framework: A tuberculosis case study

INTRODUCTION

The ICH E9 addendum outlining the estimand framework for clinical trials was published in 2019 but provides limited guidance around how to handle intercurrent events for non-inferiority studies. Once an estimand is defined, it is also unclear how to deal with missing values using principled analyses for non-inferiority studies.

METHODS

Using a tuberculosis clinical trial as a case study, we propose a primary estimand, and an additional estimand suitable for non-inferiority studies. For estimation, multiple imputation methods that align with the estimands for both primary and sensitivity analysis are proposed. We demonstrate estimation methods using the twofold fully conditional specification multiple imputation algorithm and then extend and use reference-based multiple imputation for a binary outcome to target the relevant estimands, proposing sensitivity analyses under each. We compare the results from using these multiple imputation methods with those from the original study.

RESULTS

Consistent with the ICH E9 addendum, estimands can be constructed for a non-inferiority trial which improves on the per-protocol/intention-to-treat-type analysis population previously advocated, involving respectively a hypothetical or treatment policy strategy to handle relevant intercurrent events. Results from using the 'twofold' multiple imputation approach to estimate the primary hypothetical estimand, and using reference-based methods for an additional treatment policy estimand, including sensitivity analyses to handle the missing data, were consistent with the original study's reported per-protocol and intention-to-treat analysis in failing to demonstrate non-inferiority.

CONCLUSIONS

Using carefully constructed estimands and appropriate primary and sensitivity estimators, using all the information available, results in a more principled and statistically rigorous approach to analysis. Doing so provides an accurate interpretation of the estimand.

Retrieved-Dropout-Based multiple imputation for time-to-event data in cardiovascular outcome trials

Recently, retrieved-dropout-based multiple imputation has been used in some therapeutic areas to address the treatment policy estimand, mostly for continuous endpoints. In this approach, data from subjects who discontinued study treatment but remained in study were used to construct a model for multiple imputation for the missing data of subjects in the same treatment arm who discontinued study. We extend this approach to time-to-event endpoints and provide a practical guide for its implementation. We use a cardiovascular outcome trial dataset to illustrate the method and compare the results with those from Cox proportional hazard and reference-based multiple imputation methods.

SMIM: A unified framework of survival sensitivity analysis using multiple imputation and martingale

Censored survival data are common in clinical trial studies. We propose a unified framework for sensitivity analysis to censoring at random in survival data using multiple imputation and martingale, called SMIM. The proposed framework adopts the δ-adjusted and control-based models, indexed by the sensitivity parameter, entailing censoring at random and a wide collection of censoring not at random assumptions. Also, it targets a broad class of treatment effect estimands defined as functionals of treatment-specific survival functions, taking into account missing data due to censoring. Multiple imputation facilitates the use of simple full-sample estimation; however, the standard Rubin's combining rule may overestimate the variance for inference in the sensitivity analysis framework. We decompose the multiple imputation estimator into a martingale series based on the sequential construction of the estimator and propose the wild bootstrap inference by resampling the martingale series. The new bootstrap inference has a theoretical guarantee for consistency and is computationally efficient compared to the nonparametric bootstrap counterpart. We evaluate the finite-sample performance of the proposed SMIM through simulation and an application on an HIV clinical trial.

Estimation of a treatment policy estimand for time to event data using data collected post discontinuation of randomised treatment

Discontinuation from randomised treatment is a common intercurrent event in clinical trials. When the target estimand uses a treatment policy strategy to deal with this intercurrent event, data after cessation of treatment is relevant to estimate the estimand and all efforts should be made to collect such data. Missing data may nevertheless occur due to participants withdrawing from the study and assumptions regarding the values for data that are missing are required for estimation. A missing-at-random assumption is commonly made in this setting, but it may not always be viewed as appropriate. Another potential approach is to assume missing values are similar to data collected after treatment discontinuation. This idea has been previously proposed in the context of recurrent event data. Here we extend this approach to time-to-event outcomes using the hazard function. We propose imputation models that allow for different hazard rates before and after treatment discontinuation and use the posttreatment discontinuation hazard to impute events for participants with missing follow-up periods due to study withdrawal. The imputation models are fitted as Andersen-Gill models. We illustrate the proposed methods with an example of a clinical trial in patients with chronic obstructive pulmonary disease.

Lutetium-177-PSMA-617 for Metastatic Castration-Resistant Prostate Cancer

BACKGROUND

Metastatic castration-resistant prostate cancer remains fatal despite recent advances. Prostate-specific membrane antigen (PSMA) is highly expressed in metastatic castration-resistant prostate cancer. Lutetium-177 (¹⁷⁷Lu)-PSMA-617 is a radioligand therapy that delivers beta-particle radiation to PSMA-expressing cells and the surrounding microenvironment.

METHODS

We conducted an international, open-label, phase 3 trial evaluating ¹⁷⁷Lu-PSMA-617 in patients who had metastatic castration-resistant prostate cancer previously treated with at least one androgen-receptor-pathway inhibitor and one or two taxane regimens and who had PSMA-positive gallium-68 (⁶⁸Ga)-labeled PSMA-11 positron-emission tomographic-computed tomographic scans. Patients were randomly assigned in a 2:1 ratio to receive either ¹⁷⁷Lu-PSMA-617 (7.4 GBq every 6 weeks for four to six cycles) plus protocol-permitted standard care or standard care alone. Protocol-permitted standard care excluded chemotherapy, immunotherapy, radium-223 (²²³Ra), and investigational drugs. The alternate primary end points were imaging-based progression-free survival and overall survival, which were powered for hazard ratios of 0.67 and 0.73, respectively. Key secondary end points were objective response, disease control, and time to symptomatic skeletal events. Adverse events during treatment were those occurring no more than 30 days after the last dose and before subsequent anticancer treatment.

RESULTS

From June 2018 to mid-October 2019, a total of 831 of 1179 screened patients underwent randomization. The baseline characteristics of the patients were balanced between the groups. The median follow-up was 20.9 months. ¹⁷⁷Lu-PSMA-617 plus standard care significantly prolonged, as compared with standard care, both imaging-based progression-free survival (median, 8.7 vs. 3.4 months; hazard ratio for progression or death, 0.40; 99.2% confidence interval [CI], 0.29 to 0.57; P<0.001) and overall survival (median, 15.3 vs. 11.3 months; hazard ratio for death, 0.62; 95% CI, 0.52 to 0.74; P<0.001). All the key secondary end points significantly favored ¹⁷⁷Lu-PSMA-617. The incidence of adverse events of grade 3 or above was higher with ¹⁷⁷Lu-PSMA-617 than without (52.7% vs. 38.0%), but quality of life was not adversely affected.

CONCLUSIONS

Radioligand therapy with ¹⁷⁷Lu-PSMA-617 prolonged imaging-based progression-free survival and overall survival when added to standard care in patients with advanced PSMA-positive metastatic castration-resistant prostate cancer. (Funded by Endocyte, a Novartis company; VISION ClinicalTrials.gov number, NCT03511664.).

Lutetium-177-PSMA-617 for Metastatic Castration-Resistant Prostate Cancer

BACKGROUND

Metastatic castration-resistant prostate cancer remains fatal despite recent advances. Prostate-specific membrane antigen (PSMA) is highly expressed in metastatic castration-resistant prostate cancer. Lutetium-177 (¹⁷⁷Lu)-PSMA-617 is a radioligand therapy that delivers beta-particle radiation to PSMA-expressing cells and the surrounding microenvironment.

METHODS

We conducted an international, open-label, phase 3 trial evaluating ¹⁷⁷Lu-PSMA-617 in patients who had metastatic castration-resistant prostate cancer previously treated with at least one androgen-receptor-pathway inhibitor and one or two taxane regimens and who had PSMA-positive gallium-68 (⁶⁸Ga)-labeled PSMA-11 positron-emission tomographic-computed tomographic scans. Patients were randomly assigned in a 2:1 ratio to receive either ¹⁷⁷Lu-PSMA-617 (7.4 GBq every 6 weeks for four to six cycles) plus protocol-permitted standard care or standard care alone. Protocol-permitted standard care excluded chemotherapy, immunotherapy, radium-223 (²²³Ra), and investigational drugs. The alternate primary end points were imaging-based progression-free survival and overall survival, which were powered for hazard ratios of 0.67 and 0.73, respectively. Key secondary end points were objective response, disease control, and time to symptomatic skeletal events. Adverse events during treatment were those occurring no more than 30 days after the last dose and before subsequent anticancer treatment.

RESULTS

From June 2018 to mid-October 2019, a total of 831 of 1179 screened patients underwent randomization. The baseline characteristics of the patients were balanced between the groups. The median follow-up was 20.9 months. ¹⁷⁷Lu-PSMA-617 plus standard care significantly prolonged, as compared with standard care, both imaging-based progression-free survival (median, 8.7 vs. 3.4 months; hazard ratio for progression or death, 0.40; 99.2% confidence interval [CI], 0.29 to 0.57; P<0.001) and overall survival (median, 15.3 vs. 11.3 months; hazard ratio for death, 0.62; 95% CI, 0.52 to 0.74; P<0.001). All the key secondary end points significantly favored ¹⁷⁷Lu-PSMA-617. The incidence of adverse events of grade 3 or above was higher with ¹⁷⁷Lu-PSMA-617 than without (52.7% vs. 38.0%), but quality of life was not adversely affected.

CONCLUSIONS

Radioligand therapy with ¹⁷⁷Lu-PSMA-617 prolonged imaging-based progression-free survival and overall survival when added to standard care in patients with advanced PSMA-positive metastatic castration-resistant prostate cancer. (Funded by Endocyte, a Novartis company; VISION ClinicalTrials.gov number, NCT03511664.).

Missing data: A statistical framework for practice

Missing data are ubiquitous in medical research, yet there is still uncertainty over when restricting to the complete records is likely to be acceptable, when more complex methods (e.g. maximum likelihood, multiple imputation and Bayesian methods) should be used, how they relate to each other and the role of sensitivity analysis. This article seeks to address both applied practitioners and researchers interested in a more formal explanation of some of the results. For practitioners, the framework, illustrative examples and code should equip them with a practical approach to address the issues raised by missing data (particularly using multiple imputation), alongside an overview of how the various approaches in the literature relate. In particular, we describe how multiple imputation can be readily used for sensitivity analyses, which are still infrequently performed. For those interested in more formal derivations, we give outline arguments for key results, use simple examples to show how methods relate, and references for full details. The ideas are illustrated with a cohort study, a multi-centre case control study and a randomised clinical trial.

A review of the use of controlled multiple imputation in randomised controlled trials with missing outcome data

Background

Missing data are common in randomised controlled trials (RCTs) and can bias results if not handled appropriately. A statistically valid analysis under the primary missing-data assumptions should be conducted, followed by sensitivity analysis under alternative justified assumptions to assess the robustness of results. Controlled Multiple Imputation (MI) procedures, including delta-based and reference-based approaches, have been developed for analysis under missing-not-at-random assumptions. However, it is unclear how often these methods are used, how they are reported, and what their impact is on trial results. This review evaluates the current use and reporting of MI and controlled MI in RCTs.

Methods

A targeted review of phase II-IV RCTs (non-cluster randomised) published in two leading general medical journals (The Lancet and New England Journal of Medicine) between January 2014 and December 2019 using MI. Data was extracted on imputation methods, analysis status, and reporting of results. Results of primary and sensitivity analyses for trials using controlled MI analyses were compared.

Results

A total of 118 RCTs (9% of published RCTs) used some form of MI. MI under missing-at-random was used in 110 trials; this was for primary analysis in 43/118 (36%), and in sensitivity analysis for 70/118 (59%) (3 used in both). Sixteen studies performed controlled MI (1.3% of published RCTs), either with a delta-based (n = 9) or reference-based approach (n = 7). Controlled MI was mostly used in sensitivity analysis (n = 14/16). Two trials used controlled MI for primary analysis, including one reporting no sensitivity analysis whilst the other reported similar results without imputation. Of the 14 trials using controlled MI in sensitivity analysis, 12 yielded comparable results to the primary analysis whereas 2 demonstrated contradicting results. Only 5/110 (5%) trials using missing-at-random MI and 5/16 (31%) trials using controlled MI reported complete details on MI methods.

Conclusions

Controlled MI enabled the impact of accessible contextually relevant missing data assumptions to be examined on trial results. The use of controlled MI is increasing but is still infrequent and poorly reported where used. There is a need for improved reporting on the implementation of MI analyses and choice of controlled MI parameters.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01261-6.

A four-step strategy for handling missing outcome data in randomised trials affected by a pandemic

BACKGROUND

The coronavirus pandemic (Covid-19) presents a variety of challenges for ongoing clinical trials, including an inevitably higher rate of missing outcome data, with new and non-standard reasons for missingness. International drug trial guidelines recommend trialists review plans for handling missing data in the conduct and statistical analysis, but clear recommendations are lacking.

METHODS

We present a four-step strategy for handling missing outcome data in the analysis of randomised trials that are ongoing during a pandemic. We consider handling missing data arising due to (i) participant infection, (ii) treatment disruptions and (iii) loss to follow-up. We consider both settings where treatment effects for a 'pandemic-free world' and 'world including a pandemic' are of interest.

RESULTS

In any trial, investigators should; (1) Clarify the treatment estimand of interest with respect to the occurrence of the pandemic; (2) Establish what data are missing for the chosen estimand; (3) Perform primary analysis under the most plausible missing data assumptions followed by; (4) Sensitivity analysis under alternative plausible assumptions. To obtain an estimate of the treatment effect in a 'pandemic-free world', participant data that are clinically affected by the pandemic (directly due to infection or indirectly via treatment disruptions) are not relevant and can be set to missing. For primary analysis, a missing-at-random assumption that conditions on all observed data that are expected to be associated with both the outcome and missingness may be most plausible. For the treatment effect in the 'world including a pandemic', all participant data is relevant and should be included in the analysis. For primary analysis, a missing-at-random assumption - potentially incorporating a pandemic time-period indicator and participant infection status - or a missing-not-at-random assumption with a poorer response may be most relevant, depending on the setting. In all scenarios, sensitivity analysis under credible missing-not-at-random assumptions should be used to evaluate the robustness of results. We highlight controlled multiple imputation as an accessible tool for conducting sensitivity analyses.

CONCLUSIONS

Missing data problems will be exacerbated for trials active during the Covid-19 pandemic. This four-step strategy will facilitate clear thinking about the appropriate analysis for relevant questions of interest.

Sensitivity analysis for clinical trials with missing continuous outcome data using controlled multiple imputation: A practical guide

Missing data due to loss to follow-up or intercurrent events are unintended, but unfortunately inevitable in clinical trials. Since the true values of missing data are never known, it is necessary to assess the impact of untestable and unavoidable assumptions about any unobserved data in sensitivity analysis. This tutorial provides an overview of controlled multiple imputation (MI) techniques and a practical guide to their use for sensitivity analysis of trials with missing continuous outcome data. These include δ- and reference-based MI procedures. In δ-based imputation, an offset term, δ, is typically added to the expected value of the missing data to assess the impact of unobserved participants having a worse or better response than those observed. Reference-based imputation draws imputed values with some reference to observed data in other groups of the trial, typically in other treatment arms. We illustrate the accessibility of these methods using data from a pediatric eczema trial and a chronic headache trial and provide Stata code to facilitate adoption. We discuss issues surrounding the choice of δ in δ-based sensitivity analysis. We also review the debate on variance estimation within reference-based analysis and justify the use of Rubin's variance estimator in this setting, since as we further elaborate on within, it provides information anchored inference.

Reference-based sensitivity analysis for time-to-event data

The analysis of time‐to‐event data typically makes the censoring at random assumption, ie, that—conditional on covariates in the model—the distribution of event times is the same, whether they are observed or unobserved (ie, right censored). When patients who remain in follow‐up stay on their assigned treatment, then analysis under this assumption broadly addresses the de jure, or “while on treatment strategy” estimand. In such cases, we may well wish to explore the robustness of our inference to more pragmatic, de facto or “treatment policy strategy,” assumptions about the behaviour of patients post‐censoring.

This is particularly the case when censoring occurs because patients change, or revert, to the usual (ie, reference) standard of care. Recent work has shown how such questions can be addressed for trials with continuous outcome data and longitudinal follow‐up, using reference‐based multiple imputation. For example, patients in the active arm may have their missing data imputed assuming they reverted to the control (ie, reference) intervention on withdrawal. Reference‐based imputation has two advantages: (a) it avoids the user specifying numerous parameters describing the distribution of patients' postwithdrawal data and (b) it is, to a good approximation, information anchored, so that the proportion of information lost due to missing data under the primary analysis is held constant across the sensitivity analyses. In this article, we build on recent work in the survival context, proposing a class of reference‐based assumptions appropriate for time‐to‐event data. We report a simulation study exploring the extent to which the multiple imputation estimator (using Rubin's variance formula) is information anchored in this setting and then illustrate the approach by reanalysing data from a randomized trial, which compared medical therapy with angioplasty for patients presenting with angina.