Treatment effect quantification for time‐to‐event endpoints–Estimands, analysis strategies, and beyond

Principles for Defining Estimands in Clinical Trials-A Proposal

The ICH E9(R1) guideline outlines the estimand framework, which aligns planning, design, conduct, analysis, and interpretation of a clinical trial. The benefits and value of using this framework in clinical trials have been outlined in the literature, and guidance has been provided on how to choose the estimand and define the estimand attributes. Although progress has been made in the implementation of estimands in clinical trials, to the best of our knowledge, there is no published discussion on the basic principles that estimands in clinical trials should fulfill to be well defined and consistent with the ideas presented in the ICH E9(R1) guideline. Therefore, in this Viewpoint article, we propose four key principles for defining an estimand. These principles form a basis for well-defined treatment effects that reflect the estimand thinking process. We hope that this Viewpoint will complement ICH E9(R1) and stimulate a discussion on which fundamental properties an estimand in a clinical trial should have and that such discussions will eventually lead to an improved clarity and precision for defining estimands in clinical trials.

Causal interpretation of the hazard ratio in randomized clinical trials

BACKGROUND

Although the hazard ratio has no straightforward causal interpretation, clinical trialists commonly use it as a measure of treatment effect.

METHODS

We review the definition and examples of causal estimands. We discuss the causal interpretation of the hazard ratio from a two-arm randomized clinical trial, and the implications of proportional hazards assumptions in the context of potential outcomes. We illustrate the application of these concepts in a synthetic model and in a model of the time-varying effects of COVID-19 vaccination.

RESULTS

We define causal estimands as having either an individual-level or population-level interpretation. Difference-in-expectation estimands are both individual-level and population-level estimands, whereas without strong untestable assumptions the causal rate ratio and hazard ratio have only population-level interpretations. We caution users against making an incorrect individual-level interpretation, emphasizing that in general a hazard ratio does not on average change each individual's hazard by a factor. We discuss a potentially valid interpretation of the constant hazard ratio as a population-level causal effect under the proportional hazards assumption.

CONCLUSION

We conclude that the population-level hazard ratio remains a useful estimand, but one must interpret it with appropriate attention to the underlying causal model. This is especially important for interpreting hazard ratios over time.

Current developments of the estimand concept

Since the introduction of the estimand in therapeutical studies, several adaptions have been developed. This short article highlights the important aspects of the estimand concept. A literature research was conducted to identify different extensions to this framework. Different modified strategies for intercurrent events are presented, as well as examples of methods to implement the estimand in clinical studies. The article reflects that the estimand is an ongoing research field with further exploration.

Tree-temporal scan statistics for safety signal detection in vaccine clinical trials

The evaluation of safety is critical in all clinical trials. However, the quantitative analysis of safety data in clinical trials poses statistical difficulties because of multiple potentially overlapping endpoints. Tree-temporal scan statistic approaches address this issue and have been widely employed in other data sources, but not to date in clinical trials. We evaluated the performance of three complementary scan statistical methods for routine quantitative safety signal detection: the self-controlled tree-temporal scan (SCTTS), a tree-temporal scan based on group comparison (BGTTS), and a log-rank based tree-temporal scan (LgRTTS). Each method was evaluated using data from two phase III clinical trials, and simulated data (simulation study). In the case study, the reference set was adverse events (AEs) in the Reference Safety Information of the evaluated vaccine. The SCTTS method had higher sensitivity than other methods, and after dose 1 detected 80 true positives (TP) with a positive predictive value (PPV) of 60%. The LgRTTS detected 49 TPs with 69% PPV. The BGTTS had 90% of PPV with 38 TPs. In the simulation study, with simulated reference sets of AEs, the SCTTS method had good sensitivity to detect transient effects. The LgRTTS method showed the best performance for the detection of persistent effects, with high sensitivity and expected probability of type I error. These three methods provide complementary approaches to safety signal detection in clinical trials or across clinical development programmes. All three methods formally adjust for multiple testing of large numbers of overlapping endpoints without being excessively conservative.

Duration of and time to response in oncology clinical trials from the perspective of the estimand framework

Duration of response (DOR) and time to response (TTR) are typically evaluated as secondary endpoints in early-stage clinical studies in oncology when efficacy is assessed by the best overall response and presented as the overall response rate. Despite common use of DOR and TTR in particular in single-arm studies, the definition of these endpoints and the questions they are intended to answer remain unclear. Motivated by the estimand framework, we present relevant scientific questions of interest for DOR and TTR and propose corresponding estimand definitions. We elaborate on how to deal with relevant intercurrent events which should follow the same considerations as implemented for the primary response estimand. A case study in mantle cell lymphoma illustrates the implementation of relevant estimands of DOR and TTR. We close the paper with practical recommendations to implement DOR and TTR in clinical study protocols.

The net benefit for time-to-event outcome in oncology clinical trials with treatment switching

BACKGROUND

The net benefit is an effect measure for any type of endpoint, including the time-to-event outcome, and can provide intuitive and clinically meaningful interpretation. It is defined as the probability of a randomly selected subject from the experimental arm surviving by at least a clinically relevant time longer than a randomly selected subject from the control arm. In oncology clinical trials, an intercurrent event such as treatment switching is common, which potentially causes informative censoring; nevertheless, conventional methods for the net benefit are not able to deal with it. In this study, we proposed a new estimator using the inverse probability of censoring weighting (IPCW) method and illustrated an oncology clinical trial with treatment switching (the SHIVA study) to apply the proposed method under the estimand framework.

METHODS

The net benefit can be estimated using the survival functions of each treatment group. The proposed estimator was based on the survival functions estimated by the inverse probability of the censoring weighting method that can handle covariate-dependent censoring. The simulation study was undertaken to evaluate the operating characteristics of the proposed estimator under several scenarios; we varied the shapes of the survival curves, treatment effect, covariates effect on censoring, proportion of the censoring, threshold of the net benefit, and sample size. We also applied conventional methods (the scoring rules by Péron or Gehan) and the proposed method to the SHIVA study.

RESULTS

Our simulation study showed that the proposed estimator provided less biased results under the covariate-dependent censoring than existing estimators. When applying the proposed method to the SHIVA study, we were able to estimate the net benefit by incorporating the information of the covariates with different estimand strategies to address the intercurrent event of the treatment switching. However, the estimates of the proposed method and those of the aforementioned conventional methods were similar under the hypothetical strategy.

CONCLUSIONS

We proposed a new estimator of the net benefit that can include covariates to account for the possibly informative censoring. We also provided an illustrative analysis of the proposed method for the oncology clinical trial with treatment switching using the estimand framework. Our proposed new estimator is suitable for handling the intercurrent events that can potentially cause covariate-dependent censoring.

Two-sample inference procedures under nonproportional hazards

We introduce a new two-sample inference procedure to assess the relative performance of two groups over time. Our model-free method does not assume proportional hazards, making it suitable for scenarios where nonproportional hazards may exist. Our procedure includes a diagnostic tau plot to identify changes in hazard timing and a formal inference procedure. The tau-based measures we develop are clinically meaningful and provide interpretable estimands to summarize the treatment effect over time. Our proposed statistic is a U-statistic and exhibits a martingale structure, allowing us to construct confidence intervals and perform hypothesis testing. Our approach is robust with respect to the censoring distribution. We also demonstrate how our method can be applied for sensitivity analysis in scenarios with missing tail information due to insufficient follow-up. Without censoring, Kendall's tau estimator we propose reduces to the Wilcoxon-Mann-Whitney statistic. We evaluate our method using simulations to compare its performance with the restricted mean survival time and log-rank statistics. We also apply our approach to data from several published oncology clinical trials where nonproportional hazards may exist.

Tell me what you want, what you really really want: Estimands in observational pharmacoepidemiologic comparative effectiveness and safety studies

PURPOSE

Ideally, the objectives of a pharmacoepidemiologic comparative effectiveness or safety study should dictate its design and data analysis. This paper discusses how defining an estimand is instrumental to this process.

METHODS

We applied the ICH-E9 (Statistical Principles for Clinical Trials) R1 addendum on estimands - which originally focused on randomized trials - to three examples of observational pharmacoepidemiologic comparative effectiveness and safety studies. Five key elements specify the estimand: the population, contrasted treatments, endpoint, intercurrent events, and population-level summary measure.

RESULTS

Different estimands were defined for case studies representing three types of pharmacological treatments: (1) single-dose treatments using a case study about the effect of influenza vaccination versus no vaccination on mortality risk in an adult population of ≥60 years of age; (2) sustained-treatments using a case study about the effect of dipeptidyl peptidase 4 inhibitor versus glucagon-like peptide-1 agonist on hypoglycemia risk in treatment of uncontrolled diabetes; and (3) as needed treatments using a case study on the effect of nitroglycerin spray as-needed versus no nitroglycerin on syncope risk in treatment of stabile angina pectoris.

CONCLUSIONS

The case studies illustrated that a seemingly clear research question can still be open to multiple interpretations. Defining an estimand ensures that the study targets a treatment effect that aligns with the treatment decision the study aims to inform. Estimand definitions further help to inform choices regarding study design and data-analysis and clarify how to interpret study findings.

Eliminating Ambiguous Treatment Effects Using Estimands

Most reported treatment effects in medical research studies are ambiguously defined, which can lead to misinterpretation of study results. This is because most authors do not attempt to describe what the treatment effect represents, and instead require readers to deduce this based on the reported statistical methods. However, this approach is challenging, because many methods provide counterintuitive results. For example, some methods include data from all patients, yet the resulting treatment effect applies only to a subset of patients, whereas other methods will exclude certain patients while results will apply to everyone. Additionally, some analyses provide estimates pertaining to hypothetical settings in which patients never die or discontinue treatment. Herein we introduce estimands as a solution to the aforementioned problem. An estimand is a clear description of what the treatment effect represents, thus saving readers the necessity of trying to infer this from study methods and potentially getting it wrong. We provide examples of how estimands can remove ambiguity from reported treatment effects and describe their current use in practice. The crux of our argument is that readers should not have to infer what investigators are estimating; they should be told explicitly.

On logistic regression with right censored data, with or without competing risks, and its use for estimating treatment effects

Simple logistic regression can be adapted to deal with right-censoring by inverse probability of censoring weighting (IPCW). We here compare two such IPCW approaches, one based on weighting the outcome, the other based on weighting the estimating equations. We study the large sample properties of the two approaches and show that which of the two weighting methods is the most efficient depends on the censoring distribution. We show by theoretical computations that the methods can be surprisingly different in realistic settings. We further show how to use the two weighting approaches for logistic regression to estimate causal treatment effects, for both observational studies and randomized clinical trials (RCT). Several estimators for observational studies are compared and we present an application to registry data. We also revisit interesting robustness properties of logistic regression in the context of RCTs, with a particular focus on the IPCW weighting. We find that these robustness properties still hold when the censoring weights are correctly specified, but not necessarily otherwise.

New late‐emphasis and combination tests based on infimum and supremum logrank statistics with application in oncology trials

Immunotherapy cancer clinical trials routinely feature an initial period during which the treatment is given without evident therapeutic benefit, which may be followed by a period during which an effective therapy reduces the hazard for event occurrence. The nature of this treatment effect is incompatible with the proportional hazards assumption, which has prompted much work on the development of alternative effect measures of frameworks for testing. We consider tests based on individual and combination of early- and late-emphasis infimum and supremum logrank statistics, describe how they can be implemented, and evaluate their performance in simulation studies. Through this work and illustrative applications we conclude that this class of test statistics offers a new and powerful framework for assessing treatment effects in cancer clinical trials involving immunotherapies.

Estimands for factorial trials

Factorial trials offer an efficient method to evaluate multiple interventions in a single trial, however the use of additional treatments can obscure research objectives, leading to inappropriate analytical methods and interpretation of results. We define a set of estimands for factorial trials, and describe a framework for applying these estimands, with the aim of clarifying trial objectives and ensuring appropriate primary and sensitivity analyses are chosen. This framework is intended for use in factorial trials where the intent is to conduct "two-trials-in-one" (ie, to separately evaluate the effects of treatments A and B), and is comprised of four steps: (i) specifying how additional treatment(s) (eg, treatment B) will be handled in the estimand, and how intercurrent events affecting the additional treatment(s) will be handled; (ii) designating the appropriate factorial estimator as the primary analysis strategy; (iii) evaluating the interaction to assess the plausibility of the assumptions underpinning the factorial estimator; and (iv) performing a sensitivity analysis using an appropriate multiarm estimator to evaluate to what extent departures from the underlying assumption of no interaction may affect results. We show that adjustment for other factors is necessary for noncollapsible effect measures (such as odds ratio), and through a trial re-analysis we find that failure to consider the estimand could lead to inappropriate interpretation of results. We conclude that careful use of the estimands framework clarifies research objectives and reduces the risk of misinterpretation of trial results, and should become a standard part of both the protocol and reporting of factorial trials.

A Review of Causal Inference for External Comparator Arm Studies

Randomized controlled trials (RCTs) are the gold standard design to establish the efficacy of new drugs and to support regulatory decision making. However, a marked increase in the submission of single-arm trials (SATs) has been observed in recent years, especially in the field of oncology due to the trend towards precision medicine contributing to the rise of new therapeutic interventions for rare diseases. SATs lack results for control patients, and information from external sources can be compiled to provide context for better interpretability of study results. External comparator arm (ECA) studies are defined as a clinical trial (most commonly a SAT) and an ECA of a comparable cohort of patients-commonly derived from real-world settings including registries, natural history studies, or medical records of routine care. This publication aims to provide a methodological overview, to sketch emergent best practice recommendations and to identify future methodological research topics. Specifically, existing scientific and regulatory guidance for ECA studies is reviewed and appropriate causal inference methods are discussed. Further topics include sample size considerations, use of estimands, handling of different data sources regarding differential baseline covariate definitions, differential endpoint measurements and timings. In addition, unique features of ECA studies are highlighted, specifically the opportunity to address bias caused by unmeasured ECA covariates, which are available in the SAT.

Re-randomisation trials in multi-episode settings: Estimands and independence estimators

Often patients may require treatment on multiple occasions. The re-randomisation design can be used in such multi-episode settings, as it allows patients to be re-enrolled and re-randomised for each new treatment episode they experience. We propose a set of estimands that can be used in multi-episode settings, focusing on issues unique to multi-episode settings, namely how each episode should be weighted, how the patient's treatment history in previous episodes should be handled, and whether episode-specific effects or average effects across all episodes should be used. We then propose independence estimators for each estimand, and show the manner in which many re-randomisation trials have been analysed in the past (a simple comparison between all intervention episodes vs. all control episodes) corresponds to a per-episode added-benefit estimand, that is, the average effect of the intervention across all episodes, over and above any benefit conferred from the intervention in previous episodes. We show this estimator is generally unbiased, and describe when other estimators will be unbiased. We conclude that (i) consideration of these estimands can help guide the choice of which analysis method is most appropriate; and (ii) the re-randomisation design with an independence estimator can be a useful approach in multi-episode settings.

Estimation of treatment effects in short-term depression studies. An evaluation based on the ICH E9(R1) estimands framework

Estimands aim to incorporate intercurrent events in design, data collection and estimation of treatment effects in clinical trials. Our aim was to understand what estimands may correspond to efficacy analyses commonly employed in clinical trials conducted before publication of ICH E9(R1). We re‐analysed six clinical trials evaluating a new anti‐depression treatment. We selected the following analysis methods—ANCOVA on complete cases, following last observation carried forward (LOCF) imputation and following multiple imputation; mixed‐models for repeated measurements without imputation (MMRM), MMRM following LOCF imputation and following jump‐to‐reference imputation; and pattern‐mixture mixed models. We included a principal stratum analysis based on the predicted subset of the study population who would not discontinue due to adverse events or lack of efficacy. We translated each analysis into the implicitly targeted estimand, and formulated corresponding clinical questions. We could map six estimands to analysis methods. The same analysis method could be mapped to more than one estimand. The major difference between estimands was the strategy for intercurrent events, with other attributes mostly the same across mapped estimands. The quantitative differences in MADRS10 population‐level summaries between the estimands were 4–8 points. Not all six estimands had a clinically meaningful interpretation. Only a few analyses would target the same estimand, hence only few could be used as sensitivity analyses. The fact that an analysis could estimate different estimands emphasises the importance of prospectively defining the estimands targeting the primary objective of a trial. The fact that an estimand can be targeted by different analyses emphasises the importance of prespecifying precisely the estimator for the targeted estimand.

Design and analysis of group-sequential clinical trials based on a modestly weighted log-rank test in anticipation of a delayed separation of survival curves: A practical guidance

BACKGROUND

A common feature of many recent trials evaluating the effects of immunotherapy on survival is that non-proportional hazards can be anticipated at the design stage. This raises the possibility to use a statistical method tailored towards testing the purported long-term benefit, rather than applying the more standard log-rank test and/or Cox model. Many such proposals have been made in recent years, but there remains a lack of practical guidance on implementation, particularly in the context of group-sequential designs. In this article, we aim to fill this gap.

METHODS

We illustrate how the POPLAR trial, which compared immunotherapy versus chemotherapy in non-small-cell lung cancer, might have been re-designed to be more robust to the presence of a delayed effect using the modestly-weighted log-rank test in a group-sequential setting.

CONCLUSION

We provide step-by-step instructions on how to analyse a hypothetical realization of the trial, based on this new design. Basic theory on weighted log-rank tests and group-sequential methods is covered, and an accompanying R package (including vignette) is provided.

Time to deterioration of patient-reported outcomes in non-small cell lung cancer: exploring different definitions

Purpose

The clinical relevance of different time-to-deterioration (TTD) definitions for patient-reported outcomes were explored.

Methods

TTD definitions differing by reference score and deterioration event were used to analyse data from the phase 3 FLAURA trial of first-line osimertinib versus erlotinib or gefitinib in patients with EGFR-mutated advanced non-small cell lung cancer. Pre-specified key symptoms were fatigue, appetite loss, cough, chest pain and dyspnoea, scored using the European Organisation for Research and Treatment of Cancer QLQ-C30 and QLQ-LC13 questionnaires (≥ 10-point difference = clinically relevant).

Results

No significant treatment differences in TTD (distributions) were observed using definitions based on transient or definitive deterioration alone. TTD definitions based on definitive, sustained deterioration, with death not included as an event, yielded a significant treatment difference for dyspnoea (hazard ratio [HR] 0.71; P = 0.034) when baseline was the reference, and for cough (HR 0.70; P = 0.009) and dyspnoea (HR 0.71; P = 0.004) when best previous score was the reference. With death included as an event, treatment differences were significant for dyspnoea (HR 0.70; P = 0.025) when baseline was the reference, and for cough (HR 0.70; P = 0.011), dyspnoea (HR 0.71; P = 0.003) and chest pain (HR 0.71; P = 0.038) when best previous score was the reference. Irrespective of definition, TTD for appetite loss and fatigue did not differ significantly between arms.

Conclusion

This exploratory work showed that different TTD definitions yield different magnitudes of treatment difference, highlighting the importance of pre-specifying TTD definitions upfront in clinical trials.

Clinical trial registration

ClinicalTrials.gov NCT02296125.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11136-022-03088-0.

Estimands in published protocols of randomised trials: urgent improvement needed

BACKGROUND

An estimand is a precise description of the treatment effect to be estimated from a trial (the question) and is distinct from the methods of statistical analysis (how the question is to be answered). The potential use of estimands to improve trial research and reporting has been underpinned by the recent publication of the ICH E9(R1) Addendum on the use of estimands in clinical trials in 2019. We set out to assess how well estimands are described in published trial protocols.

METHODS

We reviewed 50 trial protocols published in October 2020 in Trials and BMJ Open. For each protocol, we determined whether the estimand for the primary outcome was explicitly stated, not stated but inferable (i.e. could be constructed from the information given), or not inferable.

RESULTS

None of the 50 trials explicitly described the estimand for the primary outcome, and in 74% of trials, it was impossible to infer the estimand from the information included in the protocol. The population attribute of the estimand could not be inferred in 36% of trials, the treatment condition attribute in 20%, the population-level summary measure in 34%, and the handling of intercurrent events in 60% (the strategy for handling non-adherence was not inferable in 32% of protocols, and the strategy for handling mortality was not inferable in 80% of the protocols for which it was applicable). Conversely, the outcome attribute was stated for all trials. In 28% of trials, three or more of the five estimand attributes could not be inferred.

CONCLUSIONS

The description of estimands in published trial protocols is poor, and in most trials, it is impossible to understand exactly what treatment effect is being estimated. Given the utility of estimands to improve clinical research and reporting, this urgently needs to change.

Estimands for overall survival in clinical trials with treatment switching in oncology

An addendum of the ICH E9 guideline on Statistical Principles for Clinical Trials was released in November 2019 introducing the estimand framework. This new framework aims to align trial objectives and statistical analyses by requiring a precise definition of the inferential quantity of interest, that is, the estimand. This definition explicitly accounts for intercurrent events, such as switching to new anticancer therapies for the analysis of overall survival (OS), the gold standard in oncology. Traditionally, OS in confirmatory studies is analyzed using the intention-to-treat (ITT) approach comparing treatment groups as they were initially randomized regardless of whether treatment switching occurred and regardless of any subsequent therapy (treatment-policy strategy). Regulatory authorities and other stakeholders often consider ITT results as most relevant. However, the respective estimand only yields a clinically meaningful comparison of two treatment arms if subsequent therapies are already approved and reflect clinical practice. We illustrate different scenarios where subsequent therapies are not yet approved drugs and thus do not reflect clinical practice. In such situations the hypothetical strategy could be more meaningful from patient's and prescriber's perspective. The cross-industry Oncology Estimand Working Group (www.oncoestimand.org) was initiated to foster a common understanding and consistent implementation of the estimand framework in oncology clinical trials. This paper summarizes the group's recommendations for appropriate estimands in the presence of treatment switching, one of the key intercurrent events in oncology clinical trials. We also discuss how different choices of estimands may impact study design, data collection, trial conduct, analysis, and interpretation.

The detailed clinical objectives approach to designing clinical trials and choosing estimands

Objective setting is a necessary early step in the development of a clinical trial. ICH E9(R1) notes that the clinical objectives of a trial lead directly to the choice of estimands but barely discusses objectives themselves. Indeed, there is very little guidance anywhere in literature about objectives in clinical trials. This article identifies the substantial overlap between description of estimands and high quality definitions of objectives. It consequently shows that the estimand is decided by the precise choice of trial objective, and that therefore estimand decisions should be made at the objective level. The Detailed Clinical Objectives approach is proposed to support this. It emphasises clarity, specificity and a clinical focus when choosing and documenting objectives. Template text and examples are included to provide guidance on how it can be used in real trials. Finally, we describe objective-driven trial design, emphasising how strong objective setting establishes an important foundation for rigorous trial design discussions, logistical and operational decision-making during trial preparations, and clear communication of results and conclusions at the end of the trial. Highlighting the distinctions between objectives and estimands, we note how an objective-based framework can build on the ICH E9(R1) estimand framework to address many of its unanswered questions.

Semiparametric recurrent event vs time-to-first-event analyses in randomized trials: Estimands and model misspecification

Insights regarding the merits of recurrent event and time-to-first-event analyses are needed to provide guidance on strategies for analyzing intervention effects in randomized trials involving recurrent event responses. Using established asymptotic results we introduce a framework for studying the large sample properties of estimators arising from semiparametric proportional rate function models and Cox regression under model misspecification. The asymptotic biases and power implications are investigated for different data generating models, and we study the impact of dependent censoring on these findings. Illustrative applications are given involving data from a cystic fibrosis trial and a carcinogenicity experiment, following which we summarize findings and discuss implications for clinical trial design.

Sample Size Calculation When Planning Clinical Trials with Intercurrent Events

BACKGROUND

Missing data are not intercurrent events; missing data are consequences of intercurrent events. When planning clinical trials with potential missing data, a conventional approach is two stage: (1) to calculate a required sample size N without considering missing data; (2) to adjust the sample size using [Formula: see text], where r is the expected missing data rate. However, this approach is not estimand oriented.

METHODS

Clinical trial design should be aligned with estimand. Sample size calculation is a key step in clinical trial design, so methods for sample size calculation should be aligned with estimand.

RESULTS

ICH E9(R1) summarizes five strategies for dealing with intercurrent events. We consider five basic approaches for sample size calculation when planning clinical trials with intercurrent events, with each approach aligned with one of these five strategies. We extend the approaches to scenarios where some combination of multiple strategies is applied to deal with intercurrent events.

CONCLUSION

Being aligned with estimands and strategies for dealing with intercurrent events, these methods can be used for sample size calculations when planning clinical trials with intercurrent events.

Estimands in hematologic oncology trials

The estimand framework included in the addendum to the ICH E9 guideline facilitates discussions to ensure alignment between the key question of interest, the analysis, and interpretation. Therapeutic knowledge and drug mechanism play a crucial role in determining the strategy and defining the estimand for clinical trial designs. Clinical trials in patients with hematological malignancies often present unique challenges for trial design due to complexity of treatment options and existence of potential curative but highly risky procedures, for example, stem cell transplant or treatment sequence across different phases (induction, consolidation, maintenance). Here, we illustrate how to apply the estimand framework in hematological clinical trials and how the estimand framework can address potential difficulties in trial result interpretation. This paper is a result of a cross-industry collaboration to connect the International Conference on Harmonisation (ICH) E9 addendum concepts to applications. Three randomized phase 3 trials will be used to consider common challenges including intercurrent events in hematologic oncology trials to illustrate different scientific questions and the consequences of the estimand choice for trial design, data collection, analysis, and interpretation. Template language for describing estimand in both study protocols and statistical analysis plans is suggested for statisticians' reference.

Non-proportional hazards in immuno-oncology: Is an old perspective needed?

A fundamental concept in two-arm non-parametric survival analysis is the comparison of observed versus expected numbers of events on one of the treatment arms (the choice of which arm is arbitrary), where the expectation is taken assuming that the true survival curves in the two arms are identical. This concept is at the heart of the counting-process theory that provides a rigorous basis for methods such as the log-rank test. It is natural, therefore, to maintain this perspective when extending the log-rank test to deal with non-proportional hazards, for example, by considering a weighted sum of the "observed - expected" terms, where larger weights are given to time periods where the hazard ratio is expected to favor the experimental treatment. In doing so, however, one may stumble across some rather subtle issues, related to difficulties in the interpretation of hazard ratios, that may lead to strange conclusions. An alternative approach is to view non-parametric survival comparisons as permutation tests. With this perspective, one can easily improve on the efficiency of the log-rank test, while thoroughly controlling the false positive rate. In particular, for the field of immuno-oncology, where researchers often anticipate a delayed treatment effect, sample sizes could be substantially reduced without loss of power.

Statistical adaptation to oncology drug development evolution

Cancer treatment started with surgery at least three thousand years ago. Radiation therapy was added in 1896 with chemotherapy started 50 years later. These "cut, burn, and poison" techniques try to kill cancer cells directly and have been the main approaches in treating cancer until recently. In the past few years, immunotherapies have revolutionized cancer treatment. Instead of treating the disease, immunotherapies treat the patient with the disease; more precisely, correct the patient's immune system so that it can fight cancer in a long term, which makes the cure of metastatic cancers a real possibility. To adapt to the evolution of oncology treatment, clinical trial designs and statistical analysis methodologies are required to change accordingly in order to efficiently bring novel oncology medicines to cancer patients. For example, one of the major differences between immunotherapies and chemotherapies is that immunotherapies may take longer to have an effect but generally last longer with some patients cured. Trial design assumptions and adaptation rules (if adaptive design is used) need to take account of this delayed effect and long-term cure effect phenomenon. At the same time, more efficient statistical tests such as Fleming-Harrington test and Zmax test can be used to improve statistical power over the conventional logrank test for the analyses of time-to-event data that often exhibit non-proportional hazards. This article intends to describe how oncology drug development evolves over time and how statistical methods change accordingly.

Estimand framework: Are we asking the right questions? A case study in the solid tumor setting

The estimand framework requires a precise definition of the clinical question of interest (the estimand) as different ways of accounting for "intercurrent" events post randomization may result in different scientific questions. The initiation of subsequent therapy is common in oncology clinical trials and is considered an intercurrent event if the start of such therapy occurs prior to a recurrence or progression event. Three possible ways to account for this intercurrent event in the analysis are to censor at initiation, consider recurrence or progression events (including death) that occur before and after the initiation of subsequent therapy, or consider the start of subsequent therapy as an event in and of itself. The new estimand framework clarifies that these analyses address different questions ("does the drug delay recurrence if no patient had received subsequent therapy?" vs "does the drug delay recurrence with or without subsequent therapy?" vs "does the drug delay recurrence or start of subsequent therapy?"). The framework facilitates discussions during clinical trial planning and design to ensure alignment between the key question of interest, the analysis, and interpretation. This article is a result of a cross-industry collaboration to connect the International Council for Harmonisation E9 addendum concepts to applications. Data from previously reported randomized phase 3 studies in the renal cell carcinoma setting are used to consider common intercurrent events in solid tumor studies, and to illustrate different scientific questions and the consequences of the estimand choice for study design, data collection, analysis, and interpretation.

Survival analysis for AdVerse events with VarYing follow-up times (SAVVY): Rationale and statistical concept of a meta-analytic study

The assessment of safety is an important aspect of the evaluation of new therapies in clinical trials, with analyses of adverse events being an essential part of this. Standard methods for the analysis of adverse events such as the incidence proportion, that is the number of patients with a specific adverse event out of all patients in the treatment groups, do not account for both varying follow-up times and competing risks. Alternative approaches such as the Aalen-Johansen estimator of the cumulative incidence function have been suggested. Theoretical arguments and numerical evaluations support the application of these more advanced methodology, but as yet there is to our knowledge only insufficient empirical evidence whether these methods would lead to different conclusions in safety evaluations. The Survival analysis for AdVerse events with VarYing follow-up times (SAVVY) project strives to close this gap in evidence by conducting a meta-analytical study to assess the impact of the methodology on the conclusion of the safety assessment empirically. Here we present the rationale and statistical concept of the empirical study conducted as part of the SAVVY project. The statistical methods are presented in unified notation, and examples of their implementation in R and SAS are provided.

Treatment estimands in clinical trials of patients hospitalised for COVID-19: ensuring trials ask the right questions

When designing a clinical trial, explicitly defining the treatment estimands of interest (that which is to be estimated) can help to clarify trial objectives and ensure the questions being addressed by the trial are clinically meaningful. There are several challenges when defining estimands. Here, we discuss a number of these in the context of trials of treatments for patients hospitalised with COVID-19 and make suggestions for how estimands should be defined for key outcomes. We suggest that treatment effects should usually be measured as differences in proportions (or risk or odds ratios) for outcomes such as death and requirement for ventilation, and differences in means for outcomes such as the number of days ventilated. We further recommend that truncation due to death should be handled differently depending on whether a patient- or resource-focused perspective is taken; for the former, a composite approach should be used, while for the latter, a while-alive approach is preferred. Finally, we suggest that discontinuation of randomised treatment should be handled from a treatment policy perspective, where non-adherence is ignored in the analysis (i.e. intention to treat).

Dynamic RMST curves for survival analysis in clinical trials

BACKGROUND

The data from immuno-oncology (IO) therapy trials often show delayed effects, cure rate, crossing hazards, or some mixture of these phenomena. Thus, the proportional hazards (PH) assumption is often violated such that the commonly used log-rank test can be very underpowered. In these trials, the conventional hazard ratio for describing the treatment effect may not be a good estimand due to the lack of an easily understandable interpretation. To overcome this challenge, restricted mean survival time (RMST) has been strongly recommended for survival analysis in clinical literature due to its independence of the PH assumption as well as a more clinically meaningful interpretation. The RMST also aligns well with the estimand associated with the analysis from the recommendation in ICH E-9 (R1), and the test/estimation coherency. Currently, the Kaplan Meier (KM) curve is commonly applied to RMST related analyses. Due to some drawbacks of the KM approach such as the limitation in extrapolating to time points beyond the follow-up time, and the large variance at time points with small numbers of events, the RMST may be hindered.

METHODS

The dynamic RMST curve using a mixture model is proposed in this paper to fully enhance the RMST method for survival analysis in clinical trials. It is constructed that the RMST difference or ratio is computed over a range of values to the restriction time τ which traces out an evolving treatment effect profile over time.

RESULTS

This new dynamic RMST curve overcomes the drawbacks from the KM approach. The good performance of this proposal is illustrated through three real examples.

CONCLUSIONS

The RMST provides a clinically meaningful and easily interpretable measure for survival clinical trials. The proposed dynamic RMST approach provides a useful tool for assessing treatment effect over different time frames for survival clinical trials. This dynamic RMST curve also allows ones for checking whether the follow-up time for a study is long enough to demonstrate a treatment difference. The prediction feature of the dynamic RMST analysis may be used for determining an appropriate time point for an interim analysis, and the data monitoring committee (DMC) can use this evaluation tool for study recommendation.

What is an estimand & how does it relate to quantifying the effect of treatment on patient-reported quality of life outcomes in clinical trials?

BACKGROUND

Published in 2019, a new addendum to the ICH E9 guideline presents the estimand framework as a systematic approach to ensure alignment among clinical trial objectives, trial execution/conduct, statistical analyses, and interpretation of results. The use of the estimand framework for describing clinical trial objectives has yet to be extensively considered in the context of patient-reported outcomes (PROs). We discuss the application of the estimand framework to PRO objectives when designing clinical trials in the future, with a focus on PRO outcomes in oncology trial settings as our example. MAIN: We describe the components of an estimand and take a naïve PRO trial objective to illustrate how to apply attributes described in the estimand framework to inform construction of a detailed clinical trial objective and its related estimand. We discuss identifying potential post-randomization events that alter the interpretation of the endpoint or render its observation impossible (also defined as intercurrent events) in the context of PRO endpoints, and the implications of how to handle intercurrent events in the construction of the PRO objective. Using a simple objective statement, "What is the effect of treatment X on patient's quality of life?", we build up an example estimand statement and also use a previously published phase III oncology clinical trial to illustrate how an estimand for a PRO objective could have been written to align to the estimate framework.

CONCLUSION

The use of the estimand framework, as described in the new ICH E9 (R1) addendum guideline will become a key common framework for developing clinical trial objectives for evaluating effects of treatment. In the context of considering PROs, the framework provides an opportunity to more precisely specify and build the rationale for patient-focused objectives. This will help to ensure that clinical trials used for registration are designed and analysed appropriately, enabling all stakeholders to accurately interpret conclusions about the treatment effects for patient-focused outcomes.

Assessing the Impact of COVID-19 on the Clinical Trial Objective and Analysis of Oncology Clinical Trials-Application of the Estimand Framework

Abstract-Coronavirus disease 2019 (COVID-19) outbreak has rapidly evolved into a global pandemic. The impact of COVID-19 on patient journeys in oncology represents a new risk to interpretation of trial results and its broad applicability for future clinical practice. We identify key intercurrent events (ICEs) that may occur due to COVID-19 in oncology clinical trials with a focus on time-to-event endpoints and discuss considerations pertaining to the other estimand attributes introduced in the ICH E9 addendum. We propose strategies to handle COVID-19 related ICEs, depending on their relationship with malignancy and treatment and the interpretability of data after them. We argue that the clinical trial objective from a world without COVID-19 pandemic remains valid. The estimand framework provides a common language to discuss the impact of COVID-19 in a structured and transparent manner. This demonstrates that the applicability of the framework may even go beyond what it was initially intended for.

Statistical Issues and Recommendations for Clinical Trials Conducted During the COVID-19 Pandemic

Abstract-The COVID-19 pandemic has had and continues to have major impacts on planned and ongoing clinical trials. Its effects on trial data create multiple potential statistical issues. The scale of impact is unprecedented, but when viewed individually, many of the issues are well defined and feasible to address. A number of strategies and recommendations are put forward to assess and address issues related to estimands, missing data, validity and modifications of statistical analysis methods, need for additional analyses, ability to meet objectives and overall trial interpretability.

Mitigating bias from intermittent measurement of time-dependent covariates in failure time analysis

Cox regression models are routinely fitted to examine the association between time-dependent markers and a failure time when analyzing data from clinical registries. Typically, the marker values are measured periodically at clinic visits with the recorded value carried forward until the next assessment. We examine the asymptotic behavior of estimators from Cox regression models under this observation and data handling scheme when the true relationship is based on a Cox model using the current value of the marker. Specifically, we explore the impact of the marker process dynamics, the clinic visit intensity, and the marginal failure rate on the limiting value of the estimator of the marker effect from the Cox model. We also illustrate how a joint multistate model that accommodates intermittent observation of the time-varying marker can be formulated. Simulation studies demonstrate that the finite sample performance of the naive estimator aligns with the asymptotic results and shows good performance of the estimators from the joint model. We apply both methods to data from a study of bone markers and their effect on the development of skeletal complications in metastatic cancer.

The potential of the estimands framework for clinical pharmacology trials: Some discussion points

The recently finalised and published guideline ICH E9 (R1) introduced a new framework for the statistical analysis of clinical trials, namely that of "estimands". While the framework was originally developed for the analysis of late-phase trials, it could also provide a rigorous basis for the analysis of clinical pharmacology trials. We illustrate potential applications on two examples: a multiple dose pharmacology trial and the interpretation of confirmatory bioequivalence (BE) trials according to the current FDA and EMA BE guidelines.

Multistate modeling of clinical hold in randomized clinical trials

A clinical hold order by the Food and Drug Administration (FDA) to the sponsor of a clinical trial is a measure to delay a proposed or to suspend an ongoing clinical investigation. The phase III clinical trial START serves as motivating data example to explore implications and potential statistical approaches for a trial continuing after a clinical hold is lifted. In spite of a modified intention-to-treat (ITT) analysis introduced to account for the clinical hold by excluding patients potentially affected most by the clinical hold, results of the trial did not show a significant improvement of overall survival duration, and the question remains whether the negative result was an effect of the clinical hold. In this paper, we propose a multistate model incorporating the clinical hold as well as disease progression as intermediate events to investigate the impact of the clinical hold on the treatment effect. Moreover, we consider a simple counterfactual censoring approach as alternative strategy to the modified ITT analysis to deal with a clinical hold. Using a realistic simulation study informed by the START data and with a design based on our multistate model, we show that the modified ITT analysis used in the START trial was reasonable. However, the censoring approach will be shown to have some benefits in terms of power and flexibility.

Estimands and the Patient Journey: Addressing the Right Question in Oncology Clinical Trials

The diversity of patient journeys can raise fundamental questions regarding the evaluation of drug effects in clinical trials to inform clinical practice. When defining the treatment effect of interest in a trial, the researcher needs to account for events occurring after treatment initiation, such as the start of a new therapy, before observing the end point. We review the newly introduced estimand framework to structure discussions on the relationship between patient journeys and the treatment effect of interest in oncology trials. In 2017, the International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use released a draft addendum to its E9 guideline. The addendum introduces the concept of an estimand to precisely describe the treatment effect of interest. This estimand framework provides a structured approach to discuss how to account for intercurrent events that occur after random assignment and may affect the assessment or interpretation of the treatment effect. The framework is expected to improve coherence between trial objectives, design, analysis, and interpretation, as illustrated by examples in oncology disease settings. The estimand framework was applied to design a trial for a chimeric antigen receptor T-cell therapy. The treatment effect of interest was carefully defined considering the range of patient journeys expected for this particular indication and treatment. The trial design was developed accordingly to assess that treatment effect. All parties involved in the design of clinical trials need to consider possible patient journeys to define appropriate treatment effects and corresponding trial designs and analysis strategies. The estimand framework provides a common language to address the complexity introduced by varied patient journeys.