Calculating the Expected Net Benefit of Sampling for Survival Data: A Tutorial and Case Study

HIGHLIGHTS

The net value of reducing decision uncertainty by collecting additional data is quantified by the expected net benefit of sampling (ENBS). This tutorial presents a general-purpose algorithm for computing the ENBS for collecting survival data along with a step-by-step implementation in R.The algorithm is based on recently published methods for simulating survival data and computing expected value of sample information that do not rely on the survival data to follow any particular parametric distribution and that can take into account any arbitrary censoring process.We demonstrate in a case study based on a previous cancer technology appraisal that ENBS calculations are useful not only for designing new studies but also for optimizing reimbursement decisions for new health technologies based on immature evidence from ongoing trials.

Informing Structural Assumptions for Three State Oncology Cost-Effectiveness Models through Model Efficiency and Fit

The characteristics and relative strengths and weaknesses of partitioned survival models (PSMs) and state transition models (STMs) for three state oncology cost-effectiveness models have previously been studied. Despite clear and longstanding economic modeling guidelines, more than one structure is rarely presented, and the choice of structure appears correlated more with audience or precedent than disease, decision problem, or available data. One reason may be a lack of guidance and tools available to readily compare measures of internal validity such as the model fit and efficiency of different structures, or sensitivity of results to those choices. To address this gap, methods are presented to evaluate the fit and efficiency of three structures, with an accompanying R software package, psm3mkv. The methods are illustrated by analyzing interim and final analysis datasets of the KEYNOTE-826 randomized controlled trial. At both interim and final analyses, the STM Clock Reset structure provided the best and most efficient fit. Structural uncertainties had been reduced from interim to final analysis. Beyond measures of internal validity, guidelines highlight the importance of reflecting all available data, avoiding model selection purely on the basis of goodness of fit and strongly considering external validity. The method and software allow modelers to more easily evaluate and report model fit and efficiency, examine implicit assumptions, and reveal sensitivities to structural choices.

A multistate platform model for time-to-event endpoints in oncology clinical trials

A multistate platform model was developed to describe time-to-event (TTE) endpoints in an oncology trial through the following states: initial, tumor response (TR), progressive disease (PD), overall survival (OS) event (death), censor to the last evaluable tumor assessment (progression-free survival [PFS] censor), and censor to study end (OS censor), using an ordinary differential equation framework. Two types of piecewise functions were used to describe the hazards for different events. Piecewise surge functions were used for events that require tumor assessments at the scheduled study visit times (TR, PD, and PFS censor). Piecewise constant functions were used to describe hazards for events that occur evenly throughout the study (OS event and OS censor). The multistate TTE model was applied to describe TTE endpoints from a published phase III study. The piecewise surge functions well-described the observed surges of hazards/events for TR, PD, PFS, and OS occurring near scheduled tumor assessments and showed good agreement with all Kaplan-Meier curves. With the flexibility of piecewise hazard functions, the model was able to evaluate covariate effects in a time-variant fashion to better understand the temporal patterns of disease prognosis through different disease states. This model can be applied to advance the field of oncology trial design and optimization by: (1) enabling robust estimations of baseline hazards and covariate effects for multiple TTE endpoints, (2) providing a platform model for understanding the composition and correlations between different TTE endpoints, and (3) facilitating oncology trial design optimization through clinical trial simulations.

Spearman-like correlation measure adjusting for covariates in bivariate survival data

We propose an extension of Spearman's correlation for censored continuous and discrete data that permits covariate adjustment. Previously proposed nonparametric and semiparametric Spearman's correlation estimators require either nonparametric estimation of the bivariate survival surface or parametric assumptions about the dependence structure. In practice, nonparametric estimation of the bivariate survival surface is difficult, and parametric assumptions about the correlation structure may not be satisfied. Therefore, we propose a method that requires neither and uses only the marginal survival distributions. Our method estimates the correlation of probability-scale residuals, which has been shown to equal Spearman's correlation when there is no censoring. Because this method relies only on marginal distributions, it tends to be less variable than the previously suggested nonparametric estimators, and the confidence intervals are easily constructed. Although under censoring, it is biased for Spearman's correlation as our simulations show, it performs well under moderate censoring, with a smaller mean squared error than nonparametric approaches. We also extend it to partial (adjusted), conditional, and partial-conditional correlation, which makes it particularly relevant for practical applications. We apply our method to estimate the correlation between time to viral failure and time to regimen change in a multisite cohort of persons living with HIV in Latin America.

Multilevel joint frailty model for hierarchically clustered binary and survival data

Hierarchical data arise when observations are clustered into groups. Multilevel models are practically useful in these settings, but these models are elusive in the context of hierarchical data with mixed multivariate outcomes. In this article, we consider binary and survival outcomes and assume the hierarchical structure is induced by clustering of both outcomes within patients and clustering of patients within hospitals which frequently occur in multicenter studies. We introduce a multilevel joint frailty model that analyzes the outcomes simultaneously to jointly estimate their regression parameters and explicitly model within-patient correlation between the outcomes and within-hospital correlation separately for each outcome. Estimation is facilitated by a computationally efficient residual maximum likelihood method that further predicts cluster-specific frailties for both outcomes and circumvents the formidable challenges induced by multidimensional integration that complicates the underlying likelihood. The performance of the model and estimation procedure is investigated via extensive simulation studies. The practical utility of the model is illustrated through simultaneous modeling of disease-free survival and binary endpoint of platelet recovery in a multicenter allogeneic bone marrow transplantation dataset that motivates this study.

General-Purpose Methods for Simulating Survival Data for Expected Value of Sample Information Calculations

BACKGROUND

Expected value of sample information (EVSI) quantifies the expected value to a decision maker of reducing uncertainty by collecting additional data. EVSI calculations require simulating plausible data sets, typically achieved by evaluating quantile functions at random uniform numbers using standard inverse transform sampling (ITS). This is straightforward when closed-form expressions for the quantile function are available, such as for standard parametric survival models, but these are often unavailable when assuming treatment effect waning and for flexible survival models. In these circumstances, the standard ITS method could be implemented by numerically evaluating the quantile functions at each iteration in a probabilistic analysis, but this greatly increases the computational burden. Thus, our study aims to develop general-purpose methods that standardize and reduce the computational burden of the EVSI data-simulation step for survival data.

METHODS

We developed a discrete sampling method and an interpolated ITS method for simulating survival data from a probabilistic sample of survival probabilities over discrete time units. We compared the general-purpose and standard ITS methods using an illustrative partitioned survival model with and without adjustment for treatment effect waning.

RESULTS

The discrete sampling and interpolated ITS methods agree closely with the standard ITS method, with the added benefit of a greatly reduced computational cost in the scenario with adjustment for treatment effect waning.

CONCLUSIONS

We present general-purpose methods for simulating survival data from a probabilistic sample of survival probabilities that greatly reduce the computational burden of the EVSI data-simulation step when we assume treatment effect waning or use flexible survival models. The implementation of our data-simulation methods is identical across all possible survival models and can easily be automated from standard probabilistic decision analyses.

HIGHLIGHTS

Expected value of sample information (EVSI) quantifies the expected value to a decision maker of reducing uncertainty through a given data collection exercise, such as a randomized clinical trial. In this article, we address the problem of computing EVSI when we assume treatment effect waning or use flexible survival models, by developing general-purpose methods that standardize and reduce the computational burden of the EVSI data-generation step for survival data.We developed 2 methods for simulating survival data from a probabilistic sample of survival probabilities over discrete time units, a discrete sampling method and an interpolated inverse transform sampling method, which can be combined with a recently proposed nonparametric EVSI method to accurately estimate EVSI for collecting survival data.Our general-purpose data-simulation methods greatly reduce the computational burden of the EVSI data-simulation step when we assume treatment effect waning or use flexible survival models. The implementation of our data-simulation methods is identical across all possible survival models and can therefore easily be automated from standard probabilistic decision analyses.

A connection between survival multistate models and causal inference for external treatment interruptions

Recently, treatment interruptions such as a clinical hold in randomized clinical trials have been investigated by using a multistate model approach. The phase III clinical trial START (Stimulating Targeted Antigenic Response To non-small-cell cancer) with primary endpoint overall survival was temporarily placed on hold for enrollment and treatment by the US Food and Drug Administration (FDA). Multistate models provide a flexible framework to account for treatment interruptions induced by a time-dependent external covariate. Extending previous work, we propose a censoring and a filtering approach both aimed at estimating the initial treatment effect on overall survival in the hypothetical situation of no clinical hold. A special focus is on creating a link to causal inference. We show that calculating the matrix of transition probabilities in the multistate model after application of censoring (or filtering) yields the desired causal interpretation. Assumptions in support of the identification of a causal effect by censoring (or filtering) are discussed. Thus, we provide the basis to apply causal censoring (or filtering) in more general settings such as the COVID-19 pandemic. A simulation study demonstrates that both causal censoring and filtering perform favorably compared to a naïve method ignoring the external impact.

The heterogeneity effect of surveillance intervals on progression free survival

Progression-free survival (PFS) is an increasingly important surrogate endpoint in cancer clinical trials. However, the true time of progression is typically unknown if the evaluation of progression status is only scheduled at given surveillance intervals. In addition, comparison between treatment arms under different surveillance schema is not uncommon. Our aim is to explore whether the heterogeneity of the surveillance intervals may interfere with the validity of the conclusion of efficacy based on PFS, and the extent to which the variation would bias the results. We conduct comprehensive simulation studies to explore the aforementioned goals in a two-arm randomized control trial. We introduce three steps to simulate survival data with predefined surveillance intervals under different censoring rate considerations. We report the estimated hazard ratios and examine false positive rate, power and bias under different surveillance intervals, given different baseline median PFS, hazard ratio and censoring rate settings. Results show that larger heterogeneous lengths of surveillance intervals lead to higher false positive rate and overestimate the power, and the effect of the heterogeneous surveillance intervals may depend upon both the life expectancy of the tumor prognoses and the censoring proportion of the survival data. We also demonstrate such heterogeneity effect of surveillance intervals on PFS in a phase III metastatic colorectal cancer trial. In our opinions, adherence to consistent surveillance intervals should be favored in designing the comparative trials. Otherwise, it needs to be appropriately taken into account when analyzing data.

Design aspects of COVID-19 treatment trials: Improving probability and time of favorable events

As a reaction to the pandemic of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), a multitude of clinical trials for the treatment of SARS-CoV-2 or the resulting corona disease 2019 (COVID-19) are globally at various stages from planning to completion. Although some attempts were made to standardize study designs, this was hindered by the ferocity of the pandemic and the need to set up clinical trials quickly. We take the view that a successful treatment of COVID-19 patients (i) increases the probability of a recovery or improvement within a certain time interval, say 28 days; (ii) aims to expedite favorable events within this time frame; and (iii) does not increase mortality over this time period. On this background, we discuss the choice of endpoint and its analysis. Furthermore, we consider consequences of this choice for other design aspects including sample size and power and provide some guidance on the application of adaptive designs in this particular context.

Confirmatory adaptive group sequential designs for single-arm phase II studies with multiple time-to-event endpoints

Existing methods concerning the assessment of long-term survival outcomes in one-armed trials are commonly restricted to one primary endpoint. Corresponding adaptive designs suffer from limitations regarding the use of information from other endpoints in interim design changes. Here we provide adaptive group sequential one-sample tests for testing hypotheses on the multivariate survival distribution derived from multi-state models, while making provision for data-dependent design modifications based on all involved time-to-event endpoints. We explicitly elaborate application of the methodology to one-sample tests for the joint distribution of (i) progression-free survival (PFS) and overall survival (OS) in the context of an illness-death model, and (ii) time to toxicity and time to progression while accounting for death as a competing event. Large sample distributions are derived using a counting process approach. Small sample properties are studied by simulation. An already established multi-state model for non-small cell lung cancer is used to illustrate the adaptive procedure.

Conditional copula models for correlated survival endpoints: Individual patient data meta-analysis of randomized controlled trials

Correlations among survival endpoints are important for exploring surrogate endpoints of the true endpoint. With a valid surrogate endpoint tightly correlated with the true endpoint, the efficacy of a new drug/treatment can be measurable on it. However, the existing methods for measuring correlation between two endpoints impose an invalid assumption: correlation structure is constant across different treatment arms. In this article, we reconsider the definition of Kendall's concordance measure (tau) in the context of individual patient data meta-analyses of randomized controlled trials. According to our new definition of Kendall's tau, its value depends on the treatment arms. We then suggest extending the existing copula (and frailty) models so that their Kendall's tau can vary across treatment arms. Our newly proposed model, a joint frailty-conditional copula model, is the implementation of the new definition of Kendall's tau in meta-analyses. In order to facilitate our approach, we develop an original R function condCox.reg(.) and make it available in the R package joint.Cox (https://CRAN.R-project.org/package=joint.Cox). We apply the proposed method to a gastric cancer dataset (3288 patients in 14 randomized trials from the GASTRIC group). This data analysis concludes that Kendall's tau has different values between the surgical treatment arm and the adjuvant chemotherapy arm (p-value<0.001), whereas disease-free survival remains a valid surrogate at individual level for overall survival in these trials.

A unified approach to power and sample size determination for log-rank tests under proportional and nonproportional hazards

Log-rank tests have been widely used to compare two survival curves in biomedical research. We describe a unified approach to power and sample size calculation for the unweighted and weighted log-rank tests in superiority, noninferiority and equivalence trials. It is suitable for both time-driven and event-driven trials. A numerical algorithm is suggested. It allows flexible specification of the patient accrual distribution, baseline hazards, and proportional or nonproportional hazards patterns, and enables efficient sample size calculation when there are a range of choices for the patient accrual pattern and trial duration. A confidence interval method is proposed for the trial duration of an event-driven trial. We point out potential issues with several popular sample size formulae. Under proportional hazards, the power of a survival trial is commonly believed to be determined by the number of observed events. The belief is roughly valid for noninferiority and equivalence trials with similar survival and censoring distributions between two groups, and for superiority trials with balanced group sizes. In unbalanced superiority trials, the power depends also on other factors such as data maturity. Surprisingly, the log-rank test usually yields slightly higher power than the Wald test from the Cox model under proportional hazards in simulations. We consider various nonproportional hazards patterns induced by delayed effects, cure fractions, and/or treatment switching. Explicit power formulae are derived for the combination test that takes the maximum of two or more weighted log-rank tests to handle uncertain nonproportional hazards patterns. Numerical examples are presented for illustration.

Multistate modeling of clinical hold in randomized clinical trials

A clinical hold order by the Food and Drug Administration (FDA) to the sponsor of a clinical trial is a measure to delay a proposed or to suspend an ongoing clinical investigation. The phase III clinical trial START serves as motivating data example to explore implications and potential statistical approaches for a trial continuing after a clinical hold is lifted. In spite of a modified intention-to-treat (ITT) analysis introduced to account for the clinical hold by excluding patients potentially affected most by the clinical hold, results of the trial did not show a significant improvement of overall survival duration, and the question remains whether the negative result was an effect of the clinical hold. In this paper, we propose a multistate model incorporating the clinical hold as well as disease progression as intermediate events to investigate the impact of the clinical hold on the treatment effect. Moreover, we consider a simple counterfactual censoring approach as alternative strategy to the modified ITT analysis to deal with a clinical hold. Using a realistic simulation study informed by the START data and with a design based on our multistate model, we show that the modified ITT analysis used in the START trial was reasonable. However, the censoring approach will be shown to have some benefits in terms of power and flexibility.