Delayed treatment effects, treatment switching and heterogeneous patient populations: How to design and analyze RCTs in oncology

Assurance methods for designing a clinical trial with a delayed treatment effect

An assurance calculation is a Bayesian alternative to a power calculation. One may be performed to aid the planning of a clinical trial, specifically setting the sample size or to support decisions about whether or not to perform a study. Immuno-oncology is a rapidly evolving area in the development of anticancer drugs. A common phenomenon that arises in trials of such drugs is one of delayed treatment effects, that is, there is a delay in the separation of the survival curves. To calculate assurance for a trial in which a delayed treatment effect is likely to be present, uncertainty about key parameters needs to be considered. If uncertainty is not considered, the number of patients recruited may not be enough to ensure we have adequate statistical power to detect a clinically relevant treatment effect and the risk of an unsuccessful trial is increased. We present a new elicitation technique for when a delayed treatment effect is likely and show how to compute assurance using these elicited prior distributions. We provide an example to illustrate how this can be used in practice and develop open-source software to implement our methods. Our methodology has the potential to improve the success rate and efficiency of Phase III trials in immuno-oncology and for other treatments where a delayed treatment effect is expected to occur.

Comparison between asymptotic and re-randomisation tests under non-proportional hazards in a randomised controlled trial using the minimisation method

BACKGROUND

Pocock-Simon's minimisation method has been widely used to balance treatment assignments across prognostic factors in randomised controlled trials (RCTs). Previous studies focusing on the survival outcomes have demonstrated that the conservativeness of asymptotic tests without adjusting for stratification factors, as well as the inflated type I error rate of adjusted asymptotic tests conducted in a small sample of patients, can be relaxed using re-randomisation tests. Although several RCTs using minimisation have suggested the presence of non-proportional hazards (non-PH) effects, the application of re-randomisation tests has been limited to the log-rank test and Cox PH models, which may result in diminished statistical power when confronted with non-PH scenarios. To address this issue, we proposed two re-randomisation tests based on a maximum combination of weighted log-rank tests (MaxCombo test) and the difference in restricted mean survival time (dRMST) up to a fixed time point τ , both of which can be extended to adjust for randomisation stratification factors.

METHODS

We compared the performance of asymptotic and re-randomisation tests using the MaxCombo test, dRMST, log-rank test, and Cox PH models, assuming various non-PH situations for RCTs using minimisation, with total sample sizes of 50, 100, and 500 at a 1:1 allocation ratio. We mainly considered null, and alternative scenarios featuring delayed, crossing, and diminishing treatment effects.

RESULTS

Across all examined null scenarios, re-randomisation tests maintained the type I error rates at the nominal level. Conversely, unadjusted asymptotic tests indicated excessive conservatism, while adjusted asymptotic tests in both the Cox PH models and dRMST indicated inflated type I error rates for total sample sizes of 50. The stratified MaxCombo-based re-randomisation test consistently exhibited robust power across all examined scenarios.

CONCLUSIONS

The re-randomisation test is a useful alternative in non-PH situations for RCTs with minimisation using the stratified MaxCombo test, suggesting its robust power in various scenarios.

Methods for non-proportional hazards in clinical trials: A systematic review

For the analysis of time-to-event data, frequently used methods such as the log-rank test or the Cox proportional hazards model are based on the proportional hazards assumption, which is often debatable. Although a wide range of parametric and non-parametric methods for non-proportional hazards has been proposed, there is no consensus on the best approaches. To close this gap, we conducted a systematic literature search to identify statistical methods and software appropriate under non-proportional hazard. Our literature search identified 907 abstracts, out of which we included 211 articles, mostly methodological ones. Review articles and applications were less frequently identified. The articles discuss effect measures, effect estimation and regression approaches, hypothesis tests, and sample size calculation approaches, which are often tailored to specific non-proportional hazard situations. Using a unified notation, we provide an overview of methods available. Furthermore, we derive some guidance from the identified articles.

Avelumab first-line maintenance in advanced urothelial carcinoma: Complete screening for prognostic and predictive factors using machine learning in the JAVELIN Bladder 100 phase 3 trial

BACKGROUND

Avelumab first-line (1 L) maintenance is a standard of care for advanced urothelial carcinoma (aUC) based on the JAVELIN Bladder 100 phase 3 trial, which showed that avelumab 1 L maintenance + best supportive care (BSC) significantly prolonged overall survival (OS) and progression-free survival (PFS) vs BSC alone in patients who were progression free after receiving 1 L platinum-containing chemotherapy. Here, we comprehensively screened JAVELIN Bladder 100 trial datasets to identify prognostic factors that define subpopulations of patients with longer or shorter OS irrespective of treatment, and predictive factors that select patients who could obtain a greater OS benefit from avelumab 1 L maintenance treatment.

METHODS

We performed machine learning analyses to screen a large set of baseline covariates, including patient demographics, disease characteristics, laboratory values, molecular biomarkers, and patient-reported outcomes. Covariates were identified from previously reported analyses and established prognostic and predictive markers. Variables selected from random survival forest models were processed further in univariate Cox models with treatment interaction and visually inspected using correlation analysis and Kaplan-Meier curves. Results were summarized in a multivariable Cox model.

RESULTS

Prognostic baseline covariates associated with OS included in the final model were assignment to avelumab 1 L maintenance treatment, Eastern Cooperative Oncology Group performance status, site of metastasis, sum of longest target lesion diameters, levels of C-reactive protein and alkaline phosphatase in blood, lymphocyte proportion in intratumoral stroma, tumor mutational burden, and tumor CD8+ T-cell infiltration. Potential predictive factors included site of metastasis, tumor mutation burden, and tumor CD8+ T-cell infiltration. An analysis in patients with PD-L1+ tumors had similar findings to those in the overall population.

CONCLUSIONS

Machine learning analyses of data from the JAVELIN Bladder 100 trial identified potential prognostic and predictive factors for avelumab 1 L maintenance treatment in patients with aUC, which warrant further evaluation in other clinical datasets.

Simultaneous inference procedures for the comparison of multiple characteristics of two survival functions

Survival time is the primary endpoint of many randomized controlled trials, and a treatment effect is typically quantified by the hazard ratio under the assumption of proportional hazards. Awareness is increasing that in many settings this assumption is a priori violated, for example, due to delayed onset of drug effect. In these cases, interpretation of the hazard ratio estimate is ambiguous and statistical inference for alternative parameters to quantify a treatment effect is warranted. We consider differences or ratios of milestone survival probabilities or quantiles, differences in restricted mean survival times, and an average hazard ratio to be of interest. Typically, more than one such parameter needs to be reported to assess possible treatment benefits, and in confirmatory trials, the according inferential procedures need to be adjusted for multiplicity. A simple Bonferroni adjustment may be too conservative because the different parameters of interest typically show considerable correlation. Hence simultaneous inference procedures that take into account the correlation are warranted. By using the counting process representation of the mentioned parameters, we show that their estimates are asymptotically multivariate normal and we provide an estimate for their covariance matrix. We propose according to the parametric multiple testing procedures and simultaneous confidence intervals. Also, the logrank test may be included in the framework. Finite sample type I error rate and power are studied by simulation. The methods are illustrated with an example from oncology. A software implementation is provided in the R package nph.

Futility monitoring for randomized clinical trials with non-proportional hazards: An optimal conditional power approach

BACKGROUND

Standard futility analyses designed for a proportional hazards setting may have serious drawbacks when non-proportional hazards are present. One important type of non-proportional hazards occurs when the treatment effect is delayed. That is, there is little or no early treatment effect but a substantial later effect.

METHODS

We define optimality criteria for futility analyses in this setting and propose simple search procedures for deriving such rules in practice.

RESULTS

We demonstrate the advantages of the optimal rules over commonly used rules in reducing the average number of events, the average sample size, or the average study duration under the null hypothesis with minimal power loss under the alternative hypothesis.

CONCLUSION

Optimal futility rules can be derived for a non-proportional hazards setting that control the loss of power under the alternative hypothesis while maximizing the gain in early stopping under the null hypothesis.

A win ratio approach for comparing crossing survival curves in clinical trials

Many clinical trials include time-to-event or survival data as an outcome. To compare two survival distributions, the log-rank test is often used to produce a P-value for a statistical test of the null hypothesis that the two survival curves are identical. However, such a P-value does not provide the magnitude of the difference between the curves regarding the treatment effect. As a result, the P-value is often accompanied by an estimate of the hazard ratio from the proportional hazards model or Cox model as a measurement of treatment difference. However, one of the most important assumptions for Cox model is that the hazard functions for the two treatment groups are proportional. When the hazard curves cross, the Cox model could lead to misleading results and the log-rank test could also perform poorly. To address the problem of crossing curves in survival analysis, we propose the use of the win ratio method put forward by Pocock et al. as an estimand for analysing such data. The subjects in the test and control treatment groups are formed into all possible pairs. For each pair, the test treatment subject is labelled a winner or a loser if it is known who had the event of interest such as death. The win ratio is the total number of winners divided by the total number of losers and its standard error can be estimated using Bebu and Lachin method. Using real trial datasets and Monte Carlo simulations, this study investigates the power and type I error and compares the win ratio method with the log-rank test and Cox model under various scenarios of crossing survival curves with different censoring rates and distribution parameters. The results show that the win ratio method has similar power as the log-rank test and Cox model to detect the treatment difference when the assumption of proportional hazards holds true, and that the win ratio method outperforms log-rank test and Cox model in terms of power to detect the treatment difference when the survival curves cross.

New late‐emphasis and combination tests based on infimum and supremum logrank statistics with application in oncology trials

Immunotherapy cancer clinical trials routinely feature an initial period during which the treatment is given without evident therapeutic benefit, which may be followed by a period during which an effective therapy reduces the hazard for event occurrence. The nature of this treatment effect is incompatible with the proportional hazards assumption, which has prompted much work on the development of alternative effect measures of frameworks for testing. We consider tests based on individual and combination of early- and late-emphasis infimum and supremum logrank statistics, describe how they can be implemented, and evaluate their performance in simulation studies. Through this work and illustrative applications we conclude that this class of test statistics offers a new and powerful framework for assessing treatment effects in cancer clinical trials involving immunotherapies.

Radiotherapy and paclitaxel plus pazopanib or placebo in anaplastic thyroid cancer (NRG/RTOG 0912): a randomised, double-blind, placebo-controlled, multicentre, phase 2 trial

BACKGROUND

Anaplastic thyroid cancer is a rare and aggressive cancer with no standard radiotherapy-based local treatment. Based on data suggesting synergy between pazopanib and paclitaxel in anaplastic thyroid cancer, NRG Oncology did a double-blind, placebo-controlled, randomised phase 2 clinical trial comparing concurrent paclitaxel and intensity-modulated radiotherapy (IMRT) with the addition of pazopanib or placebo with the aim of improving overall survival in this patient population.

METHODS

Eligible patients were aged 18 years or older with a pathological diagnosis of anaplastic thyroid cancer, any TNM stage, Zubrod performance status of 0-2, no recent haemoptysis or bleeding, and no brain metastases. Patients were enrolled from 34 centres in the USA. Initially, a run-in was done to establish safety. In the randomised phase 2 trial, patients in the experimental group (pazopanib) received 2-3 weeks of weekly paclitaxel (80 mg/m2) intravenously and daily pazopanib suspension 400 mg orally followed by concurrent weekly paclitaxel (50 mg/m2), daily pazopanib (300 mg), and IMRT 66 Gy given in 33 daily fractions (2 Gy fractions). In the control group (placebo), pazopanib was replaced by matching placebo. Patients were randomly assigned (1:1) to the two treatment groups by permuted block randomisation by NRG Oncology with stratification by metastatic disease. All investigators, patients, and funders of the study were masked to group allocation. The primary endpoint was overall survival in the intention-to-treat population. Safety was assessed in all patients who received at least one dose of study treatment. This trial is registered with Clinicaltrials.gov, NCT01236547, and is complete.

FINDINGS

The safety run-showed the final dosing regimen to be safe based on two out of nine participants having adverse events of predefined concern. Between June 23, 2014, and Dec 30, 2016, 89 patients were enrolled to the phase 2 trial, of whom 71 were eligible (36 in the pazopanib group and 35 in the placebo group; 34 [48%] males and 37 [52%] females). At the final analysis (data cutoff March 9, 2020), with a median follow-up of 2·9 years (IQR 0·002-4·0), 61 patients had died. Overall survival was not significantly improved with pazopanib versus placebo, with a median overall survival of 5·7 months (95% CI 4·0-12·8) in the pazopanib group versus 7·3 months (4·3-10·6) in the placebo group (hazard ratio 0·86, 95% CI 0·52-1·43; one-sided log-rank p=0·28). 1-year overall survival was 37·1% (95% CI 21·1-53·2) in the pazopanib group and 29·0% (13·2-44·8) in the placebo group. The incidence of grade 3-5 adverse events did not differ significantly between the treatment groups (pazopanib 88·9% [32 of 36 patients] and placebo 85·3% [29 of 34 patients]; p=0·73). The most common clinically significant grade 3-4 adverse events in the 70 eligible treated patients (36 in the pazopanib group and 34 in the placebo group) were dysphagia (13 [36%] vs 10 [29%]), radiation dermatitis (8 [22%] vs 13 [38%]), increased alanine aminotransferase (12 [33%] vs none), increased aspartate aminotransferase (eight [22%] vs none), and oral mucositis (five [14%] vs eight [24%]). Treatment-related serious adverse events were reported for 16 (44%) patients on pazopanib and 12 (35%) patients on placebo. The most common serious adverse events were dehydration and thromboembolic event (three [8%] each) in patients on pazopanib and oral mucositis (three [8%]) in those on placebo. There was one treatment-related death in each group (sepsis in the pazopanib group and pneumonitis in the placebo group).

INTERPRETATION

To our knowledge, this study is the largest randomised anaplastic thyroid cancer study that has completed accrual showing feasibility in a multicenter NCI National Clinical Trials Network setting. Although no significant improvement in overall survival was recorded in the pazopanib group, the treatment combination was shown to be feasible and safe, and hypothesis-generating data that might warrant further investigation were generated.

FUNDING

National Cancer Institute and Novartis.

Examining evidence of time-dependent treatment effects: an illustration using regression methods

BACKGROUND

For the design and analysis of clinical trials with time-to-event outcomes, the Cox proportional hazards model and the logrank test have been the cornerstone methods for many decades. Increasingly, the key assumption of proportionality-or time-fixed effects-that underpins these methods has been called into question. The availability of novel therapies with new mechanisms of action and clinical trials of longer duration mean that non-proportional hazards are now more frequently encountered.

METHODS

We compared several regression-based methods to model time-dependent treatment effects. For illustration purposes, we used selected endpoints from a large, community-based clinical trial of low dose daily aspirin in older persons. Relative and absolute estimands were defined, and analyses were conducted in all participants. Additional exploratory analyses were undertaken by selected subgroups of interest using interaction terms in the regression models.

DISCUSSION

In the trial with median 4.7 years follow-up, we found evidence for non-proportionality and a time-dependent treatment effect of aspirin on cancer mortality not previously reported in trial findings. We also found some evidence of time-dependence to an aspirin by age interaction for major adverse cardiovascular events. For other endpoints, time-fixed treatment effect estimates were confirmed as appropriate.

CONCLUSIONS

The consideration of treatment effects using both absolute and relative estimands enhanced clinical insights into potential dynamic treatment effects. We recommend these analytical approaches as an adjunct to primary analyses to fully explore findings from clinical trials.

Adjuvant nivolumab plus ipilimumab or nivolumab alone versus placebo in patients with resected stage IV melanoma with no evidence of disease (IMMUNED): final results of a randomised, double-blind, phase 2 trial

BACKGROUND

The IMMUNED trial previously showed significant improvements in recurrence-free survival for adjuvant nivolumab plus ipilimumab as well as for adjuvant nivolumab alone in patients with stage IV melanoma with no evidence of disease after resection or radiotherapy. Here, we report the final analysis, including overall survival data.

METHODS

IMMUNED was an investigator-sponsored, double-blind, placebo-controlled, three-arm, phase 2 trial conducted in 20 academic medical centres in Germany. Eligible patients were aged 18-80 years with stage IV melanoma with no evidence of disease after surgery or radiotherapy. Patients were randomly assigned (1:1:1) to either nivolumab plus ipilimumab (nivolumab 1 mg/kg plus ipilimumab 3 mg/kg every 3 weeks for four doses followed by nivolumab 3 mg/kg every 2 weeks), nivolumab monotherapy (nivolumab 3 mg/kg every 2 weeks), or matching placebo, for up to 1 year. The primary endpoint was recurrence-free survival in the intention-to-treat population. Secondary endpoints were time-to-recurrence, overall survival, progression-free survival or recurrence-free survival 2 (in patients in the placebo group who crossed over to nivolumab monotherapy after experiencing disease recurrence), and safety endpoints. This trial is registered on ClinicalTrials.gov (NCT02523313), and is complete.

FINDINGS

Between Sept 2, 2015, and Nov 20, 2018, 175 patients were enrolled in the study, and 167 were randomly assigned to receive either nivolumab plus ipilimumab (n=56), nivolumab plus ipilimumab-matching placebo (n=59), or double placebo control (n=52). At a median follow-up of 49·2 months (IQR 34·9-58·1), 4-year recurrence-free survival was 64·2% (95% CI 49·2-75·9) in the nivolumab plus ipilimumab group, 31·4% (19·7-43·8) in the nivolumab alone group, and 15·0% (6·7-26·6) in the placebo group. The hazard ratio (HR) for recurrence for the nivolumab plus ipilimumab group versus placebo was 0·25 (97·5% CI 0·13-0·48; p<0·0001), and for the nivolumab group versus placebo was 0·60 (0·36-1·00; p=0·024). Median overall survival was not reached in any treatment group. The HR for overall survival was significantly in favour of the nivolumab plus ipilimumab group versus placebo (HR 0·41; 95% CI 0·17-0·99; p=0·040), but not for the nivolumab group versus placebo (HR 0·75; 0·36-1·56; p=0·44). 4-year overall survival was 83·8% (95% CI 68·8-91·9) in the nivolumab plus ipilimumab group, 72·6% (57·4-83·2) in the nivolumab alone group, and 63·1% (46·9-75·6) in the placebo group. The median progression-free survival or recurrence-free survival 2 of patients in the placebo group who crossed over to nivolumab monotherapy after experiencing disease recurrence was not reached (95% CI 21·2 months to not reached). Rates of grade 3-4 treatment-related adverse events remained largely unchanged compared with our previous report, occurring in 71% (95% CI 57-82) of the nivolumab plus ipilimumab group, and 29% (95% CI 17-42) of patients receiving nivolumab alone. There were no treatment-related deaths.

INTERPRETATION

Both active regimens continued to show significantly improved recurrence-free survival compared with placebo in patients with stage IV melanoma with no evidence of disease who were at high risk of recurrence. Overall survival was significantly improved for patients receiving nivolumab plus ipilimumab compared with placebo. Use of subsequent anti-PD-1-based therapy was high in patients in the placebo group after recurrence and most likely impacted the overall survival comparison of nivolumab alone versus placebo. The recurrence-free and overall survival benefit of nivolumab plus ipilimumab over placebo reinforces the change of practice already initiated for the treatment of patients with stage IV melanoma with no evidence of disease.

FUNDING

Bristol-Myers Squibb.

Robust group sequential designs for trials with survival endpoints and delayed response

Randomized clinical trials in oncology typically utilize time-to-event endpoints such as progression-free survival or overall survival as their primary efficacy endpoints, and the most commonly used statistical test to analyze these endpoints is the log-rank test. The power of the log-rank test depends on the behavior of the hazard ratio of the treatment arm to the control arm. Under the assumption of proportional hazards, the log-rank test is asymptotically fully efficient. However, this proportionality assumption does not hold true if there is a delayed treatment effect. Cancer immunology has evolved over time and several cancer vaccines are available in the market for treating existing cancers. This includes sipuleucel-T for metastatic hormone-refractory prostate cancer, nivolumab for metastatic melanoma, and pembrolizumab for advanced nonsmall-cell lung cancer. As cancer vaccines require some time to elicit an immune response, a delayed treatment effect is observed, resulting in a violation of the proportional hazards assumption. Thus, the traditional log-rank test may not be optimal for testing immuno-oncology drugs in randomized clinical trials. Moreover, the new immuno-oncology compounds have been shown to be very effective in prolonging overall survival. Therefore, it is desirable to implement a group sequential design with the possibility of early stopping for overwhelming efficacy. In this paper, we investigate the max-combo test, which utilizes the maximum of two weighted log-rank statistics, as a robust alternative to the log-rank test. The new test is implemented for two-stage designs with possible early stopping at the interim analysis time point. Two classes of weights are investigated for the max-combo test: the Fleming and Harrington (1981) G ρ , γ weights and the Magirr and Burman (2019) modest ( τ ∗ )  weights.

Complex survival trial design by the product integration method

Nonproportional hazards (NPHs) are often observed in survival trials such as the immunotherapy cancer trials. Under NPH, the classical log-rank test can be inefficient, and the estimated hazards ratio from the Cox model is difficult to interpret. The weighted log-rank test, and the tests for comparing the restricted mean survival time or the milestone survival become increasingly popular in handling NPH. The sample size calculation for these tests may require high-dimensional numerical integration. We present a sample size determination method for survival trials via product integration on the basis of a continuous-time multistate Markov model. The main challenge of the method lies in the design of the multistate model under a complex NPH pattern, and this is illustrated for NPH induced by delayed effect with individual heterogeneity in the lag duration, cure fractions, and treatment switching due to disease progression or noncompliance. Numerical examples are presented to demonstrate the accuracy of the proposed method. We obtain the following findings. The powers of the tests for milestone survival and RMST depend on both the trial duration and milestone timepoint, and may not increase as the milestone timepoint increases. If the milestone timepoint is appropriately chosen, the RMST test can be more powerful than the conventional log-rank test in the presence of diminishing treatment effect or in the proportional hazards cure model. In general, the RMST test yields lower power than a proper Fleming-Harrington weighted log-rank test.

Progression-free survival in oncological clinical studies: Assessment time bias and methods for its correction

Progression-free survival (PFS) is a frequently used endpoint in oncological clinical studies. In case of PFS, potential events are progression and death. Progressions are usually observed delayed as they can be diagnosed not before the next study visit. For this reason potential bias of treatment effect estimates for progression-free survival is a concern. In randomized trials and for relative treatment effects measures like hazard ratios, bias-correcting methods are not necessarily required or have been proposed before. However, less is known on cross-trial comparisons of absolute outcome measures like median survival times. This paper proposes a new method for correcting the assessment time bias of progression-free survival estimates to allow a fair cross-trial comparison of median PFS. Using median PFS for example, the presented method approximates the unknown posterior distribution by a Bayesian approach based on simulations. It is shown that the proposed method leads to a substantial reduction of bias as compared to estimates derived from maximum likelihood or Kaplan-Meier estimates. Bias could be reduced by more than 90% over a broad range of considered situations differing in assessment times and underlying distributions. By coverage probabilities of at least 94% based on the credibility interval of the posterior distribution the resulting parameters hold common confidence levels. In summary, the proposed approach is shown to be useful for a cross-trial comparison of median PFS.

Delayed treatment effects, treatment switching and heterogeneous patient populations: How to design and analyze RCTs in oncology

In the analysis of survival times, the logrank test and the Cox model have been established as key tools, which do not require specific distributional assumptions. Under the assumption of proportional hazards, they are efficient and their results can be interpreted unambiguously. However, delayed treatment effects, disease progression, treatment switchers or the presence of subgroups with differential treatment effects may challenge the assumption of proportional hazards. In practice, weighted logrank tests emphasizing either early, intermediate or late event times via an appropriate weighting function may be used to accommodate for an expected pattern of non‐proportionality. We model these sources of non‐proportional hazards via a mixture of survival functions with piecewise constant hazard. The model is then applied to study the power of unweighted and weighted log‐rank tests, as well as maximum tests allowing different time dependent weights. Simulation results suggest a robust performance of maximum tests across different scenarios, with little loss in power compared to the most powerful among the considered weighting schemes and huge power gain compared to unfavorable weights. The actual sources of non‐proportional hazards are not obvious from resulting populationwise survival functions, highlighting the importance of detailed simulations in the planning phase of a trial when assuming non‐proportional hazards.We provide the required tools in a software package, allowing to model data generating processes under complex non‐proportional hazard scenarios, to simulate data from these models and to perform the weighted logrank tests.