Methods for Accommodating Nonproportional Hazards in Clinical Trials: Ready for the Primary Analysis?

Comparison between asymptotic and re-randomisation tests under non-proportional hazards in a randomised controlled trial using the minimisation method

BACKGROUND

Pocock-Simon's minimisation method has been widely used to balance treatment assignments across prognostic factors in randomised controlled trials (RCTs). Previous studies focusing on the survival outcomes have demonstrated that the conservativeness of asymptotic tests without adjusting for stratification factors, as well as the inflated type I error rate of adjusted asymptotic tests conducted in a small sample of patients, can be relaxed using re-randomisation tests. Although several RCTs using minimisation have suggested the presence of non-proportional hazards (non-PH) effects, the application of re-randomisation tests has been limited to the log-rank test and Cox PH models, which may result in diminished statistical power when confronted with non-PH scenarios. To address this issue, we proposed two re-randomisation tests based on a maximum combination of weighted log-rank tests (MaxCombo test) and the difference in restricted mean survival time (dRMST) up to a fixed time point τ , both of which can be extended to adjust for randomisation stratification factors.

METHODS

We compared the performance of asymptotic and re-randomisation tests using the MaxCombo test, dRMST, log-rank test, and Cox PH models, assuming various non-PH situations for RCTs using minimisation, with total sample sizes of 50, 100, and 500 at a 1:1 allocation ratio. We mainly considered null, and alternative scenarios featuring delayed, crossing, and diminishing treatment effects.

RESULTS

Across all examined null scenarios, re-randomisation tests maintained the type I error rates at the nominal level. Conversely, unadjusted asymptotic tests indicated excessive conservatism, while adjusted asymptotic tests in both the Cox PH models and dRMST indicated inflated type I error rates for total sample sizes of 50. The stratified MaxCombo-based re-randomisation test consistently exhibited robust power across all examined scenarios.

CONCLUSIONS

The re-randomisation test is a useful alternative in non-PH situations for RCTs with minimisation using the stratified MaxCombo test, suggesting its robust power in various scenarios.

Exploratory subgroup identification in the heterogeneous Cox model: A relatively simple procedure

For survival analysis applications we propose a novel procedure for identifying subgroups with large treatment effects, with focus on subgroups where treatment is potentially detrimental. The approach, termed forest search, is relatively simple and flexible. All-possible subgroups are screened and selected based on hazard ratio thresholds indicative of harm with assessment according to the standard Cox model. By reversing the role of treatment one can seek to identify substantial benefit. We apply a splitting consistency criteria to identify a subgroup considered "maximally consistent with harm." The type-1 error and power for subgroup identification can be quickly approximated by numerical integration. To aid inference we describe a bootstrap bias-corrected Cox model estimator with variance estimated by a Jacknife approximation. We provide a detailed evaluation of operating characteristics in simulations and compare to virtual twins and generalized random forests where we find the proposal to have favorable performance. In particular, in our simulation setting, we find the proposed approach favorably controls the type-1 error for falsely identifying heterogeneity with higher power and classification accuracy for substantial heterogeneous effects. Two real data applications are provided for publicly available datasets from a clinical trial in oncology, and HIV.

Evaluation of the proportional hazards assumption and covariate adjustment methods in comparative surgical observational studies with time-to-event endpoints

INTRODUCTION

Comparative studies on surgical treatments with time-to-event endpoints have provided substantial evidence for clinical practice, but the accurate use of survival data analysis and the control of confounding bias remain big challenges.

METHODS

This was a survey of surgical studies with survival outcomes published in four general medical journals and five general surgical journals in 2021. The two most concerned statistical issues were evaluated, including confounding control by propensity score analysis (PSA) or multivariable analysis and testing of proportional hazards (PH) assumption in Cox model.

RESULTS

A total of 74 studies were included, comprising 63 observational studies and 11 randomized controlled trials. Among the observational studies, the proportion of studies utilizing PSA in surgical oncology and non-oncology studies was similar (40.9 % versus 36.8 %, P = 0.762). However, the former reported a significantly lower proportion of PH assumption assessments compared to the latter (13.6 % versus 42.1 %, P = 0.020). Twenty-five observational studies (25/63) used PSA methods, but two-thirds of them (17/25) showed unclear balance of baseline data after PSA. And the proportion of PH assumption testing after PSA was slightly lower than that before PSA, but the difference was not statistically significant (24.0 % versus 28.0 %, P = 0.317). Comprehensive suggestions were given on confounding control in survival analysis and alternative resolutions for non-compliance with PH assumption.

CONCLUSION

This study highlights suboptimal reporting of PH assumption evaluation in observational surgical studies both before and after PSA. Efforts and consensus are needed with respect to the underlying assumptions of statistical methods.

Scaling and interpreting treatment effects in clinical trials using restricted mean survival time

BACKGROUND

Restricted mean survival time is the expected duration of survival up to a chosen time of restriction τ . For comparison studies, the difference in restricted mean survival times between two groups provides a summary measure of the treatment effect that is free of assumptions regarding the relative shape of the two survival curves, such as proportional hazards. However, it can be difficult to judge the magnitude of the effect from a comparison of restricted means due to the truncation of observation at time τ .

METHODS

In this article, we describe additional ways of expressing the treatment effect based on restricted means that can be helpful in this regard. These include the ratio of restricted means, the ratio of life-years (or time) lost, and the average integrated difference between the survival curves, equal to the difference in restricted means divided by τ . These alternative metrics are straightforward to calculate and provide a means for scaling the effect size as an aid to interpretation. Examples from two randomized, multicenter clinical trials in prostate cancer, NRG/RTOG 0521 and NRG/RTOG 0534, with primary endpoints of overall survival and biochemical/radiological progression-free survival, respectively, are presented to illustrate the ideas.

RESULTS

The four effect measures (restricted mean survival time difference, restricted mean survival time ratio, time lost ratio, and average survival rate difference) were 0.45 years, 1.05, 0.81, and 0.038 for RTOG 0521 and 1.36 years, 1.17, 0.56, and 0.12 for RTOG 0534 with τ  = 12 and 11 years, respectively. Thus, for example, the 0.45-year difference in the first trial translates into a 19% reduction in time lost and a 3.8% average absolute difference between the survival curves over the 12-year horizon, a modest effect size, whereas the 1.36-year difference in the second trial corresponds to a 44% reduction in time lost and a 12% absolute survival difference, a rather large effect.

CONCLUSIONS

In addition to the difference in restricted mean survival times, these alternative measures can be helpful in determining whether the magnitude of the treatment effect is clinically meaningful.

Group sequential methods based on supremum logrank statistics under proportional and nonproportional hazards

Despite the widespread use of Cox regression for modeling treatment effects in clinical trials, in immunotherapy oncology trials and other settings therapeutic benefits are not immediately realized thereby violating the proportional hazards assumption. Weighted logrank tests and the so-called Maxcombo test involving the combination of multiple logrank test statistics have been advocated to increase power for detecting effects in these and other settings where hazards are nonproportional. We describe a testing framework based on supremum logrank statistics created by successively analyzing and excluding early events, or obtained using a moving time window. We then describe how such tests can be conducted in a group sequential trial with interim analyses conducted for potential early stopping of benefit. The crossing boundaries for the interim test statistics are determined using an easy-to-implement Monte Carlo algorithm. Numerical studies illustrate the good frequency properties of the proposed group sequential methods.

Visualizing hypothesis tests in survival analysis under anticipated delayed effects

What can be considered an appropriate statistical method for the primary analysis of a randomized clinical trial (RCT) with a time-to-event endpoint when we anticipate non-proportional hazards owing to a delayed effect? This question has been the subject of much recent debate. The standard approach is a log-rank test and/or a Cox proportional hazards model. Alternative methods have been explored in the statistical literature, such as weighted log-rank tests and tests based on the Restricted Mean Survival Time (RMST). While weighted log-rank tests can achieve high power compared to the standard log-rank test, some choices of weights may lead to type-I error inflation under particular conditions. In addition, they are not linked to a mathematically unambiguous summary measure. Test statistics based on the RMST, on the other hand, allow one to investigate the average difference between two survival curves up to a pre-specified time pointτ $$ \tau $$ -a mathematically unambiguous summary measure. However, by emphasizing differences prior toτ $$ \tau $$ , such test statistics may not fully capture the benefit of a new treatment in terms of long-term survival. In this article, we introduce a graphical approach for direct comparison of weighted log-rank tests and tests based on the RMST. This new perspective allows a more informed choice of the analysis method, going beyond power and type I error comparison.

Adaptive promising zone design for cancer immunotherapy with heterogeneous delayed treatment effect

Indirect mechanisms of cancer immunotherapies result in delayed treatment effects that vary among patients. Consequently, the use of the log-rank test in trial design and analysis can lead to significant power loss and pose additional challenges for interim decisions in adaptive designs. In this paper, we describe patients' survival using a piecewise proportional hazard model with random lag time and propose an adaptive promising zone design for cancer immunotherapy with heterogeneous delayed effects. We provide solutions for calculating conditional power and adjusting the critical value for the log-rank test with interim data. We divide the sample space into three zones - unfavourable, promising, and favourable -based on re-estimations of the survival parameters, the log-rank test statistic at the interim analysis, and the initial and maximum sample sizes. If the interim results fall into the promising zone, the sample size is increased; otherwise, it remains unchanged. We show through simulations that our proposed approach has greater overall power than the fixed sample design and similar power to the matched group sequential trial. Furthermore, we confirm that critical value adjustment effectively controls the type I error rate inflation. Finally, we provide recommendations on the implementation of our proposed method in cancer immunotherapy trials.

Simultaneous inference procedures for the comparison of multiple characteristics of two survival functions

Survival time is the primary endpoint of many randomized controlled trials, and a treatment effect is typically quantified by the hazard ratio under the assumption of proportional hazards. Awareness is increasing that in many settings this assumption is a priori violated, for example, due to delayed onset of drug effect. In these cases, interpretation of the hazard ratio estimate is ambiguous and statistical inference for alternative parameters to quantify a treatment effect is warranted. We consider differences or ratios of milestone survival probabilities or quantiles, differences in restricted mean survival times, and an average hazard ratio to be of interest. Typically, more than one such parameter needs to be reported to assess possible treatment benefits, and in confirmatory trials, the according inferential procedures need to be adjusted for multiplicity. A simple Bonferroni adjustment may be too conservative because the different parameters of interest typically show considerable correlation. Hence simultaneous inference procedures that take into account the correlation are warranted. By using the counting process representation of the mentioned parameters, we show that their estimates are asymptotically multivariate normal and we provide an estimate for their covariance matrix. We propose according to the parametric multiple testing procedures and simultaneous confidence intervals. Also, the logrank test may be included in the framework. Finite sample type I error rate and power are studied by simulation. The methods are illustrated with an example from oncology. A software implementation is provided in the R package nph.

Assessing Long-term Treatment Benefits Using Complementary Statistical Approaches: An In Silico Analysis of the Phase III Keynote-045 and Checkmate-214 Immune Checkpoint Inhibitor Trials

BACKGROUND

The Keynote-045 trial illustrates that the long-term benefit (LTB) of treatment does not always translate to improved progression-free survival (PFS). Milestone survival and flexible parametric survival model with cure (FPCM) have been proposed as complementary statistical approaches to more comprehensively evaluate LTBs of treatments.

OBJECTIVE

The current study compares milestone survival and FPCM analyses to evaluate treatment effects of immune checkpoint inhibitor (ICI) phase III trials.

DESIGN, SETTING, AND PARTICIPANTS

Individual patient data, from initial and follow-up analyses of Keynote-045 (urothelial cancer) and Checkmate-214 (advanced renal cell carcinoma), were reconstructed for PFS.

OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS

Each trial was reanalyzed using the Cox proportional hazard regression and two complementary methods (milestone survival and FPCM) to estimate treatment impact on the LTB.

RESULTS AND LIMITATIONS

For each trial, there was evidence of nonproportional hazards. For the long-term analysis of the Keynote-045 trial, FPCM identified a time-dependent effect on PFS, but the Cox model found no statistical difference in PFS (hazard ratio, 0.90; 95% confidence interval, 0.75-1.08). Milestone survival and FPCM identified improvements in the LTB fractions. This was consistent with the results from the reanalysis of Keynote-045, based on the shorter follow-up, although the LTB fraction was not retained. The increase in PFS in Checkmate-214 was identified by both Cox model and FPCM. Experimental treatment-dependent improvement in the LTB fraction was demonstrated using milestone survival and FPCM. The LTB fraction estimated with FPCM was consistent with the results from the reanalysis of the shorter follow-up period.

CONCLUSIONS

Although ICIs show substantial shifts toward LTBs in terms of PFS, based on a conventional Kaplan-Meier or Cox model analysis, our approach provides an alternative assessment of benefit-risk ratios for new therapeutics and facilitates communicating risk to patients. Kidney patients treated with ICIs can be counseled that they are potentially cured, but future work will need to definitively validate this conclusion.

PATIENT SUMMARY

Although immune checkpoint inhibitor treatments show substantial shifts toward long-term benefits in terms of progression-free survival, a more rigorous attempt to quantify this shift, rather than simply using a Kaplan-Meier estimate or comparing progression-free survival curves using the classic Cox model, is warranted. Our results suggest that advanced renal cell carcinoma patients who had not received a previous treatment are functionally cured by nivolumab and ipilimumab, which is not the case for second-line urothelial carcinoma.

Efficacy and Safety of 5-Day Oral Ensitrelvir for Patients With Mild to Moderate COVID-19: The SCORPIO-SR Randomized Clinical Trial

Importance

Treatment options for COVID-19 are warranted irrespective of the presence of risk factors for severe disease.

Objective

To assess the efficacy and safety of ensitrelvir in patients with mild to moderate COVID-19.

Design, Setting, and Participants

This phase 3 part of a phase 2/3, double-blind, placebo-controlled randomized clinical trial was conducted from February 10 to July 10, 2022, with a 28-day follow-up period, at 92 institutions in Japan, Vietnam, and South Korea. Patients (aged 12 to <70 years) with mild to moderate COVID-19 within 120 hours of positive viral test results were studied.

Interventions

Patients were randomized (1:1:1) to receive 125 mg of once-daily ensitrelvir (375 mg on day 1), 250 mg of once-daily ensitrelvir (750 mg on day 1), or placebo for 5 days.

Main Outcomes and Measures

The primary end point was the time to resolution of the composite of 5 characteristic symptoms of SARS-CoV-2 Omicron infection, assessed using a Peto-Prentice generalized Wilcoxon test stratified by vaccination history. Virologic efficacy and safety were also assessed.

Results

A total of 1821 patients were randomized, of whom 1030 (347 in the 125-mg ensitrelvir group, 340 in the 250-mg ensitrelvir group, and 343 in the placebo group) were randomized in less than 72 hours of disease onset (primary analysis population). The mean (SD) age in this population was 35.2 (12.3) years, and 552 (53.6%) were men. A significant difference was observed between the 125-mg ensitrelvir group and the placebo group (P = .04 with a Peto-Prentice generalized Wilcoxon test). The difference in median time was approximately 1 day between the 125-mg ensitrelvir group and the placebo group (167.9 vs 192.2 hours; difference, -24.3 hours; 95% CI, -78.7 to 11.7 hours). Adverse events were observed in 267 of 604 patients (44.2%) in the 125-mg ensitrelvir group, 321 of 599 patients (53.6%) in the 250-mg ensitrelvir group, and 150 of 605 patients (24.8%) in the placebo group, which included a decrease in high-density lipoprotein level (188 [31.1%] in the 125-mg ensitrelvir group, 231 [38.6%] in the 250-mg ensitrelvir group, and 23 [3.8%] in the placebo group). No treatment-related serious adverse events were reported.

Conclusions and Relevance

In this randomized clinical trial, 125-mg ensitrelvir treatment reduced the time to resolution of the 5 typical COVID-19 symptoms compared with placebo in patients treated in less than 72 hours of disease onset; the absolute difference in median time to resolution was approximately 1 day. Ensitrelvir demonstrated clinical and antiviral efficacy without new safety concerns. Generalizability to populations outside Asia should be confirmed.

Trial Registration

Japan Registry of Clinical Trials Identifier: jRCT2031210350.

Imputation methods for informative censoring in survival analysis with time dependent covariates

Cox proportional hazards model has been an established model for survival analysis. The flexibility of incorporating time dependent covariates has made the analysis more suitable in many clinical trials when the time dependent covariates may be predictive factors for the events. Subjects are censored for various reasons, but they are usually nonnormatively censored in the analysis. Methods for informative censoring are not well studied for settings with time dependent covariates. In this paper, we propose a few methods for informative censoring in survival analysis by Cox model with time dependent covariates, including tipping point method and Reference Based Imputation (Jump to Reference and Copy Reference). The implementation of these methods by multiple imputation is described and illustrated with two data examples.

The Inverse Log-Rank Test: A Versatile Procedure for Late Separating Survival Curves

Often in the planning phase of a clinical trial, a researcher will need to choose between a standard versus weighted log-rank test (LRT) for investigating right-censored survival data. While a standard LRT is optimal for analyzing evenly distributed but distinct survival events (proportional hazards), an appropriately weighted LRT test may be better suited for handling non-proportional, delayed treatment effects. The "a priori" misspecification of this alternative may result in a substantial loss of power when determining the effectiveness of an experimental drug. In this paper, the standard unweighted and inverse log-rank tests (iLRTs) are compared with the multiple weight, default Max-Combo procedure for analyzing differential late survival outcomes. Unlike combination LRTs that depend on the arbitrary selection of weights, the iLRT by definition is a single weight test and does not require implicit multiplicity correction. Empirically, both weighted methods have reasonable flexibility for assessing continuous survival curve differences from the onset of a study. However, the iLRT may be preferable for accommodating delayed separating survival curves, especially when one arm finishes first. Using standard large-sample methods, the power and sample size for the iLRT are easily estimated without resorting to complex and timely simulations.

Two-sample inference procedures under nonproportional hazards

We introduce a new two-sample inference procedure to assess the relative performance of two groups over time. Our model-free method does not assume proportional hazards, making it suitable for scenarios where nonproportional hazards may exist. Our procedure includes a diagnostic tau plot to identify changes in hazard timing and a formal inference procedure. The tau-based measures we develop are clinically meaningful and provide interpretable estimands to summarize the treatment effect over time. Our proposed statistic is a U-statistic and exhibits a martingale structure, allowing us to construct confidence intervals and perform hypothesis testing. Our approach is robust with respect to the censoring distribution. We also demonstrate how our method can be applied for sensitivity analysis in scenarios with missing tail information due to insufficient follow-up. Without censoring, Kendall's tau estimator we propose reduces to the Wilcoxon-Mann-Whitney statistic. We evaluate our method using simulations to compare its performance with the restricted mean survival time and log-rank statistics. We also apply our approach to data from several published oncology clinical trials where nonproportional hazards may exist.

Dabrafenib plus trametinib versus anti-PD-1 monotherapy as adjuvant therapy in BRAF V600-mutant stage III melanoma after definitive surgery: a multicenter, retrospective cohort study

Background

Both dabrafenib/trametinib (D/T) and anti-PD-1 monotherapy (PD-1) are approved adjuvant therapies for patients with stage III BRAF V600-mutant melanoma. However, there is still a lack of head-to-head comparative data. We aimed to describe efficacy and toxicity outcomes for these two standard therapies across melanoma centers.

Methods

This multicenter, retrospective cohort study was conducted in 15 melanoma centers in Australia, China, Germany, Italy, Japan, UK, and US. We included adult patients with resected stage III BRAF V600-mutant melanoma who received either adjuvant D/T or PD-1 between Jul 2015 and Oct 2022. The primary endpoint was relapse-free survival (RFS). Secondary endpoints included overall survival (OS), recurrence pattern and toxicity.

Findings

We included 598 patients with stage III BRAF V600-mutant melanoma who received either adjuvant D/T (n = 393 [66%]) or PD-1 (n = 205 [34%]) post definitive surgery between Jul 2015 and Oct 2022. At a median follow-up of 33 months (IQR 21-43), the median RFS was 51.0 months (95% CI 41.0-not reached [NR]) in the D/T group, significantly longer than PD-1 (44.8 months [95% CI 28.5-NR]) (univariate: HR 0.66, 95% CI 0.50-0.87, P = 0.003; multivariate: HR 0.58, 95% CI 0.39-0.86, P = 0.007), with comparable OS with PD-1 (multivariate, HR 0.90, 95% CI 0.48-1.70, P = 0.75). Similar findings were observed using a restricted-mean-survival-time model. Among those who experienced recurrence, the proportion of distant metastases was higher in the D/T cohort. D/T had a higher incidence of treatment modification due to adverse events (AEs) than PD-1, but fewer persistent AEs.

Interpretation

In patients with stage III BRAF V600-mutant melanoma post definitive surgery, D/T yielded better RFS than PD-1, with higher transient but lower persistent toxicity, and comparable OS. D/T seems to provide a better outcome compared with PD-1, but a longer follow-up and ideally a large prospective trial are needed.

Funding

Dr. Xue Bai was supported by the Beijing Hospitals Authority Youth Programme (QMS20211101) for her efforts devoted to this study. Dr. Keith T. Flaherty was funded by Adelson Medical Research Foundation for the efforts devoted to this study.

Dealing With Nonproportional Hazards in Coronary Revascularisation Studies

The Cox proportional hazards model is one of the most popular statistical tools to model time to event outcomes without the need for specifying the hazards or survival time distributions. The Cox model requires that the ratio of the hazards of the occurrence of the outcome for any 2 individuals remains constant during the entire follow-up. Studies comparing coronary revascularisation strategies, however, might be prone to violations of proportionality by the crossing of the hazard functions over time. Early increases in the risk of cardiovascular outcomes are commonly observed when comparing coronary artery bypass grafting vs percutaneous coronary intervention, whereas decreased risk might be observed later during the follow-up. The same is valid for comparisons between invasive vs conservative coronary revascularisation strategies. In these situations, the statistical power of the Cox model is reduced, and hazard ratios might not be an informative summary measure of treatment effect. In this article, we discuss methods to identify and account for nonproportionality. We illustrate the use of these methods in a case study based on reconstructed data from a coronary revascularisation clinical trial. And finally, we review the cardiovascular literature to estimate how the proportionality assumption has been reported in coronary revascularisation studies recently.

Challenges, Complexities, and Considerations in the Design and Interpretation of Late-Phase Oncology Trials

Optimal management of cancer patients relies heavily on late-phase oncology randomized controlled trials. A comprehensive understanding of the key considerations in designing and interpreting late-phase trials is crucial for improving subsequent trial design, execution, and clinical decision-making. In this review, we explore important aspects of late-phase oncology trial design. We begin by examining the selection of primary endpoints, including the advantages and disadvantages of using surrogate endpoints. We address the challenges involved in assessing tumor progression and discuss strategies to mitigate bias. We define informative censoring bias and its impact on trial results, including illustrative examples of scenarios that may lead to informative censoring. We highlight the traditional roles of the log-rank test and hazard ratio in survival analyses, along with their limitations in the presence of nonproportional hazards as well as an introduction to alternative survival estimands, such as restricted mean survival time or MaxCombo. We emphasize the distinctions between the design and interpretation of superiority and noninferiority trials, and compare Bayesian and frequentist statistical approaches. Finally, we discuss appropriate utilization of phase II and phase III trial results in shaping clinical management recommendations and evaluate the inherent risks and benefits associated with relying on phase II data for treatment decisions.

On assessing survival benefit of immunotherapy using long-term restricted mean survival time

The pattern of the difference between two survival curves we often observe in randomized clinical trials for evaluating immunotherapy is not proportional hazards; the treatment effect typically appears several months after the initiation of the treatment (ie, delayed difference pattern). The commonly used logrank test and hazard ratio estimation approach will be suboptimal concerning testing and estimation for those trials. The long-term restricted mean survival time (LT-RMST) approach is a promising alternative for detecting the treatment effect that potentially appears later in the study. A challenge in employing the LT-RMST approach is that it must specify a lower end of the time window in addition to a truncation time point that the RMST requires. There are several investigations and suggestions regarding the choice of the truncation time point for the RMST. However, little has been investigated to address the choice of the lower end of the time window. In this paper, we propose a flexible LT-RMST-based test/estimation approach that does not require users to specify a lower end of the time window. Numerical studies demonstrated that the potential power loss by adopting this flexibility was minimal, compared to the standard LT-RMST approach using a prespecified lower end of the time window. The proposed method is flexible and can offer higher power than the RMST-based approach when the delayed treatment effect is expected. Also, it provides a robust estimate of the magnitude of the treatment effect and its confidence interval that corresponds to the test result.

New late‐emphasis and combination tests based on infimum and supremum logrank statistics with application in oncology trials

Immunotherapy cancer clinical trials routinely feature an initial period during which the treatment is given without evident therapeutic benefit, which may be followed by a period during which an effective therapy reduces the hazard for event occurrence. The nature of this treatment effect is incompatible with the proportional hazards assumption, which has prompted much work on the development of alternative effect measures of frameworks for testing. We consider tests based on individual and combination of early- and late-emphasis infimum and supremum logrank statistics, describe how they can be implemented, and evaluate their performance in simulation studies. Through this work and illustrative applications we conclude that this class of test statistics offers a new and powerful framework for assessing treatment effects in cancer clinical trials involving immunotherapies.

Clinical effectiveness reporting of novel cancer drugs in the context of non-proportional hazards: a review of nice single technology appraisals

OBJECTIVES

The hazard ratio (HR) is a commonly used summary statistic when comparing time to event (TTE) data between trial arms, but assumes the presence of proportional hazards (PH). Non-proportional hazards (NPH) are increasingly common in NICE technology appraisals (TAs) due to an abundance of novel cancer treatments, which have differing mechanisms of action compared with traditional chemotherapies. The goal of this study is to understand how pharmaceutical companies, evidence review groups (ERGs) and appraisal committees (ACs) test for PH and report clinical effectiveness in the context of NPH.

METHODS

A thematic analysis of NICE TAs concerning novel cancer treatments published between 1 January 2020 and 31 December 2021 was undertaken. Data on PH testing and clinical effectiveness reporting for overall survival (OS) and progression-free survival (PFS) were obtained from company submissions, ERG reports, and final appraisal determinations (FADs).

RESULTS

NPH were present for OS or PFS in 28/40 appraisals, with log-cumulative hazard plots the most common testing methodology (40/40), supplemented by Schoenfeld residuals (20/40) and/or other statistical methods (6/40). In the context of NPH, the HR was ubiquitously reported by companies, inconsistently critiqued by ERGs (10/28), and commonly reported in FADs (23/28).

CONCLUSIONS

There is inconsistency in PH testing methodology used in TAs. ERGs are inconsistent in critiquing use of the HR in the context of NPH, and even when critiqued it remains a commonly reported outcome measure in FADs. Other measures of clinical effectiveness should be considered, along with guidance on clinical effectiveness reporting when NPH are present.

Stratified modestly weighted log-rank tests in settings with an anticipated delayed separation of survival curves

Delayed separation of survival curves is a common occurrence in confirmatory studies in immuno-oncology. Many novel statistical methods that aim to efficiently capture potential long-term survival improvements have been proposed in recent years. However, the vast majority do not consider stratification, which is a major limitation considering that most large confirmatory studies currently employ a stratified primary analysis. In this article, we combine recently proposed weighted log-rank tests that have been designed to work well under a delayed separation of survival curves, with stratification by a baseline variable. The aim is to increase the efficiency of the test when the stratifying variable is highly prognostic for survival. As there are many potential ways to combine the two techniques, we compare several possibilities in an extensive simulation study. We also apply the techniques retrospectively to two recent randomized clinical trials.

Individual Patient Data Meta-Analysis of 10-Year Follow-Up after Endovascular and Open Repair for Ruptured Abdominal AorticAneurysms

BACKGROUND

Endovascular aortic repair (EVAR) has conferred an early survival advantage compared to an open surgical repair (OSR) in patients with ruptured abdominal aortic aneurysms (rAAA). However, the long-term survival benefit after EVAR was not displayed among randomized controlled trials (RCTs), whereas many non-RCTs have provided conflicting results. We conducted a time-to-event individual patient data (IPD) meta-analysis on long-term rAAA data.

METHODS

All studies comparing mortality after EVAR versus OSR for rAAA were included. We used restricted mean survival times (RMSTs) as a measure of life expectancy for EVAR and OSR.

RESULTS

A total of 21 studies, including 12,187 patients (4952 EVAR and 7235 OSR) were finally deemed eligible. A secondary IPD analysis included 725 (372 EVAR and 353 OSR) patients only from the 3 RCTs (Immediate Management of the Patient With Rupture : Open Versus Endovascular Repair, Endovasculaire ou Chirurgie dans les Anévrysmes aorto-iliaques Rompus and Amsterdam Acute Aneurysm Trial trials). Among all studies, the median survival was 4.20 (95% confidence interval [CI]: 3.70-4.58) years for EVAR and 1.91 (95% CI: 1.57-2.39) years for OSR. Although EVAR presented with increased hazard risk from 4 to 7 years, which peaked at 6 years after the operation, the RMST difference was 0.54 (95% CI: 0.35-0.73; P < 0.001) years gained with EVAR at the end of the 10-year follow-up. IPD meta-analysis of RCTs did not demonstrate significant differences.

CONCLUSIONS

At 10-years follow-up, EVAR was associated with a 6.5 month increase in life expectancy when compared to OSR after analyzing all eligible studies. Evidence from our study suggests that a strict follow-up program would be desirable, especially for patients with long-life expectancy.

Treatment effect measures under nonproportional hazards

In a clinical trial with a time-to-event endpoint the treatment effect can be measured in various ways. Under proportional hazards all reasonable measures (such as the hazard ratio and the difference in restricted mean survival time) are consistent in the following sense: Take any control group survival distribution such that the hazard rate remains above zero; if there is no benefit by any measure there is no benefit by all measures, and as the magnitude of treatment benefit increases by any measure it increases by all measures. Under nonproportional hazards, however, survival curves can cross, and the direction of the effect for any pair of measures can be inconsistent. In this paper we critically evaluate a variety of treatment effect measures in common use and identify flaws with them. In particular, we demonstrate that a treatment's benefit has two distinct and independent dimensions which can be measured by the difference in the survival rate at the end of follow-up and the difference in restricted mean survival time, and that commonly used measures do not adequately capture both dimensions. We demonstrate that a generalized hazard difference, which can be estimated by the difference in exposure-adjusted subject incidence rates, captures both dimensions, and that its inverse, the number of patient-years of follow-up that results in one fewer event (the NYNT), is an easily interpretable measure of the magnitude of clinical benefit.

Log-Rank Test vs MaxCombo and Difference in Restricted Mean Survival Time Tests for Comparing Survival Under Nonproportional Hazards in Immuno-oncology Trials: A Systematic Review and Meta-analysis

Importance

The log-rank test is considered the criterion standard for comparing 2 survival curves in pivotal registrational trials. However, with novel immunotherapies that often violate the proportional hazards assumptions over time, log-rank can lose power and may fail to detect treatment benefit. The MaxCombo test, a combination of weighted log-rank tests, retains power under different types of nonproportional hazards. The difference in restricted mean survival time (dRMST) test is frequently proposed as an alternative to the log-rank under nonproportional hazard scenarios.

Objective

To compare the log-rank with the MaxCombo and dRMST in immuno-oncology trials to evaluate their performance in practice.

Data Sources

Comprehensive literature review using Google Scholar, PubMed, and other sources for randomized clinical trials published in peer-reviewed journals or presented at major clinical conferences before December 2019 assessing efficacy of anti-programmed cell death protein-1 or anti-programmed death/ligand 1 monoclonal antibodies.

Study Selection

Pivotal studies with overall survival or progression-free survival as the primary or key secondary end point with a planned statistical comparison in the protocol. Sixty-three studies on anti-programmed cell death protein-1 or anti-programmed death/ligand 1 monoclonal antibodies used as monotherapy or in combination with other agents in 35 902 patients across multiple solid tumor types were identified.

Data Extraction and Synthesis

Statistical comparisons (n = 150) were made between the 3 tests using the analysis populations as defined in the original protocol of each trial.

Main Outcomes and Measures

Nominal significance based on a 2-sided .05-level test was used to evaluate concordance. Case studies featuring different types of nonproportional hazards were used to discuss more robust ways of characterizing treatment benefit instead of sole reliance on hazard ratios.

Results

In this systematic review and meta-analysis of 63 studies including 35 902 patients, between the log-rank and MaxCombo, 135 of 150 comparisons (90%) were concordant; MaxCombo achieved nominal significance in 15 of 15 discordant cases, while log-rank did not. Several cases appeared to have clinically meaningful benefits that would not have been detected using log-rank. Between the log-rank and dRMST tests, 137 of 150 comparisons (91%) were concordant; log-rank was nominally significant in 5 of 13 cases, while dRMST was significant in 8 of 13. Among all 3 tests, 127 comparisons (85%) were concordant.

Conclusions and Relevance

The findings of this review show that MaxCombo may provide a pragmatic alternative to log-rank when departure from proportional hazards is anticipated. Both tests resulted in the same statistical decision in most comparisons. Discordant studies had modest to meaningful improvements in treatment effect. The dRMST test provided no added sensitivity for detecting treatment differences over log-rank.

Testing for Differences in Survival When Treatment Effects Are Persistent, Decaying, or Delayed

A statistical test for the presence of treatment effects on survival will be based on a null hypothesis (absence of effects) and an alternative (presence of effects). The null is very simply expressed. The most common alternative, also simply expressed, is that of proportional hazards. For this situation, not only do we have a very powerful test in the log-rank test but also the outcome is readily interpreted. However, many modern treatments fall outside this relatively straightforward paradigm and, as such, have attracted attention from statisticians eager to do their best to avoid losing power as well as to maintain interpretability when the alternative hypothesis is less simple. Examples include trials where the treatment effect decays with time, immunotherapy trials where treatment effects may be slow to manifest themselves as well as the so-called crossing hazards problem. We review some of the solutions that have been proposed to deal with these issues. We pay particular attention to the integrated log-rank test and how it can be combined with the log-rank test itself to obtain powerful tests for these more complex situations.

Versatile tests for window mean survival time

Window mean survival time (WMST) evaluates the mean survival between a lower time horizon, τ 0 $$ {\tau}_0 $$ , and an upper time horizon, τ 1 $$ {\tau}_1 $$ . As a flexible extension of restricted mean survival time, specific clinically relevant windows of time can be assessed for survival difference accompanied by a communicable interpretation of estimates and tests. In its original application, WMST required the pre-specification of a window through the selection of appropriate window bounds, τ 0 $$ {\tau}_0 $$ and τ 1 $$ {\tau}_1 $$ . In the instance of severe window misspecification of τ 0 $$ {\tau}_0 $$ and τ 1 $$ {\tau}_1 $$ , the analysis may suffer from low power and a less meaningful interpretation. In this article, we introduce versatile tests whose procedures are based on the simultaneous use of multiple WMST test statistics that are asymptotically normal under the null hypothesis of no difference between two groups. Simulations are performed to examine the power of the tests in moderate sample sizes when the data are uncensored to heavily censored with a ramp-up enrollment period. The survival scenarios chosen for simulation are intended to imitate those which are commonly encountered in oncology, especially in trials involving immunotherapies. Implementation of the procedures is discussed in two real data examples for illustration. Functions for performing versatile WMST tests are provided in the survWMST package in R.

Clinical Benefit Scales and Trial Design: Some Statistical Issues

Recently developed clinical-benefit outcome scales by the European Society for Medical Oncology (ESMO) and the American Society of Clinical Oncology (ASCO) allow standardized objective evaluation of outcomes of randomized clinical trials. However, incorporation of clinical-benefit outcome scales into trial designs highlights a number of statistical issues: the relationship between minimal clinical benefit and the target treatment-effect alternative used in the trial design, designing trials to assess long-term benefit, potential problems with using a trial endpoint that is not overall survival, and how to incorporate subgroup analyses into the trial design. Using the ESMO Magnitude of Clinical Benefit Scale as a basis for discussion, we review what these issues are and how they can guide the choice of trial-design target effects, appropriate endpoints, and pre-specified subgroup analyses to increase the chances that the resulting trial outcomes can be appropriately evaluated for clinical benefit.

The Predictive Individual Effect for Survival Data

BACKGROUND

The call for patient-focused drug development is loud and clear, as expressed in the twenty-first Century Cures Act and in recent guidelines and initiatives of regulatory agencies. Among the factors contributing to modernized drug development and improved health-care activities are easily interpretable measures of clinical benefit. In addition, special care is needed for cancer trials with time-to-event endpoints if the treatment effect is not constant over time.

OBJECTIVE

To quantify the potential clinical survival benefit for a new patient, would he/she be treated with the test or control treatment.

METHODS

We propose the predictive individual effect which is a patient-centric and tangible measure of clinical benefit under a wide variety of scenarios. It can be obtained by standard predictive calculations under a rank preservation assumption that has been used previously in trials with treatment switching.

RESULTS

We discuss four recent Oncology trials that cover situations with proportional as well as non-proportional hazards (delayed treatment effect or crossing of survival curves). It is shown that the predictive individual effect offers valuable insights beyond p-values, estimates of hazard ratios or differences in median survival.

CONCLUSION

Compared to standard statistical measures, the predictive individual effect is a direct, easily interpretable measure of clinical benefit. It facilitates communication among clinicians, patients, and other parties and should therefore be considered in addition to standard statistical results.

Group sequential design for randomized trials using "first hitting time" model

Group sequential design (GSD) has become a popular choice in recent clinical trials as it improves trial efficiency by providing options for early termination. The implementation of traditional tests for survival analysis (eg, the log-rank test and the Cox proportional hazard (PH) model) in the GSD setting has been widely discussed. The PH assumption is required for conventional (sequential) design, it is, however, often violated in practice. As an alternative, some generalized tests have been proposed (eg, the Max-Combo test) and their efficacies have been established. In this article, we explore the application of a more flexible, "first hitting time" based threshold regression (TR) model to GSD. TR assumes that subjects' health status is a latent (unobservable) process, and the clinical event of interest occurs when the latent health process hits a pre-specified boundary. The simulation results supported our findings that, in most cases, this comparable new method can successfully control type I error while providing higher early stopping opportunities in the sequential design, even when non-proportional hazard presents.

Design and analysis of group-sequential clinical trials based on a modestly weighted log-rank test in anticipation of a delayed separation of survival curves: A practical guidance

BACKGROUND

A common feature of many recent trials evaluating the effects of immunotherapy on survival is that non-proportional hazards can be anticipated at the design stage. This raises the possibility to use a statistical method tailored towards testing the purported long-term benefit, rather than applying the more standard log-rank test and/or Cox model. Many such proposals have been made in recent years, but there remains a lack of practical guidance on implementation, particularly in the context of group-sequential designs. In this article, we aim to fill this gap.

METHODS

We illustrate how the POPLAR trial, which compared immunotherapy versus chemotherapy in non-small-cell lung cancer, might have been re-designed to be more robust to the presence of a delayed effect using the modestly-weighted log-rank test in a group-sequential setting.

CONCLUSION

We provide step-by-step instructions on how to analyse a hypothetical realization of the trial, based on this new design. Basic theory on weighted log-rank tests and group-sequential methods is covered, and an accompanying R package (including vignette) is provided.

OUP accepted manuscript

When designing a comparative oncology trial for an overall or progression-free survival endpoint, investigators often quantify the treatment effect using a difference in median survival times. However, rather than directly designing the study to estimate this difference, it is almost always converted to a hazard ratio (HR) to determine the study size. At the analysis stage, the hazard ratio is utilized for formal analysis, yet because it may be difficult to interpret clinically, especially when the proportional hazards assumption is not met, the observed medians are also reported descriptively. The hazard ratio and median difference contrast different aspects of the survival curves. Whereas the hazard ratio places greater emphasis on late-occurring separation, the median difference focuses locally on the centers of the distributions and cannot capture either short- or long-term differences. Having 2 sets of summaries (a hazard ratio and the medians) may lead to incoherent conclusions regarding the treatment effect. For instance, the hazard ratio may suggest a treatment difference whereas the medians do not, or vice versa. In this commentary, we illustrate these commonly encountered issues using examples from recent oncology trials. We present a coherent alternative strategy that, unlike relying on the hazard ratio, does not require modeling assumptions and always results in clinically interpretable summaries of the treatment effect.

Assessment of Treatment Effects and Long-term Benefits in Immune Checkpoint Inhibitor Trials Using the Flexible Parametric Cure Model: A Systematic Review

IMPORTANCE

Compared with standard cytotoxic therapies, randomized immune checkpoint inhibitor (ICI) phase 3 trials reveal delayed benefits in terms of patient survival and/or long-term response. Such outcomes generally violate the assumption of proportional hazards, and the classical Cox proportional hazards regression model is therefore unsuitable for these types of analyses.

OBJECTIVE

To evaluate the ability of the flexible parametric cure model (FPCM) to estimate treatment effects and long-term responder fractions (LRFs) independently of prespecified time points.

EVIDENCE REVIEW

This systematic review used reconstructed individual patient data from ICI advanced or metastatic melanoma and lung cancer phase 3 trials extracted from the literature. Trials published between January 1, 2010, and October 1, 2019, with long-term follow-up periods (maximum follow-up, ≥36 months in first line and ≥30 months otherwise) were selected to identify LRFs. Individual patient data for progression-free survival were reconstructed from the published randomized ICI phase 3 trial results. The FPCM was applied to estimate treatment effects on the overall population and on the following components of the population: LRF and progression-free survival in non-long-term responders. Results obtained were compared with treatment effects estimated using the Cox proportional hazards regression model.

FINDINGS

In this systematic review, among the 23 comparisons studied using the FPCM, a statistically significant association between the time-to-event component and experimental treatment was observed in the main analyses and confirmed in the sensitivity analyses of 18 comparisons. Results were discordant for 4 comparisons that were not significant by the Cox proportional hazards regression model. The LRFs varied from 1.5% to 12.7% for the control arms and from 4.6% to 38.8% for the experimental arms. Differences in LRFs varied from 2% to 29% and were significantly increased in the experimental compared with the control arms, except for 4 comparisons.

CONCLUSIONS AND RELEVANCE

This systematic review of reconstructed individual patient data found that the FPCM was a complementary approach that provided a comprehensive and pertinent evaluation of benefit and risk by assessing whether ICI treatment was associated with an increased probability of patients being long-term responders or with an improved progression-free survival in patients who were not long-term responders.

Window mean survival time

We propose a class of alternative estimates and tests to restricted mean survival time (RMST) which improves power in numerous survival scenarios while maintaining a level of interpretability. The industry standards for interpretable hypothesis tests in survival analysis, RMST and logrank tests (LRTs), can suffer from low power in cases where the proportional hazards assumption fails. In particular, when late differences occur between survival curves, our proposed estimate and class of tests, window mean survival time (WMST), outperforms both RMST and LRT without sacrificing interpretability, unlike weighted rank tests (WRTs). WMST has the added advantage of maintaining high power when the proportional hazards assumption is met, while WRTs do not. With testing methods often being chosen in advance of data collection, WMST can ensure adequate power without distributional assumptions and is robust to the choice of its restriction parameters. Functions for performing WMST analysis are provided in the survWM2 package in R.

Cost-effectiveness analysis of oral nutritional supplements with nutritional counselling in head and neck cancer patients undergoing radiotherapy

OBJECTIVE

There is limited evidence regarding the economic effects of nutrition support in cancer patients. This study aims at investigating the cost-effectiveness profile of systematic oral nutritional supplementation (ONS) in head and neck cancer (HNC) patients undergoing radiotherapy (RT) and receiving nutritional counseling.

METHODS

A cost-effectiveness analysis based on a RCT was performed to estimate direct medical costs, life years gained (LYG) and Quality-Adjusted Life Years (QALY) for nutritional counseling with or without ONS at 5-month and 6-year follow up time. Value of information analysis was performed to value the expected gain from reducing uncertainty through further data collection.

RESULTS

ONS with nutritional counseling produced higher QALY than nutritional counseling alone (0.291 ± 0.087 vs 0.288 ± 0.087), however the difference was not significant (0.0027, P = 0.84). Mean costs were €987.60 vs €996.09, respectively in the treatment and control group (-€8.96, P = 0.98). The Incremental Cost Effectiveness Ratio (ICER) was -€3,277/QALY, with 55.4% probabilities of being cost-effective at a cost-effectiveness threshold of €30,000/QALY. The Expected Incremental Benefit was €95.16 and the Population Expected Value of Perfect Information was €8.6 million, implying that additional research is likely to be worthwhile. At a median 6-year follow up, the treatment group had a significantly better survival rate when adjusting for late effect (P = 0.039).

CONCLUSION

Our findings provide the first evidence to inform decisions about funding and reimbursement of ONS in combination with nutritional counseling in HNC patients undergoing RT. ONS may improve quality of cancer care at no additional costs, however further research on the cost-effectiveness of nutritional supplementation is recommended.

TRIAL REGISTRATION

ClinicalTrials.gov: NCT02055833. Registered 5th February 2014 https://clinicaltrials.gov/ct2/show/NCT02055833.

Are restricted mean survival time methods especially useful for noninferiority trials?

BACKGROUND

Restricted mean survival time methods compare the areas under the Kaplan-Meier curves up to a time τ for the control and experimental treatments. Extraordinary claims have been made about the benefits (in terms of dramatically smaller required sample sizes) when using restricted mean survival time methods as compared to proportional hazards methods for analyzing noninferiority trials, even when the true survival distributions satisfy proportional hazardss.

METHODS

Through some limited simulations and asymptotic power calculations, the authors compare the operating characteristics of restricted mean survival time and proportional hazards methods for analyzing both noninferiority and superiority trials under proportional hazardss to understand what relative power benefits there are when using restricted mean survival time methods for noninferiority testing.

RESULTS

In the setting of low-event rates, very large targeted noninferiority margins, and limited follow-up past τ, restricted mean survival time methods have more power than proportional hazards methods. For superiority testing, proportional hazards methods have more power. This is not a small-sample phenomenon but requires a low-event rate and a large noninferiority margin.

CONCLUSION

Although there are special settings where restricted mean survival time methods have a power advantage over proportional hazards methods for testing noninferiority, the larger issue in these settings is defining appropriate noninferiority margins. We find the restricted mean survival time methods lacking in these regards.

Immune checkpoint blockade in the treatment of advanced non-small cell lung cancer - predictors of response and impact of previous radiotherapy

BACKGROUND

The implementation of immune checkpoint inhibitors (ICI) into the standard care of advanced non-small cell lung cancer (NSCLC) has improved prognosis for this group of patients. However, long-term survival is rare. The aim of the study was to identify predictors of response and, especially, to investigate the impact radiotherapy might have on duration of response.

MATERIAL AND METHODS

The association between pretreatment patient/tumor characteristics and progression-free survival (PFS), overall survival (OS), and lung cancer-specific survival was investigated in 78 patients receiving an ICI as ≥2nd line treatment for advanced NSCLC, using Cox regression analysis. Due to competing risk, cause-specific deaths were also examined with cumulative incidence plots.

RESULTS

Median OS was 12.6 months (95% CI 7.8-18.2) and median PFS 4.1 months (95% CI 3.0-6.2), after median follow-up time of 49.7 months (range 20.9-51.5). Increasing CRP and neutrophil/lymphocyte ratio (NLR), were associated with poor PFS (CRP: HR 1.49, 95% CI 1.12-1.98; NLR: HR 1.59, 95% CI 1.22-1.85) and OS (CRP: HR 1.94, 95% CI 1.47-2.56; NLR: HR 1.54, 95% CI 1.27-1.87). Radiotherapy prior to immunotherapy was not significantly associated with patient outcome. However, when the dataset was split at 6 months of follow-up, to be able to identify early and late predictors of prognosis, we found that patients receiving radiotherapy <6 months prior to immunotherapy had better PFS (HR: 0.27, 95% CI 0.09-0.84) and lung cancer-specific survival (HR: 0.41, 95% CI 0.18-0.95) after the first 6 months of follow-up, while increasing CRP (PFS: HR1.61, 95% CI 1.21-2.14; OS: HR2.04, 95% CI 1.51-2.74) and NLR (PFS: HR 1.57, 95% CI 1.29-1.91; OS: HR 1.63, 95% CI 1.35-1.97) were predictors of poor short-term prognosis.

CONCLUSIONS

Radiotherapy may be of importance to achieve a long-lasting response to immunotherapy, while indicators of systemic inflammation can help in identifying patients with poor short-term prognosis.

Time-dependent treatment effects of metronomic chemotherapy in unfit AML patients: a secondary analysis of a randomised controlled trial

OBJECTIVES

To examine the presence of the time-dependent effect of metronomic chemotherapy for the treatment of older patients with acute myeloid leukemia (AML) who were unfit for standard chemotherapy and to reanalyze the data using an appropriate statistical approach in the presence of non-proportional hazards, the restricted mean survival time (RMST).

RESULTS

This was a secondary analysis of a multi-center, open-label, randomized controlled trial, which was conducted in seven tertiary care hospitals across Thailand. A total of 81 unfit AML patients were randomized into two treatment groups, metronomic chemotherapy and palliative treatment. The hazard ratio of metronomic chemotherapy over palliative treatment was time-dependent. At three landmark time points of 90, 180, 365 days, the restricted mean survival time differences were 13.3 (95% CI 1.9-24.7) days, 28.9 (95% CI 3.3-54.4) days, and 40.4 (95% CI - 1.3 to 82.0) days, respectively. With non-proportional hazards modeling and RMST analysis, we were able to conclude that metronomic chemotherapy is a potentially effective alternative treatment for elderly AML patients who were medically unfit for intensive chemotherapy. In the future clinical trials, non-proportional hazards should be carefully inspected and properly handled with appropriate statistical methods. Trial registration Randomized clinical trial TCTR20150918001; registration date: 15/09/2015. Retrospectively registered.

Non-proportional hazards in immuno-oncology: Is an old perspective needed?

A fundamental concept in two-arm non-parametric survival analysis is the comparison of observed versus expected numbers of events on one of the treatment arms (the choice of which arm is arbitrary), where the expectation is taken assuming that the true survival curves in the two arms are identical. This concept is at the heart of the counting-process theory that provides a rigorous basis for methods such as the log-rank test. It is natural, therefore, to maintain this perspective when extending the log-rank test to deal with non-proportional hazards, for example, by considering a weighted sum of the "observed - expected" terms, where larger weights are given to time periods where the hazard ratio is expected to favor the experimental treatment. In doing so, however, one may stumble across some rather subtle issues, related to difficulties in the interpretation of hazard ratios, that may lead to strange conclusions. An alternative approach is to view non-parametric survival comparisons as permutation tests. With this perspective, one can easily improve on the efficiency of the log-rank test, while thoroughly controlling the false positive rate. In particular, for the field of immuno-oncology, where researchers often anticipate a delayed treatment effect, sample sizes could be substantially reduced without loss of power.

Comparative efficacy of chemoimmunotherapy versus immunotherapy for advanced non-small cell lung cancer: A network meta-analysis of randomized trials

BACKGROUND

To the authors' knowledge, in the absence of head-to-head trials, it is unclear whether chemoimmunotherapy provides an additional overall survival (OS) benefit compared with immunotherapy alone in the first-line treatment of patients with advanced non-small cell lung cancer (NSCLC). The authors conducted a systematic literature review and network meta-analysis (NMA) to compare the efficacy of chemoimmunotherapy versus ICI.

METHODS

MEDLINE, Excerpta Medica dataBASE (EMBASE), Cochrane Central Register of Controlled Trials, and ClinicalTrials.gov were searched from inception to April 2020. Phase 3 trials evaluating the efficacy of first-line ICI or chemoimmunotherapy and reporting efficacy outcomes (OS, progression-free survival [PFS], and the overall response rate [ORR]) stratified by programmed death-ligand 1 (PD-L1) status were included. NMA with a Bayesian random effects model was performed.

RESULTS

A total of 12 eligible trials comprising 7845 patients were included. In patients who were negative for PD-L1 (tumor proportion score [TPS] <1%), NMA comparing chemoimmunotherapy with dual-agent ICI failed to demonstrate a statistically significant difference with regard to OS, PFS, or the ORR. In patients with low PD-L1 (TPS 1%-49%), there was no statistically significant difference observed between chemoimmunotherapy compared with either single-agent ICI or dual-agent ICI with regard to OS or the ORR. In patients with high PD-L1 (TPS ≥50%), chemoimmunotherapy was found to be associated with an improved PFS and ORR compared with single-agent ICI, but not with dual-agent ICI. No differences in OS were observed with chemoimmunotherapy when compared with either single-agent or dual-agent ICIs.

CONCLUSIONS

Although chemoimmunotherapy appears to improve the ORR and PFS in patients with PD-L1-high tumors when compared with single-agent ICI, it does not appear to confer an OS benefit over single-agent or dual-agent ICI for patients with advanced NSCLC regardless of PD-L1 status. Prospective trials are needed to validate these findings.

Empirical power comparison of statistical tests in contemporary phase III randomized controlled trials with time-to-event outcomes in oncology

BACKGROUND

More than 95% of recent cancer randomized controlled trials used the log-rank test to detect a treatment difference making it the predominant tool for comparing two survival functions. As with other tests, the log-rank test has both advantages and disadvantages. One advantage is that it offers the highest power against proportional hazards differences, which may be a major reason why alternative methods have rarely been employed in practice. The performance of statistical tests has traditionally been investigated both theoretically and numerically for several patterns of difference between two survival functions. However, to the best of our knowledge, there has been no attempt to compare the performance of various statistical tests using empirical data from past oncology randomized controlled trials. So, it is unknown whether the log-rank test offers a meaningful power advantage over alternative testing methods in contemporary cancer randomized controlled trials. Focusing on recently reported phase III cancer randomized controlled trials, we assessed whether the log-rank test gave meaningfully greater power when compared with five alternative testing methods: generalized Wilcoxon, test based on maximum of test statistics from multiple weighted log-rank tests, difference in t-year event rate, and difference in restricted mean survival time with fixed and adaptive τ.

METHODS

Using manuscripts from cancer randomized controlled trials recently published in high-tier clinical journals, we reconstructed patient-level data for overall survival (69 trials) and progression-free survival (54 trials). For each trial endpoint, we estimated the empirical power of each test. Empirical power was measured as the proportion of trials for which a test would have identified a significant result (p value < .05).

RESULTS

For overall survival, t-year event rate offered the lowest (30.4%) empirical power and restricted mean survival time with fixed τ offered the highest (43.5%). The empirical power of the other types of tests was almost identical (36.2%-37.7%). For progression-free survival, the tests we investigated offered numerically equivalent empirical power (55.6%-61.1%). No single test consistently outperformed any other test.

CONCLUSION

The empirical power assessment with the past cancer randomized controlled trials provided new insights on the performance of statistical tests. Although the log-rank test has been used in almost all trials, our study suggests that the log-rank test is not the only option from an empirical power perspective. Near universal use of the log-rank test is not supported by a meaningful difference in empirical power. Clinical trial investigators could consider alternative methods, beyond the log-rank test, for their primary analysis when designing a cancer randomized controlled trial. Factors other than power (e.g. interpretability of the estimated treatment effect) should garner greater consideration when selecting statistical tests for cancer randomized controlled trials.

Quantifying treatment differences in confirmatory trials under non-proportional hazards

Proportional hazards are a common assumption when designing confirmatory clinical trials in oncology. With the emergence of immunotherapy and novel targeted therapies, departure from the proportional hazard assumption is not rare in nowadays clinical research. Under non-proportional hazards, the hazard ratio does not have a straightforward clinical interpretation, and the log-rank test is no longer the most powerful statistical test even though it is still valid. Nevertheless, the log-rank test and the hazard ratio are still the primary analysis tools, and traditional approaches such as sample size increase are still proposed to account for the impact of non-proportional hazards. The weighed log-rank test and the test based on the restricted mean survival time (RMST) are receiving a lot of attention as a potential alternative to the log-rank test. We conduct a simulation study comparing the performance and operating characteristics of the log-rank test, the weighted log-rank test and the test based on the RMST, including a treatment effect estimation, under different non-proportional hazards patterns. Results show that, under non-proportional hazards, the hazard ratio and weighted hazard ratio have no straightforward clinical interpretation whereas the RMST ratio can be interpreted regardless of the proportional hazards assumption. In terms of power, the RMST achieves a similar performance when compared to the log-rank test.