Planning and evaluating clinical trials with composite time-to-first-event endpoints in a competing risk framework

Re-analysis of ventilator-free days (VFD) in acute respiratory distress syndrome (ARDS) studies

BACKGROUND

Over recent decades, improvements in healthcare have reduced mortality and morbidity rates in many conditions. This has resulted, in part, from the identification of effective interventions in randomised trials, and in conducting such trials, a composite outcome measure (COM) with multiple components will increase event rates, which allows study completion with a smaller sample size. In critical care research, the COM "ventilator-free days" (VFD) combines mortality and duration of mechanical ventilation (MV) into a single continuous measure, which can be analysed in a variety of ways. This study investigates the usefulness of Poisson and two-part Poisson models compared to t-distribution for the analysis of VFD.

METHODS

Data from four studies (ALbuterol for the Treatment of ALI (ALTA), Early vs. Delayed Enteral Nutrition (EDEN), Hydroxymethylglutaryl-CoA reductase inhibition with simvastatin in Acute Lung Injury (ALI) to reduce pulmonary dysfunction (HARP-2), Statins for Acutely Injured Lungs from Sepsis (SAILS)) were used for analysis, with the VFD results summarised using mean, standard deviation (SD), median, interquartile range (25th and 75th percentiles) and minimum and maximum values. The statistical analyses that are compared used the t-test, Poisson, zero-inflated Poisson (ZIP) and two-part Logit-Poisson hurdle models. The analyses were exploratory in nature, and the significance level for differences in the estimates was set to 0.05.

RESULTS

In HARP-2, which compared simvastatin and placebo, the mean (SD) VFD for all patients was 12.0 (10.2), but this mean value did not represent the data distribution as it falls in a zone between two peaks, with the lowest frequency of occurrence. The mean (SD) VFD after excluding patients who died before day 28 and patients who did not achieve unassisted breathing were 15.9 (8.7) and 18.2 (6.6), respectively. The mean difference (95% CI) between the two groups was 1.1 (95% CI: 0.7 to 2.8; p = 0.20) based on an independent t-test. However, when the two-part hurdle model was used, the simvastatin arm had a significantly higher number of non-zero values compared to the placebo group, which indicated that more patients were alive and free of mechanical ventilation in the simvastatin group. Similarly, in ALTA, this model found that significantly more patients were alive and free of MV in the control group. In EDEN and SAILS, there was no significant difference between the control and intervention groups.

CONCLUSION

Our analyses show that the t-test and Poisson model are not appropriate for bi-modal data (such as VFD) where there is a large number of zero events. The two-part hurdle model was the most promising approach. There is a need for future research to investigate other analysis techniques, such as two-part quantile regression and to determine the impact on sample size requirements for comparative effectiveness trials.

Weighted composite time to event endpoints with recurrent events: comparison of three analytical approaches

Background

In clinical trials the study interest often lies in the comparison of a treatment to a control regarding a time to event endpoint. A composite endpoint allows to consider several time to event endpoints at once. Usually, only the time to the first occurring event for a patient is thereby analyzed. However, an individual may experience more than one non-fatal event. Including all observed events in the analysis can increase the power and provides a more complete picture of the disease. Thus, analytical methods for recurrent events are required. A challenge is that the different event types belonging to the composite often are of different clinical relevance. In this case, weighting the event types according to their clinical relevance is an option. Different weight-based methods for composite time to event endpoints were proposed. So far, there exists no systematic comparison of these methods.

Methods

Within this work we provide a systematic comparison of three methods proposed for weighted composite endpoints in a recurrent event setting combining non-fatal and fatal events of different clinical relevance. We consider an extension of an approach proposed by Wei and Lachin, an approach by Rauch et al., and an approach by Bakal et al.. Comparison is done based on a simulation study and based on a clinical study example.

Results

For all three approaches closed formula test statistics are available. The Wei-Lachin approach and the approach by Rauch et al. show similar results in mean squared error. For the approach by Wei and Lachin confidence intervals are provided. The approach by Bakal et al. is not related to a quantifiable estimand. The relevance weights of the different approaches work on different level, i.e. either on cause-specific hazard ratios or on event count.

Conclusion

The provided comparison and simulations can help to guide applied researchers to choose an adequate method for the analysis of composite endpoints combining (recurrent) events of different clinical relevance. The approach by Wei and Lachin and Rauch et al. can be recommended in scenarios where the composite effect is time-independent. The approach by Bakal et al. should be applied carefully.

Supplementary Information

The online version contains supplementary material available at (10.1186/s12874-022-01511-1).

ALS/SURV: a modification of the CAFS statistic

We present a composite endpoint that can be used in amyotrophic lateral sclerosis (ALS) trials, which combines functional status (via the ALS functional rating scale) and survival, denoted ALS/SURV. ALS/SURV modifies and extends the combined assessment of function and survival (CAFS) score and assigns rankings to participants that withdraw or are lost to follow up in a way that does not disproportionately lower and skew ranks for those participants that reach study endpoint (either death or study completion). ALS/SURV has properties of: (1) ordering participants that completed the study from the shortest surviving participant to the last observed death followed by worst function to best function; (2) ordering participants withdrawing at time of withdrawal by their decline in functional status relative to all the participants still in the study; and (3) then maintaining this ordering at time of withdrawal relative to participants still in the study. These properties allow ALS/SURV to better account for participant drop out compared to CAFS. We derive and compare the rankings of participants from the ceftriaxone treatment trial for ALS/SURV and CAFS and demonstrate that ALS/SURV does not modify the ordering of participants that complete a study by the results of participants who withdraw. Additionally, ALS/SURV can be summarized as either median functional status or median survival along with interquartile range, thereby adding clinical meaning to the statistic. Finally, by applying normal deviates, confidence intervals can be computed and used to estimate power for future studies. In summary, the above properties support the role for ALS/SURV as a new ALS composite statistic.

Introducing a new estimator and test for the weighted all-cause hazard ratio

Background

The rationale for the use of composite time-to-event endpoints is to increase the number of expected events and thereby the power by combining several event types of clinical interest. The all-cause hazard ratio is the standard effect measure for composite endpoints where the all-cause hazard function is given as the sum of the event-specific hazards. However, the effect of the individual components might differ, in magnitude or even in direction, which leads to interpretation difficulties. Moreover, the individual event types often are of different clinical relevance which further complicates interpretation. Our working group recently proposed a new weighted effect measure for composite endpoints called the ‘weighted all-cause hazard ratio’. By imposing relevance weights for the components, the interpretation of the composite effect becomes more ‘natural’. Although the weighted all-cause hazard ratio seems an elegant solution to overcome interpretation problems, the originally published approach has several shortcomings: First, the proposed point estimator requires pre-specification of a parametric survival model. Second, no closed formula for a corresponding test statistic was provided. Instead, a permutation test was proposed. Third, no clear guidance for the choice of the relevance weights was provided. In this work, we will overcome these problems.

Methods

Within this work a new non-parametric estimator and a related closed formula test statistic are presented. Performance of the new estimator and test is compared to the original ones by a Monte-Carlo simulation study.

Results

The original parametric estimator is sensible to miss-specifications of the survival model. The new non-parametric estimator turns out to be very robust even if the required assumptions are not met. The new test shows considerably better power properties than the permutation test, is computationally much less expensive but might not preserve type one error in all situations. A scheme for choosing the relevance weights in the planning stage is provided.

Conclusion

We recommend to use the non-parametric estimator along with the new test to assess the weighted all-cause hazard ratio. Concrete guidance for the choice of the relevance weights is now available. Thus, applying the weighted all-cause hazard ratio in clinical applications is both - feasible and recommended.

Electronic supplementary material

The online version of this article (10.1186/s12874-019-0765-1) contains supplementary material, which is available to authorized users.

Methods for the analysis of multiple endpoints in small populations: A review

While current guidelines generally recommend single endpoints for primary analyses of confirmatory clinical trials, it is recognized that certain settings require inference on multiple endpoints for comprehensive conclusions on treatment effects. Furthermore, combining treatment effect estimates from several outcome measures can increase the statistical power of tests. Such an efficient use of resources is of special relevance for trials in small populations. This paper reviews approaches based on a combination of test statistics or measurements across endpoints as well as multiple testing procedures that allow for confirmatory conclusions on individual endpoints. We especially focus on feasibility in trials with small sample sizes and do not solely rely on asymptotic considerations. A systematic literature search in the Scopus database, supplemented by a manual search, was performed to identify research papers on analysis methods for multiple endpoints with relevance to small populations. The identified methods were grouped into approaches that combine endpoints into a single measure to increase the power of statistical tests and methods to investigate differential treatment effects in several individual endpoints by multiple testing.

Dealing with competing risks in clinical trials: How to choose the primary efficacy analysis?

We investigate different primary efficacy analysis approaches for a 2-armed randomized clinical trial when interest is focused on a time to event primary outcome that is subject to a competing risk. We extend the work of Friedlin and Korn (2005) by considering estimation as well as testing and by simulating the primary and competing events' times from both a cause-specific hazards model as well as a joint subdistribution-cause-specific hazards model. We show that the cumulative incidence function can provide useful prognostic information for a particular patient but is not advisable for the primary efficacy analysis. Instead, it is preferable to fit a Cox model for the primary event which treats the competing event as an independent censoring. This is reasonably robust for controlling type I error and treatment effect bias with respect to the true primary and competing events' cause-specific hazards model, even when there is a shared, moderately prognostic, unobserved baseline frailty for the primary and competing events in that model. However, when it is plausible that a strongly prognostic frailty exists, combining the primary and competing events into a composite event should be considered. Finally, when there is an a priori interest in having both the primary and competing events in the primary analysis, we compare a bivariate approach for establishing overall treatment efficacy to the composite event approach. The ideas are illustrated by analyzing the Women's Health Initiative clinical trials sponsored by the National Heart, Lung, and Blood Institute.

Time-to-first-event versus recurrent-event analysis: points to consider for selecting a meaningful analysis strategy in clinical trials with composite endpoints

BACKGROUND

Composite endpoints combining several event types of clinical interest often define the primary efficacy outcome in cardiologic trials. They are commonly evaluated as time-to-first-event, thereby following the recommendations of regulatory agencies. However, to assess the patient's full disease burden and to identify preventive factors or interventions, subsequent events following the first one should be considered as well. This is especially important in cohort studies and RCTs with a long follow-up leading to a higher number of observed events per patients. So far, there exist no recommendations which approach should be preferred.

DESIGN

Recently, the Cardiovascular Round Table of the European Society of Cardiology indicated the need to investigate "how to interpret results if recurrent-event analysis results differ […] from time-to-first-event analysis" (Anker et al., Eur J Heart Fail 18:482-489, 2016). This work addresses this topic by means of a systematic simulation study.

METHODS

This paper compares two common analysis strategies for composite endpoints differing with respect to the incorporation of recurrent events for typical data scenarios motivated by a clinical trial.

RESULTS

We show that the treatment effects estimated from a time-to-first-event analysis (Cox model) and a recurrent-event analysis (Andersen-Gill model) can systematically differ, particularly in cardiovascular trials. Moreover, we provide guidance on how to interpret these results and recommend points to consider for the choice of a meaningful analysis strategy.

CONCLUSIONS

When planning trials with a composite endpoint, researchers, and regulatory agencies should be aware that the model choice affects the estimated treatment effect and its interpretation.

A weighted combined effect measure for the analysis of a composite time-to-first-event endpoint with components of different clinical relevance

Composite endpoints combine several events within a single variable, which increases the number of expected events and is thereby meant to increase the power. However, the interpretation of results can be difficult as the observed effect for the composite does not necessarily reflect the effects for the components, which may be of different magnitude or even point in adverse directions. Moreover, in clinical applications, the event types are often of different clinical relevance, which also complicates the interpretation of the composite effect. The common effect measure for composite endpoints is the all-cause hazard ratio, which gives equal weight to all events irrespective of their type and clinical relevance. Thereby, the all-cause hazard within each group is given by the sum of the cause-specific hazards corresponding to the individual components. A natural extension of the standard all-cause hazard ratio can be defined by a "weighted all-cause hazard ratio" where the individual hazards for each component are multiplied with predefined relevance weighting factors. For the special case of equal weights across the components, the weighted all-cause hazard ratio then corresponds to the standard all-cause hazard ratio. To identify the cause-specific hazard of the individual components, any parametric survival model might be applied. The new weighted effect measure can be tested for deviations from the null hypothesis by means of a permutation test. In this work, we systematically compare the new weighted approach to the standard all-cause hazard ratio by theoretical considerations, Monte-Carlo simulations, and by means of a real clinical trial example.

Sample size for a noninferiority clinical trial with time-to-event data in the presence of competing risks

The analysis and planning methods for competing risks model have been described in the literature in recent decades, and noninferiority clinical trials are helpful in current pharmaceutical practice. Analytical methods for noninferiority clinical trials in the presence of competing risks (NiCTCR) were investigated by Parpia et al., who indicated that the proportional sub-distribution hazard (SDH) model is appropriate in the context of biological studies. However, the analytical methods of the competing risks model differ from those appropriate for analyzing noninferiority clinical trials with a single outcome; thus, a corresponding method for planning such trials is necessary. A sample size formula for NiCTCR based on the proportional SDH model is presented in this paper. The primary endpoint relies on the SDH ratio. A total of 120 simulations and an example based on a randomized controlled trial verified the empirical performance of the presented formula. The results demonstrate that the empirical power of sample size formulas based on the Weibull distribution for noninferiority clinical trials with competing risks can reach the targeted power.

A DAG-based comparison of interventional effect underestimation between composite endpoint and multi-state analysis in cardiovascular trials

Background

Composite endpoints comprising hospital admissions and death are the primary outcome in many cardiovascular clinical trials. For statistical analysis, a Cox proportional hazards model for the time to first event is commonly applied. There is an ongoing debate on whether multiple episodes per individual should be incorporated into the primary analysis. While the advantages in terms of power are readily apparent, potential biases have been mostly overlooked so far.

Methods

Motivated by a randomized controlled clinical trial in heart failure patients, we use directed acyclic graphs (DAG) to investigate potential sources of bias in treatment effect estimates, depending on whether only the first or multiple episodes are considered. The biases first are explained in simplified examples and then more thoroughly investigated in simulation studies that mimic realistic patterns.

Results

Particularly the Cox model is prone to potentially severe selection bias and direct effect bias, resulting in underestimation when restricting the analysis to first events. We find that both kinds of bias can simultaneously be reduced by adequately incorporating recurrent events into the analysis model. Correspondingly, we point out appropriate proportional hazards-based multi-state models for decreasing bias and increasing power when analyzing multiple-episode composite endpoints in randomized clinical trials.

Conclusions

Incorporating multiple episodes per individual into the primary analysis can reduce the bias of a treatment’s total effect estimate. Our findings will help to move beyond the paradigm of considering first events only for approaches that use more information from the trial and augment interpretability, as has been called for in cardiovascular research.

A Multi-state Model for Designing Clinical Trials for Testing Overall Survival Allowing for Crossover after Progression

In designing a clinical trial for comparing two or more treatments with respect to overall survival (OS), a proportional hazards assumption is commonly made. However, in many cancer clinical trials, patients pass through various disease states prior to death and because of this may receive treatments other than originally assigned. For example, patients may crossover from the control treatment to the experimental treatment at progression. Even without crossover, the survival pattern after progression may be very different than the pattern prior to progression. The proportional hazards assumption will not hold in these situations and the design power calculated on this assumption will not be correct. In this paper we describe a simple and intuitive multi-state model allowing for progression, death before progression, post-progression survival and crossover after progression and apply this model to the design of clinical trials for comparing the OS of two treatments. For given values of the parameters of the multi-state model, we simulate the required number of deaths to achieve a specified power and the distribution of time required to achieve the requisite number of deaths. The results may be quite different from those derived using the usual PH assumption.

Statistical issues in the analysis of adverse events in time-to-event data

The aim of this work is to shed some light on common issues in the statistical analysis of adverse events (AEs) in clinical trials, when the main outcome is a time-to-event endpoint. To begin, we show that AEs are always subject to competing risks. That is, the occurrence of a certain AE may be precluded by occurrence of the main time-to-event outcome or by occurrence of another (fatal) AE. This has raised concerns on 'informative' censoring. We show that, in general, neither simple proportions nor Kaplan-Meier estimates of AE occurrence should be used, but common survival techniques for hazards that censor the competing event are still valid, but incomplete analyses. They must be complemented by an analogous analysis of the competing event for inference on the cumulative AE probability. The commonly used incidence rate (or incidence density) is a valid estimator of the AE hazard assuming it to be time constant. An estimator of the cumulative AE probability can be derived if the incidence rate of AE is combined with an estimator of the competing hazard. We discuss less restrictive analyses using non-parametric and semi-parametric approaches. We first consider time-to-first-AE analyses and then briefly discuss how they can be extended to the analysis of recurrent AEs. We will give a practical presentation with illustration of the methods by a simple example. Copyright © 2016 John Wiley & Sons, Ltd.

Multiplicity in confirmatory clinical trials: a case study with discussion from a JSM panel

An invited panel session was conducted in the 2012 Joint Statistical Meetings, San Diego, California, USA, to stimulate the discussion on multiplicity issues in confirmatory clinical trials for drug development. A total of 11 expert panel members were invited and 9 participated. Prior to the session, a case study was previously provided to the panel members to facilitate the discussion, focusing on the key components of the study design and multiplicity. The Phase 3 development program for this new experimental treatment was based on a single randomized controlled trial alone. Each panelist was asked to clarify if he or she responded as if he or she were a pharmaceutical drug sponsor, an academic panelist or a health regulatory scientist.

Opportunities and challenges of clinical trials in cardiology using composite primary endpoints

In clinical trials, the primary efficacy endpoint often corresponds to a so-called “composite endpoint”. Composite endpoints combine several events of interest within a single outcome variable. Thereby it is intended to enlarge the expected effect size and thereby increase the power of the study. However, composite endpoints also come along with serious challenges and problems. On the one hand, composite endpoints may lead to difficulties during the planning phase of a trial with respect to the sample size calculation, as the expected clinical effect of an intervention on the composite endpoint depends on the effects on its single components and their correlations. This may lead to wrong assumptions on the sample size needed. Too optimistic assumptions on the expected effect may lead to an underpowered of the trial, whereas a too conservatively estimated effect results in an unnecessarily high sample size. On the other hand, the interpretation of composite endpoints may be difficult, as the observed effect of the composite does not necessarily reflect the effects of the single components. Therefore the demonstration of the clinical efficacy of a new intervention by exclusively evaluating the composite endpoint may be misleading. The present paper summarizes results and recommendations of the latest research addressing the above mentioned problems in the planning, analysis and interpretation of clinical trials with composite endpoints, thereby providing a practical guidance for users.

Opportunities and challenges of combined effect measures based on prioritized outcomes

Many authors have proposed different approaches to combine multiple endpoints in a univariate outcome measure in the literature. In case of binary or time-to-event variables, composite endpoints, which combine several event types within a single event or time-to-first-event analysis are often used to assess the overall treatment effect. A main drawback of this approach is that the interpretation of the composite effect can be difficult as a negative effect in one component can be masked by a positive effect in another. Recently, some authors proposed more general approaches based on a priority ranking of outcomes, which moreover allow to combine outcome variables of different scale levels. These new combined effect measures assign a higher impact to more important endpoints, which is meant to simplify the interpretation of results. Whereas statistical tests and models for binary and time-to-event variables are well understood, the latter methods have not been investigated in detail so far. In this paper, we will investigate the statistical properties of prioritized combined outcome measures. We will perform a systematical comparison to standard composite measures, such as the all-cause hazard ratio in case of time-to-event variables or the absolute rate difference in case of binary variables, to derive recommendations for different clinical trial scenarios. We will discuss extensions and modifications of the new effect measures, which simplify the clinical interpretation. Moreover, we propose a new method on how to combine the classical composite approach with a priority ranking of outcomes using a multiple testing strategy based on the closed test procedure.