An alternative approach to confidence interval estimation for the win ratio statistic

Analysis of composite time-to-event endpoints in cardiovascular outcome trials

Composite time-to-event endpoints are commonly used in cardiovascular outcome trials. For example, the IMPROVE-IT trial comparing ezetimibe+simvastatin to placebo+simvastatin in 18,144 patients with acute coronary syndrome used a primary composite endpoint with five component outcomes: (1) cardiovascular death, (2) non-fatal stroke, (3) non-fatal myocardial infarction, (4) coronary revascularization ≥30 days after randomization, and (5) unstable angina requiring hospitalization. In such settings, the traditional analysis compares treatments using the observed time to the occurrence of the first (i.e. earliest) component outcome for each patient. This approach ignores information for subsequent outcome(s), possibly leading to reduced power to demonstrate the benefit of the test versus the control treatment. We use real data examples and simulations to contrast the traditional approach with several alternative approaches that use data for all the intra-patient component outcomes, not just the first.

Defining estimand for the win ratio: Separate the true effect from censoring

The win ratio has been increasingly used in trials with hierarchical composite endpoints. While the outcomes involved and the rule for their comparisons vary with the application, there is invariably little attention to the estimand of the resulting statistic, causing difficulties in interpretation and cross-trial comparison. We make the case for articulating the estimand as a first step to win ratio analysis and establish that the root cause for its elusiveness is its intrinsic dependency on the time frame of comparison, which, if left unspecified, is set haphazardly by trial-specific censoring. From the statistical literature, we summarize two general approaches to overcome this uncertainty-a nonparametric one that pre-specifies the time frame for all comparisons, and a semiparametric one that posits a constant win ratio across all times-each with publicly available software and real examples. Finally, we discuss unsolved challenges, such as estimand construction and inference in the presence of intercurrent events.

A win ratio-based framework to combine multiple clinical endpoints in exploratory basket trials

In contemporary exploratory phase of oncology drug development, there has been an increasing interest in evaluating investigational drug or drug combination in multiple tumor indications in a single basket trial to expedite drug development. There has been extensive research on more efficiently borrowing information across tumor indications in early phase drug development including Bayesian hierarchical modeling and the pruning-and-pooling methods. Despite the fact that the Go/No-Go decision for subsequent Phase 2 or Phase 3 trial initiation is almost always a multi-facet consideration, the statistical literature of basket trial design and analysis has largely been limited to a single binary endpoint. In this paper we explore the application of considering clinical priorities of multiple endpoints based on matched win ratio to the basket trial design and analysis. The control arm data will be simulated for each tumor indication based on the corresponding null assumptions that could be heterogeneous across tumor indications. The matched win ratio matching on the tumor indication can be performed for individual tumor indication, pooled data, or the pooled data after pruning depending on whether an individual evaluation or a simple pooling or a pruning-and-pooling method is used. We conduct the simulation studies to evaluate the performance of proposed win ratio-based framework and the results suggest the proposed framework could provide desirable operating characteristics.

Use of Win Statistics to Analyze Outcomes in the DAPA-HF and DELIVER Trials

BACKGROUND: The primary end point in most heart failure (HF) trials is a composite of time to a first worsening HF event or cardiovascular death. Prevention of recurrent events and improvements in symptoms/quality of life are also important for patients but are usually analyzed separately. Win statistics can integrate all these outcomes into a single composite end point, which is analyzed in hierarchical order, reflecting the clinical importance of each component. METHODS: The Dapagliflozin and Prevention of Adverse Outcomes in Heart Failure (DAPA-HF, n=4744) and Dapagliflozin Evaluation to Improve the Lives of Patients with Preserved Ejection Fraction Heart Failure (DELIVER, n=6263) trials enrolled patients with New York Heart Association class II, III, or IV HF, elevated natriuretic peptides, and either an ejection fraction of 40% or less (DAPA-HF) or greater than 40% and left atrial enlargement/left ventricular hypertrophy (DELIVER). We examined the effects of dapagliflozin compared with placebo on a hierarchical composite outcome, including cardiovascular death, total (first and recurrent) HF hospitalizations, total urgent HF visits, and improvement/deterioration in Kansas City Cardiomyopathy Questionnaire total symptom score (KCCQ-TSS; range from 0 to 100, with a higher score indicating fewer symptoms and physical limitations) at 8 months. RESULTS: For this composite outcome, the win ratio was 1.30 (95% confidence interval [CI], 1.23 to 1.36) in the pooled cohort, 1.33 (95% CI, 1.23 to 1.43) in the DAPA-HF trial, and 1.27 (95% CI, 1.18 to 1.36) in the DELIVER trial. Win odds and net benefit in overall patients were 1.19 (95% CI, 1.14 to 1.24) and 8.7% (95% CI, 6.6 to 10.9%), respectively. In the overall pooled cohort, the majority of wins and losses were accounted for by KCCQ-TSS; 52.4% were settled by the KCCQ-TSS tier in the pooled cohort. CONCLUSIONS: In both the DAPA-HF and DELIVER trials, dapagliflozin led to a significant improvement in composite outcomes that incorporated patient-reported outcomes along with total HF events, as well as cardiovascular deaths. These analyses provided a comprehensive presentation of win statistics and illustrated the utility and flexibility of win statistics in describing the effects of dapagliflozin in two recent clinical trials in patients with HF. (Funded by British Heart Foundation Centre of Research Excellence and others; clinical trial registration numbers, NCT03036124 and NCT03619213.)

A win ratio approach for comparing crossing survival curves in clinical trials

Many clinical trials include time-to-event or survival data as an outcome. To compare two survival distributions, the log-rank test is often used to produce a P-value for a statistical test of the null hypothesis that the two survival curves are identical. However, such a P-value does not provide the magnitude of the difference between the curves regarding the treatment effect. As a result, the P-value is often accompanied by an estimate of the hazard ratio from the proportional hazards model or Cox model as a measurement of treatment difference. However, one of the most important assumptions for Cox model is that the hazard functions for the two treatment groups are proportional. When the hazard curves cross, the Cox model could lead to misleading results and the log-rank test could also perform poorly. To address the problem of crossing curves in survival analysis, we propose the use of the win ratio method put forward by Pocock et al. as an estimand for analysing such data. The subjects in the test and control treatment groups are formed into all possible pairs. For each pair, the test treatment subject is labelled a winner or a loser if it is known who had the event of interest such as death. The win ratio is the total number of winners divided by the total number of losers and its standard error can be estimated using Bebu and Lachin method. Using real trial datasets and Monte Carlo simulations, this study investigates the power and type I error and compares the win ratio method with the log-rank test and Cox model under various scenarios of crossing survival curves with different censoring rates and distribution parameters. The results show that the win ratio method has similar power as the log-rank test and Cox model to detect the treatment difference when the assumption of proportional hazards holds true, and that the win ratio method outperforms log-rank test and Cox model in terms of power to detect the treatment difference when the survival curves cross.

Generalised pairwise comparisons for trend: An extension to the win ratio and win odds for dose-response and prognostic variable analysis with arbitrary statements of outcome preference

The win ratio is a novel approach for handling complex patient outcomes that have seen considerable interest in the medical statistics literature, and operates by considering all-to-all pairwise statements of preference on outcomes. Recent extensions to the method have focused on the two-group case, with few developments made for considering the impact of a well-ordered explanatory variable, which would allow for dose-response analysis or the analysis of links between complex patient outcomes and prognostic variables. Where such methods have been developed, they are semiparametric methods that can only be applied to survival outcomes. In this article, we introduce the generalised pairwise comparison for trend, a modified form of Agresti's generalised odds ratio. This approach is capable of considering arbitrary statements of preference, thus enabling its use across all types of outcome data. We provide a simulation study validating the approach and illustrate it with three clinical applications in stroke research.

Investigations of methods for multiple time-to-event endpoints: A chronic myeloid leukemia data analysis

BACKGROUND

In randomized controlled trials, multiple time-to-event endpoints are commonly used to determine treatment effects. However, choosing an appropriate method to address multiple endpoints, according to different purposes of clinical practice, is a challenge for researchers.

METHODS

We applied single endpoint, composite endpoint and win ratio analysis to chronic myeloid leukemia (CML) data to illustrate the distinctions with different multiple endpoints, including relapse, recovery and death after transplantation.

RESULTS

Regarding relapse and death, the hazard ratio in single endpoint analysis (HR_s ) were 1.281 (95% CI: 1.061-1.546) and hazard ratio in composite endpoint analysis (HR_c ) were 1.286 (95% CI: 1.112-1.486) and 1/WR (win ratio) was 1.292 (95% CI: 1.115-1.497) indicated a similar negative effect for non-prophylaxis patients. However, when considering recovery and death, the corresponding HR_s  = 1.280 (95% CI: 1.056-1.552) may not be enough to describe the effect on death with nonproportional hazards (p < 0.05), and for the composite endpoint analysis, the HR_c  = 0.828 (95% CI: 0.740-0.926) cannot quantify and interpret the clinical effect on the composite endpoint with the combination of recovery and death, while the 1/WR = 1.351 (95% CI: 1.207-1.513) showed an unfavourable effect for non-prophylaxis patients CONCLUSIONS: When dealing with multiple endpoints, single endpoints, researchers may choose single endpoints, composite endpoints and WR analysis due to different clinical applications and purposes. However, both single and composite endpoint analyses are hazard-based measures, and thus, the proportional hazards assumption should be considered. Moreover, composite endpoint analysis should be applied for endpoints with similar clinical meanings but not opposing implications. Win ratio analysis can be considered for different clinical importance of multiple endpoints, but the meaning of 'winner' needs to be specified for desired or undesired endpoints.

Statistical power considerations in the use of win ratio in cardiovascular outcome trials

BACKGROUND

In cardiovascular outcome trials, the win ratio (WR) method models the composite endpoint under a hierarchical structure to account for clinical priorities. It also can be applied to both survival and nonsurvival outcomes.

METHODS

In this article, we assess the performance of the WR method via extensive simulation studies and real data analyses and discuss power considerations of the method with respect to hierarchical order, variable type, magnitude of treatment effect, and event rates when applied to clinical studies.

RESULTS AND CONCLUSION

In the hierarchy of the WR method, the first-ordered component (e.g., death) plays a dominant role in statistical power, especially when that component has a large treatment effect and a high event rate. This is in contrast with the score test of the Cox proportional hazards model in which the power is more likely affected by the nonfatal events that are usually observed earlier. Furthermore, when adding an additional component to the composite endpoint, the performance of the WR method varies depending on the treatment effect, event rate, and hierarchical position of the component. If the additional component has a relatively smaller or no treatment effect, the statistical power will decrease; if the additional component has a relatively larger treatment effect and higher event rate, the statistical power will increase. When adding a nonsurvival continuous outcome (e.g., 6-min walk distance) with even a tiny treatment effect, the statistical power could dramatically increase.

Win statistics (win ratio, win odds, and net benefit) can complement one another to show the strength of the treatment effect on time-to-event outcomes

Conventional analyses of a composite of multiple time-to-event outcomes use the time to the first event. However, the first event may not be the most important outcome. To address this limitation, generalized pairwise comparisons and win statistics (win ratio, win odds, and net benefit) have become popular and have been applied to clinical trial practice. However, win ratio, win odds, and net benefit have typically been used separately. In this article, we examine the use of these three win statistics jointly for time-to-event outcomes. First, we explain the relation of point estimates and variances among the three win statistics, and the relation between the net benefit and the Mann-Whitney U statistic. Then we explain that the three win statistics are based on the same win proportions, and they test the same null hypothesis of equal win probabilities in two groups. We show theoretically that the Z-values of the corresponding statistical tests are approximately equal; therefore, the three win statistics provide very similar p-values and statistical powers. Finally, using simulation studies and data from a clinical trial, we demonstrate that, when there is no (or little) censoring, the three win statistics can complement one another to show the strength of the treatment effect. However, when the amount of censoring is not small, and without adjustment for censoring, the win odds and the net benefit may have an advantage for interpreting the treatment effect; with adjustment (e.g., IPCW adjustment) for censoring, the three win statistics can complement one another to show the strength of the treatment effect. For calculations we use the R package WINS, available on the CRAN (Comprehensive R Archive Network).

Stratified proportional win-fractions regression analysis

The recently proposed proportional win-fractions (PW) model extends the two-sample win ratio analysis of prioritized composite endpoints to regression. Its proportionality assumption ensures that the covariate-specific win ratios are invariant to the follow-up time. However, this assumption is strong and may not be satisfied by every covariate in the model. We develop a stratified PW model that adjusts for certain prognostic factors without setting them as covariates, thus bypassing the proportionality requirement. We formulate the stratified model based on pairwise comparisons within each stratum, with a common win ratio across strata modeled as a multiplicative function of the covariates. Correspondingly, we construct an estimating function for the regression coefficients in the form of an incomplete U $$ U $$ -statistic consisting of within-stratum pairs. Two types of asymptotic variance estimators are developed depending on the number of strata relative to the sample size. This in particular allows valid inference even when the strata are extremely small, such as with matched pairs. Simulation studies in realistic settings show that the stratified model outperforms the unstratified version in robustness and efficiency. Finally, real data from a major cardiovascular trial are analyzed to illustrate the potential benefits of stratification. The proposed methods are implemented in the R package WR, publicly available on the Comprehensive R Archive Network (CRAN).

Evidence synthesis analysis with prioritized benefit outcomes in oncology clinical trials

Overall survival, progression-free survival, objective response/complete response, and duration of (complete) response are frequently used as the primary and secondary efficacy endpoints for designs and analyses of oncology clinical trials. However, these endpoints are typically analyzed separately. In this article, we introduce an evidence synthesis approach to prioritize the benefit outcomes by applying the generalized pairwise comparisons (GPC) method, and use win statistics (win ratio, win odds and net benefit) to quantify treatment benefit. Under the framework of GPC, the main advantage of this evidence synthesis approach is the ability to combine relevant outcomes of various types into a single summary statistic without relying on any parametric assumptions. It is particularly relevant since health authorities and the pharmaceutical industry are increasingly incorporating structured quantitative methodologies in their benefit-risk assessment. We apply this evidence synthesis approach to an oncology phase 3 study in first-line renal cell carcinoma to assess the overall effect of an investigational treatment by ranking the most clinically relevant endpoints in cancer drug development. This application and a simulation study demonstrate that the proposed approach can synthesize the evidence of treatment effect from multiple prioritized benefit outcomes, and has substantial advantage over conventional methods that analyze each individual endpoint separately. We also introduce a newly developed R package WINS for statistical inference based on win statistics.

Power and Sample Size Calculations for the Restricted Mean Time Analysis of Prioritized Composite Endpoints

As a new way of reporting treatment effect, the restricted mean time in favor (RMT-IF) of treatment measures the net average time the treated have had a less serious outcome than the untreated over a specified time window. With multiple outcomes of differing severity, this offers a more interpretable and data-efficient alternative to the prototypical restricted mean (event-free) survival time. To facilitate its adoption in actual trials, we develop simple approaches to power and sample size calculations and implement them in user-friendly R programs. In doing so we model the bivariate outcomes of death and a nonfatal event using a Gumbel-Hougaard copula with component-wise proportional hazards structures, under which the RMT-IF estimand is derived in closed form. In a standard set-up for censoring, the variance of the nonparametric effect-size estimator is simplified and computed via a hybrid of numerical and Monte Carlo integrations, allowing us to compute the power and sample size as functions of component-wise hazard ratios. Simulation studies show that these formulas provide accurate approximations in realistic settings. To illustrate our methods, we consider designing a new trial to evaluate treatment effect on the composite outcomes of death and cancer relapse in lymph node-positive breast cancer patients, with baseline parameters calculated from a previous study.

Sample size formula for general win ratio analysis

Originally proposed for the analysis of prioritized composite endpoints, the win ratio has now expanded into a broad class of methodology based on general pairwise comparisons. Complicated by the non-i.i.d. structure of the test statistic, however, sample size estimation for the win ratio has lagged behind. In this article, we develop general and easy-to-use formulas to calculate sample size for win ratio analysis of different outcome types. In a nonparametric setting, the null variance of the test statistic is derived using U-statistic theory in terms of a dispersion parameter called the standard rank deviation, an intrinsic characteristic of the null outcome distribution and the user-defined rule of comparison. The effect size can be hypothesized either on the original scale of the population win ratio, or on the scale of a "usual" effect size suited to the outcome type. The latter approach allows one to measure the effect size by, for example, odds/continuation ratio for totally/partially ordered outcomes and hazard ratios for composite time-to-event outcomes. Simulation studies show that the derived formulas provide accurate estimates for the required sample size across different settings. As illustration, real data from two clinical studies of hepatic and cardiovascular diseases are used as pilot data to calculate sample sizes for future trials.

The win ratio: A novel approach to define and analyze postoperative composite outcomes to reflect patient and clinician priorities

BACKGROUND

The "win ratio" (WR) is a novel statistical technique that hierarchically weighs various postoperative outcomes (eg, mortality weighted more than complications) into a composite metric to define an overall benefit or "win." We sought to use the WR to assess the impact of social vulnerability on the likelihood of achieving a "win" after hepatopancreatic surgery.

METHODS

Individuals who underwent an elective hepatopancreatic procedure between 2013 and 2017 were identified using the Medicare database, which was merged with the Center for Disease Control and Prevention's Social Vulnerability Index. The win ratio was defined based on a hierarchy of postoperative outcomes: 90-day mortality, perioperative complications, 90-day readmissions, and length of stay. Patients matched based on procedure type, race, sex, age, and Charlson Comorbidity Index score were compared and assessed relative to win ratio.

RESULTS

Among 32,557 Medicare beneficiaries who underwent hepatectomy (n = 11,621, 35.7%) or pancreatectomy (n = 20,936, 64.3%), 16,846 (51.7%) patients were male with median age of 72 years (interquartile range 68-77) and median Charlson Comorbidity Index of 3 (interquartile range 2-8), and a small subset of patients were a racial/ethnic minority (n = 3,759, 11.6%). Adverse events associated with lack of a postoperative optimal outcome included 90-day mortality (n = 2,222, 6.8%), postoperative complication (n = 8,029, 24.7%), readmission (n = 6,349, 19.5%), and length of stay (median: 7 days, interquartile range 5-11). Overall, the patients from low Social Vulnerability Index areas were more likely to "win" with a textbook outcome (win ratio 1.07, 95% confidence interval 1.01-1.12) compared with patients from high social vulnerability counties; in contrast, there was no difference in the win ratio among patients living in average versus high Social Vulnerability Index (win ratio 1.04, 95% confidence interval 0.98-1.10). In assessing surgeon volume, patients who had a liver or pancreas procedure performed by a high-volume surgeon had a higher win ratio versus patients who were treated by a low-volume surgeon (win ratio 1.21, 95% confidence interval 1.16-1.25). In contrast, there was no difference in the win ratio (win ratio 1.01, 95% confidence interval 0.97-1.06) among patients relative to teaching hospital status.

CONCLUSION

Using a novel statistical approach, the win ratio ranked outcomes to create a composite measure to assess a postoperative "win." The WR demonstrated that social vulnerability was an important driver in explaining disparate postoperative outcomes.

Designing a longitudinal clinical trial based on a composite endpoint: Sample size, monitoring, and adaptation

Longitudinal clinical trials are often designed to compare treatments on the basis of multiple outcomes. For example in the case of cardiac trials, the outcomes of interest include mortality as well as cardiac events and hospitalization. For a COVID-19 trial, the outcomes of interest include mortality, time on ventilator, and time in hospital. Earlier work by these authors proposed a non-parametric test based on a composite of multiple endpoints referred to as the Finkelstein-Schoenfeld (FS) test (Finkelstein and Schoenfeld. Stat Med. 1999;18(11):1341-1354.). More recently, an estimate of the treatment comparison based on multiple endpoints (related to the FS test) was proposed (Pocock et al. Eur Heart J. 2011;33(2):176-182.). This estimate, which summarized the ratio of the number of patients who fared better vs worse on the experimental arm was coined the win ratio. The aim of this article is to provide guidance in the design of a trial that will use the FS test or the win ratio. The issues that will be considered are the sample size, sequential monitoring, and adaptive designs.

Win ratio approach for analyzing composite time-to-event endpoint with opposite treatment effects in its components

There is an increasing interest in the use of win ratio with composite time-to-event due to its flexibility in combining component endpoints. Exploring this flexibility further, one interesting question is in assessing the impact when there is a difference in treatment effect in the component endpoints. For example, the active treatment may prolong the time to occurrence of the negative event such as death or ventilation; meanwhile, the treatment effect may also shorten the time to achieving positive events, such as recovery or improvement. Notably, this portrays a situation where the treatment effect on time to recovery is in a different direction of benefit compared to the time to ventilation or death. Under such circumstances, if a single endpoint is used, the benefit gained for other individual outcomes is not counted and is diminished. As consequence, the study may need a larger sample size to detect a significant effect of treatment. Such a scenario can be handled by win ratio in a novel way by ranking component events, which is different from the usual composite endpoint approach such as time-to-first event. To evaluate how the different directions of treatment effect on component endpoints will impact the win ratio analysis, we use a Clayton copula-based bivariate survival simulation to investigate the correlation of component time-to-event. Through simulation, we found that compared to the marginal model using single endpoints, the win ratio analysis on composite endpoint performs better, especially when the correlation between two events is weak. Then, we applied the methodology to an infectious disease progression simulated study motivated by COVID-19. The application demonstrates that the win ratio approach offers advantages in empirical power compared to the traditional Cox proportional hazard approach when there is a difference in treatment effect in the marginal events.

On recurrent-event win ratio

The win ratio approach proposed by Pocock et al. (2012) has become a popular tool for analyzing composite endpoints of death and non-fatal events like hospitalization. Its standard version, however, draws on the non-fatal event only through the first occurrence. For statistical efficiency and clinical interpretability, we construct and compare different win ratio variants that make fuller use of recurrent events. We pay special attention to a variant called last-event-assisted win ratio, which compares two patients on the cumulative frequency of the non-fatal event, with ties broken by the time of its latest episode. It is shown that last-event-assisted win ratio uses more data than the standard win ratio does but reduces to the latter when the non-fatal event occurs at most once. We further prove that last-event-assisted win ratio rejects the null hypothesis with large probability if the treatment stochastically delays all events. Simulations under realistic settings show that the last-event-assisted win ratio test consistently enjoys higher power than the standard win ratio and other competitors. Analysis of a real cardiovascular trial provides further evidence for the practical advantages of the last-event-assisted win ratio. Finally, we discuss future work to develop meaningful effect size estimands based on the extended rules of comparison. The R-code for the proposed methods is included in the package WR openly available on the Comprehensive R Archive Network.

Common scale minimal sufficient balance: An improved method for covariate-adaptive randomization based on the Wilcoxon-Mann-Whitney odds ratio statistic

Minimal sufficient balance (MSB) is a recently suggested method for adaptively controlling covariate imbalance in randomized controlled trials in a manner which reduces the impact on randomness of allocation over other approaches by only intervening when the imbalance is sufficiently significant. Despite its improvements, the approach is unable to consider the relative clinical importance or magnitude of imbalance in each covariate weight, and ignores any imbalance which is not statistically significant, even when these imbalances may collectively justify intervention. We propose the common scale MSB (CS‐MSB) method which addresses these limitations, and present simulation studies comparing our proposed method to MSB. We demonstrate that CS‐MSB requires less intervention than MSB to achieve the same level of covariate balance, and does not adversely impact either statistical power or Type‐I error.

Sample size formula for a win ratio endpoint

The win ratio composite endpoint, which organizes the components of the composite hierarchically, is becoming popular in late-stage clinical trials. The method involves comparing data in a pair-wise manner starting with the endpoint highest in priority (eg, cardiovascular death). If the comparison is a tie, the endpoint next highest in priority (eg, hospitalizations for heart failure) is compared, and so on. Its sample size is usually calculated through complex simulations because there does not exist in the literature a simple sample size formula. This article provides a formula that depends on the probability that a randomly selected patient from one group does better than a randomly selected patient from another group, and on the probability of a tie. We compare the published 95% confidence intervals, which require patient-level data, with that calculated from the formula, requiring only summary-level data, for 17 composite or single win ratio endpoints. The two sets of results are similar. Simulations show the sample size formula performs well. The formula provides important insights. It shows when adding an endpoint to the hierarchy can increase power even if the added endpoint has low power by itself. It provides relevant information to modify an on-going blinded trial if necessary. The formula allows a non-specialist to quickly determine the size of the trial with a win ratio endpoint whose use is expected to increase over time.

The asymptotic distribution of the Net Benefit estimator in presence of right-censoring

The benefit-risk balance is a critical information when evaluating a new treatment. The Net Benefit has been proposed as a metric for the benefit-risk assessment, and applied in oncology to simultaneously consider gains in survival and possible side effects of chemotherapies. With complete data, one can construct a U-statistic estimator for the Net Benefit and obtain its asymptotic distribution using standard results of the U-statistic theory. However, real data is often subject to right-censoring, e.g. patient drop-out in clinical trials. It is then possible to estimate the Net Benefit using a modified U-statistic, which involves the survival time. The latter can be seen as a nuisance parameter affecting the asymptotic distribution of the Net Benefit estimator. We present here how existing asymptotic results on U-statistics can be applied to estimate the distribution of the net benefit estimator, and assess their validity in finite samples. The methodology generalizes to other statistics obtained using generalized pairwise comparisons, such as the win ratio. It is implemented in the R package BuyseTest (version 2.3.0 and later) available on Comprehensive R Archive Network.

Statistical models for composite endpoints of death and non-fatal events: a review

The proper analysis of composite endpoints consisting of both death and non-fatal events is an intriguing and sometimes contentious topic. The current practice of analyzing time to the first event often draws criticisms for ignoring the unequal importance between component events and for leaving recurrent-event data unused. Novel methods that address these limitations have recently been proposed. To compare the novel versus traditional approaches, we review three typical models for composite endpoints based on time to the first event, composite event process, and pairwise hierarchical comparisons. The pros and cons of these models are discussed with reference to the relevant regulatory guidelines, such as the recently released ICH-E9(R1) Addendum "Estimands and Sensitivity Analysis in Clinical Trials". We also discuss the impact of censoring when the model assumptions are violated and explore sensitivity analysis strategies. Simulation studies are conducted to assess the performance of the reviewed methods under different settings. As demonstration, we use publicly available R-packages to analyze real data from a major cardiovascular trial.

Transcatheter Edge-to-Edge Repair in Proportionate Versus Disproportionate Functional Mitral Regurgitation

BACKGROUND

Functional mitral regurgitation (FMR) can be subclassified on the basis of its proportionality relative to left ventricular (LV) volume and function, indicating potential differences in underlying etiology. The aim of this study was to evaluate the association of FMR proportionality with FMR reduction, heart failure hospitalization and mortality after transcatheter edge-to-edge mitral valve repair (TEER).

METHODS

This multicenter registry included 241 patients with symptomatic heart failure with reduced LV ejection fraction treated with TEER for moderate to severe or greater FMR. FMR proportionality was graded on preprocedural transthoracic echocardiography using the ratio of the effective regurgitant orifice area to LV end-diastolic volume. Baseline characteristics, follow-up transthoracic echocardiography, and 2-year clinical outcomes were compared between groups.

RESULTS

Median LV ejection fraction, effective regurgitant orifice area and LV end-diastolic volume index were 30% (interquartile range [IQR], 25%-35%), 27 mm², and 107 mL/m² (IQR, 90-135 mL/m²), respectively. Median effective regurgitant orifice area/LV end-diastolic volume ratio was 0.13 (IQR, 0.10-0.18). Proportionate FMR (pFMR) and disproportionate FMR (dFMR) was present in 123 and 118 patients, respectively. Compared with patients with pFMR, those with dFMR had higher baseline LV ejection fractions (median, 32% [IQR, 27%-39%] vs 26% [IQR, 22%-33%]; P < .01). Early FMR reduction with TEER was more pronounced in patients with dFMR (odds ratio, 0.45; 95% CI, 0.28-0.74; P < .01) than those with pFMR, but not at 12 months (odds ratio, 0.93; 95% CI, 0.53-1.63; P = .80). Overall, in 35% of patients with initial FMR reduction after TEER, FMR deteriorated again at 1-year follow-up. Rates of 2-year all-cause mortality and heart failure hospitalization were 30% (n = 66) and 37% (n = 76), with no differences between dFMR and pFMR.

CONCLUSIONS

TEER resulted in more pronounced early FMR reduction in patients with dFMR compared with those with pFMR. Yet after initial improvement, FMR deteriorated in a substantial number of patients, calling into question durable mitral regurgitation reductions with TEER in selected patients. The proportionality framework may not identify durable TEER responders.

Win odds: An adaptation of the win ratio to include ties

The win ratio, a recently proposed measure for comparing the benefit of two treatment groups, allows ties in the data but ignores ties in the inference. In this article, we highlight some difficulties that this can lead to, and we propose to focus on the win odds instead, a modification of the win ratio which takes ties into account. We construct hypothesis tests and confidence intervals for the win odds, and we investigate their properties through simulations and in a case study. We conclude that the win odds should be preferred over the win ratio.

Statistical methods for composite endpoints

Composite endpoints are commonly used in clinical trials, and time-to-first-event analysis has been the usual standard. Time-to-first-event analysis treats all components of the composite endpoint as having equal severity and is heavily influenced by short-term components. Over the last decade, novel statistical approaches have been introduced to overcome these limitations. We reviewed win ratio analysis, competing risk regression, negative binomial regression, Andersen-Gill regression, and weighted composite endpoint (WCE) analysis. Each method has both advantages and limitations. The advantage of win ratio and WCE analyses is that they take event severity into account by assigning weights to each component of the composite endpoint. These weights should be pre-specified because they strongly influence treatment effect estimates. Negative binomial regression and Andersen-Gill analyses consider all events for each patient -rather than only the first event - and tend to have more statistical power than time-to-first-event analysis. Pre-specified novel statistical methods may enhance our understanding of novel therapy when components vary substantially in severity and timing. These methods consider the specific types of patients, drugs, devices, events, and follow-up duration.

The win ratio approach for composite endpoints: practical guidance based on previous experience

The win ratio was introduced in 2012 as a new method for examining composite endpoints and has since been widely adopted in cardiovascular (CV) trials. Improving upon conventional methods for analysing composite endpoints, the win ratio accounts for relative priorities of the components and allows the components to be different types of outcomes. For example, the win ratio can combine the time to death with the number of occurrences of a non-fatal outcome such as CV-related hospitalizations (CVHs) in a single hierarchical composite endpoint. The win ratio can provide greater statistical power to detect and quantify a treatment difference by using all available information contained in the component outcomes. The win ratio can also incorporate quantitative outcomes such as exercise tests or quality-of-life scores. There is a need for more practical guidance on how best to design trials using the win ratio approach. This manuscript provides an overview of the principles behind the win ratio and provides insights into how to implement the win ratio in CV trial design and reporting, including how to determine trial size.

Adjusting win statistics for dependent censoring

For composite outcomes whose components can be prioritized on clinical importance, the win ratio, the net benefit and the win odds apply that order in comparing patients pairwise to produce wins and subsequently win proportions. Because these three statistics are derived using the same win proportions and they test the same hypothesis of equal win probabilities in the two treatment groups, we refer to them as win statistics. These methods, particularly the win ratio and the net benefit, have received increasing attention in methodological research and in design and analysis of clinical trials. For time-to-event outcomes, however, censoring may introduce bias. Previous work has shown that inverse-probability-of-censoring weighting (IPCW) can correct the win ratio for bias from independent censoring. The present article uses the IPCW approach to adjust win statistics for dependent censoring that can be predicted by baseline covariates and/or time-dependent covariates (producing the CovIPCW-adjusted win statistics). Theoretically and with examples and simulations, we show that the CovIPCW-adjusted win statistics are unbiased estimators of treatment effect in the presence of dependent censoring.

A class of proportional win-fractions regression models for composite outcomes

The win ratio is gaining traction as a simple and intuitive approach to analysis of prioritized composite endpoints in clinical trials. To extend it from two-sample comparison to regression, we propose a novel class of semiparametric models that includes as special cases both the two-sample win ratio and the traditional Cox proportional hazards model on time to the first event. Under the assumption that the covariate-specific win and loss fractions are proportional over time, the regression coefficient is unrelated to the censoring distribution and can be interpreted as the log win ratio resulting from one-unit increase in the covariate. U-statistic estimating functions, in the form of an arbitrary covariate-specific weight process integrated by a pairwise residual process, are constructed to obtain consistent estimators for the regression parameter. The asymptotic properties of the estimators are derived using uniform weak convergence theory for U-processes. Visual inspection of a "score" process provides useful clues as to the plausibility of the proportionality assumption. Extensive numerical studies using both simulated and real data from a major cardiovascular trial show that the regression methods provide valid inference on covariate effects and outperform the two-sample win ratio in both efficiency and robustness. The proposed methodology is implemented in the R-package WR, publicly available from the Comprehensive R Archive Network (CRAN).

Adjusted win ratio with stratification: Calculation methods and interpretation

The win ratio is a general method of comparing locations of distributions of two independent, ordinal random variables, and it can be estimated without distributional assumptions. In this paper we provide a unified theory of win ratio estimation in the presence of stratification and adjustment by a numeric variable. Building step by step on the estimate of the crude win ratio we compare corresponding tests with well known non-parametric tests of group difference (Wilcoxon rank-sum test, Fligner-Policello test, van Elteren test, test based on the regression on ranks, and the rank analysis of covariance test). We show that the win ratio gives an interpretable treatment effect measure with corresponding test to detect treatment effect difference under minimal assumptions.

The inverse-probability-of-censoring weighting (IPCW) adjusted win ratio statistic: an unbiased estimator in the presence of independent censoring

The win ratio method has received much attention in methodological research, ad hoc analyses, and designs of prospective studies. As the primary analysis, it supported the approval of tafamidis for the treatment of cardiomyopathy to reduce cardiovascular mortality and cardiovascular-related hospitalization. However, its dependence on censoring is a potential shortcoming. In this article, we propose the inverse-probability-of-censoring weighting (IPCW) adjusted win ratio statistic (i.e., the IPCW-adjusted win ratio statistic) to overcome censoring issues. We consider independent censoring, common censoring across endpoints, and right censoring. We develop an asymptotic variance estimator for the logarithm of the IPCW-adjusted win ratio statistic and evaluate it via simulation. Our simulation studies show that, as the amount of censoring increases, the unadjusted win proportions may decrease greatly. Consequently, the bias of the unadjusted win ratio estimate may increase greatly, producing either an overestimate or an underestimate. We demonstrate theoretically and through simulation that the IPCW-adjusted win ratio statistic gives an unbiased estimate of treatment effect.

A new approach for sizing trials with composite binary endpoints using anticipated marginal values and accounting for the correlation between components

Composite binary endpoints are increasingly used as primary endpoints in clinical trials. When designing a trial, it is crucial to determine the appropriate sample size for testing the statistical differences between treatment groups for the primary endpoint. As shown in this work, when using a composite binary endpoint to size a trial, one needs to specify the event rates and the effect sizes of the composite components as well as the correlation between them. In practice, the marginal parameters of the components can be obtained from previous studies or pilot trials; however, the correlation is often not previously reported and thus usually unknown. We first show that the sample size for composite binary endpoints is strongly dependent on the correlation and, second, that slight deviations in the prior information on the marginal parameters may result in underpowered trials for achieving the study objectives at a pre-specified significance level. We propose a general strategy for calculating the required sample size when the correlation is not specified and accounting for uncertainty in the marginal parameter values. We present the web platform CompARE to characterize composite endpoints and to calculate the sample size just as we propose in this paper. We evaluate the performance of the proposal with a simulation study and illustrate it by means of a real case study using CompARE.

Properties of composite time to first event versus joint marginal analyses of multiple outcomes

Many clinical studies (eg, cardiovascular outcome trials) investigate the effect of an intervention on multiple event time outcomes. The most common method of analysis is a so-called "composite" analysis of a composite outcome defined as the time to the first component event. Other approaches have been proposed, including the win ratio (or win difference) for ordered outcomes and the application of the Wei-Lachin test. Herein, we assess the influence of the marginal and joint distributions of the component events, and their correlation structures, on the operating characteristics of these methods for the analysis of multiple events. The operating characteristics are investigated using a bivariate exponential model with a shared frailty, under which these properties are obtained in closed form. While the composite time-to-first-event analysis provides an unbiased test of the hypothesis of equality of the distribution of the time to first event, we show that it can provide a biased test of the joint null hypothesis of equal marginal hazards when the correlation of event times differs between groups. The same applies to the win ratio. However, the operating characteristics of the Wei-Lachin or other tests of the joint equality of the marginal hazards are unaffected. Furthermore, when the correlations are equal, the Wei-Lachin test is more powerful to detect a difference in marginal hazards than the composite analysis test. Careful consideration of the properties of the various methods for analysis of composite outcome measures are in order before adopting one as primary analysis in a clinical study.

Methods for the analysis of multiple endpoints in small populations: A review

While current guidelines generally recommend single endpoints for primary analyses of confirmatory clinical trials, it is recognized that certain settings require inference on multiple endpoints for comprehensive conclusions on treatment effects. Furthermore, combining treatment effect estimates from several outcome measures can increase the statistical power of tests. Such an efficient use of resources is of special relevance for trials in small populations. This paper reviews approaches based on a combination of test statistics or measurements across endpoints as well as multiple testing procedures that allow for confirmatory conclusions on individual endpoints. We especially focus on feasibility in trials with small sample sizes and do not solely rely on asymptotic considerations. A systematic literature search in the Scopus database, supplemented by a manual search, was performed to identify research papers on analysis methods for multiple endpoints with relevance to small populations. The identified methods were grouped into approaches that combine endpoints into a single measure to increase the power of statistical tests and methods to investigate differential treatment effects in several individual endpoints by multiple testing.

Weighted win loss approach for analyzing prioritized outcomes

To analyze prioritized outcomes, Buyse (2010) and Pocock et al. (2012) proposed the win loss approach. In this paper, we first study the relationship between the win loss approach and the traditional survival analysis on the time to the first event. We then propose the weighted win loss statistics to improve the efficiency of the unweighted methods. A closed-form variance estimator of the weighted win loss statistics is derived to facilitate hypothesis testing and study design. We also calculated the contribution index to better interpret the results of the weighted win loss approach. Simulation studies and real data analysis demonstrated the characteristics of the proposed statistics. Copyright © 2017 John Wiley & Sons, Ltd.

Win Ratio -An Intuitive and Easy-To-Interpret Composite Outcome in Medical Studies

In medical studies with multiple outcomes, researchers always need to make choices as to whether to use a composite outcome (after combining multiple outcomes) as their primary outcome. In this paper we review a new measurement of the treatment effect - win ratio, which can be easily used in studies with prioritized multiple outcomes. We also propose some research topics to be done in this area.

An extension of generalized pairwise comparisons for prioritized outcomes in the presence of censoring

Generalized pairwise comparisons have been proposed to permit a comprehensive assessment of several prioritized outcomes between two groups of observations. This procedure estimates Δ, the net chance of a better outcome with treatment than with control by comparing the patients outcomes among all possible pairs taking one patient from the treatment group and one patient from the control group. For time to event outcomes, the standard procedure of generalized pairwise comparisons is analogous to the Gehan’s modification of the Mann-Whitney test which is biased in presence of censored observation and less powerful than Efron’s modification of this test. We adapt Efron’s modification to generalized pairwise comparisons. We show how a pairwise contribution to Δ can be calculated from the estimates of the survival function in the presence of right-censored data. We performed a simulation study to assess the bias, the type I error and the power of the new procedure. The estimate of Δ with the new procedure is only slightly biased even in presence of heavy censoring. We also show how this bias can be corrected when only one time-to-event outcome is analyzed. The new procedure has higher power in most cases compared to the standard procedure.