Statistical challenges in the evaluation of surrogate endpoints in randomized trials

Assessing heterogeneity in surrogacy using censored data

Determining whether a surrogate marker can be used to replace a primary outcome in a clinical study is complex. While many statistical methods have been developed to formally evaluate a surrogate marker, they generally do not provide a way to examine heterogeneity in the utility of a surrogate marker. Similar to treatment effect heterogeneity, where the effect of a treatment varies based on a patient characteristic, heterogeneity in surrogacy means that the strength or utility of the surrogate marker varies based on a patient characteristic. The few methods that have been recently developed to examine such heterogeneity cannot accommodate censored data. Studies with a censored outcome are typically the studies that could most benefit from a surrogate because the follow-up time is often long. In this paper, we develop a robust nonparametric approach to assess heterogeneity in the utility of a surrogate marker with respect to a baseline variable in a censored time-to-event outcome setting. In addition, we propose and evaluate a testing procedure to formally test for heterogeneity at a single time point or across multiple time points simultaneously. Finite sample performance of our estimation and testing procedure are examined in a simulation study. We use our proposed method to investigate the complex relationship between change in fasting plasma glucose, diabetes, and sex hormones using data from the diabetes prevention program study.

Challenges and Opportunities in Establishing Appropriate Intermediate Endpoints Reflecting Patient Benefit: A Roadmap for Research and Clinical Application in Nonmetastatic Prostate Cancer

Defining meaningful endpoints for research of early-stage high-risk prostate cancer is challenging, with established measures such as overall survival and metastasis-free survival facing limitations related to feasibility and adequate reflection of patient relevance. Developing endpoints must cater to diverse perspectives across scientific, clinical, regulatory, and patient viewpoints. Endpoints such as pathological complete response, no evidence of disease, and prevention of prostate-specific antigen relapse may reflect patient benefit by accounting for diagnostic and treatment burdens.

Design a Clinical Research Protocol: Influence of Real-World Setting

The design of a clinical research protocol to evaluate new therapies, devices, patient quality of life, and medical practices from scratch is probably one of the greatest challenges for the majority of novice researchers. This is especially true since a high-quality methodology is required to achieve success and effectiveness in academic and hospital research centers. This review discusses the concrete steps and necessary guidelines needed to create and structure a research protocol. Along with the methodology, some administrative challenges (ethics, regulatory and people-management barriers) and possible time-saving recommendations (standardized procedures, collaborative training, and centralization) are discussed.

Surrogacy Beyond Prognosis: The Importance of “Trial-Level” Surrogacy

Many candidate surrogate endpoints are currently assessed using a 2-level statistical approach, which consists in checking whether (1) the potential surrogate is associated with the final endpoint in individual patients and (2) the effect of treatment on the surrogate can be used to reliably predict the effect of treatment on the final endpoint. In some situations, condition (1) is fulfilled but condition (2) is not. We use concepts of causal inference to explain this apparently paradoxical situation, illustrating this review with 2 contrasting examples in operable breast cancer: the example of pathological complete response (pCR) and that of disease-free survival (DFS). In a previous meta-analysis, pCR has been shown to be a strong and independent prognostic factor for event-free survival (EFS) and overall survival (OS) after neoadjuvant treatment of operable breast cancer. Yet, in randomized trials, the effects of experimental treatments on pCR have not translated into predictable effects on EFS or OS, making pCR an “individual-level” surrogate, but not a “trial-level” surrogate. In contrast, DFS has been shown to be an acceptable surrogate for OS at both the individual and trial levels in early, HER2-positive breast cancer. The distinction between the prognostic and predictive roles of a tentative surrogate, not always made in the literature, avoids unnecessary confusion and allows better understanding of what it takes to validate a surrogate endpoint that is truly able to replace a final endpoint.

Refining neoadjuvant therapy clinical trial design for muscle-invasive bladder cancer before cystectomy: a joint US Food and Drug Administration and Bladder Cancer Advocacy Network workshop

The success of the use of novel therapies in the treatment of advanced urothelial carcinoma has contributed to growing interest in evaluating these therapies at earlier stages of the disease. However, trials evaluating these therapies in the neoadjuvant setting must have clearly defined study elements and appropriately selected end points to ensure the applicability of the trial and enable interpretation of the study results. To advance the development of rational trial design, a public workshop jointly sponsored by the US Food and Drug Administration and the Bladder Cancer Advocacy Network convened in August 2019. Clinicians, clinical trialists, radiologists, biostatisticians, patients, advocates and other stakeholders discussed key elements and end points when designing trials of neoadjuvant therapy for muscle-invasive bladder cancer (MIBC), identifying opportunities to refine eligibility, design and end points for neoadjuvant trials in MIBC. Although pathological complete response (pCR) is already being used as a co-primary end point, both individual-level and trial-level surrogacy for time-to-event end points, such as event-free survival or overall survival, remain incompletely characterized in MIBC. Additionally, use of pCR is limited by heterogeneity in pathological evaluation and the fact that the magnitude of pCR improvement that might translate into a meaningful clinical benefit remains unclear. Given existing knowledge gaps, capture of highly granular patient-related, tumour-related and treatment-related characteristics in the current generation of neoadjuvant MIBC trials will be critical to informing the design of future trials.

Counterfactual mediation analysis in the multistate model framework for surrogate and clinical time-to-event outcomes in randomized controlled trials

In cancer randomized controlled trials, surrogate endpoints are frequently time-to-event endpoints, subject to the competing risk from the time-to-event clinical outcome. In this context, we introduce a counterfactual-based mediation analysis for a causal assessment of surrogacy. We use a multistate model for risk prediction to account for both direct transitions towards the clinical outcome and indirect transitions through the surrogate outcome. Within the counterfactual framework, we define natural direct and indirect effects with a causal interpretation. Based on these measures, we define the proportion of the treatment effect on the clinical outcome mediated by the surrogate outcome. We estimate the proportion for both the cumulative risk and restricted mean time lost. We illustrate our approach by using 18-year follow-up data from the SPCG-4 randomized trial of radical prostatectomy for prostate cancer. We assess time to metastasis as a surrogate outcome for prostate cancer-specific mortality.

Quantifying the feasibility of shortening clinical trial duration using surrogate markers

The potential benefit of using a surrogate marker in place of a long-term primary outcome is very attractive in terms of the impact on study length and cost. Many available methods for quantifying the effectiveness of a surrogate endpoint either rely on strict parametric modeling assumptions or require that the primary outcome and surrogate marker are fully observed that is, not subject to censoring. Moreover, available methods for quantifying surrogacy typically provide a proportion of treatment effect explained (PTE) measure and do not directly address the important questions of whether and how the trial can be ended earlier using the surrogate marker. In this article, we specifically address these important questions by proposing a PTE measure to quantify the feasibility of ending trials early based on endpoint information collected at an earlier landmark point t 0 in a time-to-event outcome setting. We provide a framework for deriving an optimally predicted outcome for individual patients at t 0 based on a combination of surrogate marker and event time information in the presence of censoring. We propose a non-parametric estimator for the PTE measure and derive the asymptotic properties of our estimators. Finite sample performance of our estimators are illustrated via extensive simulation studies and a real data application examining the potential of hemoglobin A1c and fasting plasma glucose to predict treatment effects on long term diabetes risk based on the Diabetes Prevention Program study.

Measuring Surrogacy in Clinical Research: With an application to studying surrogate markers for HIV Treatment-as-Prevention

In clinical research, validated surrogate markers are highly desirable in study design, monitoring, and analysis, as they do not only reduce the required sample size and follow-up duration, but also facilitate scientific discoveries. However, challenges exist to identify a reliable marker. One particular statistical challenge arises on how to measure and rank the surrogacy of potential markers quantitatively. We review the main statistical methods for evaluating surrogate markers. In addition, we suggest a new measure, the so-called "population surrogacy fraction of treatment effect," or simply the p-measure, in the setting of clinical trials. The p-measure carries an appealing population impact interpretation and supplements the existing statistical measures of surrogacy by providing "absolute" information. We apply the new measure along with other prominent measures to the HIV Prevention Trial Network 052 Study, a landmark trial for HIV/AIDS treatment-as-prevention.

Simple Methods for Evaluating 4 Types of Biomarkers: Surrogate Endpoint, Prognostic, Predictive, and Cancer Screening

We review simple methods for evaluating 4 types of biomarkers. First, we discuss the evaluation of surrogate endpoint biomarkers (to shorten a randomized trial) using 2 statistical and 3 biological criteria. Second, we discuss the evaluation of prognostic biomarkers (to predict the risk of disease) by comparing data collection costs with the anticipated net benefit of risk prediction. Third, we discuss the evaluation of predictive markers (to search for a promising subgroup in a randomized trial) using a multivariate subpopulation treatment effect pattern plot involving a risk difference or responders-only benefit function. Fourth, we discuss the evaluation of cancer screening biomarkers (to predict cancer in asymptomatic persons) using methodology to substantially reduce the sample size with stored specimens.

Longitudinal serum biomarker screening identifies malate dehydrogenase 2 as candidate prognostic biomarker for Duchenne muscular dystrophy

BACKGROUND

Duchenne muscular dystrophy (DMD) is a fatal disease for which no cure is available. Clinical trials have shown to be largely underpowered due to inter-individual variability and noisy outcome measures. The availability of biomarkers able to anticipate clinical benefit is highly needed to improve clinical trial design and facilitate drug development.

METHODS

In this study, we aimed to appraise the value of protein biomarkers to predict prognosis and monitor disease progression or treatment outcome in patients affected by DMD. We collected clinical data and 303 blood samples from 157 DMD patients in three clinical centres; 78 patients contributed multiple blood samples over time, with a median follow-up time of 2 years. We employed linear mixed models to identify biomarkers that are associated with disease progression, wheelchair dependency, and treatment with corticosteroids and performed survival analysis to find biomarkers whose levels are associated with time to loss of ambulation.

RESULTS

Our analysis led to the identification of 21 proteins whose levels significantly decrease with age and nine proteins whose levels significantly increase. Seven of these proteins are also differentially expressed in non-ambulant patients, and three proteins are differentially expressed in patients treated with glucocorticosteroids. Treatment with corticosteroids was found to partly counteract the effect of disease progression on two biomarkers, namely, malate dehydrogenase 2 (MDH2, P = 0.0003) and ankyrin repeat domain 2 (P = 0.0005); however, patients treated with corticosteroids experienced a further reduction on collagen 1 serum levels (P = 0.0003), especially following administration of deflazacort. A time to event analysis allowed to further support the use of MDH2 as a prognostic biomarker as it was associated with an increased risk of wheelchair dependence (P = 0.0003). The obtained data support the prospective evaluation of the identified biomarkers in natural history and clinical trials as exploratory biomarkers.

CONCLUSIONS

We identified a number of serum biomarkers associated with disease progression, loss of ambulation, and treatment with corticosteroids. The identified biomarkers are promising candidate prognostic and surrogate biomarkers, which may support drug developers if confirmed in prospective studies. The serum levels of MDH2 are of particular interest, as they correlate with disease stage and response to treatment with corticosteroids, and are also associated with the risk of wheelchair dependency and pulmonary function.

Evaluation of surrogacy in the multi-trial setting based on information theory: an extension to ordinal outcomes

In clinical trials, surrogate outcomes are early measures of treatment effect that are used to predict treatment effect on a later primary outcome of interest: the primary outcome therefore does not need to be observed and trials can be shortened. Evaluating surrogates is a complex area as a given treatment can act through multiple pathways, some of which may circumvent the surrogate. One of the best established and practically sound approaches to surrogacy evaluation is based on information theory. We have extended this approach to the case of ordinal outcomes, which are used as primary outcomes in many medical areas. This extension provides researchers with the means of evaluating surrogates in this setting, which expands the usefulness of the information theory approach while also demonstrating its versatility.

Summary of the Third Annual Blood and Marrow Transplant Clinical Trials Network Myeloma Intergroup Workshop on Minimal Residual Disease and Immune Profiling

The third annual Blood and Marrow Transplant Clinical Trials Network (BMT CTN) Myeloma Intergroup Workshop on Minimal Residual Disease and Immune Profiling was held on November 29, 2018, at the American Society of Hematology (ASH) annual meeting. This workshop featured the latest research focused on minimal residual disease (MRD) assessment and immune profiling (IP) in myeloma as well as discussion of the statistical and regulatory issues intrinsic to the development of MRD as a surrogate endpoint. In this report, we provide a summary of the workshop and focus on the integration of MRD and IP assessment into trial design and clinical practice.

Neoadjuvant rectal score as individual-level surrogate for disease-free survival in rectal cancer in the CAO/ARO/AIO-04 randomized phase III trial

Background

Surrogate end points in rectal cancer after preoperative chemoradiation are lacking as their statistical validation poses major challenges, including confirmation based on large phase III trials. We examined the prognostic role and individual-level surrogacy of neoadjuvant rectal (NAR) score that incorporates weighted cT, ypT and ypN categories for disease-free survival (DFS) in 1191 patients with rectal carcinoma treated within the CAO/ARO/AIO-04 phase III trial.

Patients and methods

Cox regression models adjusted for treatment arm, resection status, and NAR score were used in multivariable analysis. The four Prentice criteria (PC1-4) were used to assess individual-level surrogacy of NAR for DFS.

Results

After a median follow-up of 50 months, the addition of oxaliplatin to fluorouracil-based chemoradiotherapy (CRT) significantly improved 3-year DFS [75.9% (95% confidence interval [CI] 72.30% to 79.50%) versus 71.3% (95% CI 67.60% to 74.90%); P = 0.034; PC 1) and resulted in a shift toward lower NAR groups (P = 0.034, PC 2) compared with fluorouracil-only CRT. The 3-year DFS was 91.7% (95% CI 88.2% to 95.2%), 81.8% (95% CI 78.4% to 85.1%), and 58.1% (95% CI 52.4% to 63.9%) for low, intermediate, and high NAR score, respectively (P < 0.001; PC 3). NAR score remained an independent prognostic factor for DFS [low versus high NAR: hazard ratio (HR) 4.670; 95% CI 3.106-7.020; P < 0.001; low versus intermediate NAR: HR 1.971; 95% CI 1.303-2.98; P = 0.001] in multivariable analysis. Notwithstanding the inherent methodological difficulty in interpretation of PC 4 to establish surrogacy, the treatment effect on DFS was captured by NAR, supporting satisfaction of individual-level PC 4.

Conclusion

Our study validates the prognostic role and individual-level surrogacy of NAR score for DFS within a large randomized phase III trial. NAR score could help oncologists to speed up response-adapted therapeutic decision, and further large phase III trial data sets should aim to confirm trial-level surrogacy.

Radiation dose-response (a Bayesian model) in the radiotherapy of the localized prostatic adenocarcinoma: the reliability of PSA slope changes as a response surrogate endpoint

Purpose

One of the characteristics of Prostate-Specific Antigen (PSA) is PSA slope. It is the rate of diminishing PSA marker over time after radiotherapy (RT) in prostate cancer (PC) patients. The purpose of this study was to evaluate the relationship between increasing RT doses and PSA slope as a potential surrogate for PC recurrence.

Patients and Methods

This retrospective study was conducted on PC patients who were treated by radiotherapy in the Cancer Institute of Iran during 2007–2012. By reviewing the records of these patients, the baseline PSA measurement before treatment (iPSA), Gleason score (GS), clinical T stage (T. stage), and periodic PSA measurements after RT and the total radiation dose received were extracted for each patient separately. We used a Bayesian dose-response model, analysis of variance, Kruskal–Wallis test, Kaplan–Meier product-limit method for analysis. Probability values less 0.05 were considered statistically significant.

Results

Based on the D’Amico risk assessment system, 13.34% of patients were classified as “Low Risk”, 51.79% were “Intermediate Risk”, and 34.87% were “High Risk”. In terms of radiation doses, 12.31% of the patients received fewer than 50 Gy, 15.38% received 50 to 69 Gy, 61.03% received 70 Gy, and 11.28% received more than 70 Gy. The PSA values decreased after RT for all dose levels. The slope of PSA changes was negative for 176 of 195 patients. By increasing the dosage of radiation, the PSA decreased but these changes were not statistically significant (p = 0.701) and PSA slope as a surrogate end point cannot met the Prentice’s criteria for PC recurrence.

Conclusion

Significant changes in the dose-response relationship were not observed when the PSA slope was considered as the response criterion. Therefore, although the absolute value of the PSA decreased with increasing doses of RT, the relationship between PSA slope changes and increasing doses was not clear and cannot be used as a reliable response surrogate endpoint.

Five criteria for using a surrogate endpoint to predict treatment effect based on data from multiple previous trials

A surrogate endpoint in a randomized clinical trial is an endpoint that occurs after randomization and before the true, clinically meaningful, endpoint that yields conclusions about the effect of treatment on true endpoint. A surrogate endpoint can accelerate the evaluation of new treatments but at the risk of misleading conclusions. Therefore, criteria are needed for deciding whether to use a surrogate endpoint in a new trial. For the meta-analytic setting of multiple previous trials, each with the same pair of surrogate and true endpoints, this article formulates 5 criteria for using a surrogate endpoint in a new trial to predict the effect of treatment on the true endpoint in the new trial. The first 2 criteria, which are easily computed from a zero-intercept linear random effects model, involve statistical considerations: an acceptable sample size multiplier and an acceptable prediction separation score. The remaining 3 criteria involve clinical and biological considerations: similarity of biological mechanisms of treatments between the new trial and previous trials, similarity of secondary treatments following the surrogate endpoint between the new trial and previous trials, and a negligible risk of harmful side effects arising after the observation of the surrogate endpoint in the new trial. These 5 criteria constitute an appropriately high bar for using a surrogate endpoint to make a definitive treatment recommendation.

Relapse-Free Survival as a Surrogate for Overall Survival in the Evaluation of Stage II-III Melanoma Adjuvant Therapy

Background

We assessed whether relapse-free survival (RFS; time until recurrence/death) is a valid surrogate for overall survival (OS) among resected stage II-III melanoma patients through a meta-analysis of randomized controlled trials.

Methods

Individual patient data (IPD) on RFS and OS were collected from 5826 patients enrolled in 11 randomized adjuvant trials comparing interferon (IFN) to observation. In addition, IPD from two studies comparing IFN and vaccination in 989 patients were included. A two-level modeling approach was used for assessing Spearman's patient-level correlation (rho) of RFS and OS and the trial-level coefficient of determination (R²) of the treatment effects on RFS and on OS. The results were validated externally in 13 adjuvant studies without available IPD. We then tested the results on the European Organisation for Research and Treatment of Cancer (EORTC) 18071 double-blind trial comparing ipilimumab 10 mg/kg with placebo, which showed a statistically significant impact of the checkpoint inhibitor on RFS and OS. All statistical tests were two-sided.

Results

With a median follow-up of seven years, 12 of 13 trials showed a consistency between the IFN vs No IFN differences regarding RFS (hazard ratio [HR]RFS = 0.88) and OS (HROS = 0.91), but the small trial, Eastern Cooperative Oncology Group 2696, was an outlier (HRRFS = 0.72 vs HROS = 1.11). Therefore, even if rho was high, R² was low and could not reliably be estimated. Based on the 12 trials, rho remained high (0.89), and the hazard ratios for RFS and OS were strongly correlated (R² = 0.91). The surrogate threshold effect for RFS was estimated to be 0.77. For the EORTC 18071 trial, the hazard ratio for RFS was 0.75, predicting an effect of ipilimumab on OS. This was subsequently confirmed (HROS = 0.72, 95.1% confidence interval = 0.58 to 0.88, P = .001).

Conclusions

In high-risk stage II-III melanoma, RFS appeared to be a valid surrogate end point for OS for adjuvant randomized studies assessing interferon or a checkpoint inhibitor. In future similar adjuvant studies, a hazard ratio for RFS of 0.77 or less would predict a treatment impact on OS.

Use of Surrogate Outcomes in Nephrology Research

Clinical trials are large and expensive and could require exceedingly long-term follow-up for subjects to reach clinically meaningful end points. To combat these methodologic issues, researchers sometimes use biomarkers as surrogate end points. A biomarker is an objectively measured characteristic that is indicative of some underlying phenomenon or process, while a surrogate is a biomarker that "takes the place" of a clinically meaningful outcome, usually earlier in the disease process. This paper reviews the history, strengths, and weaknesses of surrogate outcome use in clinical research and then discusses potential surrogate outcomes in nephrology research.

Progression-Free Survival, Response Rate, and Disease Control Rate as Predictors of Overall Survival in Phase III Randomized Controlled Trials Evaluating the First-Line Chemotherapy for Advanced, Locally Advanced, and Recurrent Non-Small Cell Lung Carcinoma

INTRODUCTION

Recent improvements in chemotherapy agents have prolonged postprogression survival of non-small cell lung cancer. Thus, primary outcomes other than overall survival (OS) have been frequently used for recent phase III trials to obtain quick results. However, no systematic review had assessed whether progression-free survival (PFS), response rate (RR), and disease control rate (DCR) can serve as surrogates for OS at the trial level in the phase III first-line chemotherapy setting.

METHODS

We included phase III randomized clinical trials (RCTs) comparing two arms that were reported as a full article regardless of their primary end point. We included only RCTs that evaluated chemonaive patients with advanced, locally advanced, or metastatic non-small cell lung cancer and were published after January 1, 2005. We systematically searched four public electronic databases. Two investigators independently screened and scrutinized candidate articles. How surrogate outcomes represented hazard ratios (HRs) for OS was examined.

RESULTS

Among 1907 articles, we ultimately found 44 eligible articles covering 22,709 subjects. HR for PFS, median PFS in the experimental arm minus median PFS in the control arm in months, OR for RR (ORrr), and OR for DCR were evaluated in 34, 35, 44, and 35 RCTs, respectively. HR for OS (HRos), median PFS in the experimental arm minus median PFS in the control arm, ORrr, and OR for DCR had weighted Spearman's rank correlation coefficients with an HRos of 0.496, 0.477, 0.570, and 0.470, respectively; the standardized weighted regression coefficients were 0.439, -0.376, -0.605, and -0.381, respectively; and the adjusted weighted coefficients of determination were 0.224, 0.161, 0.350, and 0.176, respectively.

CONCLUSIONS

ORrr, followed by HRpfs, had the strongest association with HRos at the trial level. However, these measures were not strong enough to replace OS.

Robust estimation of the proportion of treatment effect explained by surrogate marker information

In randomized treatment studies where the primary outcome requires long follow-up of patients and/or expensive or invasive obtainment procedures, the availability of a surrogate marker that could be used to estimate the treatment effect and could potentially be observed earlier than the primary outcome would allow researchers to make conclusions regarding the treatment effect with less required follow-up time and resources. The Prentice criterion for a valid surrogate marker requires that a test for treatment effect on the surrogate marker also be a valid test for treatment effect on the primary outcome of interest. Based on this criterion, methods have been developed to define and estimate the proportion of treatment effect on the primary outcome that is explained by the treatment effect on the surrogate marker. These methods aim to identify useful statistical surrogates that capture a large proportion of the treatment effect. However, current methods to estimate this proportion usually require restrictive model assumptions that may not hold in practice and thus may lead to biased estimates of this quantity. In this paper, we propose a nonparametric procedure to estimate the proportion of treatment effect on the primary outcome that is explained by the treatment effect on a potential surrogate marker and extend this procedure to a setting with multiple surrogate markers. We compare our approach with previously proposed model-based approaches and propose a variance estimation procedure based on a perturbation-resampling method. Simulation studies demonstrate that the procedure performs well in finite samples and outperforms model-based procedures when the specified models are not correct. We illustrate our proposed procedure using a data set from a randomized study investigating a group-mediated cognitive behavioral intervention for peripheral artery disease participants.

Variation over time and interdependence between disease progression and death among patients with glioblastoma on RTOG 0525

BACKGROUND

We assessed the longitudinal hazard characteristics for death and progression in patients with glioblastoma, evaluated the impact of prognostic factors and treatment on the hazard within different time intervals to determine if effects are time varying, and quantified the influence of progression on survival.

METHODS

Among patients randomized to Radiation Therapy Oncology Group trial 0525, which compared dose-dense with standard-dose temozolomide, we estimated the hazards of death and treatment failure (death or progression) over time and their interdependence.

RESULTS

The peak hazard of death was reached at around 16 months with a slow decline after that; the hazard of progression/death reached a peak at around 6 months and decreased dramatically thereafter. The survival advantages for patients with MGMT gene promoter methylation and recursive partitioning analysis class III were substantial in the first 2 years, but lessened thereafter. The progression-free survival benefit of dose-dense over standard-dose temozolomide occurred in the first 6 months (hazard ratio: 0.70; 95% CI: 0.58-0.86; P < .001), although it diminished thereafter. After adjusting for recursive partitioning analysis class and MGMT methylation status, the hazard ratio of death for patients who had progressed over nonprogressors was 6.59 (95% CI: 5.15-8.43; P < .001).

CONCLUSION

After the peak hazard of death, a consistently high hazard remains, but it is lower than in the peak period. The progression hazard peak is earlier, and then hazard consistently declines. The rate of dying after disease progression is about 6.59 times the rate for nonprogressors, suggesting that progression-free survival may be a relevant clinical endpoint.

Center-Within-Trial Versus Trial-Level Evaluation of Surrogate Endpoints

Evaluation of candidate surrogate endpoints using individual patient data from multiple clinical trials is considered the gold standard approach to validate surrogates at both patient and trial levels. However, this approach assumes the availability of patient-level data from a relatively large collection of similar trials, which may not be possible to achieve for a given disease application. One common solution to the problem of too few similar trials involves performing trial-level surrogacy analyses on trial sub-units (e.g., centers within trials), thereby artificially increasing the trial-level sample size for feasibility of the multi-trial analysis. To date, the practical impact of treating trial sub-units (centers) identically to trials in multi-trial surrogacy analyses remains unexplored, and conditions under which this ad hoc solution may in fact be reasonable have not been identified. We perform a simulation study to identify such conditions, and demonstrate practical implications using a multi-trial dataset of patients with early stage colon cancer.

Surrogate measures and consistent surrogates

Surrogates which allow one to predict the effect of the treatment on the outcome of interest from the effect of the treatment on the surrogate are of importance when it is difficult or expensive to measure the primary outcome. Unfortunately, the use of such surrogates can give rise to paradoxical situations in which the effect of the treatment on the surrogate is positive, the surrogate and outcome are strongly positively correlated, but the effect of the treatment on the outcome is negative, a phenomenon sometimes referred to as the "surrogate paradox." New results are given for consistent surrogates that extend the existing literature on sufficient conditions that ensure the surrogate paradox is not manifest. Specifically, it is shown that for the surrogate paradox to be manifest it must be the case that either there is (i) a direct effect of treatment on the outcome not through the surrogate and in the opposite direction as that through the surrogate or (ii) confounding for the effect of the surrogate on the outcome, or (iii) a lack of transitivity so that treatment does not positively affect the surrogate for all the same individuals for whom the surrogate positively affects the outcome. The conditions for consistent surrogates and the results of the article are important because they allow investigators to predict the direction of the effect of the treatment on the outcome simply from the direction of the effect of the treatment on the surrogate. These results on consistent surrogates are then related to the four approaches to surrogate outcomes described by Joffe and Greene (2009, Biometrics 65, 530-538) to assess whether the standard criteria used by these approaches to assess whether a surrogate is "good" suffice to avoid the surrogate paradox.

Evaluation of early efficacy endpoints for proof-of-concept trials

A Phase II proof-of-concept (POC) trial usually uses an early efficacy endpoint other than a clinical endpoint as the primary endpoint. Because of the advancement in bioscience and technology, which has yielded a number of new surrogate biomarkers, drug developers often have more candidate endpoints to choose from than they can handle. As a result, selection of endpoint and its effect size as well as choice of type I/II error rates are often at the center of heated debates in design of POC trials. While optimization of the trade-off between benefit and cost is the implicit objective in such a decision-making process, it is seldom explicitly accounted for in practice. In this research note, motivated by real examples from the oncology field, we provide practical measures for evaluation of early efficacy endpoints (E4) for POC trials. We further provide optimal design strategies for POC trials that include optimal Go-No Go decision criteria for initiation of Phase III and optimal resource allocation strategies for conducting multiple POC trials in a portfolio under fixed resources. Although oncology is used for illustration purpose, the same idea developed in this research note also applies to similar situations in other therapeutic areas or in early-stage drug development in that a Go-No Go decision has to rely on limited data from an early efficacy endpoint and cost-effectiveness is the main concern.

Accommodating missingness when assessing surrogacy via principal stratification

BACKGROUND

When an outcome of interest in a clinical trial is late-occurring or difficult to obtain, surrogate markers can extract information about the effect of the treatment on the outcome of interest. Understanding associations between the causal effect (CE) of treatment on the outcome and the causal effect of treatment on the surrogate is critical to understanding the value of a surrogate from a clinical perspective.

PURPOSE

Traditional regression approaches to determine the proportion of the treatment effect explained by surrogate markers suffer from several shortcomings: they can be unstable and can lie outside the 0-1 range. Furthermore, they do not account for the fact that surrogate measures are obtained post randomization, and thus, the surrogate-outcome relationship may be subject to unmeasured confounding.

METHODS

to avoid these problems are of key importance. Methods Frangakis and Rubin suggested assessing the CE within prerandomization 'principal strata' defined by the counterfactual joint distribution of the surrogate marker under the different treatment arms, with the proportion of the overall outcome CE attributable to subjects for whom the treatment affects the proposed surrogate as the key measure of interest. Li et al. developed this 'principal surrogacy' approach for dichotomous markers and outcomes, utilizing Bayesian methods that accommodated nonidentifiability in the model parameters. Because the surrogate marker is typically observed early, outcome data are often missing. Here, we extend Li et al. to accommodate missing data in the observable final outcome under ignorable and nonignorable settings. We also allow for the possibility that missingness has a counterfactual component, a feature that previous literature has not addressed.

RESULTS

We apply the proposed methods to a trial of glaucoma control comparing surgery versus medication, where intraocular pressure (IOP) control at 12 months is a surrogate for IOP control at 96 months. We also conduct a series of simulations to consider the impacts of nonignorability, as well as sensitivity to priors and the ability of the decision information criterion (DIC) to choose the correct model when parameters are not fully identified.

LIMITATIONS

Because model parameters cannot be fully identified from data, informative priors can introduce nontrivial bias in moderate sample size settings, while more noninformative priors can yield wide credible intervals.

CONCLUSIONS

Assessing the linkage between CEs of treatment on a surrogate marker and CEs of a treatment on an outcome is important to understanding the value of a marker. These CEs are not fully identifiable; hence, we explore the sensitivity and identifiability aspects of these models and show that relatively weak assumptions can still yield meaningful results.

Review of meta-analyses evaluating surrogate endpoints for overall survival in oncology

Overall survival (OS) is the gold standard in measuring the treatment effect of new drug therapies for cancer. However, practical factors may preclude the collection of unconfounded OS data, and surrogate endpoints are often used instead. Meta-analyses have been widely used for the validation of surrogate endpoints, specifically in oncology. This research reviewed published meta-analyses on the types of surrogate measures used in oncology studies and examined the extent of correlation between surrogate endpoints and OS for different cancer types. A search was conducted in October 2010 to compile available published evidence in the English language for the validation of disease progression-related endpoints as surrogates of OS, based on meta-analyses. We summarize published meta-analyses that quantified the correlation between progression-based endpoints and OS for multiple advanced solid-tumor types. We also discuss issues that affect the interpretation of these findings. Progression-free survival is the most commonly used surrogate measure in studies of advanced solid tumors, and correlation with OS is reported for a limited number of cancer types. Given the increased use of crossover in trials and the availability of second-/third-line treatment options available to patients after progression, it will become increasingly more difficult to establish correlation between effects on progression-free survival and OS in additional tumor types.

Surrogate end points in secondary analyses of cardiovascular trials

A surrogate end point is one that is used as a substitute for a clinical end point of more direct interest, usually for reasons of practicality, and that is expected to predict clinical benefit. Surrogate end points play a critical role in the advancement of all medical research, and cardiovascular (CV) research in particular. However, the relationship between a surrogate end point and its clinical end point is usually complex, and there are many examples where results based on surrogates have proved to be misleading. Secondary analyses of existing clinical trial data are likely to involve surrogate end points, if only because clinical end points will have been extensively studied as part of the primary analysis of a trial large enough to collect useful clinical end point data. Validation of a surrogate end point is a laudable goal for a secondary analysis of a large clinical end point trial (or meta-analysis of multiple smaller trials), and the result may be an important new tool for further study of a class of compounds in a particular disease context. Secondary analyses using surrogate end points may also provide new insight into disease or treatment mechanism, but as with any surrogate end point analysis, the results can mislead, and the existing literature is heavy on application and light on methodology. Surrogate end points often substitute efficiency for clarity, and while many interesting and potentially informative secondary analyses of CV trials will involve surrogates, results are likely to be ambiguous and should be interpreted with care.

Comparing biomarkers as principal surrogate endpoints

Recently a new definition of surrogate endpoint, the "principal surrogate," was proposed based on causal associations between treatment effects on the biomarker and on the clinical endpoint. Despite its appealing interpretation, limited research has been conducted to evaluate principal surrogates, and existing methods focus on risk models that consider a single biomarker. How to compare principal surrogate value of biomarkers or general risk models that consider multiple biomarkers remains an open research question. We propose to characterize a marker or risk model's principal surrogate value based on the distribution of risk difference between interventions. In addition, we propose a novel summary measure (the standardized total gain) that can be used to compare markers and to assess the incremental value of a new marker. We develop a semiparametric estimated-likelihood method to estimate the joint surrogate value of multiple biomarkers. This method accommodates two-phase sampling of biomarkers and is more widely applicable than existing nonparametric methods by incorporating continuous baseline covariates to predict the biomarker(s), and is more robust than existing parametric methods by leaving the error distribution of markers unspecified. The methodology is illustrated using a simulated example set and a real data set in the context of HIV vaccine trials.

Magnetic resonance active lesions as individual-level surrogate for relapses in multiple sclerosis

Background: Use of quantitative magnetic resonance imaging (MRI) metrics as surrogates for clinical outcomes in multiple sclerosis (MS) trials is controversial.

Objectives: We sought to validate, at the individual-patient level, the number of MRI active lesions, as a surrogate marker for relapses in MS.

Methods: Individual-patient data from two large, placebo-controlled clinical trials of subcutaneous interferon β-1a in patients with relapsing–remitting or secondary progressive (SP) MS were analysed separately and as pooled data. The four Prentice criteria were applied to assess surrogacy for the number of new T2 MRI lesions. The predictive value of short-term treatment effects on this MRI marker for longer-term clinical relapses was also assessed.

Results: All Prentice criteria were satisfied. The number of new T2 MRI lesions correlated with the number of relapses over the follow-up period. The proportion of treatment effect on relapses accounted for by the effect of treatment on new T2 MRI lesions over 2 years was 53% in patients with relapsing–remitting MS, 67% in patients with secondary progressive MS, and 62% in pooled data. In the pooled data, treatment effects on new lesions over 1 year mediated a good proportion (70%) of effects on relapses over the subsequent year.

Conclusions: This study provides evidence that new T2 MRI lesion count is a surrogate for relapses in patients with MS treated with interferon or drugs with a similar mechanism of action. Short-term treatment effects on this MRI measure can predict longer-term effects on relapses.

Use of meta-analysis for the validation of surrogate endpoints and biomarkers in cancer trials

This article discusses statistical approaches to the validation of surrogate biomarkers and endpoints. One approach that has been successfully used in oncology consists of estimating associations at two levels: the association between the surrogate and the clinical endpoint, called the individual-level association, and the association between the effects of treatment on the surrogate and the clinical endpoint, called the trial-level association. This approach requires data to be available from multiple randomized trials, such as in a meta-analysis of trials based on individual patient data. The approach is illustrated using randomized trials of first-line treatments for advanced tumors of the colon, breast, ovary, and prostate. Data from several meta-analyses suggest that progression-free survival is an acceptable surrogate in advanced colorectal and ovarian cancer, but not in breast and prostate cancer.

Biomarkers and surrogate end points--the challenge of statistical validation

Biomarkers and surrogate end points have great potential for use in clinical oncology, but their statistical validation presents major challenges, and few biomarkers have been robustly confirmed. Provisional supportive data for prognostic biomarkers, which predict the likely outcome independently of treatment, is possible through small retrospective studies, but it has proved more difficult to achieve robust multi-site validation. Predictive biomarkers, which predict the likely response of patients to specific treatments, require more extensive data for validation, specifically large randomized clinical trials and meta-analysis. Surrogate end points are even more challenging to validate, and require data demonstrating both that the surrogate is prognostic for the true end point independently of treatment, and that the effect of treatment on the surrogate reliably predicts its effect on the true end point. In this Review, we discuss the nature of prognostic and predictive biomarkers and surrogate end points, and examine the statistical techniques and designs required for their validation. In cases where the statistical requirements for validation cannot be rigorously achieved, the biological plausibility of an end point or surrogate might support its adoption. No consensus yet exists on processes or standards for pragmatic evaluation and adoption of biomarkers and surrogate end points in the absence of robust statistical validation.

Evidence for endoscopic ulcers as meaningful surrogate endpoint for clinically significant upper gastrointestinal harm

BACKGROUND & AIMS

Surrogate endpoints are biomarkers intended to substitute for a clinical endpoint. Are endoscopic ulcers a useful surrogate endpoint for a biological progression to clinical endpoints of ulcer complications (perforation, ulcers, and bleeds), hospital admission, or death?

METHODS

Review of randomized trials, meta-analyses, clinical outcomes trials, and observational studies.

RESULTS

No large study examined both endoscopic and clinical endpoints. Endoscopic ulcers and clinically significant ulcer complications were affected in the same direction and to about the same extent in 4 distinct circumstances: (1) by risk factors-age, previous history of symptomatic ulcer or bleeding, Helicobacter pylori, aspirin; (2) in studies of antiulcer treatments with differing modes of action, especially in relation to nonsteroidal anti-inflammatory drug toxicity, and Helicobacter pylori infection; (3) in studies evaluating ulcer complications with Cox-2 selective drugs and nonsteroidal anti-inflammatory drugs; and (4) in studies of interventions in patients with high risk of recurrent ulcer bleed needing nonsteroidal anti-inflammatory drug therapy. All study designs showed consistent and reproducible effects on gastrointestinal ulcer complications paralleling endoscopy.

CONCLUSIONS

Consistent and plausible findings from disparate populations and designs make endoscopic ulcers a strong candidate for surrogacy, though direct progression from endoscopic ulcers to ulcer complications cannot be demonstrated. Large outcome studies are needed to establish the power of the surrogacy, absolute risk of clinical outcomes, and to identify the totality of risks and benefits of new pharmacologic therapies.

On assessing surrogacy in a single trial setting using a semicompeting risks paradigm

SUMMARY

There has been a recent emphasis on the identification of biomarkers and other biologic measures that may be potentially used as surrogate endpoints in clinical trials. We focus on the setting of data from a single clinical trial. In this article, we consider a framework in which the surrogate must occur before the true endpoint. This suggests viewing the surrogate and true endpoints as semicompeting risks data; this approach is new to the literature on surrogate endpoints and leads to an asymmetrical treatment of the surrogate and true endpoints. However, such a data structure also conceptually complicates many of the previously considered measures of surrogacy in the literature. We propose novel estimation and inferential procedures for the relative effect and adjusted association quantities proposed by Buyse and Molenberghs (1998, Biometrics 54, 1014-1029). The proposed methodology is illustrated with application to simulated data, as well as to data from a leukemia study.

Meta-analysis for the evaluation of surrogate endpoints in cancer clinical trials

The identification and validation of putative surrogate endpoints in oncology is a great challenge to medical investigators, statisticians, and regulators. A putative surrogate endpoint must be validated at both individual-level and trial-level before it can be used to replace the clinical endpoint in a future clinical trial. Recently, meta-analytic methods for evaluating potential surrogates have become widely accepted in cancer clinical trials. In this review, after addressing multiple complications and general issues surrounding surrogate endpoints, we review various proposed and adopted meta-analytic methodologies pertaining to the application of these methods to oncology clinical trials with different tumor types. In oncology, several applications have successfully identified useful surrogates. For example, disease-free survival and progression-free survival have been validated through meta-analyses as acceptable surrogates for overall survival in adjuvant colon cancer and advanced colorectal cancer, respectively. We also discuss several limitations of surrogate endpoints, including the critical issues that the extrapolation of the validity of a surrogate is always context-dependent and that such extrapolation should be exercised with caution.

Evaluating a surrogate endpoint at three levels, with application to vaccine development

Identification of an immune response to vaccination that reliably predicts protection from clinically significant infection, i.e. an immunological surrogate endpoint, is a primary goal of vaccine research. Using this problem of evaluating an immunological surrogate as an illustration, we describe a hierarchy of three criteria for a valid surrogate endpoint and statistical analysis frameworks for evaluating them. Based on a placebo-controlled vaccine efficacy trial, the first level entails assessing the correlation of an immune response with a study endpoint in the study groups, and the second level entails evaluating an immune response as a surrogate for the study endpoint that can be used for predicting vaccine efficacy for a setting similar to that of the vaccine trial. We show that baseline covariates, innovative study design, and a potential outcomes formulation can be helpful for this assessment. The third level entails validation of a surrogate endpoint via meta-analysis, where the goal is to evaluate how well the immune response can be used to predict vaccine efficacy for new settings (building bridges). A simulated vaccine trial and two example vaccine trials are presented, one supporting that certain anti-influenza antibody levels are an excellent surrogate for influenza illness and another supporting that certain anti-HIV antibody levels are not useful as a surrogate for HIV infection.