Assessing surrogates as trial endpoints using mixed models

"Heterogeneity of treatment effect on patients' long-term outcome according to pathological response type in neoadjuvant RCTs for breast cancer."

INTRODUCTION

To provide evidence explaining the poor association between pCR and patients' long-term outcome at trial-level in neoadjuvant RCTs for breast cancer (BC), we performed a systematic-review and meta-analysis of all RCTs testing neoadjuvant treatments for early-BC and reporting the hazard ratio of DFS (HRDFS) for the intervention versus control arm stratified by pathological response type (i.e., pCR yes versus no).

METHODS

The objective was to explore differences of treatment effects on DFS across patients with and without pCR. We calculated the pooled HRDFS in the two strata of pathological response (i.e., pCR yes versus no) using a random-effects model, and assessed the difference between these two estimates using an interaction test.

RESULTS

Ten RCTs and 8496 patients were included in the analysis. Patients obtaining pCR in the intervention-arm had a higher, although not statistically significant, risk of DFS-event as compared with patients obtaining pCR in the control-arm: the pooled HRDFS for the experimental versus control arm was 1.23 (95%CI, 0.91-1.65). On the opposite, the risk of DFS-event was higher for control as compared with the intervention-arm in the stratum of patients without pCR: the pooled HRDFS was 0.86 (95%CI, 0.78-0.95). Treatment effect on DFS was significantly different according to pathological response type (interaction test p: 0.014).

CONCLUSION

We reported new evidence that contributes to explaining the poor surrogacy value of pCR at trial-level in neoadjuvant RCTs for early-BC.

Flexible evaluation of surrogacy in platform studies

Trial-level surrogates are useful tools for improving the speed and cost effectiveness of trials but surrogates that have not been properly evaluated can cause misleading results. The evaluation procedure is often contextual and depends on the type of trial setting. There have been many proposed methods for trial-level surrogate evaluation, but none, to our knowledge, for the specific setting of platform studies. As platform studies are becoming more popular, methods for surrogate evaluation using them are needed. These studies also offer a rich data resource for surrogate evaluation that would not normally be possible. However, they also offer a set of statistical issues including heterogeneity of the study population, treatments, implementation, and even potentially the quality of the surrogate. We propose the use of a hierarchical Bayesian semiparametric model for the evaluation of potential surrogates using nonparametric priors for the distribution of true effects based on Dirichlet process mixtures. The motivation for this approach is to flexibly model relationships between the treatment effect on the surrogate and the treatment effect on the outcome and also to identify potential clusters with differential surrogate value in a data-driven manner so that treatment effects on the surrogate can be used to reliably predict treatment effects on the clinical outcome. In simulations, we find that our proposed method is superior to a simple, but fairly standard, hierarchical Bayesian method. We demonstrate how our method can be used in a simulated illustrative example (based on the ProBio trial), in which we are able to identify clusters where the surrogate is, and is not useful. We plan to apply our method to the ProBio trial, once it is completed.

Handling missing within-study correlations in the evaluation of surrogate endpoints

Rigorous evaluation of surrogate endpoints is performed in a trial-level analysis in which the strength of the association between treatment effects on the clinical and surrogate endpoints is quantified across a collection of previously conducted trials. To reduce bias in measures of the performance of the surrogate, the statistical model must account for the sampling error in each trial's estimated treatment effects and their potential correlation. Unfortunately, these within-study correlations can be difficult to obtain, especially for meta-analysis of published trial results where individual patient data is not available. As such, these terms are frequently partially or completely missing in the analysis. We show that improper handling of these missing terms can meaningfully alter the perceived quality of the surrogate and we introduce novel strategies to handle the missingness.

Lessons Learned from Phase II and Phase III Trials Investigating Therapeutic Agents for Cerebral Ischemia Associated with Aneurysmal Subarachnoid Hemorrhage

One of the challenges in bringing new therapeutic agents (since nimodipine) in for the treatment of cerebral ischemia associated with aneurysmal subarachnoid hemorrhage (aSAH) is the incongruence in therapeutic benefit observed between phase II and subsequent phase III clinical trials. Therefore, identifying areas for improvement in the methodology and interpretation of results is necessary to increase the value of phase II trials. We performed a systematic review of phase II trials that continued into phase III trials, evaluating a therapeutic agent for the treatment of cerebral ischemia associated with aSAH. We followed the Preferred Reporting Items for Systematic reviews and Meta-Analyses guidelines for systematic reviews, and review was based on a peer-reviewed protocol (International Prospective Register of Systematic Reviews no. 222965). A total of nine phase III trials involving 7,088 patients were performed based on eight phase II trials involving 1558 patients. The following therapeutic agents were evaluated in the selected phase II and phase III trials: intravenous tirilazad, intravenous nicardipine, intravenous clazosentan, intravenous magnesium, oral statins, and intraventricular nimodipine. Shortcomings in several design elements of the phase II aSAH trials were identified that may explain the incongruence between phase II and phase III trial results. We suggest the consideration of the following strategies to improve phase II design: increased focus on the selection of surrogate markers of efficacy, selection of the optimal dose and timing of intervention, adjustment for exaggerated estimate of treatment effect in sample size calculations, use of prespecified go/no-go criteria using futility design, use of multicenter design, enrichment of the study population, use of concurrent control or placebo group, and use of innovative trial designs such as seamless phase II to III design. Modifying the design of phase II trials on the basis of lessons learned from previous phase II and phase III trial combinations is necessary to plan more effective phase III trials.

Evaluation of pathological complete response as surrogate endpoint in neoadjuvant randomised clinical trials of early stage breast cancer: systematic review and meta-analysis

OBJECTIVE

To evaluate pathological complete response as a surrogate endpoint for disease-free survival and overall survival in regulatory neoadjuvant trials of early stage breast cancer.

DESIGN

Systematic review and meta-analysis.

DATA SOURCES

Medline, Embase, and Scopus to 1 December 2020.

ELIGIBILITY CRITERIA FOR STUDY SELECTION

Randomised clinical trials that tested neoadjuvant chemotherapy given alone or combined with other treatments, including anti-human epidermal growth factor 2 (anti-HER2) drugs, targeted treatments, antivascular agents, bisphosphonates, and immune checkpoint inhibitors.

DATA EXTRACTION AND SYNTHESIS

Trial level associations between the surrogate endpoint pathological complete response and disease-free survival and overall survival.

METHODS

A weighted regression analysis was performed on log transformed treatment effect estimates (hazard ratio for disease-free survival and overall survival and relative risk for pathological complete response), and the coefficient of determination (R2) was used to quantify the association. The secondary objective was to explore heterogeneity of results in preplanned subgroups analysis, stratifying trials according treatment type in the experimental arm, definition used for pathological complete response (breast and lymph nodes v breast only), and biological features of the disease (HER2 positive or triple negative breast cancer). The surrogate threshold effect was also evaluated, indicating the minimum value of the relative risk for pathological complete response necessary to confidently predict a non-null effect on hazard ratio for disease-free survival or overall survival.

RESULTS

54 randomised clinical trials comprising a total of 32 611 patients were included in the analysis. A weak association was observed between the log(relative risk) for pathological complete response and log(hazard ratio) for both disease-free survival (R2=0.14, 95% confidence interval 0.00 to 0.29) and overall survival (R2 =0.08, 0.00 to 0.22). Similar results were found across all subgroups evaluated, independently of the definition used for pathological complete response, treatment type in the experimental arm, and biological features of the disease. The surrogate threshold effect was 5.19 for disease-free survival but was not estimable for overall survival. Consistent results were confirmed in three sensitivity analyses: excluding small trials (<200 patients enrolled), excluding trials with short median follow-up (<24 months), and replacing the relative risk for pathological complete response with the absolute difference of pathological complete response rates between treatment arms.

CONCLUSION

A lack of surrogacy of pathological complete response was identified at trial level for both disease-free survival and overall survival. The findings suggest that pathological complete response should not be used as primary endpoint in regulatory neoadjuvant trials of early stage breast cancer.

Assurance in vaccine efficacy clinical trial design based on immunological responses

The assurance of a future clinical trial is a key quantitative tool for decision-making in drug development. It is derived from prior knowledge (Bayesian approach) about the clinical endpoint of interest, typically from previous clinical trials. In this paper, we examine assurance in the specific context of vaccine development, where early development (Phase 2) is often based on immunological endpoints (e.g., antibody levels), while the confirmatory trial (Phase 3) is based on the clinical endpoint (very large sample sizes and long follow-up). Our proposal is to use the Phase 2 vaccine efficacy predicted by the immunological endpoint (using a model estimated from epidemiological studies) as prior information for the calculation of the assurance.

Summary of the Third Annual Blood and Marrow Transplant Clinical Trials Network Myeloma Intergroup Workshop on Minimal Residual Disease and Immune Profiling

The third annual Blood and Marrow Transplant Clinical Trials Network (BMT CTN) Myeloma Intergroup Workshop on Minimal Residual Disease and Immune Profiling was held on November 29, 2018, at the American Society of Hematology (ASH) annual meeting. This workshop featured the latest research focused on minimal residual disease (MRD) assessment and immune profiling (IP) in myeloma as well as discussion of the statistical and regulatory issues intrinsic to the development of MRD as a surrogate endpoint. In this report, we provide a summary of the workshop and focus on the integration of MRD and IP assessment into trial design and clinical practice.

Surrogate Endpoints and Drug Regulation: What Is Needed to Clarify the Evidence

The FDA's new table of surrogate endpoints used for drug approvals is an important step forward for overseeing the use of biomarkers in clinical trials. Nevertheless, we present several ways in which the table can be improved.

Surrogate and Intermediate Endpoints in Randomized Trials: What's the Goal?

Establishing trial-level surrogacy of an intermediate endpoint for predicting survival benefit in future trials is extremely challenging because of the extrapolations required, but there are other useful drug development and patient management applications of intermediate endpoints. Clin Cancer Res; 24(10); 2239-40. ©2018 AACRSee related article by Mushti et al., p. 2268.

Beyond Correlations, Sensitivities, and Specificities: A Roadmap for Demonstrating Utility of Advanced Imaging in Oncology Treatment and Clinical Trial Design

Despite the widespread belief that advanced imaging should be very helpful in guiding oncology treatment decision and improving efficiency and success rates in treatment clinical trials, its acceptance has been slow. Part of this is likely attributable to gaps in study design and statistical methodology for these imaging studies. Also, results supporting the performance of the imaging in these roles have largely been insufficient to justify their use within the design of a clinical trial or in treatment decision making. Statistically significant correlations between the imaging results and clinical outcomes are often incorrectly taken as evidence of adequate performance. Assessments of whether the imaging can outperform standard techniques or meaningfully supplement them are also frequently neglected. This paper provides guidance on study designs and statistical analyses for evaluating the performance of advanced imaging in the various roles in treatment decision guidance and clinical trial conduct. Relevant methodology from the imaging literature is reviewed; gaps in the literature are addressed using related concepts from the more extensive genomic and in vitro biomarker literature.

[Not Available]

HOW A BIOMARKER CAN BECOME AN ACCEPTABLE SUBSTITUTION CRITERIA ?: Numerous biomarkers of the treatment activity are now available as a result of the fascinating progresses in biology and biotechnology. Together with the rapidly growing understanding of the mechanisms of action of new agents, these biomarkers provide promising tools to evaluate early the effect of treatments against cancer. It is tempting to use these new markers of activity as primary endpoints to evaluate new treatments in the context of randomized clinical trials. Nevertheless, a mandatory preliminary step is to demonstrate that the two endpoints carry the same information in order to validate whether the biomarker is a surrogate of the final endpoint. We illustrate on several examples in prostate, gastric and early breast cancer that it is important to distinguish two levels of information: the individual level that allows to monitor a patient to anticipate treatment failure, and the trial level that enables to predict the treatment effect on the final endpoint based on the treatment effect measured on the surrogate endpoint. In several cases, the formal validation turned out to be disappointing despite strong biological rational.

Comparing biomarkers as trial level general surrogates

An intermediate response measure that accurately predicts efficacy in a new setting can reduce trial cost and time to product licensure. In this article, we define a trial level general surrogate, which is an intermediate response that can be used to accurately predict efficacy in a new setting. Methods for evaluating general surrogates have been developed previously. Many methods in the literature use trial level intermediate responses for prediction. However, all existing methods focus on surrogate evaluation and prediction in new settings, rather than comparison of candidate general surrogates, and few formalize the use of cross validation to quantify the expected prediction error. Our proposed method uses Bayesian non-parametric modeling and cross-validation to estimate the absolute prediction error for use in evaluating and comparing candidate trial level general surrogates. Simulations show that our method performs well across a variety of scenarios. We use our method to evaluate and to compare candidate trial level general surrogates in several multi-national trials of a pentavalent rotavirus vaccine. We identify at least one immune measure that has potential value as a trial level general surrogate and use it to predict efficacy in a new trial where the clinical outcome was not measured.

Early Change in Urine Protein as a Surrogate End Point in Studies of IgA Nephropathy: An Individual-Patient Meta-analysis

BACKGROUND

The role of change in proteinuria as a surrogate end point for randomized trials in immunoglobulin A nephropathy (IgAN) has previously not been thoroughly evaluated.

STUDY DESIGN

Individual patient-level meta-analysis.

SETTING & POPULATION

Individual-patient data for 830 patients from 11 randomized trials evaluating 4 intervention types (renin-angiotensin system [RAS] blockade, fish oil, immunosuppression, and steroids) examining associations between changes in urine protein and clinical end points at the individual and trial levels.

SELECTION CRITERIA FOR STUDIES

Randomized controlled trials of IgAN with measurements of proteinuria at baseline and a median of 9 (range, 5-12) months follow-up, with at least 1 further year of follow-up for the clinical outcome.

PREDICTOR

9-month change in proteinuria.

OUTCOME

Doubling of serum creatinine level, end-stage renal disease, or death.

RESULTS

Early decline in proteinuria at 9 months was associated with lower risk for the clinical outcome (HR per 50% reduction in proteinuria, 0.40; 95% CI, 0.32-0.48) and was consistent across studies. Proportions of treatment effect on the clinical outcome explained by early decline in proteinuria were estimated at 11% (95% CI, -19% to 41%) for RAS blockade and 29% (95% CI, 6% to 53%) for steroid therapy. The direction of the pooled treatment effect on early change in proteinuria was in accord with the direction of the treatment effect on the clinical outcome for steroids and RAS blockade. Trial-level analyses estimated that the slope for the regression line for the association of treatment effects on the clinical end points and for the treatment effect on proteinuria was 2.15 (95% Bayesian credible interval, 0.10-4.32).

LIMITATIONS

Study population restricted to 11 trials, all having fewer than 200 patients each with a limited number of clinical events.

CONCLUSIONS

Results of this analysis offer novel evidence supporting the use of an early reduction in proteinuria as a surrogate end point for clinical end points in IgAN in selected settings.

Statistical controversies in clinical research: assessing pathologic complete response as a trial-level surrogate end point for early-stage breast cancer

BACKGROUND

A trial-level surrogate end point for a randomized clinical trial may allow assessment of the relative benefits of the treatment to be performed at an earlier time point and potentially with a smaller sample size. However, determining whether an end point is a reliable trial-level surrogate based on results of previous trials is not straightforward. The question of trial-level surrogacy is easily confused with the question of individual-level surrogacy, and this confusion can lead to controversy. A recent example concerns the evaluation of pathologic complete response (pCR) as a surrogate for event-free survival (EFS) and overall survival (OS) in early-stage breast cancer.

MATERIALS AND METHODS

The differences between individual-level surrogacy (i.e. for patients receiving a specific treatment, the surrogate end point predicts the definitive end point) and trial-level surrogacy (the results of the trial for the surrogate end point predict the results of the trial for the definitive end point) are discussed. Trial-level data used in two previous meta-analyses evaluating pCR as a trial-level surrogate for EFS and OS were re-analyzed using methods that appropriately account for the variability in pCR rates as well as the variability in the hazard ratios for EFS and OS.

RESULTS

There is no evidence that pCR is a trial-level surrogate for EFS or OS, nor is there evidence that pCR could be used reliably to screen out nonpromising agents from further drug development.

CONCLUSIONS

At present, neoadjuvant RCTs should continue to follow patients to observe EFS and OS to assess clinical benefit, and they should be designed with sufficient sample size to reliably assess EFS. However, one cannot rule out the possibility that future meta-analyses involving more trials and in which the patient population or class of treatments is restricted could suggest the validity of pCR as a trial-level surrogate for EFS or OS in some focused settings.

Design Issues in Randomized Clinical Trials of Maintenance Therapies

A potential therapeutic strategy for patients who respond (or have stable disease) on a fixed-duration induction therapy is to receive maintenance therapy, typically given for a prolonged period of time. To enable patients and clinicians to make informed treatment decisions, the designs of phase III randomized clinical trials (RCTs) assessing maintenance strategies need to be such that their results will provide clear assessment of the relevant risks and benefits of these strategies. We review the key aspects of maintenance RCT designs. Important design considerations include choice of first-line and second-line therapies, minimizing between-arm differences in follow-up schedules, and choice of the primary endpoint. In order to change clinical practice, RCTs should be designed to accurately isolate and quantify the clinical benefit of maintenance as compared with the standard approach of fixed-duration induction followed by the second-line treatment at progression. To accomplish this, RCTs need to utilize an overall survival (or quality of life) endpoint or, in settings where this is not feasible, endpoints that incorporate the effects of the subsequent line of therapy (eg, time from randomization to second progression or death). Toxicity and symptom information over both the study treatment (maintenance) and the second-line treatment should also be collected and reported.

A Bayesian prediction model between a biomarker and the clinical endpoint for dichotomous variables

Background

Early biomarkers are helpful for predicting clinical endpoints and for evaluating efficacy in clinical trials even if the biomarker cannot replace clinical outcome as a surrogate. The building and evaluation of an association model between biomarkers and clinical outcomes are two equally important concerns regarding the prediction of clinical outcome. This paper is to address both issues in a Bayesian framework.

Methods

A Bayesian meta-analytic approach is proposed to build a prediction model between the biomarker and clinical endpoint for dichotomous variables. Compared with other Bayesian methods, the proposed model only requires trial-level summary data of historical trials in model building. By using extensive simulations, we evaluate the link function and the application condition of the proposed Bayesian model under scenario (i) equal positive predictive value (PPV) and negative predictive value (NPV) and (ii) higher NPV and lower PPV. In the simulations, the patient-level data is generated to evaluate the meta-analytic model. PPV and NPV are employed to describe the patient-level relationship between the biomarker and the clinical outcome. The minimum number of historical trials to be included in building the model is also considered.

Results

It is seen from the simulations that the logit link function performs better than the odds and cloglog functions under both scenarios. PPV/NPV ≥0.5 for equal PPV and NPV, and PPV + NPV ≥1 for higher NPV and lower PPV are proposed in order to predict clinical outcome accurately and precisely when the proposed model is considered. Twenty historical trials are required to be included in model building when PPV and NPV are equal. For unequal PPV and NPV, the minimum number of historical trials for model building is proposed to be five. A hypothetical example shows an application of the proposed model in global drug development.

Conclusions

The proposed Bayesian model is able to predict well the clinical endpoint from the observed biomarker data for dichotomous variables as long as the conditions are satisfied. It could be applied in drug development. But the practical problems in applications have to be studied in further research.

Electronic supplementary material

The online version of this article (doi:10.1186/1745-6215-15-500) contains supplementary material, which is available to authorized users.

Early change in proteinuria as a surrogate end point for kidney disease progression: an individual patient meta-analysis

BACKGROUND

It is controversial whether proteinuria is a valid surrogate end point for randomized trials in chronic kidney disease.

STUDY DESIGN

Meta-analysis of individual patient-level data.

SETTING & POPULATION

Individual patient data for 9,008 patients from 32 randomized trials evaluating 5 intervention types.

SELECTION CRITERIA FOR STUDIES

Randomized controlled trials of kidney disease progression until 2007 with measurements of proteinuria both at baseline and during the first year of follow-up, with at least 1 further year of follow-up for the clinical outcome.

PREDICTOR

Early change in proteinuria.

OUTCOMES

Doubling of serum creatinine level, end-stage renal disease, or death.

RESULTS

Early decline in proteinuria was associated with lower risk of the clinical outcome (pooled HR, 0.74 per 50% reduction in proteinuria); this association was stronger at higher levels of baseline proteinuria. Pooled estimates for the proportion of treatment effect on the clinical outcome explained by early decline in proteinuria ranged from -7.0% (95%CI, -40.6% to 26.7%) to 43.9% (95%CI, 25.3% to 62.6%) across 5 intervention types. The direction of the pooled treatment effects on early change in proteinuria agreed with the direction of the treatment effect on the clinical outcome for all 5 intervention types, with the magnitudes of the pooled treatment effects on the 2 end points agreeing for 4 of the 5 intervention types. The pooled treatment effects on both end points were simultaneously stronger at higher levels of proteinuria. However, statistical power was insufficient to determine whether differences in treatment effects on the clinical outcome corresponded to differences in treatment effects on proteinuria between individual studies.

LIMITATIONS

Limited variety of interventions tested and low statistical power for many chronic kidney disease clinical trials.

CONCLUSIONS

These results provide new evidence supporting the use of an early reduction in proteinuria as a surrogate end point, but do not provide sufficient evidence to establish its validity in all settings.

Meta-analysis of the technical performance of an imaging procedure: guidelines and statistical methodology

Medical imaging serves many roles in patient care and the drug approval process, including assessing treatment response and guiding treatment decisions. These roles often involve a quantitative imaging biomarker, an objectively measured characteristic of the underlying anatomic structure or biochemical process derived from medical images. Before a quantitative imaging biomarker is accepted for use in such roles, the imaging procedure to acquire it must undergo evaluation of its technical performance, which entails assessment of performance metrics such as repeatability and reproducibility of the quantitative imaging biomarker. Ideally, this evaluation will involve quantitative summaries of results from multiple studies to overcome limitations due to the typically small sample sizes of technical performance studies and/or to include a broader range of clinical settings and patient populations. This paper is a review of meta-analysis procedures for such an evaluation, including identification of suitable studies, statistical methodology to evaluate and summarize the performance metrics, and complete and transparent reporting of the results. This review addresses challenges typical of meta-analyses of technical performance, particularly small study sizes, which often causes violations of assumptions underlying standard meta-analysis techniques. Alternative approaches to address these difficulties are also presented; simulation studies indicate that they outperform standard techniques when some studies are small. The meta-analysis procedures presented are also applied to actual [18F]-fluorodeoxyglucose positron emission tomography (FDG-PET) test-retest repeatability data for illustrative purposes.

New challenges for comparative effectiveness in oncology: choice of primary end points for randomized clinical trials

Recent advances have led to a steady improvement in cancer treatments. The increasing number of therapeutic options and the corresponding improvement in outcomes pose a number of challenges for comparative effectiveness research in oncology. This review is focused on the choice of primary end points and their interpretation in randomized clinical trials that are designed to inform patients and clinicians on the relative benefits of cancer therapies. We discuss end points that directly measure clinical benefit as well as end points that are thought to be surrogates for clinical benefit. Particular attention is given to the issues associated with the use of overall survival as the primary end point in randomized clinical trials.

Prostate-specific antigen changes as surrogate for overall survival in men with metastatic castration-resistant prostate cancer treated with second-line chemotherapy

PURPOSE

Prostate-specific antigen (PSA) kinetics, and more specifically a ≥ 30% decline in PSA within 3 months after initiation of first-line chemotherapy with docetaxel, are associated with improvement in overall survival (OS) in men with metastatic castration-resistant prostate cancer (mCRPC). The objective of this analysis was to evaluate post-treatment PSA kinetics as surrogates for OS in patients receiving second-line chemotherapy.

PATIENTS AND METHODS

Data from a phase III trial of patients with mCRPC randomly assigned to cabazitaxel plus prednisone (C + P) or mitoxantrone plus prednisone were used. PSA decline (≥ 30% and ≥ 50%), velocity, and rise within the first 3 months of treatment were evaluated as surrogates for OS. The Prentice criteria, proportion of treatment explained (PTE), and meta-analytic approaches were used as measures of surrogacy.

RESULTS

The observed hazard ratio (HR) for death for patients treated with C + P was 0.66 (95% CI, 0.55 to 0.79; P < .001). Furthermore, a ≥ 30% decline in PSA was a statistically significant predictor of OS (HR for death, 0.52; 95% CI, 0.43 to 0.64; P < .001). Adjusting for treatment effect, the HR for a ≥ 30% PSA decline was 0.50 (95% CI, 0.40 to 0.62; P < .001), but treatment remained statistically significant, thus failing the third Prentice criterion. The PTE for a ≥ 30% decline in PSA was 0.34 (95% CI, 0.11 to 0.56), indicating a lack of surrogacy for OS. The values of R(2) were < 1, suggesting that PSA decline was not surrogate for OS.

CONCLUSION

Surrogacy for any PSA-based end point could not be demonstrated in this analysis. Thus, the benefits of cabazitaxel in mediating a survival benefit are not fully captured by early PSA changes.

Surrogate endpoint analysis: an exercise in extrapolation

Surrogate endpoints offer the hope of smaller or shorter cancer trials. It is, however, important to realize they come at the cost of an unverifiable extrapolation that could lead to misleading conclusions. With cancer prevention, the focus is on hypothesis testing in small surrogate endpoint trials before deciding whether to proceed to a large prevention trial. However, it is not generally appreciated that a small surrogate endpoint trial is highly sensitive to a deviation from the key Prentice criterion needed for the hypothesis-testing extrapolation. With cancer treatment, the focus is on estimation using historical trials with both surrogate and true endpoints to predict treatment effect based on the surrogate endpoint in a new trial. Successively leaving out one historical trial and computing the predicted treatment effect in the left-out trial yields a standard error multiplier that summarizes the increased uncertainty in estimation extrapolation. If this increased uncertainty is acceptable, three additional extrapolation issues (biological mechanism, treatment following observation of the surrogate endpoint, and side effects following observation of the surrogate endpoint) need to be considered. In summary, when using surrogate endpoint analyses, an appreciation of the problems of extrapolation is crucial.

Predicting treatment effect from surrogate endpoints and historical trials: an extrapolation involving probabilities of a binary outcome or survival to a specific time

Using multiple historical trials with surrogate and true endpoints, we consider various models to predict the effect of treatment on a true endpoint in a target trial in which only a surrogate endpoint is observed. This predicted result is computed using (1) a prediction model (mixture, linear, or principal stratification) estimated from historical trials and the surrogate endpoint of the target trial and (2) a random extrapolation error estimated from successively leaving out each trial among the historical trials. The method applies to either binary outcomes or survival to a particular time that is computed from censored survival data. We compute a 95% confidence interval for the predicted result and validate its coverage using simulation. To summarize the additional uncertainty from using a predicted instead of true result for the estimated treatment effect, we compute its multiplier of standard error. Software is available for download.

Overall survival as the outcome for randomized clinical trials with effective subsequent therapies

We review how overall survival (OS) comparisons should be interpreted with increasing availability of effective therapies that can be given subsequently to the treatment assigned in a randomized clinical trial (RCT). We examine in detail how effective subsequent therapies influence OS comparisons under varying conditions in RCTs. A subsequent therapy given after tumor progression (or relapse) in an RCT that works better in the standard arm than the experimental arm will lead to a smaller OS difference (possibly no difference) than one would see if the subsequent therapy was not available. Subsequent treatments that are equally effective in the treatment arms would not be expected to affect the absolute OS benefit of the experimental treatment but will make the relative improvement in OS smaller. In trials in which control arm patients cross over to the experimental treatment after their condition worsens, a smaller OS difference could be observed than one would see without cross-overs. In particular, use of cross-over designs in the first definitive evaluation of a new agent in a given disease compromises the ability to assess clinical benefit. In disease settings in which there is not an intermediate end point that directly measures clinical benefit, OS should be the primary end point of an RCT. The observed difference in OS should be considered the measure of clinical benefit to the patients, regardless of subsequent therapies, provided that the subsequent therapies used in both treatment arms follow the current standard of care.

Statistical assessment of biosimilar products

Biological products or medicines are therapeutic agents that are produced using a living system or organism. Access to these life-saving biological products is limited because of their expensive costs. Patents on the early biological products will soon expire in the next few years. This allows other biopharmaceutical/biotech companies to manufacture the generic versions of the biological products, which are referred to as follow-on biological products by the U.S. Food and Drug Administration (FDA) or as biosimilar medicinal products by the European Medicine Agency (EMEA) of the European Union (EU). Competition of cost-effective follow-on biological products with equivalent efficacy and safety can cut down the costs and hence increase patients' access to the much-needed biological pharmaceuticals. Unlike for the conventional pharmaceuticals of small molecules, the complexity and heterogeneity of the molecular structure, complicated manufacturing process, different analytical methods, and possibility of severe immunogenicity reactions make evaluation of equivalence (similarity) between the biosimilar products and their corresponding innovator product a great challenge for both the scientific community and regulatory agencies. In this paper, we provide an overview of the current regulatory requirements for approval of biosimilar products. A review of current criteria for evaluation of bioequivalence for the traditional chemical generic products is provided. A detailed description of the differences between the biosimilar and chemical generic products is given with respect to size and structure, immunogenicity, product quality attributed, and manufacturing processes. In addition, statistical considerations including design criteria, fundamental biosimilar assumptions, and statistical methods are proposed. The possibility of using genomic data in evaluation of biosimilar products is also explored.

Moving from correlative science to predictive oncology

Many diagnostic entities traditionally viewed as individual diseases are heterogeneous in molecular pathogenesis and treatment responsiveness. This results in treatment of many patients with ineffective drugs, the conduct of large clinical trials to identify small average treatment benefits for heterogeneous groups of patients. In oncology, genomic technologies provide powerful tools for identification of patients who require systemic treatment and for selecting the most appropriate drug. Development of drugs with companion diagnostics, however, increases the complexity of clinical development and requires new approaches to the design and analysis of clinical trials. Adapting to the fundamental importance of tumor genomics will require paradigm changes for clinical and statistical investigators in academia, industry and government. In this paper we attempt to address some of these issues and to comment specifically on the design of clinical studies for evaluating the clinical utility and robustness of prognostic and predictive biomarkers.

Phase II clinical trials in oncology: are we hitting the target?

The number of novel and molecularly targeted agents in the last decade that need screening for preliminary efficacy in Phase II trials has increased. Many of these agents have a cytostatic mode of action that is difficult to assess using traditional Phase II designs. These new agents require detailed evaluation to optimize their dosing, to evaluate their effects on their target and to define early markers that predict for a definitive benefit. This review focuses on the options for Phase II trial designs. The different end points, single versus multiarm and randomized designs, the use of biomarkers and Bayesian approaches are also reviewed. The final design chosen will depend on the characteristics and circumstances of each individual study.

Meta-analysis for the evaluation of surrogate endpoints in cancer clinical trials

The identification and validation of putative surrogate endpoints in oncology is a great challenge to medical investigators, statisticians, and regulators. A putative surrogate endpoint must be validated at both individual-level and trial-level before it can be used to replace the clinical endpoint in a future clinical trial. Recently, meta-analytic methods for evaluating potential surrogates have become widely accepted in cancer clinical trials. In this review, after addressing multiple complications and general issues surrounding surrogate endpoints, we review various proposed and adopted meta-analytic methodologies pertaining to the application of these methods to oncology clinical trials with different tumor types. In oncology, several applications have successfully identified useful surrogates. For example, disease-free survival and progression-free survival have been validated through meta-analyses as acceptable surrogates for overall survival in adjuvant colon cancer and advanced colorectal cancer, respectively. We also discuss several limitations of surrogate endpoints, including the critical issues that the extrapolation of the validity of a surrogate is always context-dependent and that such extrapolation should be exercised with caution.

Effective incorporation of biomarkers into phase II trials

The incorporation of biomarkers into the drug development process will improve understanding of how new therapeutics work and allow for more accurate identification of patients who will benefit from those therapies. Strategically planned biomarker evaluations in phase II studies may allow for the design of more efficient phase III trials and better screening of therapeutics for entry into phase III development, hopefully leading to increased chances of positive phase III trial results. Some examples of roles that a biomarker can play in a phase II trial include predictor of response or resistance to specific therapies, patient enrichment, correlative endpoint, or surrogate endpoint. Considerations for using biomarkers most effectively in these roles are discussed in the context of several examples. The substantial technical, logistic, and ethical challenges that can be faced when trying to incorporate biomarkers into phase II trials are also addressed. A rational and coordinated approach to the inclusion of biomarker studies throughout the drug development process will be the key to attaining the goal of personalized medicine.

Validation of novel imaging methodologies for use as cancer clinical trial end-points

The success or failure of a clinical trial, of any phase, depends critically on the choice of an appropriate primary end-point. In the setting of phases II and III cancer clinical trials, imaging end-points have historically, and continue presently to play a major role in determining therapeutic efficacy. The primary goal of this paper is to discuss the validation of imaging-based markers as end-points for phase II clinical trials of cancer therapy. Specifically, we outline the issues that must be considered, and the criteria that would need to be satisfied, for an imaging end-point to supplement or potentially replace RECIST- defined tumour status as a phase II clinical trial end-point. The key criteria proposed to judge the utility of a new end-point primarily relate to its ability to accurately and reproducibly predict the eventual phase III end-point for treatment effect, which is usually assessed by a difference between two arms on progression free or overall survival, both at the patient and more importantly at the trial level. As will be demonstrated, the level of evidence required to formally and fully validate a new imaging marker as an appropriate end-point for phase II trials is substantial. In many cases, this level of evidence will only become available by conducting a series of coordinated prospectively designed multicentre clinical trials culminating in a formal meta-analysis. We also include a discussion of situations where flexibility may be required, relative to the ideal rigorous evaluation, to accommodate inevitable real-world feasibility constraints.

LDL-cholesterol differences predicted survival benefit in statin trials by the surrogate threshold effect (STE)

OBJECTIVE

We describe a new statistical method called the surrogate threshold effect (STE) that estimates the threshold level of a surrogate needed in a clinical trial to predict a benefit in the target clinical outcome. In this article, we apply this method to the LDL-cholesterol biomarker surrogate and survival benefit-target outcome in statin trials.

STUDY DESIGN AND SETTING

We identified randomized trials comparing statin treatment to placebo treatment or no treatment and reporting all-cause and cardiovascular mortality. Trials with fewer than five all-cause deaths in at least one arm were excluded. Multiple regression modeled the reduction in all-cause and cardiovascular mortality as a function of LDL-cholesterol difference. The 95% confidence and 95% prediction bands were calculated and graphed to determine the minimum LDL-cholesterol difference (the surrogate threshold) below which there would be no predicted survival benefit.

RESULTS

In 16 qualifying trials, regression analysis yielded an all-cause mortality model whose prediction bands demonstrated no overall survival gain with LDL-cholesterol difference values below 1.5 mmol/L. The cardiovascular mortality model yielded prediction bands that demonstrated no cardiovascular survival benefit with LDL-cholesterol difference values below 1.4 mmol/L.

CONCLUSIONS

In a multitrial setting, the STE approach is a promising yet straightforward statistical method for evaluating the surrogate validity of biomarkers.

Lack of generalizability of sensitivity and specificity with treatment effects

In biomarker development, two types of summary measures are often used to describe marker accuracy. Positive and negative predictive values describe how well a marker predicts clinical states of interest, while sensitivity and specificity describe how well a marker discriminates between the two states. Insofar as predictive values depend heavily on the prevalence of the clinical states and sensitivity and specificity may not, sensitivity and specificity are preferred in early biomarker development. In many applications, an ideal property of a biomarker is fulfillment of the first Prentice criterion. Under this condition, predictive values do not depend on a covariate (such as treatment) because the biomarker captures all relevant information about the clinical state of interest. A similar condition can be defined for sensitivity and specificity which states that these measures do not depend on a covariate (e.g. treatment). This condition, which we refer to as the equal discriminatory accuracy (EDA) condition, is desirable because it allows sensitivity and specificity from one treatment setting (or covariate value) to be applied to a different setting. We demonstrate, however, that the Prentice condition and EDA are incompatible. Further, under a simple proportional hazards model for a time-to-event outcome, EDA will not be satisfied. We present numerical examples as well as examples of a potential marker in late-stage prostate cancer and another for cervical cancer screening. These results demonstrate that evaluating sensitivity and specificity within treatment (or other covariate) groups is necessary even when simple proportional hazards models or the Prentice criterion holds.

[Are mortality or morbidity risks appropriate endpoints for interventional studies in primary or secondary prevention with shared decision-making?]

The choice of primary and secondary endpoints for the individual studies of a research and development program is a classical biometric problem. Far-reaching decisions have to be made on the basis of incomplete information. Early studies in a program usually have functional or symptom-driven endpoints; the larger subsequent trials will usually focus on quality of life, morbidity or mortality. For program acceleration and cost reduction surrogate parameters are welcome. Unfortunately, there are only a few generally accepted surrogates for morbidity or mortality. At first glance, risk scores seem to be a suitable means for filling this gap. They are easily calculated, comprehensive and thus form a solid base for shared decision-making. However, further reasoning reveals that risk scores do not meet the usual standards for surrogates. What is more, a treatment targeting the reduction of a risk score cannot be considered an evidence-based intervention, which is due to a lack of randomized trials that compare risk score reduction to conventional interventions focussing on isolated risk factors. Thus, risk scores are unsuitable for primary endpoints, whereas they play an important role as comprehensive explanatory variables in study evaluation, i.e., for the description of population characteristics and as potential control variables or effect modifiers.

Surrogate threshold effect: an alternative measure for meta-analytic surrogate endpoint validation

In many therapeutic areas, the identification and validation of surrogate endpoints is of prime interest to reduce the duration and/or size of clinical trials. Buyse et al. [Biostatistics 2000; 1:49-67] proposed a meta-analytic approach to the validation. In this approach, the validity of a surrogate is quantified by the coefficient of determination Rtrial2 obtained from a model, which allows for prediction of the treatment effect on the endpoint of interest ('true' endpoint) from the effect on the surrogate. One problem related to the use of Rtial2 is the difficulty in interpreting its value. To address this difficulty, in this paper we introduce a new concept, the so-called surrogate threshold effect (STE), defined as the minimum treatment effect on the surrogate necessary to predict a non-zero effect on the true endpoint. One of its interesting features, apart from providing information relevant to the practical use of a surrogate endpoint, is its natural interpretation from a clinical point of view.

The Biomarker-Surrogacy Evaluation Schema: a review of the biomarker-surrogate literature and a proposal for a criterion-based, quantitative, multidimensional hierarchical levels of evidence schema for evaluating the status of biomarkers as surrogate endpoints

There are clear advantages to using biomarkers and surrogate endpoints, but concerns about clinical and statistical validity and systematic methods to evaluate these aspects hinder their efficient application. Section 2 is a systematic, historical review of the biomarker-surrogate endpoint literature with special reference to the nomenclature, the systems of classification and statistical methods developed for their evaluation. In Section 3 an explicit, criterion-based, quantitative, multidimensional hierarchical levels of evidence schema — Biomarker-Surrogacy Evaluation Schema — is proposed to evaluate and co-ordinate the multiple dimensions (biological, epidemiological, statistical, clinical trial and risk-benefit evidence) of the biomarker clinical endpoint relationships. The schema systematically evaluates and ranks the surrogacy status of biomarkers and surrogate endpoints using defined levels of evidence. The schema incorporates the three independent domains: Study Design, Target Outcome and Statistical Evaluation. Each domain has items ranked from zero to five. An additional category called Penalties incorporates additional considerations of biological plausibility, risk-benefit and generalizability. The total score (0—15) determines the level of evidence, with Level 1 the strongest and Level 5 the weakest. The term `surrogate' is restricted to markers attaining Levels 1 or 2 only. Surrogacy status of markers can then be directly compared within and across different areas of medicine to guide individual, trial-based or drug-development decisions. This schema would facilitate communication between clinical, researcher, regulatory, industry and consumer participants necessary for evaluation of the biomarker-surrogate-clinical endpoint relationship in their different settings.

An alternative model for bivariate random-effects meta-analysis when the within-study correlations are unknown

Multivariate meta-analysis models can be used to synthesize multiple, correlated endpoints such as overall and disease-free survival. A hierarchical framework for multivariate random-effects meta-analysis includes both within-study and between-study correlation. The within-study correlations are assumed known, but they are usually unavailable, which limits the multivariate approach in practice. In this paper, we consider synthesis of 2 correlated endpoints and propose an alternative model for bivariate random-effects meta-analysis (BRMA). This model maintains the individual weighting of each study in the analysis but includes only one overall correlation parameter, rho, which removes the need to know the within-study correlations. Further, the only data needed to fit the model are those required for a separate univariate random-effects meta-analysis (URMA) of each endpoint, currently the common approach in practice. This makes the alternative model immediately applicable to a wide variety of evidence synthesis situations, including studies of prognosis and surrogate outcomes. We examine the performance of the alternative model through analytic assessment, a realistic simulation study, and application to data sets from the literature. Our results show that, unless rho is very close to 1 or -1, the alternative model produces appropriate pooled estimates with little bias that (i) are very similar to those from a fully hierarchical BRMA model where the within-study correlations are known and (ii) have better statistical properties than those from separate URMAs, especially given missing data. The alternative model is also less prone to estimation at parameter space boundaries than the fully hierarchical model and thus may be preferred even when the within-study correlations are known. It also suitably estimates a function of the pooled estimates and their correlation; however, it only provides an approximate indication of the between-study variation. The alternative model greatly facilitates the utilization of correlation in meta-analysis and should allow an increased application of BRMA in practice.

Bivariate random-effects meta-analysis and the estimation of between-study correlation

Background

When multiple endpoints are of interest in evidence synthesis, a multivariate meta-analysis can jointly synthesise those endpoints and utilise their correlation. A multivariate random-effects meta-analysis must incorporate and estimate the between-study correlation (ρ_B).

Methods

In this paper we assess maximum likelihood estimation of a general normal model and a generalised model for bivariate random-effects meta-analysis (BRMA). We consider two applied examples, one involving a diagnostic marker and the other a surrogate outcome. These motivate a simulation study where estimation properties from BRMA are compared with those from two separate univariate random-effects meta-analyses (URMAs), the traditional approach.

Results

The normal BRMA model estimates ρ_B as -1 in both applied examples. Analytically we show this is due to the maximum likelihood estimator sensibly truncating the between-study covariance matrix on the boundary of its parameter space. Our simulations reveal this commonly occurs when the number of studies is small or the within-study variation is relatively large; it also causes upwardly biased between-study variance estimates, which are inflated to compensate for the restriction on ρ^ MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaaiiGacuWFbpGCgaqcaaaa@2E83@_B. Importantly, this does not induce any systematic bias in the pooled estimates and produces conservative standard errors and mean-square errors. Furthermore, the normal BRMA is preferable to two normal URMAs; the mean-square error and standard error of pooled estimates is generally smaller in the BRMA, especially given data missing at random. For meta-analysis of proportions we then show that a generalised BRMA model is better still. This correctly uses a binomial rather than normal distribution, and produces better estimates than the normal BRMA and also two generalised URMAs; however the model may sometimes not converge due to difficulties estimating ρ_B.

Conclusion

A BRMA model offers numerous advantages over separate univariate synthesises; this paper highlights some of these benefits in both a normal and generalised modelling framework, and examines the estimation of between-study correlation to aid practitioners.

New versus established drugs in venous thromboprophylaxis: efficacy and safety considerations related to timing of administration

European surgeons generally administer thromboprophylaxis with low-molecular-weight heparins (LMWHs) at high doses 12 hours preoperatively in response to findings that surgery-related deep-vein thrombosis typically originates at the time of major orthopedic surgery or shortly afterwards. North American surgeons, in contrast, generally administer LMWHs at an almost 50% higher dose than that given in Europe 12-24 hours postoperatively, even though both pre- and postoperative administration are considered suitable in current guidelines. This review therefore examines how close to major orthopedic surgery thromboprophylaxis is administered, and the subsequent effect of timing on clinically relevant efficacy and safety parameters. The trials examined involve fondaparinux sodium (fondaparinux) and (xi)melagatran, in comparison with the established LMWHs enoxaparin sodium (enoxaparin) and dalteparin. In key trials, fondaparinux reduced the risk of asymptomatic and clinical venous thromboembolism (VTE) by 55% compared with enoxaparin, at the expense of a 1.6-fold higher risk of bleeding. While the studies were not designed to compare efficacy endpoints based on clinical outcomes, no significant difference was demonstrated for symptomatic VTE. The fact that the enoxaparin regimen was started at the upper limits of its recommended initiation timeframe may have significantly influenced the results of comparative studies, given that several meta-analyses found that the timing of LMWH initiation significantly influenced its effectiveness on asymptomatic VTE and major bleedings. Compared with once-daily LMWH in European trials, early postoperative doses/regimens of twice-daily (xi)melagatran did not increase severe bleeding and was significantly less effective at preventing asymptomatic total VTE in patients who had undergone total hip-replacement surgery. When used according to the 'knife-to-skin' protocol, the melagatran regimen was superior to enoxaparin in preventing major asymptomatic VTE, but at the cost of a higher rate of major bleeding. In North America, the delayed postoperative administration of (xi)melagatran (oral only) was less effective than the postoperative twice-daily enoxaparin regimen with regard to asymptomatic total and major VTE. Our analysis highlights the fact that differences in efficacy and safety data in clinical trials of thromboprophylaxis might also be linked to differences in the timing of initiation. However, it is not possible to assess the importance of this 'time effect' among other factors considered as drug-specific properties (pharmacokinetics, mode of action, dosage) and evaluate their respective contribution in the observed differences. To avoid unbiased comparison in further studies, the possible effect of timing should be taken into account and, when feasible, both therapies started at the same time. For instance, harmonizing the initiation of thromboprophylaxis 6-8 or 12 hours postoperatively could be two acceptable harmonized options for scheduling in clinical trials.

Disease-free survival versus overall survival as a primary end point for adjuvant colon cancer studies: individual patient data from 20,898 patients on 18 randomized trials

PURPOSE

A traditional end point for colon adjuvant clinical trials is overall survival (OS), with 5 years demonstrating adequate follow-up. A shorter-term end point providing convincing evidence to allow treatment comparisons could significantly speed the translation of advances into practice.

METHODS

Individual patient data were pooled from 18 randomized phase III colon cancer adjuvant clinical trials. Trials included 43 arms, with a pooled sample size of 20,898 patients. The primary hypothesis was that disease-free survival (DFS), with 3 years of follow-up, is an appropriate primary end point to replace OS with 5 years of follow-up.

RESULTS

The recurrence rates for years 1 through 5 were 12%, 14%, 8%, 5%, and 3%, respectively. Median time from recurrence to death was 12 months. Eighty percent of recurrences were in the first 3 years; 91% of patients with recurrence by 3 years died before 5 years. Correlation between 3-year DFS and 5-year OS was 0.89. Comparing control versus experimental arms within each trial, the correlation between hazard ratios for DFS and OS was 0.92. Within-trial log-rank testing using both DFS and OS provided the same conclusion in 23 (92%) of 25 cases. Formal measures of surrogacy were satisfied.

CONCLUSION

In patients treated on phase III adjuvant colon clinical trials, DFS and OS are highly correlated, both within patients and across trials. These results suggest that DFS after 3 years of median follow-up is an appropriate end point for adjuvant colon cancer clinical trials of fluorouracil-based regimens, although marginally significant DFS improvements may not translate into significant OS benefits.

A simple meta-analytic approach for using a binary surrogate endpoint to predict the effect of intervention on true endpoint

A surrogate endpoint is an endpoint that is obtained sooner, at lower cost, or less invasively than the true endpoint for a health outcome and is used to make conclusions about the effect of intervention on the true endpoint. In this approach, each previous trial with surrogate and true endpoints contributes an estimated predicted effect of intervention on true endpoint in the trial of interest based on the surrogate endpoint in the trial of interest. These predicted quantities are combined in a simple random-effects meta-analysis to estimate the predicted effect of intervention on true endpoint in the trial of interest. Validation involves comparing the average prediction error of the aforementioned approach with (i) the average prediction error of a standard meta-analysis using only true endpoints in the other trials and (ii) the average clinically meaningful difference in true endpoints implicit in the trials. Validation is illustrated using data from multiple randomized trials of patients with advanced colorectal cancer in which the surrogate endpoint was tumor response and the true endpoint was median survival time.