Surrogate Marker Evaluation from an Information Theory Perspective

Assessing the Operational Characteristics of the Individual Causal Association as a Metric of Surrogacy in the Binary Continuous Setting

In a causal inference framework, a new metric has been proposed to quantify surrogacy for a continuous putative surrogate and a binary true endpoint, based on information theory. The proposed metric, termed the individual causal association (ICA), was quantified using a joint causal inference model for the corresponding potential outcomes. Due to the non-identifiability inherent in this type of models, a sensitivity analysis was introduced to study the behavior of the ICA as a function of the non-identifiable parameters characterizing the aforementioned model. In this scenario, to reduce uncertainty, several plausible yet untestable assumptions like monotonicity, independence, conditional independence or homogeneous variance-covariance, are often incorporated into the analysis. We assess the robustness of the methodology regarding these simplifying assumptions via simulation. The practical implications of the findings are demonstrated in the analysis of a randomized clinical trial evaluating an inactivated quadrivalent influenza vaccine.

A joint normal-ordinal (probit) model for ordinal and continuous longitudinal data

In biomedical studies, continuous and ordinal longitudinal variables are frequently encountered. In many of these studies it is of interest to estimate the effect of one of these longitudinal variables on the other. Time-dependent covariates have, however, several limitations; they can, for example, not be included when the data is not collected at fixed intervals. The issues can be circumvented by implementing joint models, where two or more longitudinal variables are treated as a response and modeled with a correlated random effect. Next, by conditioning on these response(s), we can study the effect of one or more longitudinal variables on another. We propose a normal-ordinal(probit) joint model. First, we derive closed-form formulas to estimate the model-based correlations between the responses on their original scale. In addition, we derive the marginal model, where the interpretation is no longer conditional on the random effects. As a consequence, we can make predictions for a subvector of one response conditional on the other response and potentially a subvector of the history of the response. Next, we extend the approach to a high-dimensional case with more than two ordinal and/or continuous longitudinal variables. The methodology is applied to a case study where, among others, a longitudinal ordinal response is predicted with a longitudinal continuous variable.

Heart rate variability is not suitable as surrogate marker for pain intensity in patients with chronic pain

ABSTRACT

The search towards more objective outcome measurements and consequently surrogate markers for pain started decades ago; however, no generally accepted biomarker for pain has qualified yet. The goal is to explore the value of heart rate variability (HRV) as surrogate marker for pain intensity chronic pain setting. Pain intensity scores and HRV were collected in 366 patients with chronic pain, through a cross-sectional multicenter study. Pain intensity was measured with both the Visual Analogue Scale and Numeric Rating Scale, while 16 statistical HRV parameters were derived. Canonical correlation analysis was performed to evaluate the correlation between the dependent pain variables and the HRV parameters. Surrogacy was determined for each HRV parameter with point estimates between 0 and 1 whereby values close to 1 indicate a strong association between the surrogate and the true endpoint at the patient level. Weak correlations were revealed between HRV parameters and pain intensity scores. The highest surrogacy point estimate was found for mean heart rate as marker for average pain intensity on the Numeric Rating Scale with point estimates of 0.0961 (95% CI from 0.0384 to 0.1537) and 0.0209 (95% CI from 0 to 0.05) for patients without medication use, and medication use respectively. This study indicated that HRV parameters as separate entities are no suitable surrogacy candidates for pain intensity, in a population of chronic pain patients. Further potential surrogate candidates and clinical robust true endpoints should be explored, in order to find a surrogate measure for the highly individual pain experience.

Informed decision-making: Statistical methodology for surrogacy evaluation and its role in licensing and reimbursement assessments

The desire, by patients and society, for faster access to therapies has driven a long tradition of the use of surrogate endpoints in the evaluation of pharmaceuticals and, more recently, biologics and other innovative medical technologies. The consequent need for statistical validation of potential surrogate outcome measures is a prime example on the theme of statistical support for decision-making in health technology assessment (HTA). Following the pioneering methodology based on hypothesis testing that Prentice presented in 1989, a host of further methods, both frequentist and Bayesian, have been developed to enable the value of a putative surrogate outcome to be determined. This rich methodological seam has generated practical methods for surrogate evaluation, the most recent of which are based on the principles of information theory and bring together ideas from the causal effects and causal association paradigms. Following our synopsis of statistical methods, we then consider how regulatory authorities (on licensing) and payer and HTA agencies (on reimbursement) use clinical trial evidence based on surrogate outcomes. We review existing HTA surrogate outcome evaluative frameworks. We conclude with recommendations for further steps: (1) prioritisation by regulators and payers of the application of formal surrogate outcome evaluative frameworks, (2) application of formal Bayesian decision-analytic methods to support reimbursement decisions, and (3) greater utilization of conditional surrogate-based licensing and reimbursement approvals, with subsequent reassessment of treatments in confirmatory trials based on final patient-relevant outcomes.

Evaluation of Surrogate Endpoints Using Information-Theoretic Measure of Association Based on Havrda and Charvat Entropy

Surrogate endpoints have been used to assess the efficacy of a treatment and can potentially reduce the duration and/or number of required patients for clinical trials. Using information theory, Alonso et al. (2007) proposed a unified framework based on Shannon entropy, a new definition of surrogacy that departed from the hypothesis testing framework. In this paper, a new family of surrogacy measures under Havrda and Charvat (H-C) entropy is derived which contains Alonso’s definition as a particular case. Furthermore, we extend our approach to a new model based on the information-theoretic measure of association for a longitudinally collected continuous surrogate endpoint for a binary clinical endpoint of a clinical trial using H-C entropy. The new model is illustrated through the analysis of data from a completed clinical trial. It demonstrates advantages of H-C entropy-based surrogacy measures in the evaluation of scheduling longitudinal biomarker visits for a phase 2 randomized controlled clinical trial for treatment of multiple sclerosis.

Separation and the information theory surrogate evaluation approach: A penalised likelihood solution

Surrogate evaluation is an important topic in clinical trials research, the use of a surrogate in place of a primary endpoint of interest is a common occurrence but also a contentious issue that is much debated. Statistical techniques to assess potential surrogates are closely scrutinised by the research community given the complexities of such an assessment. One such technique is the information theory surrogate evaluation approach which is well-established, practical and theoretically sound. In the context of discrete outcomes, we investigated issues of bias due to inefficiency, overfitting and separation (sparse data) that have not been recognised or addressed previously. The most serious cause of bias is separation in trial information. We outline the concerns surrounding this bias and conduct a simulation study to investigate whether a penalised likelihood technique provides an appropriate solution. We found that removing trials with separation from surrogacy evaluation resulted in a large amount of discarded data. Conversely, the penalised likelihood technique allows retention of all trial information and enables precise and reliable surrogate estimation. The information theory approach is a critical tool for conducting surrogate evaluation. This work strengthens the practical application of the information theory approach, allowing analyses to be adapted or the results summarised with appropriate caution to mitigate the biases highlighted. This is especially true where separation occurs. The adoption of the penalised likelihood technique into information theory surrogate evaluation is a useful addition that solves an issue likely to arise frequently in the context of categorical endpoints.

Assessment of the information theory approach to evaluating time-to-event surrogate and true endpoints in a meta-analytic setting

In many disease areas, commonly used long-term clinical endpoints are becoming increasingly difficult to implement due to long follow-up times and/or increased costs. Shorter-term surrogate endpoints are urgently needed to expedite drug development, the evaluation of which requires robust and reliable statistical methodology to drive meaningful clinical conclusions about the strength of relationship with the true long-term endpoint. This paper uses a simulation study to explore one such previously proposed method, based on information theory, for evaluation of time to event surrogate and long-term endpoints, including the first examination within a meta-analytic setting of multiple clinical trials with such endpoints. The performance of the information theory method is examined for various scenarios including different dependence structures, surrogate endpoints, censoring mechanisms, treatment effects, trial and sample sizes, and for surrogate and true endpoints with a natural time-ordering. Results allow us to conclude that, contrary to some findings in the literature, the approach provides estimates of surrogacy that may be substantially lower than the true relationship between surrogate and true endpoints, and rarely reach a level that would enable confidence in the strength of a given surrogate endpoint. As a result, care is needed in the assessment of time to event surrogate and true endpoints based only on this methodology.

Is Disease Intensity a Good Surrogate for Yield Loss or Toxin Contamination? A Case Study with Fusarium Head Blight of Wheat

Sometimes plant pathologists assess disease intensity when they are primarily interested in other response variables, such as yield loss or toxin concentration in harvested products. In these situations, disease intensity potentially could be considered a surrogate of yield or toxin. A surrogate is a variable which can be used instead of the variable of interest in the evaluation of experimental treatments or in making predictions. Surrogates can be measured earlier, more conveniently, or more cheaply than the variable of primary interest, but the reliability or validity of the surrogate must be shown. We demonstrate ways of quantifying two facets of surrogacy by using a protocol originally developed by Buyse and colleagues for medical research. Coefficient-of-determination type statistics can be used to conveniently assess the strength of surrogacy on a unitless scale. As a case study, we evaluated whether field severity of Fusarium head blight (i.e., FHB index) can be used as a surrogate for yield loss and deoxynivalenol (DON) toxin concentration in harvested wheat grain. Bivariate mixed models and corresponding approximations were fitted to data from 82 uniform fungicide trials conducted from 2008 to 2013. Individual-level surrogacy-for predicting the variable of interest (yield or DON) from the surrogate (index) in plots with the same treatment-was very low. Trial-level surrogacy-for predicting the effect of treatment (e.g., mean difference) for the variable of interest based on the effect of the treatment on the surrogate (index)-was moderate for yield, and only low for DON. Challenges in using disease severity as a surrogate for yield and toxin are discussed.

Evaluation of surrogacy in the multi-trial setting based on information theory: an extension to ordinal outcomes

In clinical trials, surrogate outcomes are early measures of treatment effect that are used to predict treatment effect on a later primary outcome of interest: the primary outcome therefore does not need to be observed and trials can be shortened. Evaluating surrogates is a complex area as a given treatment can act through multiple pathways, some of which may circumvent the surrogate. One of the best established and practically sound approaches to surrogacy evaluation is based on information theory. We have extended this approach to the case of ordinal outcomes, which are used as primary outcomes in many medical areas. This extension provides researchers with the means of evaluating surrogates in this setting, which expands the usefulness of the information theory approach while also demonstrating its versatility.

How to use frailtypack for validating failure-time surrogate endpoints using individual patient data from meta-analyses of randomized controlled trials

Background and Objective

The use of valid surrogate endpoints can accelerate the development of phase III trials. Numerous validation methods have been proposed with the most popular used in a context of meta-analyses, based on a two-step analysis strategy. For two failure time endpoints, two association measures are usually considered, Kendall’s τ at individual level and adjusted R2 (adjRtrial2) at trial level. However, adjRtrial2 is not always available mainly due to model estimation constraints. More recently, we proposed a one-step validation method based on a joint frailty model, with the aim of reducing estimation issues and estimation bias on the surrogacy evaluation criteria. The model was quite robust with satisfactory results obtained in simulation studies. This study seeks to popularize this new surrogate endpoints validation approach by making the method available in a user-friendly R package.

Methods

We provide numerous tools in the frailtypack R package, including more flexible functions, for the validation of candidate surrogate endpoints using data from multiple randomized clinical trials.

Results

We implemented the surrogate threshold effect which is used in combination with Rtrial2 to make decisions concerning the validity of the surrogate endpoints. It is also possible thanks to frailtypack to predict the treatment effect on the true endpoint in a new trial using the treatment effect observed on the surrogate endpoint. The leave-one-out cross-validation is available for assessing the accuracy of the prediction using the joint surrogate model. Other tools include data generation, simulation study and graphic representations. We illustrate the use of the new functions with both real data and simulated data.

Conclusion

This article proposes new attractive and well developed tools for validating failure time surrogate endpoints.

Assessing correlates of protection in vaccine trials: statistical solutions in the context of high vaccine efficacy

Background

The use of correlates of protection (CoPs) in vaccination trials offers significant advantages as useful clinical endpoint substitutes. Vaccines with very high vaccine efficacy (VE) are documented in the literature (VE ≥95%). The rare events (number of infections) observed in the vaccinated groups of these trials posed challenges when applying conventionally-used statistical methods for CoP assessment. In this paper, we describe the nature of these challenges, and propose easy-to-implement and uniquely-tailored statistical solutions for the assessment of CoPs in the specific context of high VE.

Methods

The Prentice criteria and meta-analytic frameworks are standard statistical methods for assessing vaccine CoPs, but can be problematic in high VE cases due to the rare events data available. As a result, lack of fit and the problem of infinite estimates may arise, in the former and latter methods respectively. The use of flexible models within the Prentice framework, and penalized-likelihood methods to solve the issue of infinite estimates can improve the performance of both methods in high VE settings.

Results

We have 1) devised flexible non-linear models to counteract the Prentice framework lack of fit, providing sufficient statistical power to the method, and 2) proposed the use of penalised likelihood approaches to make the meta-analytic framework applicable on randomized subgroups, such as regions. The performance of the proposed methods for high VE cases was evaluated by running simulations.

Conclusions

As vaccines with high efficacy are documented in the literature, there is a need to identify effective statistical solutions to assess CoPs. Our proposed adaptations are straight-forward and improve the performance of conventional statistical methods for high VE data, leading to more reliable CoP assessments in the context of high VE settings.

A closed-form estimator for meta-analysis and surrogate markers evaluation

Estimating complex linear mixed models using an iterative full maximum likelihood estimator can be cumbersome in some cases. With small and unbalanced datasets, convergence problems are common. Also, for large datasets, iterative procedures can be computationally prohibitive. To overcome these computational issues, an unbiased two-stage closed-form estimator for the multivariate linear mixed model is proposed. It is rooted in pseudo-likelihood-based split-sample methodology and useful, for example, when evaluating normally distributed endpoints in a meta-analytic context. However, applications go well beyond this framework. Its statistical and computational performance is assessed via simulation. The method is applied to a study in schizophrenia.

surrosurv: An R package for the evaluation of failure time surrogate endpoints in individual patient data meta-analyses of randomized clinical trials

BACKGROUND AND OBJECTIVE

Surrogate endpoints are attractive for use in clinical trials instead of well-established endpoints because of practical convenience. To validate a surrogate endpoint, two important measures can be estimated in a meta-analytic context when individual patient data are available: the R_indiv² or the Kendall's τ at the individual level, and the R_trial² at the trial level. We aimed at providing an R implementation of classical and well-established as well as more recent statistical methods for surrogacy assessment with failure time endpoints. We also intended incorporating utilities for model checking and visualization and data generating methods described in the literature to date.

METHODS

In the case of failure time endpoints, the classical approach is based on two steps. First, a Kendall's τ is estimated as measure of individual level surrogacy using a copula model. Then, the R_trial² is computed via a linear regression of the estimated treatment effects; at this second step, the estimation uncertainty can be accounted for via measurement-error model or via weights. In addition to the classical approach, we recently developed an approach based on bivariate auxiliary Poisson models with individual random effects to measure the Kendall's τ and treatment-by-trial interactions to measure the R_trial². The most common data simulation models described in the literature are based on: copula models, mixed proportional hazard models, and mixture of half-normal and exponential random variables.

RESULTS

The R package surrosurv implements the classical two-step method with Clayton, Plackett, and Hougaard copulas. It also allows to optionally adjusting the second-step linear regression for measurement-error. The mixed Poisson approach is implemented with different reduced models in addition to the full model. We present the package functions for estimating the surrogacy models, for checking their convergence, for performing leave-one-trial-out cross-validation, and for plotting the results. We illustrate their use in practice on individual patient data from a meta-analysis of 4069 patients with advanced gastric cancer from 20 trials of chemotherapy.

CONCLUSIONS

The surrosurv package provides an R implementation of classical and recent statistical methods for surrogacy assessment of failure time endpoints. Flexible simulation functions are available to generate data according to the methods described in the literature.

An investigation into the two-stage meta-analytic copula modelling approach for evaluating time-to-event surrogate endpoints which comprise of one or more events of interest

Clinical trials of experimental treatments must be designed with primary endpoints that directly measure clinical benefit for patients. In many disease areas, the recognised gold standard primary endpoint can take many years to mature, leading to challenges in the conduct and quality of clinical studies. There is increasing interest in using shorter-term surrogate endpoints as substitutes for costly long-term clinical trial endpoints; such surrogates need to be selected according to biological plausibility, as well as the ability to reliably predict the unobserved treatment effect on the long-term endpoint. A number of statistical methods to evaluate this prediction have been proposed; this paper uses a simulation study to explore one such method in the context of time-to-event surrogates for a time-to-event true endpoint. This two-stage meta-analytic copula method has been extensively studied for time-to-event surrogate endpoints with one event of interest, but thus far has not been explored for the assessment of surrogates which have multiple events of interest, such as those incorporating information directly from the true clinical endpoint. We assess the sensitivity of the method to various factors including strength of association between endpoints, the quantity of data available, and the effect of censoring. In particular, we consider scenarios where there exist very little data on which to assess surrogacy. Results show that the two-stage meta-analytic copula method performs well under certain circumstances and could be considered useful in practice, but demonstrates limitations that may prevent universal use.

Assessing a surrogate predictive value: a causal inference approach

Several methods have been developed for the evaluation of surrogate endpoints within the causal-inference and meta-analytic paradigms. In both paradigms, much effort has been made to assess the capacity of the surrogate to predict the causal treatment effect on the true endpoint. In the present work, the so-called surrogate predictive function (SPF) is introduced for that purpose, using potential outcomes. The relationship between the SPF and the individual causal association, a new metric of surrogacy recently proposed in the literature, is studied in detail. It is shown that the SPF, in conjunction with the individual causal association, can offer an appealing quantification of the surrogate predictive value. However, neither the distribution of the potential outcomes nor the SPF are identifiable from the data. These identifiability issues are tackled using a two-step procedure. In the first step, the region of the parametric space of the distribution of the potential outcomes, compatible with the data at hand, is geometrically characterized. Further, in a second step, a Monte Carlo approach is used to study the behavior of the SPF on the previous region. The method is illustrated using data from a clinical trial involving schizophrenic patients and a newly developed and user friendly R package Surrogate is provided to carry out the validation exercise. Copyright © 2016 John Wiley & Sons, Ltd.

Differences in symptom expression between unipolar and bipolar spectrum depression: Results from a nationally representative sample using item response theory (IRT)

BACKGROUND

The inclusion of subsyndromal forms of bipolarity in the fifth edition of the DSM has major implications for the way in which we approach the diagnosis of individuals with depressive symptoms. The aim of the present study was to use methods based on item response theory (IRT) to examine whether, when equating for levels of depression severity, there are differences in the likelihood of reporting DSM-IV symptoms of major depressive episode (MDE) between subjects with and without a lifetime history of manic symptoms.

METHODS

We conducted these analyses using a large, nationally representative sample from the USA (n=34,653), the second wave of the National Epidemiologic Survey on Alcohol and Related Conditions.

RESULTS

The items sadness, appetite disturbance and psychomotor symptoms were better indicators of depression severity in participants without a lifetime history of manic symptoms, in a clinically meaningful way. DSM-IV symptoms of MDE were substantially less informative in participants with a lifetime history of manic symptoms than in those without such history.

LIMITATIONS

Clinical information on DSM-IV depressive and manic symptoms was based on retrospective self-report

CONCLUSIONS

The clinical presentation of depressive symptoms may substantially differ in individuals with and without a lifetime history of manic symptoms. These findings alert to the possibility of atypical symptomatic presentations among individuals with co-occurring symptoms or disorders and highlight the importance of continued research into specific pathophysiology differentiating unipolar and bipolar depression.

Information-theory based surrogate marker evaluation from several randomized clinical trials with continuous true and binary surrogate endpoints

Background Surrogate endpoints potentially reduce the duration and/or increase the amount of information available in a study, thereby diminishing patient burden and cost. They may also increase the effectiveness and reliability of research, through beneficial impact on noncompliance and missingness.

Purpose In this article, we review the meta-analytic approach of Buyse et al. (2000) and its extension to mixed continuous and binary endpoints by Molenberghs Geys, and Buyse (2001).

Methods An information-theoretic alternative, based on Alonso and Molenberghs (2007a) is proposed. The method is evaluated using simulations and application to data from an ophthalmologic trial, with lines of vision lost at 6 months as candidate surrogate endpoints for lines of vision lost at 12 months. The method is implemented as an R function.

Results The information-theoretic approach is based on solid theory, easy to apply, and enjoys elegant properties. While the information-theoretic approach appears to be somewhat biased downwards, this is due to fact that it operates at explicitly observed outcomes, without the need for unobserved, latent scales. This is a desirable property.

Limitations While easy-to-use and implement, the theoretical foundation of the information-theory approach is more mathematical. It produces some bias for small to moderate trial/center sizes, and hence is recommended primarily for sufficiently large trials.

Conclusions Since the meta-analytic framework can be computationally extremely expensive, the information-theoretic approach of Alonso and Molenberghs (2007a) is a viable alternative. For the ophthalmologic case study, the conclusion is that the lines of vision lost at sixth month do have some, but not overwhelming promise as a surrogate endpoint. Clinical Trials 2007; 4: 587—597. http://ctj.sagepub.com

Biomarker Qualification: Toward a Multiple Stakeholder Framework for Biomarker Development, Regulatory Acceptance, and Utilization

The discovery, development, and use of biomarkers for a variety of drug development purposes are areas of tremendous interest and need. Biomarkers can become accepted for use through submission of biomarker data during the drug approval process. Another emerging pathway for acceptance of biomarkers is via the biomarker qualification program developed by the Center for Drug Evaluation and Research (CDER, US Food and Drug Administration). Evidentiary standards are needed to develop and evaluate various types of biomarkers for their intended use and multiple stakeholders, including academia, industry, government, and consortia must work together to help develop this evidence. The article describes various types of biomarkers that can be useful in drug development and evidentiary considerations that are important for qualification. A path forward for coordinating efforts to identify and explore needed biomarkers is proposed for consideration.

Center-Within-Trial Versus Trial-Level Evaluation of Surrogate Endpoints

Evaluation of candidate surrogate endpoints using individual patient data from multiple clinical trials is considered the gold standard approach to validate surrogates at both patient and trial levels. However, this approach assumes the availability of patient-level data from a relatively large collection of similar trials, which may not be possible to achieve for a given disease application. One common solution to the problem of too few similar trials involves performing trial-level surrogacy analyses on trial sub-units (e.g., centers within trials), thereby artificially increasing the trial-level sample size for feasibility of the multi-trial analysis. To date, the practical impact of treating trial sub-units (centers) identically to trials in multi-trial surrogacy analyses remains unexplored, and conditions under which this ad hoc solution may in fact be reasonable have not been identified. We perform a simulation study to identify such conditions, and demonstrate practical implications using a multi-trial dataset of patients with early stage colon cancer.

A unified procedure for meta-analytic evaluation of surrogate end points in randomized clinical trials

The meta-analytic approach to evaluating surrogate end points assesses the predictiveness of treatment effect on the surrogate toward treatment effect on the clinical end point based on multiple clinical trials. Definition and estimation of the correlation of treatment effects were developed in linear mixed models and later extended to binary or failure time outcomes on a case-by-case basis. In a general regression setting that covers nonnormal outcomes, we discuss in this paper several metrics that are useful in the meta-analytic evaluation of surrogacy. We propose a unified 3-step procedure to assess these metrics in settings with binary end points, time-to-event outcomes, or repeated measures. First, the joint distribution of estimated treatment effects is ascertained by an estimating equation approach; second, the restricted maximum likelihood method is used to estimate the means and the variance components of the random treatment effects; finally, confidence intervals are constructed by a parametric bootstrap procedure. The proposed method is evaluated by simulations and applications to 2 clinical trials.

Surrogate end points in secondary analyses of cardiovascular trials

A surrogate end point is one that is used as a substitute for a clinical end point of more direct interest, usually for reasons of practicality, and that is expected to predict clinical benefit. Surrogate end points play a critical role in the advancement of all medical research, and cardiovascular (CV) research in particular. However, the relationship between a surrogate end point and its clinical end point is usually complex, and there are many examples where results based on surrogates have proved to be misleading. Secondary analyses of existing clinical trial data are likely to involve surrogate end points, if only because clinical end points will have been extensively studied as part of the primary analysis of a trial large enough to collect useful clinical end point data. Validation of a surrogate end point is a laudable goal for a secondary analysis of a large clinical end point trial (or meta-analysis of multiple smaller trials), and the result may be an important new tool for further study of a class of compounds in a particular disease context. Secondary analyses using surrogate end points may also provide new insight into disease or treatment mechanism, but as with any surrogate end point analysis, the results can mislead, and the existing literature is heavy on application and light on methodology. Surrogate end points often substitute efficiency for clarity, and while many interesting and potentially informative secondary analyses of CV trials will involve surrogates, results are likely to be ambiguous and should be interpreted with care.

Causal assessment of surrogacy in a meta-analysis of colorectal cancer trials

When the true end points (T) are difficult or costly to measure, surrogate markers (S) are often collected in clinical trials to help predict the effect of the treatment (Z). There is great interest in understanding the relationship among S, T, and Z. A principal stratification (PS) framework has been proposed by Frangakis and Rubin (2002) to study their causal associations. In this paper, we extend the framework to a multiple trial setting and propose a Bayesian hierarchical PS model to assess surrogacy. We apply the method to data from a large collection of colon cancer trials in which S and T are binary. We obtain the trial-specific causal measures among S, T, and Z, as well as their overall population-level counterparts that are invariant across trials. The method allows for information sharing across trials and reduces the nonidentifiability problem. We examine the frequentist properties of our model estimates and the impact of the monotonicity assumption using simulations. We also illustrate the challenges in evaluating surrogacy in the counterfactual framework that result from nonidentifiability.

An information-theoretic approach to surrogate-marker evaluation with failure time endpoints

Over the last decades, the evaluation of potential surrogate endpoints in clinical trials has steadily been growing in importance, not only thanks to the availability of ever more potential markers and surrogate endpoints, also because more methodological development has become available. While early work has been devoted, to a large extent, to Gaussian, binary, and longitudinal endpoints, the case of time-to-event endpoints is in need of careful scrutiny as well, owing to the strong presence of such endpoints in oncology and beyond. While work had been done in the past, it was often cumbersome to use such tools in practice, because of the need for fitting copula or frailty models that were further embedded in a hierarchical or two-stage modeling approach. In this paper, we present a methodologically elegant and easy-to-use approach based on information theory. We resolve essential issues, including the quantification of "surrogacy" based on such an approach. Our results are put to the test in a simulation study and are applied to data from clinical trials in oncology. The methodology has been implemented in R.

Evaluating the Proportion of Treatment Effect Explained by a Continuous Surrogate Marker in Logistic or Probit Regression Models

Using surrogate endpoints in clinical trials is desirable for drug development because the trials can be shortened and therefore more cost-effective. Validating a surrogate for the clinical endpoint is critical in this context. One of the key steps in statistical validation of a surrogate for a single trial is to estimate the proportion of treatment effect explained (PTE or PE) by a surrogate. Often the measure for PTE is estimated from the difference in coefficients of treatment from two models with or without adjusting for the surrogate for clinical endpoint. Inherent problems with the method are: the two models may not be valid simultaneously; and the estimate can often lie outside the interval [0, 1]. In this article, we provide alternative measures for evaluating the proportion of treatment effect explained by a surrogate in logistic or probit regression models. Our measures can be estimated easily with any statistical programs capable of binary linear regression modeling, and the interpretation of the measures can be illustrated using Ordinal Dominance (OD) curves. The concept can be visually understood by any practical user. Simulation shows our alternative measures yield more accurate estimates which are less biased, less variable, and with narrower confidence intervals. A clinical trial example is provided.

Use of meta-analysis for the validation of surrogate endpoints and biomarkers in cancer trials

This article discusses statistical approaches to the validation of surrogate biomarkers and endpoints. One approach that has been successfully used in oncology consists of estimating associations at two levels: the association between the surrogate and the clinical endpoint, called the individual-level association, and the association between the effects of treatment on the surrogate and the clinical endpoint, called the trial-level association. This approach requires data to be available from multiple randomized trials, such as in a meta-analysis of trials based on individual patient data. The approach is illustrated using randomized trials of first-line treatments for advanced tumors of the colon, breast, ovary, and prostate. Data from several meta-analyses suggest that progression-free survival is an acceptable surrogate in advanced colorectal and ovarian cancer, but not in breast and prostate cancer.

Biomarkers and surrogate end points--the challenge of statistical validation

Biomarkers and surrogate end points have great potential for use in clinical oncology, but their statistical validation presents major challenges, and few biomarkers have been robustly confirmed. Provisional supportive data for prognostic biomarkers, which predict the likely outcome independently of treatment, is possible through small retrospective studies, but it has proved more difficult to achieve robust multi-site validation. Predictive biomarkers, which predict the likely response of patients to specific treatments, require more extensive data for validation, specifically large randomized clinical trials and meta-analysis. Surrogate end points are even more challenging to validate, and require data demonstrating both that the surrogate is prognostic for the true end point independently of treatment, and that the effect of treatment on the surrogate reliably predicts its effect on the true end point. In this Review, we discuss the nature of prognostic and predictive biomarkers and surrogate end points, and examine the statistical techniques and designs required for their validation. In cases where the statistical requirements for validation cannot be rigorously achieved, the biological plausibility of an end point or surrogate might support its adoption. No consensus yet exists on processes or standards for pragmatic evaluation and adoption of biomarkers and surrogate end points in the absence of robust statistical validation.

A bayesian approach to surrogacy assessment using principal stratification in clinical trials

A surrogate marker (S) is a variable that can be measured earlier and often more easily than the true endpoint (T) in a clinical trial. Most previous research has been devoted to developing surrogacy measures to quantify how well S can replace T or examining the use of S in predicting the effect of a treatment (Z). However, the research often requires one to fit models for the distribution of T given S and Z. It is well known that such models do not have causal interpretations because the models condition on a postrandomization variable S. In this article, we directly model the relationship among T, S, and Z using a potential outcomes framework introduced by Frangakis and Rubin (2002, Biometrics 58, 21-29). We propose a Bayesian estimation method to evaluate the causal probabilities associated with the cross-classification of the potential outcomes of S and T when S and T are both binary. We use a log-linear model to directly model the association between the potential outcomes of S and T through the odds ratios. The quantities derived from this approach always have causal interpretations. However, this causal model is not identifiable from the data without additional assumptions. To reduce the nonidentifiability problem and increase the precision of statistical inferences, we assume monotonicity and incorporate prior belief that is plausible in the surrogate context by using prior distributions. We also explore the relationship among the surrogacy measures based on traditional models and this counterfactual model. The method is applied to the data from a glaucoma treatment study.

A unified framework for the evaluation of surrogate endpoints in mental-health clinical trials

For a number of reasons, surrogate endpoints are considered instead of the so-called true endpoint in clinical studies, especially when such endpoints can be measured earlier, and/or with less burden for patient and experimenter. Surrogate endpoints may occur more frequently than their standard counterparts. For these reasons, it is not surprising that the use of surrogate endpoints in clinical practice is increasing. Building on the seminal work of Prentice(1) and Freedman et al.,(2) Buyse et al. (3) framed the evaluation exercise within a meta-analytic setting, in an effort to overcome difficulties that necessarily surround evaluation efforts based on a single trial. In this article, we review the meta-analytic approach for continuous outcomes, discuss extensions to non-normal and longitudinal settings, as well as proposals to unify the somewhat disparate collection of validation measures currently on the market. Implications for design and for predicting the effect of treatment in a new trial, based on the surrogate, are discussed. A case study in schizophrenia is analysed.

Meta-analysis for the evaluation of surrogate endpoints in cancer clinical trials

The identification and validation of putative surrogate endpoints in oncology is a great challenge to medical investigators, statisticians, and regulators. A putative surrogate endpoint must be validated at both individual-level and trial-level before it can be used to replace the clinical endpoint in a future clinical trial. Recently, meta-analytic methods for evaluating potential surrogates have become widely accepted in cancer clinical trials. In this review, after addressing multiple complications and general issues surrounding surrogate endpoints, we review various proposed and adopted meta-analytic methodologies pertaining to the application of these methods to oncology clinical trials with different tumor types. In oncology, several applications have successfully identified useful surrogates. For example, disease-free survival and progression-free survival have been validated through meta-analyses as acceptable surrogates for overall survival in adjuvant colon cancer and advanced colorectal cancer, respectively. We also discuss several limitations of surrogate endpoints, including the critical issues that the extrapolation of the validity of a surrogate is always context-dependent and that such extrapolation should be exercised with caution.

Alternative methods to evaluate trial level surrogacy

BACKGROUND

The evaluation and validation of surrogate endpoints have been extensively studied in the last decade. Prentice [1] and Freedman, Graubard and Schatzkin [2] laid the foundations for the evaluation of surrogate endpoints in randomized clinical trials. Later, Buyse et al. [5] proposed a meta-analytic methodology, producing different methods for different settings, which was further studied by Alonso and Molenberghs [9], in their unifying approach based on information theory.

PURPOSE

In this article, we focus our attention on the trial-level surrogacy and propose alternative procedures to evaluate such surrogacy measure, which do not pre-specify the type of association. A promising correction based on cross-validation is investigated. As well as the construction of confidence intervals for this measure.

METHODS

In order to avoid making assumption about the type of relationship between the treatment effects and its distribution, a collection of alternative methods, based on regression trees, bagging, random forests, and support vector machines, combined with bootstrap-based confidence interval and, should one wish, in conjunction with a cross-validation based correction, will be proposed and applied. We apply the various strategies to data from three clinical studies: in opthalmology, in advanced colorectal cancer, and in schizophrenia.

RESULTS

The results obtained for the three case studies are compared; they indicate that using random forest or bagging models produces larger estimated values for the surrogacy measure, which are in general stabler and the confidence interval narrower than linear regression and support vector regression. For the advanced colorectal cancer studies, we even found the trial-level surrogacy is considerably different from what has been reported.

LIMITATIONS

In general the alternative methods are more computationally demanding, and specially the calculation of the confidence intervals, require more computational time that the delta-method counterpart.

CONCLUSIONS

First, more flexible modeling techniques can be used, allowing for other type of association. Second, when no cross-validation-based correction is applied, overly optimistic trial-level surrogacy estimates will be found, thus cross-validation is highly recommendable. Third, the use of the delta method to calculate confidence intervals is not recommendable since it makes assumptions valid only in very large samples. It may also produce range-violating limits. We therefore recommend alternatives: bootstrap methods in general. Also, the information-theoretic approach produces comparable results with the bagging and random forest approaches, when cross-validation correction is applied. It is also important to observe that, even for the case in which the linear model might be a good option too, bagging methods perform well too, and their confidence intervals were more narrow.

Considerations for development of surrogate endpoints for antifracture efficacy of new treatments in osteoporosis: a perspective

Because of the broad availability of efficacious osteoporosis therapies, conduct of placebo-controlled trials in subjects at high risk for fracture is becoming increasing difficult. Alternative trial designs include placebo-controlled trials in patients at low risk for fracture or active comparator studies, both of which would require enormous sample sizes and associated financial resources. Another more attractive alternative is to develop and validate surrogate endpoints for fracture. In this perspective, we review the concept of surrogate endpoints as it has been developed in other fields of medicine and discuss how it could be applied in clinical trials of osteoporosis. We outline a stepwise approach and possible study designs to qualify a biomarker as a surrogate endpoint in osteoporosis and review the existing data for several potential surrogate endpoints to assess their success in meeting the proposed criteria. Finally, we suggest a research agenda needed to advance the development of biomarkers as surrogate endpoints for fracture in osteoporosis trials. To ensure optimal development and best use of biomarkers to accelerate drug development, continuous dialog among the health professionals, industry, and regulators is of paramount importance.

Evaluating time to cancer recurrence as a surrogate marker for survival from an information theory perspective

The last two decades have seen a lot of development in the area of surrogate marker validation. One of these approaches places the evaluation in a meta-analytic framework, leading to definitions in terms of trial- and individual-level association. A drawback of this methodology is that different settings have led to different measures at the individual level. Using information theory, Alonso et al. proposed a unified framework, leading to a new definition of surrogacy, which offers interpretational advantages and is applicable in a wide range of situations. In this work, we illustrate how this information-theoretic approach can be used to evaluate surrogacy when both endpoints are of a time-to-event type. Two meta-analyses, in early and advanced colon cancer, respectively, are then used to evaluate the performance of time to cancer recurrence as a surrogate for overall survival.