Comparing and combining biomarkers as principal surrogates for time-to-event clinical endpoints

Standardized statistical framework for comparison of biomarkers: Techniques from ADNI

INTRODUCTION

Well-chosen biomarkers have the potential to increase the efficiency of clinical trials and drug discovery and should show good precision as well as clinical validity.

METHODS

We suggest measures that operationalize these criteria and describe a general approach that can be used for inference-based comparisons of biomarker performance. The methods are applied to measures obtained from structural magnetic resonance imaging (MRI) from individuals with mild dementia (n = 70) or mild cognitive impairment (MCI; n = 303) enrolled in the Alzheimer's Disease Neuroimaging Initiative.

RESULTS

Ventricular volume and hippocampal volume showed the best precision in detecting change over time in both individuals with MCI and with dementia. Differences in clinical validity varied by group.

DISCUSSION

The methodology presented provides a standardized framework for comparison of biomarkers across modalities and across different methods used to generate similar measures and will help in the search for the most promising biomarkers.

HIGHLIGHTS

A framework for comparison of biomarkers on pre-defined criteria is presented. Criteria for comparison include precision in capturing change and clinical validity. Ventricular volume has high precision in change for both dementia and mild cognitive impairment (MCI) trials. Imaging measures' performance in clinical validity varies more for dementia than for MCI.

Surrogacy validation for time-to-event outcomes with illness-death frailty models

A common practice in clinical trials is to evaluate a treatment effect on an intermediate outcome when the true outcome of interest would be difficult or costly to measure. We consider how to validate intermediate outcomes in a causally-valid way when the trial outcomes are time-to-event. Using counterfactual outcomes, those that would be observed if the counterfactual treatment had been given, the causal association paradigm assesses the relationship of the treatment effect on the surrogate outcome with the treatment effect on the true, primary outcome. In particular, we propose illness-death models to accommodate the censored and semicompeting risk structure of survival data. The proposed causal version of these models involves estimable and counterfactual frailty terms. Via these multistate models, we characterize what a valid surrogate would look like using a causal effect predictiveness plot. We evaluate the estimation properties of a Bayesian method using Markov chain Monte Carlo and assess the sensitivity of our model assumptions. Our motivating data source is a localized prostate cancer clinical trial where the two survival outcomes are time to distant metastasis and time to death.

Solutions for surrogacy validation with longitudinal outcomes for a gene therapy

Valid surrogate endpoints S can be used as a substitute for a true outcome of interest T to measure treatment efficacy in a clinical trial. We propose a causal inference approach to validate a surrogate by incorporating longitudinal measurements of the true outcomes using a mixed modeling approach, and we define models and quantities for validation that may vary across the study period using principal surrogacy criteria. We consider a surrogate-dependent treatment efficacy curve that allows us to validate the surrogate at different time points. We extend these methods to accommodate a delayed-start treatment design where all patients eventually receive the treatment. Not all parameters are identified in the general setting. We apply a Bayesian approach for estimation and inference, utilizing more informative prior distributions for selected parameters. We consider the sensitivity of these prior assumptions as well as assumptions of independence among certain counterfactual quantities conditional on pretreatment covariates to improve identifiability. We examine the frequentist properties (bias of point and variance estimates, credible interval coverage) of a Bayesian imputation method. Our work is motivated by a clinical trial of a gene therapy where the functional outcomes are measured repeatedly throughout the trial.

Estimation of the proportion of treatment effect explained by a high-dimensional surrogate

Clinical studies examining the effectiveness of a treatment with respect to some primary outcome often require long-term follow-up of patients and/or costly or burdensome measurements of the primary outcome of interest. Identifying a surrogate marker for the primary outcome of interest may allow one to evaluate a treatment effect with less follow-up time, less cost, or less burden. While much clinical and statistical work has focused on identifying and validating surrogate markers, available approaches tend to focus on settings in which only a single surrogate marker is of interest. Limited work has been done to accommodate the high-dimensional surrogate marker setting where the number of potential surrogates is greater than the sample size. In this article, we develop methods to estimate the proportion of treatment effect explained by high-dimensional surrogates. We study the asymptotic properties of our proposed estimator, propose inference procedures, and examine finite sample performance via a simulation study. We illustrate our proposed methods using data from a randomized study comparing a novel whey-based oral nutrition supplement with a standard supplement with respect to change in body fat percentage over 12 weeks, where the surrogate markers of interest are gene expression probesets.

Evaluation of Surrogate Endpoints Using Information-Theoretic Measure of Association Based on Havrda and Charvat Entropy

Surrogate endpoints have been used to assess the efficacy of a treatment and can potentially reduce the duration and/or number of required patients for clinical trials. Using information theory, Alonso et al. (2007) proposed a unified framework based on Shannon entropy, a new definition of surrogacy that departed from the hypothesis testing framework. In this paper, a new family of surrogacy measures under Havrda and Charvat (H-C) entropy is derived which contains Alonso’s definition as a particular case. Furthermore, we extend our approach to a new model based on the information-theoretic measure of association for a longitudinally collected continuous surrogate endpoint for a binary clinical endpoint of a clinical trial using H-C entropy. The new model is illustrated through the analysis of data from a completed clinical trial. It demonstrates advantages of H-C entropy-based surrogacy measures in the evaluation of scheduling longitudinal biomarker visits for a phase 2 randomized controlled clinical trial for treatment of multiple sclerosis.

Inference on treatment effect modification by biomarker response in a three-phase sampling design

An objective in randomized clinical trials is the evaluation of "principal surrogates," which consists of analyzing how the treatment effect on a clinical endpoint varies over principal strata subgroups defined by an intermediate response outcome under both or one of the treatment assignments. The latter effect modification estimand has been termed the marginal causal effect predictiveness (mCEP) curve. This objective was addressed in two randomized placebo-controlled Phase 3 dengue vaccine trials for an antibody response biomarker whose sampling design rendered previously developed inferential methods highly inefficient due to a three-phase sampling design. In this design, the biomarker was measured in a case-cohort sample and a key baseline auxiliary strongly associated with the biomarker (the "baseline surrogate measure") was only measured in a further sub-sample. We propose a novel approach to estimation of the mCEP curve in such three-phase sampling designs that avoids the restrictive "placebo structural risk" modeling assumption common to past methods and that further improves robustness by the use of non-parametric kernel smoothing for biomarker density estimation. Additionally, we develop bootstrap-based procedures for pointwise and simultaneous confidence intervals and testing of four relevant hypotheses about the mCEP curve. We investigate the finite-sample properties of the proposed methods and compare them to those of an alternative method making the placebo structural risk assumption. Finally, we apply the novel and alternative procedures to the two dengue vaccine trial data sets.

Predictive cluster level surrogacy in the presence of interference

Surrogate evaluation is a difficult problem that is made more so by the presence of interference. Our proposed procedure can allow for relatively easy evaluation of surrogates for indirect or spill-over clinical effects at the cluster level. Our definition of surrogacy is based on the causal-association paradigm (Joffe and Greene, 2009. Related causal frameworks for surrogate outcomes. Biometrics65, 530-538), under which surrogates are evaluated by the strength of the association between a causal treatment effect on the clinical outcome and a causal treatment effect on the candidate surrogate. Hudgens and Halloran (2008, Toward causal inference with interference. Journal of the American Statistical Association103, 832-842) introduced estimators that can be used for many of the marginal causal estimands of interest in the presence of interference. We extend these to consider surrogates for not just direct effects, but indirect and total effects at the cluster level. We suggest existing estimators that can be used to evaluate biomarkers under our proposed definition of surrogacy. In our motivating setting of a transmission blocking malaria vaccine, there is expected to be no direct protection to those vaccinated and predictive surrogates are urgently needed. We use a set of simulated data examples based on the proposed Phase IIb/III trial design of transmission blocking malaria vaccine to demonstrate how our definition, proposed criteria and procedure can be used to identify biomarkers as predictive cluster level surrogates in the presence of interference on the clinical outcome.

Evaluating multiple surrogate markers with censored data

The utilization of surrogate markers offers the opportunity to reduce the length of required follow-up time and/or costs of a randomized trial examining the effectiveness of an intervention or treatment. There are many available methods for evaluating the utility of a single surrogate marker including both parametric and nonparametric approaches. However, as the dimension of the surrogate marker increases, a completely nonparametric procedure becomes infeasible due to the curse of dimensionality. In this paper, we define a quantity to assess the value of multiple surrogate markers in a time-to-event outcome setting and propose a robust estimation approach for censored data. We focus on surrogate markers that are measured at some landmark time, t₀ , which occurs earlier than the end of the study. Our approach is based on a dimension reduction procedure with an option to incorporate weights to guard against potential misspecification of the working model, resulting in three different proposed estimators, two of which can be shown to be double robust. We examine the finite sample performance of the estimators under various scenarios using a simulation study. We illustrate the estimation and inference procedures using data from the Diabetes Prevention Program (DPP) to examine multiple potential surrogate markers for diabetes.

Post-randomization Biomarker Effect Modification Analysis in an HIV Vaccine Clinical Trial

While the HVTN 505 trial showed no overall efficacy of the tested vaccine to prevent HIV infection over placebo, markers measuring immune response to vaccination were strongly correlated with infection. This finding generated the hypothesis that some marker-defined vaccinated subgroups were partially protected whereas others had their risk increased. This hypothesis can be assessed using the principal stratification framework (Frangakis and Rubin, 2002) for studying treatment effect modification by an intermediate response variable, using methods in the sub-field of principal surrogate (PS) analysis that studies multiple principal strata. Unfortunately, available methods for PS analysis require an augmented study design not available in HVTN 505, and make untestable structural risk assumptions, motivating a need for more robust PS methods. Fortunately, another sub-field of principal stratification, survivor average causal effect (SACE) analysis (Rubin, 2006) - which studies effects in a single principal stratum - provides many methods not requiring an augmented design and making fewer assumptions. We show how, for a binary intermediate response variable, methods developed for SACE analysis can be adapted to PS analysis, providing new and more robust PS methods. Application to HVTN 505 supports that the vaccine partially protected individuals with vaccine-induced T-cells expressing certain combinations of functions.

Assessing the value of a censored surrogate outcome

Assessing the potential of surrogate markers and surrogate outcomes for replacing a long term outcome is an active area of research. The interest in this topic is partly motivated by increasing pressure from stakeholders to shorten the time required to evaluate the safety and/or efficacy of a treatment or intervention such that treatments deemed safe and effective can be made available to those in need more quickly. Most existing methods in surrogacy evaluation either require strict model assumptions or that primary outcome and surrogate outcome information is available for all study participants. In this paper, we focus on a setting where the primary outcome is subject to censoring and the aim is to quantify the surrogacy of an intermediate outcome, which is also subject to censoring. We define the surrogacy as the proportion of treatment effect on the primary outcome that is explained by the intermediate surrogate outcome information and propose two robust methods to estimate this quantity. We propose both a nonparametric approach that uses a kernel smoothed Nelson-Aalen estimator of conditional survival, and a semiparametric method that derives conditional survival estimates from a landmark Cox proportional hazards model. Simulation studies demonstrate that both approaches perform well in finite samples. Our methodological development is motivated by our interest in investigating the use of a composite cardiovascular endpoint as a surrogate outcome in a randomized study of the effectiveness of angiotensin-converting enzyme inhibitors on survival. We apply the proposed methods to quantify the surrogacy of this potential surrogate outcome for the primary outcome, time to death.

A unified evaluation of differential vaccine efficacy

Many infectious diseases are well prevented by proper vaccination. However, when a vaccine is not completely efficacious, there is great interest in determining how the vaccine effect differs in subgroups conditional on measured immune responses postvaccination and also according to the type of infecting agent (eg, strain of a virus). The former is often called correlate of protection (CoP) analysis, while the latter has been called sieve analysis. We propose a unified framework for simultaneously assessing CoP and sieve effects of a vaccine in a large Phase III randomized trial. We use flexible parametric models treating times to infection from different agents as competing risks and estimated maximum likelihood to fit the models. The parametric models under competing risks allow for estimation of both cumulative incidence-based contrasts and instantaneous rates. We outline the assumptions with which we can link the observable data to the causal contrasts of interest, propose hypothesis testing procedures, and evaluate our proposed methods in an extensive simulation study.

Evaluation and comparison of predictive individual-level general surrogates

An intermediate response measure that accurately predicts efficacy in a new setting at the individual level could be used both for prediction and personalized medical decisions. In this article, we define a predictive individual-level general surrogate (PIGS), which is an individual-level intermediate response that can be used to accurately predict individual efficacy in a new setting. While methods for evaluating trial-level general surrogates, which are predictors of trial-level efficacy, have been developed previously, few, if any, methods have been developed to evaluate individual-level general surrogates, and no methods have formalized the use of cross-validation to quantify the expected prediction error. Our proposed method uses existing methods of individual-level surrogate evaluation within a given clinical trial setting in combination with cross-validation over a set of clinical trials to evaluate surrogate quality and to estimate the absolute prediction error that is expected in a new trial setting when using a PIGS. Simulations show that our method performs well across a variety of scenarios. We use our method to evaluate and to compare candidate individual-level general surrogates over a set of multi-national trials of a pentavalent rotavirus vaccine.

Ongoing Vaccine and Monoclonal Antibody HIV Prevention Efficacy Trials and Considerations for Sequel Efficacy Trial Designs

Four randomized placebo-controlled efficacy trials of a candidate vaccine or passively infused monoclonal antibody for prevention of HIV-1 infection are underway (HVTN 702 in South African men and women; HVTN 705 in sub-Saharan African women; HVTN 703/HPTN 081 in sub-Saharan African women; HVTN 704/HPTN 085 in U.S., Peruvian, Brazilian, and Swiss men or transgender persons who have sex with men). Several challenges are posed to the optimal design of the sequel efficacy trials, including: (1) how to account for the evolving mosaic of effective prevention interventions that may be part of the trial design or standard of prevention; (2) how to define viable and optimal sequel trial designs depending on the primary efficacy results and secondary "correlates of protection" results of each of the ongoing trials; and (3) how to define the primary objective of sequel efficacy trials if HIV-1 incidence is expected to be very low in all study arms such that a standard trial design has a steep opportunity cost. After summarizing the ongoing trials, I discuss statistical science considerations for sequel efficacy trial designs, both generally and specifically to each trial listed above. One conclusion is that the results of "correlates of protection" analyses, which ascertain how different host immunological markers and HIV-1 viral features impact HIV-1 risk and prevention efficacy, have an important influence on sequel trial design. This influence is especially relevant for the monoclonal antibody trials because of the focused pre-trial hypothesis that potency and coverage of serum neutralization constitutes a surrogate endpoint for HIV-1 infection. Another conclusion is that while assessing prevention efficacy against a counterfactual placebo group is fraught with risks for bias, such analysis is nonetheless important and study designs coupled with analysis methods should be developed to optimize such inferences. I draw a parallel with non-inferiority designs, which are fraught with risks given the necessity of making unverifiable assumptions for interpreting results, but nevertheless have been accepted when a superiority design is not possible and a rigorous/conservative non-inferiority margin is used. In a similar way, counterfactual placebo group efficacy analysis should use rigorous/conservative inference techniques that formally build in a rigorous/conservative margin to potential biases that could occur due to departures from unverifiable assumptions. Because reliability of this approach would require new techniques for verifying that the study cohort experienced substantial exposure to HIV-1, currently it may be appropriate as a secondary objective but not as a primary objective.

Modeling HIV vaccine trials of the future

PURPOSE OF REVIEW

Models of implementation of known-effective interventions for HIV prevention indicate that an efficacious vaccine to prevent HIV infection would be critical for controlling the HIV pandemic. Key issues in the design of future HIV vaccine trials are: first, how to develop reliable immunological correlates of vaccine efficacy, second, how to down-select candidate vaccine regimens into efficacy trials, and third, how to learn about vaccine efficacy in the context of the evolving HIV prevention landscape.

RECENT FINDINGS

Whereas in the past phase-I/-II HIV vaccine trials have addressed the first and second points using a small set of immunological assays and readouts, recently they have used a battery of assays with highly multivariate readouts. In addition, systems vaccinology studies of other pathogens measuring PBMC transcriptomics and other immunological features pre- and postfirst vaccination are demonstrating value, for example, providing discoveries that preimmunization and early postimmunization cell population markers can predict the influenza-specific antibody titer that is a correlate of vaccine protection. The HIV prevention landscape continues to evolve, and the design and analysis of vaccine trials is evolving alongside, to accommodate increasingly dynamic and regional standards of HIV prevention.

SUMMARY

Development of interpretable and robust functional assays, in addition to the associated bioinformatics and statistical analytic tools, is needed to improve the assessment of correlates of protection in efficacy trials and the down-selection of candidate vaccine regimens into efficacy trials. Moreover, high-priority trials should integrate systems vaccinology, including the analysis of prevaccination and early postvaccination markers.

Evaluating surrogate marker information using censored data

Given the long follow-up periods that are often required for treatment or intervention studies, the potential to use surrogate markers to decrease the required follow-up time is a very attractive goal. However, previous studies have shown that using inadequate markers or making inappropriate assumptions about the relationship between the primary outcome and surrogate marker can lead to inaccurate conclusions regarding the treatment effect. Currently available methods for identifying and validating surrogate markers tend to rely on restrictive model assumptions and/or focus on uncensored outcomes. The ability to use such methods in practice when the primary outcome of interest is a time-to-event outcome is difficult because of censoring and missing surrogate information among those who experience the primary outcome before surrogate marker measurement. In this paper, we propose a novel definition of the proportion of treatment effect explained by surrogate information collected up to a specified time in the setting of a time-to-event primary outcome. Our proposed approach accommodates a setting where individuals may experience the primary outcome before the surrogate marker is measured. We propose a robust non-parametric procedure to estimate the defined quantity using censored data and use a perturbation-resampling procedure for variance estimation. Simulation studies demonstrate that the proposed procedures perform well in finite samples. We illustrate the proposed procedures by investigating two potential surrogate markers for diabetes using data from the Diabetes Prevention Program. Copyright © 2017 John Wiley & Sons, Ltd.

An Introduction to Principal Surrogate Evaluation with the pseval Package

We describe a new package called pseval that implements the core methods for the evaluation of principal surrogates in a single clinical trial. It provides a flexible interface for defining models for the risk given treatment and the surrogate, the models for integration over the missing counterfactual surrogate responses, and the estimation methods. Estimated maximum likelihood and pseudo-score can be used for estimation, and the bootstrap for inference. A variety of post-estimation methods are provided, including print, summary, plot, and testing. We summarize the main statistical methods that are implemented in the package and illustrate its use from the perspective of a novice R user.

Augmented trial designs for evaluation of principal surrogates

Observation of counterfactual intermediate responses, and evaluation of them as candidate surrogates, is complicated in a standard randomized trial as half of the responses are systematically missing by design. Although some augmentation procedures exist for obtaining counterfactual responses, they are specific to vaccine trials. We outline extensions to the existing augmentations and suggest augmentations of three trial designs outside the setting of vaccines. We outline the assumptions needed to identify the causal estimands of interest under each augmented design, under which standard likelihood-based methods can be used to evaluate intermediate responses as principal surrogates. Two of these designs, crossover and individual stepped-wedge, allow for the observation of clinical endpoints under both treatment and control for some subset of subjects and can therefore improve efficiency over standard parallel trial designs. The third, the treatment run-in design, allows for the observation of a baseline measure that may be as useful a surrogate as the true counterfactual intermediate response. As the evaluation methods rely on several assumptions, we also outline a remediation analysis, which can be used to help overcome assumption violations. We illustrate our suggested methods in an example from a drug-resistant tuberculosis treatment trial.

Predicting Overall Vaccine Efficacy in a New Setting by Re-Calibrating Baseline Covariate and Intermediate Response Endpoint Effect Modifiers of Type-Specific Vaccine Efficacy

We develop a transport formula for predicting overall cumulative vaccine efficacy through time t (VE(t)) to prevent clinically significant infection with a genetically diverse pathogen (e.g., HIV infection) in a new setting for which a Phase III preventive vaccine efficacy trial that would directly estimate VE(t) has not yet been conducted. The formula integrates data from (1) a previous Phase III trial, (2) a Phase I/II immune response biomarker endpoint trial in the new setting where a follow-up Phase III trial is planned, (3) epidemiological data on background HIV infection incidence in the new setting; and (4) genomic epidemiological data on HIV sequence distributions in the previous and new settings. For (1), the randomized vaccine versus placebo Phase III trial yields estimates of vaccine efficacy to prevent particular genotypes of HIV in participant subgroups defined by baseline covariates X and immune responses to vaccination S(1) measured at a fixed time point τ (potential outcomes if assigned vaccine); often one or more immune responses to vaccination are available that modify genotype-specific vaccine efficacy. The formula focuses on subgroups defined by X and S(1) and being at-risk for HIV infection at τ under both the vaccine and placebo treatment assignments. For (2), the Phase I/II trial tests the same vaccine in a new setting, or a refined new vaccine in the same or new setting, and measures the same baseline covariates and immune responses as the original Phase III trial. For (3), epidemiological data in the new setting are used to project overall background HIV infection rates in the baseline covariate subgroups in the planned Phase III trial, hence re-calibrating for HIV incidence differences in the two settings; whereas for (4), data bases of HIV sequences measured from HIV infected individuals are used to re-calibrate for differences in the distributions of the circulating HIV genotypes in the two settings. The transport formula incorporates a user-specified bridging assumption function that measures differences in HIV genotype-specific conditional biological-susceptibility vaccine efficacies in the two settings, facilitating a sensitivity analysis. We illustrate the transport formula with application to HIV Vaccine Trials Network (HVTN) research. One application of the transport formula is to use predicted VE(t) as a rational criterion for ranking a set of candidate vaccines being studied in Phase I/II trials for their priority for down-selection into the follow-up Phase III trial.