Informed decision-making: Statistical methodology for surrogacy evaluation and its role in licensing and reimbursement assessments

The desire, by patients and society, for faster access to therapies has driven a long tradition of the use of surrogate endpoints in the evaluation of pharmaceuticals and, more recently, biologics and other innovative medical technologies. The consequent need for statistical validation of potential surrogate outcome measures is a prime example on the theme of statistical support for decision-making in health technology assessment (HTA). Following the pioneering methodology based on hypothesis testing that Prentice presented in 1989, a host of further methods, both frequentist and Bayesian, have been developed to enable the value of a putative surrogate outcome to be determined. This rich methodological seam has generated practical methods for surrogate evaluation, the most recent of which are based on the principles of information theory and bring together ideas from the causal effects and causal association paradigms. Following our synopsis of statistical methods, we then consider how regulatory authorities (on licensing) and payer and HTA agencies (on reimbursement) use clinical trial evidence based on surrogate outcomes. We review existing HTA surrogate outcome evaluative frameworks. We conclude with recommendations for further steps: (1) prioritisation by regulators and payers of the application of formal surrogate outcome evaluative frameworks, (2) application of formal Bayesian decision-analytic methods to support reimbursement decisions, and (3) greater utilization of conditional surrogate-based licensing and reimbursement approvals, with subsequent reassessment of treatments in confirmatory trials based on final patient-relevant outcomes.

Evaluation of Surrogate Endpoints Using Information-Theoretic Measure of Association Based on Havrda and Charvat Entropy

Surrogate endpoints have been used to assess the efficacy of a treatment and can potentially reduce the duration and/or number of required patients for clinical trials. Using information theory, Alonso et al. (2007) proposed a unified framework based on Shannon entropy, a new definition of surrogacy that departed from the hypothesis testing framework. In this paper, a new family of surrogacy measures under Havrda and Charvat (H-C) entropy is derived which contains Alonso’s definition as a particular case. Furthermore, we extend our approach to a new model based on the information-theoretic measure of association for a longitudinally collected continuous surrogate endpoint for a binary clinical endpoint of a clinical trial using H-C entropy. The new model is illustrated through the analysis of data from a completed clinical trial. It demonstrates advantages of H-C entropy-based surrogacy measures in the evaluation of scheduling longitudinal biomarker visits for a phase 2 randomized controlled clinical trial for treatment of multiple sclerosis.

Separation and the information theory surrogate evaluation approach: A penalised likelihood solution

Surrogate evaluation is an important topic in clinical trials research, the use of a surrogate in place of a primary endpoint of interest is a common occurrence but also a contentious issue that is much debated. Statistical techniques to assess potential surrogates are closely scrutinised by the research community given the complexities of such an assessment. One such technique is the information theory surrogate evaluation approach which is well-established, practical and theoretically sound. In the context of discrete outcomes, we investigated issues of bias due to inefficiency, overfitting and separation (sparse data) that have not been recognised or addressed previously. The most serious cause of bias is separation in trial information. We outline the concerns surrounding this bias and conduct a simulation study to investigate whether a penalised likelihood technique provides an appropriate solution. We found that removing trials with separation from surrogacy evaluation resulted in a large amount of discarded data. Conversely, the penalised likelihood technique allows retention of all trial information and enables precise and reliable surrogate estimation. The information theory approach is a critical tool for conducting surrogate evaluation. This work strengthens the practical application of the information theory approach, allowing analyses to be adapted or the results summarised with appropriate caution to mitigate the biases highlighted. This is especially true where separation occurs. The adoption of the penalised likelihood technique into information theory surrogate evaluation is a useful addition that solves an issue likely to arise frequently in the context of categorical endpoints.

Evaluation of surrogacy in the multi-trial setting based on information theory: an extension to ordinal outcomes

In clinical trials, surrogate outcomes are early measures of treatment effect that are used to predict treatment effect on a later primary outcome of interest: the primary outcome therefore does not need to be observed and trials can be shortened. Evaluating surrogates is a complex area as a given treatment can act through multiple pathways, some of which may circumvent the surrogate. One of the best established and practically sound approaches to surrogacy evaluation is based on information theory. We have extended this approach to the case of ordinal outcomes, which are used as primary outcomes in many medical areas. This extension provides researchers with the means of evaluating surrogates in this setting, which expands the usefulness of the information theory approach while also demonstrating its versatility.

Center-Within-Trial Versus Trial-Level Evaluation of Surrogate Endpoints

Evaluation of candidate surrogate endpoints using individual patient data from multiple clinical trials is considered the gold standard approach to validate surrogates at both patient and trial levels. However, this approach assumes the availability of patient-level data from a relatively large collection of similar trials, which may not be possible to achieve for a given disease application. One common solution to the problem of too few similar trials involves performing trial-level surrogacy analyses on trial sub-units (e.g., centers within trials), thereby artificially increasing the trial-level sample size for feasibility of the multi-trial analysis. To date, the practical impact of treating trial sub-units (centers) identically to trials in multi-trial surrogacy analyses remains unexplored, and conditions under which this ad hoc solution may in fact be reasonable have not been identified. We perform a simulation study to identify such conditions, and demonstrate practical implications using a multi-trial dataset of patients with early stage colon cancer.

Advances in the approach to novel drug clinical development for breast cancer

INTRODUCTION

In the post-genomic era clinical development of new agents to treat breast cancer (BC) can be a real challenge. Different from chemotherapy agents, with a broad but not specific spectrum of activity, novel drugs are being developed as 'targeted' agents, potentially benefiting a subgroup of patients. In BC, different clinically identifiable subtypes are now separately addressed in specific clinical trials.

AREAS COVERED

In this review, the authors discuss the clinical development of targeted drugs that have become part of the current treatment of BC. They also highlight the challenges that in other cases determined the failure of promising compounds. Furthermore, the article reports on how combinations of targeted agents have emerged as valid strategies to overcome acquired resistance. It also provides discussion of how 'old' therapies can be retargeted to certain patient populations or 'reinvented' as safer and more effective with the creation of drug conjugates. They also discuss how novel clinical trial designs are emerging to accelerate the successful matching of targeted drugs to the right patient population.

EXPERT OPINION

It is important not to forget that the development of BC therapeutics is a 'moving target', as its biology evolves in time under the pressure of ongoing treatments. There are currently a finite number of resources available for the development of new therapeutics, which means that resources need to be carefully allocated. There is also a need to prioritize clinical trials that can reduce the number of patients who are candidates for expensive treatments.

Endoscopic ulcers as a surrogate marker of NSAID-induced mucosal damage

The characteristic of a biomarker that makes it a useful surrogate is the ability to identify a high risk of clinically important benefits or harms occurring in the future. A number of definitions or descriptions of surrogate definition have been put forward. Most recently the Institute of Medicine of the National Academy of Sciences in the USA has put forward an evaluation scheme for biomarkers, looking at validation (assay performance), qualification (assessment of evidence), and utilisation (the context in which the surrogate is to be used). This paper examines the example of endoscopy as a surrogate marker of NSAID-induced mucosal damage using the Institute of Medicine criteria. The article finds extensive evidence that the detection of endoscopic ulcers is a valid marker. The process of qualification documents abundant evidence showing that endoscopic ulcers and serious upper gastrointestinal damage are influenced in the same direction and much the same magnitude by a variety of risk factors and interventions. Criticisms of validation and qualification for endoscopic ulcers have been examined, and dismissed. Context is the key, and in the context of serious NSAID-induced upper gastrointestinal harm, endoscopic ulcers represent a useful surrogate. Generalisability beyond this context is not considered.

On reporting of effect size in randomized clinical trials

Published reports of randomized clinical trials tend to emphasize the statistical significance of the treatment effect (p values) rather than its magnitude (effect size), although the clinical importance of the evidence depends more on the latter than on the former. We, therefore, compared the standard measures of effect size (relative and absolute risk reduction) and nonstandard composites of these measures (the product of the relative and absolute risk reductions and information content) with conventional assessments of statistical significance for 100 trials published in The New England Journal of Medicine. The p values were reported for 100% of the trials, relative risk reductions for 89%, and absolute risk reductions for 39%. Only 35% of trials reported both standard measures, and none reported either of the nonstandard measures. The standard measures correlated weakly (unexplained variance 77%). In contrast, the nonstandard measures correlated highly (unexplained variance 1.3%) but correlated weakly with statistical significance (unexplained variance 83%). Consequently, 25% of the trial results were adjudged "clinically unimportant" despite being "statistically significant." In conclusion, our results have shown that composite measures of effect size communicate the clinical importance of trial results better than do conventional assessments of risk reduction and statistical significance.

An information-theoretic approach to surrogate-marker evaluation with failure time endpoints

Over the last decades, the evaluation of potential surrogate endpoints in clinical trials has steadily been growing in importance, not only thanks to the availability of ever more potential markers and surrogate endpoints, also because more methodological development has become available. While early work has been devoted, to a large extent, to Gaussian, binary, and longitudinal endpoints, the case of time-to-event endpoints is in need of careful scrutiny as well, owing to the strong presence of such endpoints in oncology and beyond. While work had been done in the past, it was often cumbersome to use such tools in practice, because of the need for fitting copula or frailty models that were further embedded in a hierarchical or two-stage modeling approach. In this paper, we present a methodologically elegant and easy-to-use approach based on information theory. We resolve essential issues, including the quantification of "surrogacy" based on such an approach. Our results are put to the test in a simulation study and are applied to data from clinical trials in oncology. The methodology has been implemented in R.

Recommending flavanols and procyanidins for cardiovascular health: current knowledge and future needs

Data on the potential health benefits of dietary flavanols and procyanidins, especially in the context of cardiovascular health, are considerable and continue to accumulate. Significant progress has been made in flavanol analytics and the creation of phytonutrient-content food databases, and novel data emanated from epidemiological investigations as well as dietary intervention studies. However, a comprehensive understanding of the pharmacological properties of flavanols and procyanidins, including their precise mechanisms of action in vivo, and a conclusive, consensus-based accreditation of a causal relationship between intake and health benefits in the context of primary and secondary cardiovascular disease prevention is still outstanding. Thus, the objective of this review is to identify and discuss key questions and gaps that will need to be addressed in order to conclusively demonstrate whether or not dietary flavanols and procyanidins have a role in preventing, delaying the onset of, or treating cardiovascular diseases, and thus improving human life expectancy and quality of life.

A unified framework for the evaluation of surrogate endpoints in mental-health clinical trials

For a number of reasons, surrogate endpoints are considered instead of the so-called true endpoint in clinical studies, especially when such endpoints can be measured earlier, and/or with less burden for patient and experimenter. Surrogate endpoints may occur more frequently than their standard counterparts. For these reasons, it is not surprising that the use of surrogate endpoints in clinical practice is increasing. Building on the seminal work of Prentice(1) and Freedman et al.,(2) Buyse et al. (3) framed the evaluation exercise within a meta-analytic setting, in an effort to overcome difficulties that necessarily surround evaluation efforts based on a single trial. In this article, we review the meta-analytic approach for continuous outcomes, discuss extensions to non-normal and longitudinal settings, as well as proposals to unify the somewhat disparate collection of validation measures currently on the market. Implications for design and for predicting the effect of treatment in a new trial, based on the surrogate, are discussed. A case study in schizophrenia is analysed.

Meta-analysis for the evaluation of surrogate endpoints in cancer clinical trials

The identification and validation of putative surrogate endpoints in oncology is a great challenge to medical investigators, statisticians, and regulators. A putative surrogate endpoint must be validated at both individual-level and trial-level before it can be used to replace the clinical endpoint in a future clinical trial. Recently, meta-analytic methods for evaluating potential surrogates have become widely accepted in cancer clinical trials. In this review, after addressing multiple complications and general issues surrounding surrogate endpoints, we review various proposed and adopted meta-analytic methodologies pertaining to the application of these methods to oncology clinical trials with different tumor types. In oncology, several applications have successfully identified useful surrogates. For example, disease-free survival and progression-free survival have been validated through meta-analyses as acceptable surrogates for overall survival in adjuvant colon cancer and advanced colorectal cancer, respectively. We also discuss several limitations of surrogate endpoints, including the critical issues that the extrapolation of the validity of a surrogate is always context-dependent and that such extrapolation should be exercised with caution.