1
|
Jiao B. Estimating the Potential Benefits of Confirmatory Trials for Drugs with Accelerated Approval: A Comprehensive Value of Information Framework. PHARMACOECONOMICS 2023; 41:1617-1627. [PMID: 37490206 DOI: 10.1007/s40273-023-01303-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 07/04/2023] [Indexed: 07/26/2023]
Abstract
BACKGROUND The US Food and Drug Administration's Accelerated Approval (AA) policy provides a pathway for patients to access potentially life-saving drugs rapidly. However, the use of surrogate endpoints, single-arm designs, and small sample sizes in preliminary trials that support AAs can lead to uncertainty regarding the clinical benefits of such drugs. This study aims to develop a comprehensive value of information (VOI) framework for assessing the potential benefits of future confirmatory trials, accounting for the various uncertainties inherent in preliminary trials. METHODS I formulated an expected value of information from confirmatory trial (EVICT) metric, which evaluates the potential benefits of a confirmatory trial that would reduce those uncertainties by using a clinically meaningful endpoint, a randomized control, and increased sample size. The EVICT metric can quantify the expected benefits of a well-designed confirmatory trial or an inadequately designed one that continues to use surrogate endpoints or single-arm design. The framework was illustrated using a hypothetical AA drug for metastatic breast cancer. RESULTS The case study demonstrates that a highly uncertain preliminary trial of an AA drug was associated with a substantial EVICT. A confirmatory trial with an increased sample size for this AA drug, utilizing a clinically meaningful endpoint and randomized control, yielded a population-level EVICT of $12.6 million. Persistently using a surrogate endpoint and single-arm trial design would reduce the EVICT by 60%. CONCLUSIONS This framework can provide accurate VOI estimates to guide coverage policies, value-based pricing, and the design of confirmatory trials for AA drugs.
Collapse
Affiliation(s)
- Boshen Jiao
- Harvard T.H. Chan School of Public Health, 90 Smith St, Boston, MA, 02120, USA.
| |
Collapse
|
2
|
Götte H, Xiong J, Kirchner M, Demirtas H, Kieser M. Optimal decision‐making in oncology development programs based on probability of success for phase
III
utilizing phase
II
/
III
data on response and overall survival. Pharm Stat 2020; 19:861-881. [DOI: 10.1002/pst.2042] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2019] [Revised: 05/18/2020] [Accepted: 05/27/2020] [Indexed: 11/10/2022]
Affiliation(s)
| | | | - Marietta Kirchner
- Institute of Medical Biometry and Informatics University of Heidelberg Heidelberg Germany
| | - Hakan Demirtas
- Division of Epidemiology and Biostatistics University of Illinois Chicago Illinois USA
| | - Meinhard Kieser
- Institute of Medical Biometry and Informatics University of Heidelberg Heidelberg Germany
| |
Collapse
|
3
|
Baker SG. Five criteria for using a surrogate endpoint to predict treatment effect based on data from multiple previous trials. Stat Med 2018; 37:507-518. [PMID: 29164641 DOI: 10.1002/sim.7561] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2017] [Revised: 10/20/2017] [Accepted: 10/23/2017] [Indexed: 11/08/2022]
Abstract
A surrogate endpoint in a randomized clinical trial is an endpoint that occurs after randomization and before the true, clinically meaningful, endpoint that yields conclusions about the effect of treatment on true endpoint. A surrogate endpoint can accelerate the evaluation of new treatments but at the risk of misleading conclusions. Therefore, criteria are needed for deciding whether to use a surrogate endpoint in a new trial. For the meta-analytic setting of multiple previous trials, each with the same pair of surrogate and true endpoints, this article formulates 5 criteria for using a surrogate endpoint in a new trial to predict the effect of treatment on the true endpoint in the new trial. The first 2 criteria, which are easily computed from a zero-intercept linear random effects model, involve statistical considerations: an acceptable sample size multiplier and an acceptable prediction separation score. The remaining 3 criteria involve clinical and biological considerations: similarity of biological mechanisms of treatments between the new trial and previous trials, similarity of secondary treatments following the surrogate endpoint between the new trial and previous trials, and a negligible risk of harmful side effects arising after the observation of the surrogate endpoint in the new trial. These 5 criteria constitute an appropriately high bar for using a surrogate endpoint to make a definitive treatment recommendation.
Collapse
Affiliation(s)
- Stuart G Baker
- Division of Cancer Prevention, National Cancer Institute, 9609 Medical Center Dr, Room 5E606, MSC 9789, Bethesda, MD, 20892-9789, USA
| |
Collapse
|
4
|
Ensor H, Lee RJ, Sudlow C, Weir CJ. Statistical approaches for evaluating surrogate outcomes in clinical trials: A systematic review. J Biopharm Stat 2016; 26:859-79. [DOI: 10.1080/10543406.2015.1094811] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Affiliation(s)
- Hannah Ensor
- Centre for Population Health Sciences, University of Edinburgh Medical School, Edinburgh, UK
| | - Robert J. Lee
- Centre for Population Health Sciences, University of Edinburgh Medical School, Edinburgh, UK
| | - Cathie Sudlow
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Christopher J. Weir
- Centre for Population Health Sciences, University of Edinburgh Medical School, Edinburgh, UK
- Edinburgh Health Services Research Unit, University of Edinburgh, Western General Hospital, Edinburgh, UK
| |
Collapse
|
5
|
Jiang Z, Ding P, Geng Z. Principal causal effect identification and surrogate end point evaluation by multiple trials. J R Stat Soc Series B Stat Methodol 2015. [DOI: 10.1111/rssb.12135] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Affiliation(s)
- Zhichao Jiang
- Peking University; Beijing People's Republic of China
| | - Peng Ding
- University of California at Berkeley; USA
| | - Zhi Geng
- Peking University; Beijing People's Republic of China
| |
Collapse
|
6
|
Baker SG, Kramer BS, Lindeman KS. Latent class instrumental variables: a clinical and biostatistical perspective. Stat Med 2015; 35:147-60. [PMID: 26239275 DOI: 10.1002/sim.6612] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2015] [Revised: 05/13/2015] [Accepted: 07/13/2015] [Indexed: 11/06/2022]
Abstract
In some two-arm randomized trials, some participants receive the treatment assigned to the other arm as a result of technical problems, refusal of a treatment invitation, or a choice of treatment in an encouragement design. In some before-and-after studies, the availability of a new treatment changes from one time period to this next. Under assumptions that are often reasonable, the latent class instrumental variable (IV) method estimates the effect of treatment received in the aforementioned scenarios involving all-or-none compliance and all-or-none availability. Key aspects are four initial latent classes (sometimes called principal strata) based on treatment received if in each randomization group or time period, the exclusion restriction assumption (in which randomization group or time period is an instrumental variable), the monotonicity assumption (which drops an implausible latent class from the analysis), and the estimated effect of receiving treatment in one latent class (sometimes called efficacy, the local average treatment effect, or the complier average causal effect). Since its independent formulations in the biostatistics and econometrics literatures, the latent class IV method (which has no well-established name) has gained increasing popularity. We review the latent class IV method from a clinical and biostatistical perspective, focusing on underlying assumptions, methodological extensions, and applications in our fields of obstetrics and cancer research.
Collapse
Affiliation(s)
- Stuart G Baker
- Division of Cancer Prevention, National Cancer Institute, Bethesda, MD, U.S.A
| | - Barnett S Kramer
- Division of Cancer Prevention, National Cancer Institute, Bethesda, MD, U.S.A
| | - Karen S Lindeman
- Department of Anesthesiology, Johns Hopkins Medical Institutions, Baltimore, MD, U.S.A
| |
Collapse
|
7
|
Baker SG, Kramer BS. Evaluating surrogate endpoints, prognostic markers, and predictive markers: Some simple themes. Clin Trials 2015; 12:299-308. [PMID: 25385934 PMCID: PMC4451440 DOI: 10.1177/1740774514557725] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
BACKGROUND A surrogate endpoint is an endpoint observed earlier than the true endpoint (a health outcome) that is used to draw conclusions about the effect of treatment on the unobserved true endpoint. A prognostic marker is a marker for predicting the risk of an event given a control treatment; it informs treatment decisions when there is information on anticipated benefits and harms of a new treatment applied to persons at high risk. A predictive marker is a marker for predicting the effect of treatment on outcome in a subgroup of patients or study participants; it provides more rigorous information for treatment selection than a prognostic marker when it is based on estimated treatment effects in a randomized trial. METHODS We organized our discussion around a different theme for each topic. RESULTS "Fundamentally an extrapolation" refers to the non-statistical considerations and assumptions needed when using surrogate endpoints to evaluate a new treatment. "Decision analysis to the rescue" refers to use the use of decision analysis to evaluate an additional prognostic marker because it is not possible to choose between purely statistical measures of marker performance. "The appeal of simplicity" refers to a straightforward and efficient use of a single randomized trial to evaluate overall treatment effect and treatment effect within subgroups using predictive markers. CONCLUSION The simple themes provide a general guideline for evaluation of surrogate endpoints, prognostic markers, and predictive markers.
Collapse
Affiliation(s)
- Stuart G Baker
- Division of Cancer Prevention, National Cancer Institute, Bethesda MD, USA
| | - Barnett S Kramer
- Division of Cancer Prevention, National Cancer Institute, Bethesda MD, USA
| |
Collapse
|
8
|
Halabi S, Rini BI, Escudier B, Stadler WM, Small EJ. Progression-free survival: does a correlation with survival justify its role as a surrogate clinical endpoint? Cancer 2015; 121:1906. [PMID: 25677867 DOI: 10.1002/cncr.29252] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Affiliation(s)
- Susan Halabi
- Department of Biostatistics and Bioinformatics, Duke University, Alliance Statistical and Data Center, Duke University, Durham, North Carolina
| | - Brian I Rini
- Cleveland Clinic Taussig Cancer Institute, Cleveland, Ohio
| | - Bernard Escudier
- Department of Cancer Medicine, Institut Gustave Roussy, Villejuif, France
| | - Walter M Stadler
- Department of Medicine, University of Chicago Medical Center, Chicago, Illinois
| | - Eric J Small
- Department of Urology, University of California at San Francisco, San Francisco, California
| |
Collapse
|
9
|
Wilson MK, Karakasis K, Oza AM. Outcomes and endpoints in trials of cancer treatment: the past, present, and future. Lancet Oncol 2014; 16:e32-42. [PMID: 25638553 DOI: 10.1016/s1470-2045(14)70375-4] [Citation(s) in RCA: 142] [Impact Index Per Article: 14.2] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/05/2023]
Abstract
Cancer treatment should allow patients to live better or longer lives, and ideally, both. Trial endpoints should show clinically meaningful improvements in patient survival or quality of life. Alternative endpoints such as progression-free survival, disease-free survival, and objective response rate have been used to identify benefit earlier, but their true validity as surrogate endpoints is controversial. In this Review we discuss the measurement, assessment, and benefits and limitations of trial endpoints in use for cancer treatment. Many stakeholders are affected, including regulatory agencies, industry partners, clinicians, and most importantly, patients. In an accompanying Review, reflections from individual stakeholders are incorporated into a discussion of what the future holds for clinical trial endpoints and design.
Collapse
Affiliation(s)
| | | | - Amit M Oza
- Princess Margaret Cancer Centre, Toronto, Canada.
| |
Collapse
|
10
|
A Bayesian prediction model between a biomarker and the clinical endpoint for dichotomous variables. Trials 2014; 15:500. [PMID: 25528466 PMCID: PMC4307375 DOI: 10.1186/1745-6215-15-500] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2014] [Accepted: 12/04/2014] [Indexed: 12/02/2022] Open
Abstract
Background Early biomarkers are helpful for predicting clinical endpoints and for evaluating efficacy in clinical trials even if the biomarker cannot replace clinical outcome as a surrogate. The building and evaluation of an association model between biomarkers and clinical outcomes are two equally important concerns regarding the prediction of clinical outcome. This paper is to address both issues in a Bayesian framework. Methods A Bayesian meta-analytic approach is proposed to build a prediction model between the biomarker and clinical endpoint for dichotomous variables. Compared with other Bayesian methods, the proposed model only requires trial-level summary data of historical trials in model building. By using extensive simulations, we evaluate the link function and the application condition of the proposed Bayesian model under scenario (i) equal positive predictive value (PPV) and negative predictive value (NPV) and (ii) higher NPV and lower PPV. In the simulations, the patient-level data is generated to evaluate the meta-analytic model. PPV and NPV are employed to describe the patient-level relationship between the biomarker and the clinical outcome. The minimum number of historical trials to be included in building the model is also considered. Results It is seen from the simulations that the logit link function performs better than the odds and cloglog functions under both scenarios. PPV/NPV ≥0.5 for equal PPV and NPV, and PPV + NPV ≥1 for higher NPV and lower PPV are proposed in order to predict clinical outcome accurately and precisely when the proposed model is considered. Twenty historical trials are required to be included in model building when PPV and NPV are equal. For unequal PPV and NPV, the minimum number of historical trials for model building is proposed to be five. A hypothetical example shows an application of the proposed model in global drug development. Conclusions The proposed Bayesian model is able to predict well the clinical endpoint from the observed biomarker data for dichotomous variables as long as the conditions are satisfied. It could be applied in drug development. But the practical problems in applications have to be studied in further research. Electronic supplementary material The online version of this article (doi:10.1186/1745-6215-15-500) contains supplementary material, which is available to authorized users.
Collapse
|
11
|
Baker SG, Kramer BS. The risky reliance on small surrogate endpoint studies when planning a large prevention trial. JOURNAL OF THE ROYAL STATISTICAL SOCIETY. SERIES A, (STATISTICS IN SOCIETY) 2013; 176:603-608. [PMID: 23565041 PMCID: PMC3616635 DOI: 10.1111/j.1467-985x.2012.01052.x] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/02/2023]
Abstract
The definitive evaluation of treatment to prevent a chronic disease with low incidence in middle age, such as cancer or cardiovascular disease, requires a trial with a large sample size of perhaps 20,000 or more. To help decide whether to implement a large true endpoint trial, investigators first typically estimate the effect of treatment on a surrogate endpoint in a trial with a greatly reduced sample size of perhaps 200 subjects. If investigators reject the null hypothesis of no treatment effect in the surrogate endpoint trial they implicitly assume they would likely correctly reject the null hypothesis of no treatment effect for the true endpoint. Surrogate endpoint trials are generally designed with adequate power to detect an effect of treatment on surrogate endpoint. However, we show that a small surrogate endpoint trial is more likely than a large surrogate endpoint trial to give a misleading conclusion about the beneficial effect of treatment on true endpoint, which can lead to a faulty (and costly) decision about implementing a large true endpoint prevention trial. If a small surrogate endpoint trial rejects the null hypothesis of no treatment effect, an intermediate-sized surrogate endpoint trial could be a useful next step in the decision-making process for launching a large true endpoint prevention trial.
Collapse
|
12
|
Baker SG, Kramer BS. Surrogate endpoint analysis: an exercise in extrapolation. J Natl Cancer Inst 2012; 105:316-20. [PMID: 23264679 DOI: 10.1093/jnci/djs527] [Citation(s) in RCA: 24] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022] Open
Abstract
Surrogate endpoints offer the hope of smaller or shorter cancer trials. It is, however, important to realize they come at the cost of an unverifiable extrapolation that could lead to misleading conclusions. With cancer prevention, the focus is on hypothesis testing in small surrogate endpoint trials before deciding whether to proceed to a large prevention trial. However, it is not generally appreciated that a small surrogate endpoint trial is highly sensitive to a deviation from the key Prentice criterion needed for the hypothesis-testing extrapolation. With cancer treatment, the focus is on estimation using historical trials with both surrogate and true endpoints to predict treatment effect based on the surrogate endpoint in a new trial. Successively leaving out one historical trial and computing the predicted treatment effect in the left-out trial yields a standard error multiplier that summarizes the increased uncertainty in estimation extrapolation. If this increased uncertainty is acceptable, three additional extrapolation issues (biological mechanism, treatment following observation of the surrogate endpoint, and side effects following observation of the surrogate endpoint) need to be considered. In summary, when using surrogate endpoint analyses, an appreciation of the problems of extrapolation is crucial.
Collapse
Affiliation(s)
- Stuart G Baker
- National Cancer Institute, EPN 3131, 6130 Executive Blvd, MSC 7354, Bethesda, MD 20892-7354, USA.
| | | |
Collapse
|
13
|
Walter SD, Sun X, Heels-Ansdell D, Guyatt G. Treatment effects on patient-important outcomes can be small, even with large effects on surrogate markers. J Clin Epidemiol 2012; 65:940-5. [DOI: 10.1016/j.jclinepi.2012.02.012] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2011] [Revised: 02/15/2012] [Accepted: 02/19/2012] [Indexed: 11/28/2022]
|