1
|
Okada K, Tanaka S, Matsubayashi J, Takahashi K, Yokota I. Decoupling power and type I error rate considerations when incorporating historical control data using a test-then-pool approach. Biom J 2024; 66:e2200312. [PMID: 38285403 DOI: 10.1002/bimj.202200312] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2022] [Revised: 08/09/2023] [Accepted: 09/17/2023] [Indexed: 01/30/2024]
Abstract
To accelerate a randomized controlled trial, historical control data may be used after ensuring little heterogeneity between the historical and current trials. The test-then-pool approach is a simple frequentist borrowing method that assesses the similarity between historical and current control data using a two-sided test. A limitation of the conventional test-then-pool method is the inability to control the type I error rate and power for the primary hypothesis separately and flexibly for heterogeneity between trials. This is because the two-sided test focuses on the absolute value of the mean difference between the historical and current controls. In this paper, we propose a new test-then-pool method that splits the two-sided hypothesis of the conventional method into two one-sided hypotheses. Testing each one-sided hypothesis with different significance levels allows for the separate control of the type I error rate and power for heterogeneity between trials. We also propose a significance-level selection approach based on the maximum type I error rate and the minimum power. The proposed method prevented a decrease in power even when there was heterogeneity between trials while controlling type I error at a maximum tolerable type I error rate larger than the targeted type I error rate. The application of depression trial data and hypothetical trial data further supported the usefulness of the proposed method.
Collapse
Affiliation(s)
- Kazufumi Okada
- Department of Biostatistics, Graduate School of Medicine, Hokkaido University, Sapporo, Japan
| | - Shiro Tanaka
- Department of Clinical Biostatistics, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Jun Matsubayashi
- Center for Clinical Research and Advanced Medicine, Shiga University of Medical Science, Otsu, Japan
| | - Keita Takahashi
- Department of Biostatistics, Graduate School of Medicine, Hokkaido University, Sapporo, Japan
| | - Isao Yokota
- Department of Biostatistics, Graduate School of Medicine, Hokkaido University, Sapporo, Japan
| |
Collapse
|
2
|
Majumdar A, Rothwell R, Reaman G, Ahlberg C, Roy P. Utility of propensity score-based Bayesian borrowing of external adult data in pediatric trials: A pragmatic evaluation through a case study in acute lymphoblastic leukemia (ALL). J Biopharm Stat 2023; 33:737-751. [PMID: 36600441 DOI: 10.1080/10543406.2022.2162069] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2022] [Accepted: 12/19/2022] [Indexed: 01/06/2023]
Abstract
A fully powered randomized controlled cancer trial can be challenging to conduct in children because of difficulties in enrollment of pediatric patients due to low disease incidence. One way to improve the feasibility of trials in pediatric patients, when clinically appropriate, is through borrowing information from comparable external adult trials in the same disease. Bayesian analysis of a pediatric trial provides a way of seamlessly augmenting pediatric trial efficacy data with data from external adult trials. However, not all external adult trial subjects may be equally clinically relevant with respect to the baseline disease severity, prognostic factors, co-morbidities, and prior therapy observed in the pediatric trial of interest. The propensity score matching method provides a way of matching the external adult subjects to the pediatric trial subjects on a set of clinically determined baseline covariates, such as baseline disease severity, prognostic factors and prior therapy. The matching then allows Bayesian information borrowing from only the most clinically relevant external adult subjects. Through a case study in pediatric acute lymphoblastic leukemia (ALL), we examine the utility of propensity score matched mixture and power priors in bringing appropriate external adult efficacy information into pediatric trial efficacy assessment, and present considerations for scaling fixed borrowing from external adult data.
Collapse
Affiliation(s)
- Antara Majumdar
- Oncology Biostatistics, GlaxoSmithKline, Collegeville, PA, USA
| | - Rebecca Rothwell
- Office of Biostatistics, U.S. Food and Drug Administration, Silver Spring, MD, USA
| | - Gregory Reaman
- Oncology Center of Excellence, U.S. Food and Drug Administration, Silver Spring, MD, USA
| | - Corinne Ahlberg
- Acorn AI by Medidata, a Dassault Systèmes company, New York, NY, USA
| | - Pourab Roy
- Biostatistics, Regeneron Pharmaceuticals, Tarrytown, NY, USA
| |
Collapse
|
3
|
Law M, Couturier DL, Choodari-Oskooei B, Crout P, Gamble C, Jacko P, Pallmann P, Pilling M, Robertson DS, Robling M, Sydes MR, Villar SS, Wason J, Wheeler G, Williamson SF, Yap C, Jaki T. Medicines and Healthcare products Regulatory Agency's "Consultation on proposals for legislative changes for clinical trials": a response from the Trials Methodology Research Partnership Adaptive Designs Working Group, with a focus on data sharing. Trials 2023; 24:640. [PMID: 37798805 PMCID: PMC10552399 DOI: 10.1186/s13063-023-07576-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2022] [Accepted: 08/04/2023] [Indexed: 10/07/2023] Open
Abstract
In the UK, the Medicines and Healthcare products Regulatory Agency consulted on proposals "to improve and strengthen the UK clinical trials legislation to help us make the UK the best place to research and develop safe and innovative medicines". The purpose of the consultation was to help finalise the proposals and contribute to the drafting of secondary legislation. We discussed these proposals as members of the Trials Methodology Research Partnership Adaptive Designs Working Group, which is jointly funded by the Medical Research Council and the National Institute for Health and Care Research. Two topics arose frequently in the discussion: the emphasis on legislation, and the absence of questions on data sharing. It is our opinion that the proposals rely heavily on legislation to change practice. However, clinical trials are heterogeneous, and as a result some trials will struggle to comply with all of the proposed legislation. Furthermore, adaptive design clinical trials are even more heterogeneous than their non-adaptive counterparts, and face more challenges. Consequently, it is possible that increased legislation could have a greater negative impact on adaptive designs than non-adaptive designs. Overall, we are sceptical that the introduction of legislation will achieve the desired outcomes, with some exceptions. Meanwhile the topic of data sharing - making anonymised individual-level clinical trial data available to other investigators for further use - is entirely absent from the proposals and the consultation in general. However, as an aspect of the wider concept of open science and reproducible research, data sharing is an increasingly important aspect of clinical trials. The benefits of data sharing include faster innovation, improved surveillance of drug safety and effectiveness and decreasing participant exposure to unnecessary risk. There are already a number of UK-focused documents that discuss and encourage data sharing, for example, the Concordat on Open Research Data and the Medical Research Council's Data Sharing Policy. We strongly suggest that data sharing should be the norm rather than the exception, and hope that the forthcoming proposals on clinical trials invite discussion on this important topic.
Collapse
Affiliation(s)
- Martin Law
- Medical Research Council Biostatistics Unit, School of Clinical Medicine, University of Cambridge, Cambridge, UK.
- Royal Papworth Hospital NHS Foundation Trust, Cambridge, UK.
| | - Dominique-Laurent Couturier
- Medical Research Council Biostatistics Unit, School of Clinical Medicine, University of Cambridge, Cambridge, UK
- Cancer Research UK Cambridge Institute, University of Cambridge, Cambridge, UK
| | | | - Phillip Crout
- Medical Research Council Biostatistics Unit, School of Clinical Medicine, University of Cambridge, Cambridge, UK
| | - Carrol Gamble
- Liverpool Clinical Trials Centre, University of Liverpool, Liverpool, UK
| | - Peter Jacko
- Lancaster University Management School, Lancaster University, Lancaster, UK
- Berry Consultants, Abingdon, UK
| | | | - Mark Pilling
- Department of Public Health and Primary Care, School of Clinical Medicine, University of Cambridge, Cambridge, UK
| | - David S Robertson
- Medical Research Council Biostatistics Unit, School of Clinical Medicine, University of Cambridge, Cambridge, UK
| | | | - Matthew R Sydes
- University College London, London, UK
- British Heart Foundation Data Science Centre, Health Data Research UK, London, UK
| | - Sofía S Villar
- Medical Research Council Biostatistics Unit, School of Clinical Medicine, University of Cambridge, Cambridge, UK
| | - James Wason
- Biostatistics Research Group, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
| | - Graham Wheeler
- Imperial Clinical Trials Unit, Imperial College London, London, W12 7RH, UK
| | - S Faye Williamson
- Biostatistics Research Group, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
| | - Christina Yap
- Clinical Trials and Statistics Unit, The Institute of Cancer Research, London, UK
| | - Thomas Jaki
- Medical Research Council Biostatistics Unit, School of Clinical Medicine, University of Cambridge, Cambridge, UK
- Faculty for Informatics and Data Science, University of Regensburg, Regensburg, Germany
| |
Collapse
|
4
|
Li R, Lin R, Huang J, Tian L, Zhu J. A frequentist approach to dynamic borrowing. Biom J 2023; 65:e2100406. [PMID: 37189217 DOI: 10.1002/bimj.202100406] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2021] [Revised: 11/04/2022] [Accepted: 02/17/2023] [Indexed: 05/17/2023]
Abstract
There has been growing interest in leveraging external control data to augment a randomized control group data in clinical trials and enable more informative decision making. In recent years, the quality and availability of real-world data have improved steadily as external controls. However, information borrowing by directly pooling such external controls with randomized controls may lead to biased estimates of the treatment effect. Dynamic borrowing methods under the Bayesian framework have been proposed to better control the false positive error. However, the numerical computation and, especially, parameter tuning, of those Bayesian dynamic borrowing methods remain a challenge in practice. In this paper, we present a frequentist interpretation of a Bayesian commensurate prior borrowing approach and describe intrinsic challenges associated with this method from the perspective of optimization. Motivated by this observation, we propose a new dynamic borrowing approach using adaptive lasso. The treatment effect estimate derived from this method follows a known asymptotic distribution, which can be used to construct confidence intervals and conduct hypothesis tests. The finite sample performance of the method is evaluated through extensive Monte Carlo simulations under different settings. We observed highly competitive performance of adaptive lasso compared to Bayesian approaches. Methods for selecting tuning parameters are also thoroughly discussed based on results from numerical studies and an illustration example.
Collapse
Affiliation(s)
- Ruilin Li
- Institute for Computational and Mathematical Engineering, Stanford University, Stanford, California, USA
| | - Ray Lin
- Genentech, Inc., PD Data Sciences, San Francisco, California, USA
| | - Jiangeng Huang
- Genentech, Inc., PD Data Sciences, San Francisco, California, USA
| | - Lu Tian
- Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, California, USA
| | - Jiawen Zhu
- Genentech, Inc., PD Data Sciences, San Francisco, California, USA
| |
Collapse
|
5
|
Harari O, Soltanifar M, Verhoek A, Heeg B. Alone, together: On the benefits of Bayesian borrowing in a meta-analytic setting. Pharm Stat 2023; 22:903-920. [PMID: 37321565 DOI: 10.1002/pst.2318] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2022] [Revised: 04/11/2023] [Accepted: 05/26/2023] [Indexed: 06/17/2023]
Abstract
It is common practice to use hierarchical Bayesian model for the informing of a pediatric randomized controlled trial (RCT) by adult data, using a prespecified borrowing fraction parameter (BFP). This implicitly assumes that the BFP is intuitive and corresponds to the degree of similarity between the populations. Generalizing this model to any K ≥ 1 historical studies, naturally leads to empirical Bayes meta-analysis. In this paper we calculate the Bayesian BFPs and study the factors that drive them. We prove that simultaneous mean squared error reduction relative to an uninformed model is always achievable through application of this model. Power and sample size calculations for a future RCT, designed to be informed by multiple external RCTs, are also provided. Potential applications include inference on treatment efficacy from independent trials involving either heterogeneous patient populations or different therapies from a common class.
Collapse
Affiliation(s)
- Ofir Harari
- Real World and Advanced Analytics, Cytel Inc., Vancouver, British Columbia, Canada
- Core Clinical Sciences, Vancouver, British Columbia, Canada
| | - Mohsen Soltanifar
- Real World and Advanced Analytics, Cytel Inc., Vancouver, British Columbia, Canada
- Analytics Division, College of Professional Studies, Northeastern University, Vancouver, British Columbia, Canada
| | | | - Bart Heeg
- RWA & HEOR, Cytel Inc., Rotterdam, The Netherlands
| |
Collapse
|
6
|
Arfè A, Narang C, DuBois SG, Reaman G, Bourgeois FT. Clinical development of new drugs for adults and children with cancer, 2010-2020. J Natl Cancer Inst 2023; 115:917-925. [PMID: 37171887 PMCID: PMC10407707 DOI: 10.1093/jnci/djad082] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2022] [Revised: 01/30/2023] [Accepted: 05/09/2023] [Indexed: 05/14/2023] Open
Abstract
BACKGROUND Many new molecular entities enter clinical development to evaluate potential therapeutic benefits for oncology patients. We characterized adult and pediatric development of the set of new molecular entities that started clinical testing in 2010-2015 worldwide. METHODS We extracted data from AdisInsight, an extensive database of global pharmaceutical development, and the FDA.gov website. We followed the cohort of new molecular entities initiating first-in-human phase I clinical trials in 2010-2015 to the end of 2020. For each new molecular entity, we determined whether it was granted US Food and Drug Administration (FDA) approval, studied in a trial open to pediatric enrollment, or stalled during development. We characterized the cumulative incidence of these endpoints using statistical methods for censored data. RESULTS The 572 new molecular entities starting first-in-human studies in 2010-2015 were studied in 6142 trials by the end of 2020. Most new molecular entities were small molecules (n = 316, 55.2%), antibodies (n = 148, 25.9%), or antibody-drug conjugates (n = 44, 7.7%). After a mean follow-up of 8.0 years, 173 new molecular entities did not advance beyond first-in-human trials, and 39 were approved by the FDA. New molecular entities had a 10.4% estimated probability (95% confidence interval = 6.6% to 14.1%) of being approved by the FDA within 10 years of first-in-human trials. After a median of 4.6 years since start of first-in-human trials, 67 (11.7%) new molecular entities were tested in trials open to pediatric patients, and 5 (0.9%) were approved for pediatric indications. CONCLUSIONS More efficient clinical development strategies are needed to evaluate new cancer therapies, especially for children, and incorporate approaches to ensure knowledge gain from investigational products that stall in development.
Collapse
Affiliation(s)
- Andrea Arfè
- Department of Epidemiology and Biostatistics, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Claire Narang
- Pediatric Therapeutics and Regulatory Science Initiative, Computational Health Informatics Program (CHIP), Boston Children’s Hospital, Boston, MA, USA
| | - Steven G DuBois
- Dana-Farber/Boston Children’s Cancer and Blood Disorders Center, Boston, MA, USA
- Department of Pediatrics, Harvard Medical School, Boston, MA, USA
| | - Gregory Reaman
- Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, MD, USA
| | - Florence T Bourgeois
- Pediatric Therapeutics and Regulatory Science Initiative, Computational Health Informatics Program (CHIP), Boston Children’s Hospital, Boston, MA, USA
- Department of Pediatrics, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
7
|
Han Z, Zhang Q, Wang M, Ye K, Chen MH. On efficient posterior inference in normalized power prior Bayesian analysis. Biom J 2023; 65:e2200194. [PMID: 36960489 DOI: 10.1002/bimj.202200194] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Revised: 11/24/2022] [Accepted: 02/15/2023] [Indexed: 03/25/2023]
Abstract
The power prior has been widely used to discount the amount of information borrowed from historical data in the design and analysis of clinical trials. It is realized by raising the likelihood function of the historical data to a power parameterδ ∈ [ 0 , 1 ] $\delta \in [0, 1]$ , which quantifies the heterogeneity between the historical and the new study. In a fully Bayesian approach, a natural extension is to assign a hyperprior to δ such that the posterior of δ can reflect the degree of similarity between the historical and current data. To comply with the likelihood principle, an extra normalizing factor needs to be calculated and such prior is known as the normalized power prior. However, the normalizing factor involves an integral of a prior multiplied by a fractional likelihood and needs to be computed repeatedly over different δ during the posterior sampling. This makes its use prohibitive in practice for most elaborate models. This work provides an efficient framework to implement the normalized power prior in clinical studies. It bypasses the aforementioned efforts by sampling from the power prior withδ = 0 $\delta = 0$ andδ = 1 $\delta = 1$ only. Such a posterior sampling procedure can facilitate the use of a random δ with adaptive borrowing capability in general models. The numerical efficiency of the proposed method is illustrated via extensive simulation studies, a toxicological study, and an oncology study.
Collapse
Affiliation(s)
- Zifei Han
- School of Statistics, University of International Business and Economics, Beijing, China
| | - Qiang Zhang
- School of Statistics, University of International Business and Economics, Beijing, China
| | - Min Wang
- Department of Management Science and Statistics, The University of Texas at San Antonio, San Antonio, Texas, USA
| | - Keying Ye
- Department of Management Science and Statistics, The University of Texas at San Antonio, San Antonio, Texas, USA
| | - Ming-Hui Chen
- Department of Statistics, University of Connecticut, Storrs, Connecticut, USA
| |
Collapse
|
8
|
Brizzi F, Steiert B, Pang H, Diack C, Lomax M, Peck R, Morgan Z, Soubret A. A model-based approach for historical borrowing, with an application to neovascular age-related macular degeneration. Stat Methods Med Res 2023; 32:1064-1081. [PMID: 37082812 DOI: 10.1177/09622802231155597] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/22/2023]
Abstract
Bayesian historical borrowing has recently attracted growing interest due to the increasing availability of historical control data, as well as improved computational methodology and software. In this article, we argue that the statistical models used for borrowing may be suboptimal when they do not adjust for differing factors across historical studies such as covariates, dosing regimen, etc. We propose an alternative approach to address these shortcomings. We start by constructing a historical model based on subject-level historical data to accurately characterize the control treatment by adjusting for known between trials differences. This model is subsequently used to predict the control arm response in the current trial, enabling the derivation of a model-informed prior for the treatment effect parameter of another (potentially simpler) model used to analyze the trial efficacy (i.e. the trial model). Our approach is applied to neovascular age-related macular degeneration trials, employing a cross-sectional regression trial model, and a longitudinal non-linear mixed-effects drug-disease-trial historical model. The latter model characterizes the relationship between clinical response, drug exposure and baseline covariates so that the derived model-informed prior seamlessly adapts to the trial population and can be extrapolated to a different dosing regimen. This approach can yield a more accurate prior for borrowing, thus optimizing gains in efficiency (e.g. increasing power or reducing the sample size) in future trials.
Collapse
Affiliation(s)
- Francesco Brizzi
- Predictive Modelling and Data Analytics, Roche Pharma Research & Early Development, Roche Innovation Center Basel, Switzerland
| | - Bernhard Steiert
- Predictive Modelling and Data Analytics, Roche Pharma Research & Early Development, Roche Innovation Center Basel, Switzerland
| | - Herbert Pang
- Methods Collaboration & Outreach (MCO) Enabling Platform, Genentech Inc., South San Francisco, USA
| | - Cheikh Diack
- Predictive Modelling and Data Analytics, Roche Pharma Research & Early Development, Roche Innovation Center Basel, Switzerland
| | - Mark Lomax
- Data & Statistical Sciences, F. Hoffman-La Roche Ltd, Welwyn Garden City, UK
| | - Robbie Peck
- Data & Statistical Sciences, Hoffmann-La Roche AG, Basel, Switzerland
| | - Zoe Morgan
- Data & Statistical Sciences, Hoffmann-La Roche AG, Basel, Switzerland
| | - Antoine Soubret
- Predictive Modelling and Data Analytics, Roche Pharma Research & Early Development, Roche Innovation Center Basel, Switzerland
| |
Collapse
|
9
|
Harun N, Gupta N, McCormack FX, Macaluso M. Dynamic use of historical controls in clinical trials for rare disease research: A re-evaluation of the MILES trial. Clin Trials 2023; 20:223-234. [PMID: 36927115 PMCID: PMC10257755 DOI: 10.1177/17407745231158906] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/18/2023]
Abstract
BACKGROUND Randomized controlled trials offer the best design for eliminating bias in estimating treatment effects but can be slow and costly in rare disease research. Additionally, an equal randomization approach may not be optimal in studies in which prior evidence of superiority of one or more treatments exist. Supplementing prospectively enrolled, concurrent controls with historical controls can reduce recruitment requirements and provide patients a higher likelihood of enrolling in a new and possibly superior treatment arm. Appropriate methods need to be employed to ensure comparability of concurrent and historical controls to minimize bias and variability in the treatment effect estimates and reduce the chances of drawing incorrect conclusions regarding treatment benefit. METHODS MILES was a phase III placebo-controlled trial employing 1:1 randomization that led to US Food and Drug Administration approval of sirolimus for treating patients with lymphangioleiomyomatosis. We re-analyzed the MILES trial data to learn whether substituting concurrent controls with controls from a historical registry could have accelerated subject enrollment while leading to similar study conclusions. We used propensity score matching to identify exchangeable historical controls from a registry balancing the baseline characteristics of the two control groups. This allowed more new patients to be assigned to the sirolimus arm. We used trial data and simulations to estimate key outcomes under an array of alternative designs. RESULTS Borrowing information from historical controls would have allowed the trial to enroll fewer concurrent controls while leading to the same conclusion reached in the trial. Simulations showed similar statistical performance for borrowing as for the actual trial design without producing type I error inflation and preserving power for the same study size when concurrent and historical controls are comparable. CONCLUSION Substituting concurrent controls with propensity score-matched historical controls can allow more prospectively enrolled patients to be assigned to the active treatment and enable the trial to be conducted with smaller overall sample size, while maintaining covariate balance and study power and minimizing bias in response estimation. This approach does not fully eliminate the concern that introducing non-randomized historical controls in a trial may lead to bias in estimating treatment effects, and should be carefully considered on a case-by-case basis. Borrowing historical controls is best suited when conducting randomized controlled trials with conventional designs is challenging, as in rare disease research. High-quality data on covariates and outcomes must be available for candidate historical controls to ensure the validity of these designs. Additional precautions are needed to maintain blinding of the treatment assignment and to ensure comparability in the assessment of treatment safety.MILES ClinicalTrials.gov Number: NCT00414648.
Collapse
Affiliation(s)
- Nusrat Harun
- Division of Biostatistics and Epidemiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio, USA
| | - Nishant Gupta
- Division of Pulmonary Critical Care and Sleep Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio, USA
| | - Francis X McCormack
- Division of Pulmonary Critical Care and Sleep Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio, USA
| | - Maurizio Macaluso
- Division of Biostatistics and Epidemiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio, USA
- Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, Ohio, USA
| |
Collapse
|
10
|
Alt EM, Nifong B, Chen X, Psioda MA, Ibrahim JG. The scale transformed power prior for use with historical data from a different outcome model. Stat Med 2023; 42:1-14. [PMID: 36318875 PMCID: PMC9789178 DOI: 10.1002/sim.9598] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2021] [Revised: 08/26/2022] [Accepted: 10/06/2022] [Indexed: 11/05/2022]
Abstract
We develop the scale transformed power prior for settings where historical and current data involve different data types, such as binary and continuous data. This situation arises often in clinical trials, for example, when historical data involve binary responses and the current data involve some other type of continuous or discrete outcome. The power prior, proposed by Ibrahim and Chen, does not address the issue of different data types. Herein, we develop a new type of power prior, which we call the scale transformed power prior (straPP). The straPP is constructed by transforming the power prior for the historical data by rescaling the parameter using a function of the Fisher information matrices for the historical and current data models, thereby shifting the scale of the parameter vector from that of the historical to that of the current data. Examples are presented to motivate the need for such a transformation, and simulation studies are presented to illustrate the performance advantages of the straPP over the power prior and other informative and noninformative priors. A real dataset from a clinical trial undertaken to study a novel transitional care model for stroke survivors is used to illustrate the methodology.
Collapse
Affiliation(s)
- Ethan M Alt
- Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
| | - Brady Nifong
- Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
| | - Xinxin Chen
- Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
| | - Matthew A Psioda
- Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
| | - Joseph G Ibrahim
- Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
| |
Collapse
|
11
|
Kwiatkowski E, Andraca-Carrera E, Soukup M, Psioda MA. A structured framework for adaptively incorporating external evidence in sequentially monitored clinical trials. J Biopharm Stat 2022; 32:474-495. [PMID: 35797378 DOI: 10.1080/10543406.2022.2078346] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/17/2022]
Abstract
We present a Bayesian framework for sequential monitoring that allows for use of external data, and that can be applied in a wide range of clinical trial applications. The basis for this framework is the idea that, in many cases, specification of priors used for sequential monitoring and the stopping criteria can be semi-algorithmic byproducts of the trial hypotheses and relevant external data, simplifying the process of prior elicitation. Monitoring priors are defined using the family of generalized normal distributions, which comprise a flexible class of priors, naturally allowing one to construct a prior that is peaked or flat about the parameter values thought to be most likely. External data are incorporated into the monitoring process through mixing an a priori skeptical prior with an enthusiastic prior using a weight that can be fixed or adaptively estimated. In particular, we introduce an adaptive monitoring prior for efficacy evaluation that dynamically weighs skeptical and enthusiastic prior components based on the degree to which observed data are consistent with an enthusiastic perspective. The proposed approach allows for prospective and pre-specified use of external data in the monitoring procedure. We illustrate the method for both single-arm and two-arm randomized controlled trials. For the latter case, we also include a retrospective analysis of actual trial data using the proposed adaptive sequential monitoring procedure. Both examples are motivated by completed pediatric trials, and the designs incorporate information from adult trials to varying degrees. Preposterior analysis and frequentist operating characteristics of each trial design are discussed.
Collapse
Affiliation(s)
- Evan Kwiatkowski
- Department of Biostatistics, University of North Carolina, Chapel Hill, North Carolina, USA
| | - Eugenio Andraca-Carrera
- Division of Biometrics VII, Office of Biostatistics, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, Maryland, USA
| | - Mat Soukup
- Division of Biometrics VII, Office of Biostatistics, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, Maryland, USA
| | - Matthew A Psioda
- Department of Biostatistics, University of North Carolina, Chapel Hill, North Carolina, USA
| |
Collapse
|
12
|
Zhang L, Wang Z, Wang L, Cui L, Sokolove J, Chan I. A Simple Approach to Incorporating Historical Control Data in Clinical Trial Design and Analysis. STATISTICS IN BIOSCIENCES 2022. [DOI: 10.1007/s12561-022-09342-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
13
|
Magaret AS, Warden M, Simon N, Heltshe S, Retsch-Bogart GZ, Ramsey BW, Mayer-Hamblett N. A new path for CF clinical trials through the use of historical controls. J Cyst Fibros 2022; 21:293-299. [PMID: 34879997 PMCID: PMC8957493 DOI: 10.1016/j.jcf.2021.11.007] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2021] [Revised: 09/24/2021] [Accepted: 11/15/2021] [Indexed: 11/26/2022]
Abstract
BACKGROUND Given future challenges in conducting large randomized, placebo controlled trials for future CF therapeutics development, we evaluated the potential for using external historical controls to either enrich or replace traditional concurrent placebo groups in CF trials. METHODS The study included data from sequentially completed, randomized, controlled clinical trials, EPIC and OPTIMIZE respectively, evaluating optimal antibiotic therapy to reduce the risk of pulmonary exacerbation in children with early Pseudomonas aeruginosa infection. The primary treatment effect in OPTIMIZE, the risk of pulmonary exacerbation associated with azithromycin, was re-estimated in alternative designs incorporating varying numbers of participants from the earlier trial (EPIC) as historical controls. Bias and precision of these estimates were characterized. Propensity scores were derived to adjust for baseline differences across study populations, and both Poisson and Cox regression were used to estimate treatment efficacy. RESULTS Replacing 86 OPTIMIZE placebo participants with 304 controls from EPIC to mimic a fully historically controlled trial resulted an 8% reduction in risk of pulmonary exacerbations (Hazard ratio (HR):0.92 95% CI 0.61, 1.34) when not adjusting for key baseline differences between study populations. After adjustment, a 37% decrease in risk of exacerbation (HR:0.63, 95% CI 0.50, 0.80) was estimated, comparable to the estimate from the original trial comparing the 86 placebo participants to 77 azithromycin participants on azithromycin (45%, HR:0.55, 95% CI: 0.34, 0.86). Other adjusted approaches provided similar estimates for the efficacy of azithromycin in reducing exacerbation risk: pooling all controls from both studies provided a HR of 0.60 (95% x`CI 0.46, 0.77) and augmenting half the OPTIMIZE placebo participants with EPIC controls gave a HR 0.63 (95% CI 0.48, 0.82). CONCLUSIONS The potential exists for future CF trials to utilize historical control data. Careful consideration of both the comparability of controls and of optimal methods can reduce the potential for biased estimation of treatment effects.
Collapse
Affiliation(s)
- Amalia S. Magaret
- Cystic Fibrosis Therapeutics Development Network Coordinating Center, Seattle Children’s Hospital, Seattle, WA, USA,Department of Pediatrics, University of Washington, Seattle, WA, USA,Department of Biostatistics, University of Washington, Seattle, WA, USA
| | - Mark Warden
- Department of Epidemiology, University of Washington, Seattle, WA, USA
| | - Noah Simon
- Cystic Fibrosis Therapeutics Development Network Coordinating Center, Seattle Children’s Hospital, Seattle, WA, USA,Department of Biostatistics, University of Washington, Seattle, WA, USA
| | - Sonya Heltshe
- Cystic Fibrosis Therapeutics Development Network Coordinating Center, Seattle Children’s Hospital, Seattle, WA, USA,Department of Pediatrics, University of Washington, Seattle, WA, USA
| | | | - Bonnie W. Ramsey
- Cystic Fibrosis Therapeutics Development Network Coordinating Center, Seattle Children’s Hospital, Seattle, WA, USA,Department of Pediatrics, University of Washington, Seattle, WA, USA
| | - Nicole Mayer-Hamblett
- Cystic Fibrosis Therapeutics Development Network Coordinating Center, Seattle Children’s Hospital, Seattle, WA, USA,Department of Pediatrics, University of Washington, Seattle, WA, USA,Department of Biostatistics, University of Washington, Seattle, WA, USA
| |
Collapse
|
14
|
Wang X, Suttner L, Jemielita T, Li X. Propensity score-integrated Bayesian prior approaches for augmented control designs: a simulation study. J Biopharm Stat 2021; 32:170-190. [PMID: 34939894 DOI: 10.1080/10543406.2021.2011743] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Abstract
Drug development can be costly, and the availability of clinical trial participants may be limited either due to the disease setting (rare or pediatric diseases) or due to many sponsors evaluating multiple drugs or combinations in the same patient population. To maximize resource utilization, sponsors may leverage patient-level control data from historical trials. However, in a study with no control arm, it is impossible to evaluate if the historical controls are an appropriate comparator for the current study. Here, instead of conducting a single-arm trial and relying solely on historical controls, we evaluate the situation where a minimal number of patients are enrolled into a control arm, which is augmented by borrowing historical control data. Propensity score (PS) methods are commonly used to minimize bias for non-randomized data. In addition, Bayesian information borrowing with PS adjustments has been proposed when it may not be reasonable to include all available historical data. This paper proposes using PS adjustment integrated with Bayesian commensurate priors to adaptively borrow information. We then evaluate the performance of different PS adjustment methods and different Bayesian priors for augmented control using simulation studies to help inform the design of future trials. In general, we find that propensity weighting or matching combined with the commensurate prior yield reasonable statistical properties across a range of scenarios. Finally, our proposed methods are applied to a real trial with a binary outcome.Abbreviations: PS: propensity score; IPTW: inverse probability of treatment weighting; ATT: average treatment effect on those who received treatment; RCT: randomized controlled trial; CDD: covariate distribution difference; ESS: effective sample size.
Collapse
Affiliation(s)
- Xi Wang
- Department of Public Health Sciences, College of Medicine, The Pennsylvania State University, Hershey, Pennsylvania, USA
| | | | | | | |
Collapse
|
15
|
Eggleston BS, Ibrahim JG, McNeil B, Catellier D. BayesCTDesign: An R Package for Bayesian Trial Design Using Historical Control Data. J Stat Softw 2021; 100:10.18637/jss.v100.i21. [PMID: 34975350 PMCID: PMC8715862 DOI: 10.18637/jss.v100.i21] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2022] Open
Abstract
This article introduces the R (R Core Team 2019) package BayesCTDesign for two-arm randomized Bayesian trial design using historical control data when available, and simple two-arm randomized Bayesian trial design when historical control data is not available. The package BayesCTDesign, which is available on CRAN, has two simulation functions, historic_sim() and simple_sim() for studying trial characteristics under user defined scenarios, and two methods print() and plot() for displaying summaries of the simulated trial characteristics. The package BayesCTDesign works with two-arm trials with equal sample sizes per arm. The package BayesCTDesign allows a user to study Gaussian, Poisson, Bernoulli, Weibull, Lognormal, and Piecewise Exponential (pwe) outcomes. Power for two-sided hypothesis tests at a user defined alpha is estimated via simulation using a test within each simulation replication that involves comparing a 95% credible interval for the outcome specific treatment effect measure to the null case value. If the 95% credible interval excludes the null case value, then the null hypothesis is rejected, else the null hypothesis is accepted. In the article, the idea of including historical control data in a Bayesian analysis is reviewed, the estimation process of BayesCTDesign is explained, and the user interface is described. Finally, the BayesCTDesign is illustrated via several examples.
Collapse
|
16
|
Walley RJ, Grieve AP. Optimising the trade-off between type I and II error rates in the Bayesian context. Pharm Stat 2021; 20:710-720. [PMID: 33619884 DOI: 10.1002/pst.2102] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2020] [Revised: 01/28/2021] [Accepted: 01/29/2021] [Indexed: 12/13/2022]
Abstract
For any decision-making study, there are two sorts of errors that can be made, declaring a positive result when the truth is negative, and declaring a negative result when the truth is positive. Traditionally, the primary analysis of a study is a two-sided hypothesis test, the type I error rate will be set to 5% and the study is designed to give suitably low type II error - typically 10 or 20% - to detect a given effect size. These values are standard, arbitrary and, other than the choice between 10 and 20%, do not reflect the context of the study, such as the relative costs of making type I and II errors and the prior belief the drug will be placebo-like. Several authors have challenged this paradigm, typically for the scenario where the planned analysis is frequentist. When resource is limited, there will always be a trade-off between the type I and II error rates, and this article explores optimising this trade-off for a study with a planned Bayesian statistical analysis. This work provides a scientific basis for a discussion between stakeholders as to what type I and II error rates may be appropriate and some algebraic results for normally distributed data.
Collapse
|
17
|
Psioda MA, Xue X. A BAYESIAN ADAPTIVE TWO-STAGE DESIGN FOR PEDIATRIC CLINICAL TRIALS. J Biopharm Stat 2020; 30:1091-1108. [DOI: 10.1080/10543406.2020.1821704] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/30/2023]
Affiliation(s)
- Matthew A. Psioda
- Department of Biostatistics, University of North Carolina, Chapel Hill, NC, USA
| | - Xiaoqiang Xue
- Center for Statistics of Drug Development, Data Science Safety and Regulatory, IQVIA Inc., Durham, NC, USA
| |
Collapse
|
18
|
Hall KT, Vase L, Tobias DK, Dashti HT, Vollert J, Kaptchuk TJ, Cook NR. Historical Controls in Randomized Clinical Trials: Opportunities and Challenges. Clin Pharmacol Ther 2020; 109:343-351. [PMID: 32602555 DOI: 10.1002/cpt.1970] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2020] [Accepted: 06/11/2020] [Indexed: 12/19/2022]
Abstract
Randomized control trials (RCTs) with placebo are the gold standard for determining efficacy of novel pharmaceutical treatments. Since their inception, over 75 years ago, researchers have amassed a large body of underutilized data on outcomes in the placebo control arms of these trials. Although rare disease indications have used these historical placebo data as synthetic controls to reduce burden on patients and accelerate drug discovery, broad use of historical controls is in its infancy. Large-scale historical placebo data could be leveraged to benefit both drug developers and patients if warehoused and made more available to guide trial design and analysis. Here, we examine challenges in utilizing historical controls related to heterogeneity in trial design, outcome ascertainment, patient characteristics, and unmeasured pharmacogenomic effects. We then discuss the advantages and disadvantages of current approaches and propose a path forward to broader use of historical controls in RCTs.
Collapse
Affiliation(s)
- Kathryn T Hall
- Division of Preventive Medicine, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, USA.,Harvard Medical School, Boston, Massachusetts, USA
| | - Lene Vase
- Department of Psychology and Behavioral Sciences, School of Business and Social Sciences, Aarhus University, Aarhus, Denmark
| | - Deirdre K Tobias
- Division of Preventive Medicine, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, USA.,Harvard Medical School, Boston, Massachusetts, USA
| | - Hesam T Dashti
- Division of Preventive Medicine, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, USA.,Harvard Medical School, Boston, Massachusetts, USA
| | - Jan Vollert
- Pain Research, Department of Surgery and Cancer, Faculty of Medicine, Imperial College London, London, UK.,Neurophysiology, Centre for Biomedicine and Medical Technology Mannheim, Medical Faculty Mannheim, Ruprecht-Karls-University, Heidelberg, Germany
| | - Ted J Kaptchuk
- Harvard Medical School, Boston, Massachusetts, USA.,Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
| | - Nancy R Cook
- Division of Preventive Medicine, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, USA.,Harvard Medical School, Boston, Massachusetts, USA
| |
Collapse
|
19
|
Abstract
Objective: This study describes a new automated strategy to determine the detection status of an electrophysiological response.Design: Response, noise and signal-to-noise ratio of the cortical auditory evoked potential (CAEP) were characterised. Detection rules were defined: when to start testing, when to conduct subsequent statistical tests using residual noise as an objective criterion, and when to stop testing.Study sample: Simulations were run to determine optimal parameters on a large combined CAEP data set collected in 45 normal-hearing adults and 17 adults with hearing loss.Results: The proposed strategy to detect CAEPs is fully automated. The first statistical test is conducted when the residual noise level is equal to or smaller than 5.1 µV. The succeeding Hotelling's T2 statistical tests are conducted using pre-defined residual noise levels criteria ranging from 5.1 to 1.2 µV. A rule was introduced allowing to stop testing before the maximum number of recorded epochs is reached, depending on a minimum p-value criterion.Conclusion: The proposed framework can be applied to systems which involves detection of electrophysiological responses in biological systems containing background noise. The proposed detection algorithm which optimise sensitivity, specificity, and recording time has the potential to be used in clinical setting.
Collapse
Affiliation(s)
- Fabrice Bardy
- HEARing Co-operative Research Centre, Australia.,University of Auckland, New Zealand
| | - Bram Van Dun
- HEARing Co-operative Research Centre, Australia.,National Acoustic Laboratories, NSW, Australia
| | - Mark Seeto
- HEARing Co-operative Research Centre, Australia.,National Acoustic Laboratories, NSW, Australia
| | - Harvey Dillon
- HEARing Co-operative Research Centre, Australia.,Macquarie University, NSW, Australia.,University of Manchester, United Kingdom
| |
Collapse
|
20
|
Schmidli H, Häring DA, Thomas M, Cassidy A, Weber S, Bretz F. Beyond Randomized Clinical Trials: Use of External Controls. Clin Pharmacol Ther 2019; 107:806-816. [DOI: 10.1002/cpt.1723] [Citation(s) in RCA: 37] [Impact Index Per Article: 7.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2019] [Accepted: 11/07/2019] [Indexed: 12/30/2022]
|
21
|
Ollier A, Morita S, Ursino M, Zohar S. An adaptive power prior for sequential clinical trials - Application to bridging studies. Stat Methods Med Res 2019; 29:2282-2294. [PMID: 31729275 PMCID: PMC7433690 DOI: 10.1177/0962280219886609] [Citation(s) in RCA: 21] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
During drug evaluation trials, information from clinical trials previously conducted on another population, indications or schedules may be available. In these cases, it might be desirable to share information by efficiently using the available resources. In this work, we developed an adaptive power prior with a commensurability parameter for using historical or external information. It allows, at each stage, full borrowing when the data are not in conflict, no borrowing when the data are in conflict or "tuned" borrowing when the data are in between. We propose to apply our adaptive power prior method to bridging studies between Caucasians and Asians, and we focus on the sequential adaptive allocation design, although other design settings can be used. We weight the prior information in two steps: the effective sample size approach is used to set the maximum desirable amount of information to be shared from historical data at each step of the trial; then, in a sort of Empirical Bayes approach, a commensurability parameter is chosen using a measure of distribution distance. This approach avoids elicitation and computational issues regarding the usual Empirical Bayes approach. We propose several versions of our method, and we conducted an extensive simulation study evaluating the robustness and sensitivity to prior choices.
Collapse
Affiliation(s)
- Adrien Ollier
- Centre de Recherche des Cordeliers, INSERM, Sorbonne Université, USPC, Université de Paris, Paris, France
| | - Satoshi Morita
- Department of Biomedical Statistics and Bioinformatics, Kyoto University Graduate School of Medicine, Kyoto, Japan
| | - Moreno Ursino
- Centre de Recherche des Cordeliers, INSERM, Sorbonne Université, USPC, Université de Paris, Paris, France
| | - Sarah Zohar
- Centre de Recherche des Cordeliers, INSERM, Sorbonne Université, USPC, Université de Paris, Paris, France
| |
Collapse
|