1
|
Burman CF, Hermansson E, Bock D, Franzén S, Svensson D. Digital twins and Bayesian dynamic borrowing: Two recent approaches for incorporating historical control data. Pharm Stat 2024. [PMID: 38439136 DOI: 10.1002/pst.2376] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2023] [Revised: 01/29/2024] [Accepted: 02/20/2024] [Indexed: 03/06/2024]
Abstract
Recent years have seen an increasing interest in incorporating external control data for designing and evaluating randomized clinical trials (RCT). This may decrease costs and shorten inclusion times by reducing sample sizes. For small populations, with limited recruitment, this can be especially important. Bayesian dynamic borrowing (BDB) has been a popular choice as it claims to protect against potential prior data conflict. Digital twins (DT) has recently been proposed as another method to utilize historical data. DT, also known as PROCOVA™, is based on constructing a prognostic score from historical control data, typically using machine learning. This score is included in a pre-specified ANCOVA as the primary analysis of the RCT. The promise of this idea is power increase while guaranteeing strong type 1 error control. In this paper, we apply analytic derivations and simulations to analyze and discuss examples of these two approaches. We conclude that BDB and DT, although similar in scope, have fundamental differences which need be considered in the specific application. The inflation of the type 1 error is a serious issue for BDB, while more evidence is needed of a tangible value of DT for real RCTs.
Collapse
Affiliation(s)
- Carl-Fredrik Burman
- Early Biometrics & Statistical Innovation, Data Science & Artificial Intelligence, R&D, AstraZeneca, Gothenburg, Sweden
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden
| | - Erik Hermansson
- Early Biometrics & Statistical Innovation, Data Science & Artificial Intelligence, R&D, AstraZeneca, Gothenburg, Sweden
| | - David Bock
- Early Biometrics & Statistical Innovation, Data Science & Artificial Intelligence, R&D, AstraZeneca, Gothenburg, Sweden
| | - Stefan Franzén
- BMP Evidence Statistics, BioPharmaceuticals Medical, AstraZeneca, Gothenburg, Sweden
| | - David Svensson
- Early Biometrics & Statistical Innovation, Data Science & Artificial Intelligence, R&D, AstraZeneca, Gothenburg, Sweden
| |
Collapse
|
2
|
Pepić A, Stark M, Friede T, Kopp-Schneider A, Calderazzo S, Reichert M, Wolf M, Wirth U, Schopf S, Zapf A. A diagnostic phase III/IV seamless design to investigate the diagnostic accuracy and clinical effectiveness using the example of HEDOS and HEDOS II. Stat Methods Med Res 2024; 33:433-448. [PMID: 38327081 PMCID: PMC10981198 DOI: 10.1177/09622802241227951] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/09/2024]
Abstract
The development process of medical devices can be streamlined by combining different study phases. Here, for a diagnostic medical device, we present the combination of confirmation of diagnostic accuracy (phase III) and evaluation of clinical effectiveness regarding patient-relevant endpoints (phase IV) using a seamless design. This approach is used in the Thyroid HEmorrhage DetectOr Study (HEDOS & HEDOS II) investigating a post-operative hemorrhage detector named ISAR-M THYRO® in patients after thyroid surgery. Data from the phase III trial are reused as external controls in the control group of the phase IV trial. An unblinded interim analysis is planned between the two study stages which includes a recalculation of the sample size for the phase IV part after completion of the first stage of the seamless design. The study concept presented here is the first seamless design proposed in the field of diagnostic studies. Hence, the aim of this work is to emphasize the statistical methodology as well as feasibility of the proposed design in relation to the planning and implementation of the seamless design. Seamless designs can accelerate the overall trial duration and increase its efficiency in terms of sample size and recruitment. However, careful planning addressing numerous methodological and procedural challenges is necessary for successful implementation as well as agreement with regulatory bodies.
Collapse
Affiliation(s)
- Amra Pepić
- Institute of Medical Biometry and Epidemiology, University Medical Center Hamburg-Eppendorf (UKE), Hamburg, Germany
| | - Maria Stark
- Institute of Medical Biometry and Epidemiology, University Medical Center Hamburg-Eppendorf (UKE), Hamburg, Germany
| | - Tim Friede
- Department of Medical Statistics, University Medical Center Göttingen, Göttingen, Germany
| | | | - Silvia Calderazzo
- Division of Biostatistics, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | | | - Michael Wolf
- CRI—The Clinical Research Institute, Munich, Germany
| | - Ulrich Wirth
- Clinic for General, Visceral and Transplant Surgery, Hospital of the Ludwig-Maximilians-University, Munich, Germany
| | - Stefan Schopf
- RoMed Klinik Bad Aibling, Academic University Hospital of the Technical University of Munich, Bad Aibling, Germany
| | - Antonia Zapf
- Institute of Medical Biometry and Epidemiology, University Medical Center Hamburg-Eppendorf (UKE), Hamburg, Germany
| |
Collapse
|
3
|
Silva P, Janjan N, Ramos KS, Udeani G, Zhong L, Ory MG, Smith ML. External control arms: COVID-19 reveals the merits of using real world evidence in real-time for clinical and public health investigations. Front Med (Lausanne) 2023; 10:1198088. [PMID: 37484840 PMCID: PMC10359981 DOI: 10.3389/fmed.2023.1198088] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Accepted: 05/31/2023] [Indexed: 07/25/2023] Open
Abstract
Randomized controlled trials are considered the 'gold standard' to reduce bias by randomizing patients to an experimental intervention, versus placebo or standard of care cohort. There are inherent challenges to enrolling a standard of care or cohorts: costs, site engagement logistics, socioeconomic variability, patient willingness, ethics of placebo interventions, cannibalizing the treatment arm population, and extending study duration. The COVID-19 pandemic has magnified aspects of constraints in trial recruitment and logistics, spurring innovative approaches to reducing trial sizes, accelerating trial accrual while preserving statistical rigor. Using data from medical records and databases allows for construction of external control arms that reduce the costs of an external control arm (ECA) randomized to standard of care. Simultaneously examining covariates of the clinical outcomes in ECAs that are being measured in the interventional arm can be particularly useful in phase 2 trials to better understand social and genetic determinants of clinical outcomes that might inform pivotal trial design. The FDA and EMA have promulgated a number of publicly available guidance documents and qualification reports that inform the use of this regulatory science tool to streamline clinical development, of phase 4 surveillance, and policy aspects of clinical outcomes research. Availability and quality of real-world data (RWD) are a prevalent impediment to the use of ECAs given such data is not collected with the rigor and deliberateness that characterizes prospective interventional control arm data. Conversely, in the case of contemporary control arms, a clinical trial outcome can be compared to a contemporary standard of care in cases where the standard of care is evolving at a fast pace, such as the use of checkpoint inhibitors in cancer care. Innovative statistical methods are an essential aspect of an ECA strategy and regulatory paths for these innovative approaches have been navigated, qualified, and in some cases published.
Collapse
Affiliation(s)
- Patrick Silva
- Institute of Bioscience and Technology and Department of Translational Medical Sciences, College Station, TX, United States
| | - Nora Janjan
- Center for Community Health and Aging, School of Public Health, Texas A&M University, College Station, TX, United States
| | - Kenneth S. Ramos
- Institute of Bioscience and Technology and Department of Translational Medical Sciences, College Station, TX, United States
| | - George Udeani
- Department of Clinical Pharmacy, School of Pharmacy, Texas A&M University, College Station, TX, United States
| | - Lixian Zhong
- Department of Pharmaceutical Sciences, School of Pharmacy, Texas A&M University, College Station, TX, United States
| | - Marcia G. Ory
- Center for Community Health and Aging, School of Public Health, Texas A&M University, College Station, TX, United States
| | - Matthew Lee Smith
- Center for Community Health and Aging, School of Public Health, Texas A&M University, College Station, TX, United States
| |
Collapse
|
4
|
Bofill Roig M, Burgwinkel C, Garczarek U, Koenig F, Posch M, Nguyen Q, Hees K. On the use of non-concurrent controls in platform trials: a scoping review. Trials 2023; 24:408. [PMID: 37322532 DOI: 10.1186/s13063-023-07398-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2022] [Accepted: 05/19/2023] [Indexed: 06/17/2023] Open
Abstract
BACKGROUND Platform trials gained popularity during the last few years as they increase flexibility compared to multi-arm trials by allowing new experimental arms entering when the trial already started. Using a shared control group in platform trials increases the trial efficiency compared to separate trials. Because of the later entry of some of the experimental treatment arms, the shared control group includes concurrent and non-concurrent control data. For a given experimental arm, non-concurrent controls refer to patients allocated to the control arm before the arm enters the trial, while concurrent controls refer to control patients that are randomised concurrently to the experimental arm. Using non-concurrent controls can result in bias in the estimate in case of time trends if the appropriate methodology is not used and the assumptions are not met. METHODS We conducted two reviews on the use of non-concurrent controls in platform trials: one on statistical methodology and one on regulatory guidance. We broadened our searches to the use of external and historical control data. We conducted our review on the statistical methodology in 43 articles identified through a systematic search in PubMed and performed a review on regulatory guidance on the use of non-concurrent controls in 37 guidelines published on the EMA and FDA websites. RESULTS Only 7/43 of the methodological articles and 4/37 guidelines focused on platform trials. With respect to the statistical methodology, in 28/43 articles, a Bayesian approach was used to incorporate external/non-concurrent controls while 7/43 used a frequentist approach and 8/43 considered both. The majority of the articles considered a method that downweights the non-concurrent control in favour of concurrent control data (34/43), using for instance meta-analytic or propensity score approaches, and 11/43 considered a modelling-based approach, using regression models to incorporate non-concurrent control data. In regulatory guidelines, the use of non-concurrent control data was considered critical but was deemed acceptable for rare diseases in 12/37 guidelines or was accepted in specific indications (12/37). Non-comparability (30/37) and bias (16/37) were raised most often as the general concerns with non-concurrent controls. Indication specific guidelines were found to be most instructive. CONCLUSIONS Statistical methods for incorporating non-concurrent controls are available in the literature, either by means of methods originally proposed for the incorporation of external controls or non-concurrent controls in platform trials. Methods mainly differ with respect to how the concurrent and non-concurrent data are combined and temporary changes handled. Regulatory guidance for non-concurrent controls in platform trials are currently still limited.
Collapse
Affiliation(s)
- Marta Bofill Roig
- Center for Medical Data Science, Medical University of Vienna, Vienna, Austria.
| | - Cora Burgwinkel
- Center for Medical Data Science, Medical University of Vienna, Vienna, Austria
- Department of Biostatistics, Paul-Ehrlich Institut, Langen, Germany
| | | | - Franz Koenig
- Center for Medical Data Science, Medical University of Vienna, Vienna, Austria
| | - Martin Posch
- Center for Medical Data Science, Medical University of Vienna, Vienna, Austria
| | - Quynh Nguyen
- Department of Biostatistics, Paul-Ehrlich Institut, Langen, Germany
| | - Katharina Hees
- Department of Biostatistics, Paul-Ehrlich Institut, Langen, Germany.
| |
Collapse
|
5
|
Senn S. Student and the Lanarkshire milk experiment. Eur J Epidemiol 2023; 38:1-10. [PMID: 36477576 PMCID: PMC9867657 DOI: 10.1007/s10654-022-00941-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2022] [Accepted: 11/02/2022] [Indexed: 12/12/2022]
Abstract
A detailed examination of the 1930 Lanarkshire Milk Experiment (LME) by the famous statistician William Sealy Gossett ("Student"), which appeared in Biometrika in 1931, is re-examined from a more modern perspective. The LME had a complicated design whereby 67 schools in Lanarkshire were allocated to receive either raw or pasteurised milk but pupils within the schools were allocated to either receive milk or to act as controls. Student's criticisms are considered in detail and examined in terms of subsequent developments on the design and analysis of experiments, in particular as regards appropriate estimation of standard errors of treatment estimates when an incomplete blocks structure has been used. An analogy with a more modern trial in osteoarthritis is made. Suggestions are made as to how analysis might proceed if the original data were available. Some lessons for observational studies in epidemiology are drawn and it is speculated that hidden clustering structures might be an explanation as to why results may vary from observational study to observational study by more than conventionally calculated standard errors might suggest.
Collapse
Affiliation(s)
- Stephen Senn
- School of Health and Related Research, University of Sheffield, Sheffield, UK.
| |
Collapse
|
6
|
Collignon O, Schiel A, Burman C, Rufibach K, Posch M, Bretz F. Estimands and Complex Innovative Designs. Clin Pharmacol Ther 2022; 112:1183-1190. [PMID: 35253205 PMCID: PMC9790227 DOI: 10.1002/cpt.2575] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2021] [Accepted: 03/01/2022] [Indexed: 01/31/2023]
Abstract
Since the release of the ICH E9(R1) (International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use Addendum on Estimands and Sensitivity Analysis in Clinical Trials to the Guideline on Statistical Principles for Clinical Trials) document in 2019, the estimand framework has become a fundamental part of clinical trial protocols. In parallel, complex innovative designs have gained increased popularity in drug development, in particular in early development phases or in difficult experimental situations. While the estimand framework is relevant to any study in which a treatment effect is estimated, experience is lacking as regards its application to these designs. In a basket trial for example, should a different estimand be specified for each subpopulation of interest, defined, for example, by cancer site? Or can a single estimand focusing on the general population (defined, for example, by the positivity to a certain biomarker) be used? In the case of platform trials, should a different estimand be proposed for each drug investigated? In this work we discuss possible ways of implementing the estimand framework for different types of complex innovative designs. We consider trials that allow adding or selecting experimental treatment arms, modifying the control arm or the standard of care, and selecting or pooling populations. We also address the potentially data-driven, adaptive selection of estimands in an ongoing trial and disentangle certain statistical issues that pertain to estimation rather than to estimands, such as the borrowing of nonconcurrent information. We hope this discussion will facilitate the implementation of the estimand framework and its description in the study protocol when the objectives of the trial require complex innovative designs.
Collapse
Affiliation(s)
| | | | - Carl‐Fredrik Burman
- Statistical Innovation, Data Science & Artificial IntelligenceAstraZeneca Research & DevelopmentGothenburgSweden
| | - Kaspar Rufibach
- Methods, Collaboration, and Outreach Group, Product Development Data SciencesF.Hoffmann‐La RocheBaselSwitzerland
| | - Martin Posch
- Section for Medical StatisticsCenter for Medical Statistics Informatics, and Intelligent SystemsMedical University of ViennaViennaAustria
| | - Frank Bretz
- Section for Medical StatisticsCenter for Medical Statistics Informatics, and Intelligent SystemsMedical University of ViennaViennaAustria,NovartisBaselSwitzerland
| |
Collapse
|
7
|
Senn S, Schmitz S, Schritz A, Araujo A. A note regarding alternative explanations for heterogeneity in meta‐analysis. Stat Med 2022; 41:4501-4509. [DOI: 10.1002/sim.9403] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2019] [Revised: 02/25/2022] [Accepted: 03/20/2022] [Indexed: 11/12/2022]
Affiliation(s)
- Stephen Senn
- Competence Center for Methodology and Statistics Luxembourg Institute of Health Strassen Luxembourg
- School of Health and Related Research, Medical Statistics Group The University of Sheffield Sheffield UK
| | - Susanne Schmitz
- Competence Center for Methodology and Statistics Luxembourg Institute of Health Strassen Luxembourg
| | - Anna Schritz
- Competence Center for Methodology and Statistics Luxembourg Institute of Health Strassen Luxembourg
| | - Artur Araujo
- School of Health and Related Research, Medical Statistics Group The University of Sheffield Sheffield UK
| |
Collapse
|
8
|
Collignon O, Schritz A, Spezia R, Senn SJ. Implementing Historical Controls in Oncology Trials. Oncologist 2021; 26:e859-e862. [PMID: 33523511 DOI: 10.1002/onco.13696] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2020] [Accepted: 12/15/2020] [Indexed: 11/06/2022] Open
Abstract
Drug development in oncology has broadened from mainly considering randomized clinical trials to also including single-arm trials tailored for very specific subtypes of cancer. They often use historical controls, and this article discusses benefits and risks of this paradigm and provide various regulatory and statistical considerations. While leveraging the information brought by historical controls could potentially shorten development time and reduce the number of patients enrolled, a careful selection of the past studies, a prespecified statistical analysis accounting for the heterogeneity between studies, and early engagement with regulators will be key to success. Although both the European Medicines Agency and the U.S. Food and Drug Administration have already approved medicines based on nonrandomized experiments, the evidentiary package can be perceived as less comprehensive than randomized experiments. Use of historical controls, therefore, is better suited for cases of high unmet clinical need, where the disease course is well characterized and the primary endpoint is objective. IMPLICATIONS FOR PRACTICE: Incorporating historical data in single-arm oncology trials has the potential to accelerate drug development and to reduce the number of patients enrolled, compared with standard randomized controlled clinical trials. Given the lack of blinding and randomization, such an approach is better suited for cases of high unmet clinical need and/or difficult experimental situations, in which the trajectory of the disease is well characterized and the endpoint can be measured objectively. Careful pre-specification and selection of the historical data, matching of the patient characteristics with the concurrent trial data, and innovative statistical methodologies accounting for between-study variation will be needed. Early engagement with regulators (e.g., via Scientific Advice) is highly recommended.
Collapse
Affiliation(s)
- Olivier Collignon
- Luxembourg Institute of Health, Competence Center in Methodology and Statistics, Strassen, Luxembourg.,GlaxoSmithKline, Stevenage, Hertfordshire, United Kingdom
| | - Anna Schritz
- Luxembourg Institute of Health, Competence Center in Methodology and Statistics, Strassen, Luxembourg
| | | | - Stephen J Senn
- Luxembourg Institute of Health, Competence Center in Methodology and Statistics, Strassen, Luxembourg.,Medical Statistics Group, School of Health and Related Research, University of Sheffield, Sheffield, United Kingdom
| |
Collapse
|