1
|
Lee KM, Emsley R. The impact of heterogeneity on the analysis of platform trials with normally distributed outcomes. BMC Med Res Methodol 2024; 24:163. [PMID: 39080538 PMCID: PMC11290279 DOI: 10.1186/s12874-024-02293-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2023] [Accepted: 07/19/2024] [Indexed: 08/02/2024] Open
Abstract
BACKGROUND A platform trial approach allows adding arms to on-going trials to speed up intervention discovery programs. A control arm remains open for recruitment in a platform trial while intervention arms may be added after the onset of the study and could be terminated early for efficacy and/or futility when early stopping is allowed. The topic of utilising non-concurrent control data in the analysis of platform trials has been explored and discussed extensively. A less familiar issue is the presence of heterogeneity, which may exist for example due to modification of enrolment criteria and recruitment strategy. METHOD We conduct a simulation study to explore the impact of heterogeneity on the analysis of a two-stage platform trial design. We consider heterogeneity in treatment effects and heteroscedasticity in outcome data across stages for a normally distributed endpoint. We examine the performance of some hypothesis testing procedures and modelling strategies. The use of non-concurrent control data is also considered accordingly. Alongside standard regression analysis, we examine the performance of a novel method that was known as the pairwise trials analysis. It is similar to a network meta-analysis approach but adjusts for treatment comparisons instead of individual studies using fixed effects. RESULTS Several testing strategies with concurrent control data seem to control the type I error rate at the required level when there is heteroscedasticity in outcome data across stages and/or a random cohort effect. The main parameter of treatment effects in some analysis models correspond to overall treatment effects weighted by stage wise sample sizes; while others correspond to the effect observed within a single stage. The characteristics of the estimates are not affected significantly by the presence of a random cohort effect and/ or heteroscedasticity. CONCLUSION In view of heterogeneity in treatment effect across stages, the specification of null hypotheses in platform trials may need to be more subtle. We suggest employing testing procedure of adaptive design as opposed to testing the statistics from regression models; comparing the estimates from the pairwise trials analysis method and the regression model with interaction terms may indicate if heterogeneity is negligible.
Collapse
Affiliation(s)
- Kim May Lee
- Department of Biostatistics and Health Informatics, Institute of Psychiatry, Psychology and Neuroscience, King's College London, 16 De Crespigny Park, SE5 8AF, London, UK.
- Department of Biostatistics and Health Informatics, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK.
| | - Richard Emsley
- Department of Biostatistics and Health Informatics, Institute of Psychiatry, Psychology and Neuroscience, King's College London, 16 De Crespigny Park, SE5 8AF, London, UK
- Department of Biostatistics and Health Informatics, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| |
Collapse
|
2
|
Meis J, Pilz M, Bokelmann B, Herrmann C, Rauch G, Kieser M. Point estimation, confidence intervals, and P-values for optimal adaptive two-stage designs with normal endpoints. Stat Med 2024; 43:1577-1603. [PMID: 38339872 DOI: 10.1002/sim.10020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2023] [Revised: 09/25/2023] [Accepted: 12/18/2023] [Indexed: 02/12/2024]
Abstract
Due to the dependency structure in the sampling process, adaptive trial designs create challenges in point and interval estimation and in the calculation of P-values. Optimal adaptive designs, which are designs where the parameters governing the adaptivity are chosen to maximize some performance criterion, suffer from the same problem. Various analysis methods which are able to handle this dependency structure have already been developed. In this work, we aim to give a comprehensive summary of these methods and show how they can be applied to the class of designs with planned adaptivity, of which optimal adaptive designs are an important member. The defining feature of these kinds of designs is that the adaptive elements are completely prespecified. This allows for explicit descriptions of the calculations involved, which makes it possible to evaluate different methods in a fast and accurate manner. We will explain how to do so, and present an extensive comparison of the performance characteristics of various estimators between an optimal adaptive design and its group-sequential counterpart.
Collapse
Affiliation(s)
- Jan Meis
- Institute of Medical Biometry, University of Heidelberg, Heidelberg, Germany
| | - Maximilian Pilz
- Institute of Medical Biometry, University of Heidelberg, Heidelberg, Germany
| | - Björn Bokelmann
- Institute of Biometry and Clinical Epidemiology, Charité- Universitätsmedizin Berlin, Corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Carolin Herrmann
- Institute of Biometry and Clinical Epidemiology, Charité- Universitätsmedizin Berlin, Corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Geraldine Rauch
- Institute of Biometry and Clinical Epidemiology, Charité- Universitätsmedizin Berlin, Corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
- Technische Universität Berlin, Berlin, Germany
| | - Meinhard Kieser
- Institute of Medical Biometry, University of Heidelberg, Heidelberg, Germany
| |
Collapse
|
3
|
Wang P, Chow SC. The use of real-world data for clinical investigation of effectiveness in drug development. J Biopharm Stat 2024:1-24. [PMID: 38519266 DOI: 10.1080/10543406.2024.2330215] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2024] [Accepted: 03/01/2024] [Indexed: 03/24/2024]
Abstract
With the growing interest in leveraging real-world data (RWD) to support effectiveness evaluations for new indications, new target populations, and post-market performance, the United States Food and Drug Administration has published several guidance documents on RWD sources and real-world studies (RWS) to assist sponsors in generating credible real-world evidence (RWE). Meanwhile, the randomized controlled trial (RCT) remains the gold standard in drug evaluation. Along this line, we propose a hybrid two-stage adaptive design to evaluate effectiveness based on evidence from both RCT and RWS. At the first stage, a typical non-inferiority test is conducted using RCT data to test for not-ineffectiveness. Once not-ineffectiveness is established, the study proceeds to the second stage to conduct an RWS and test for effectiveness using integrated information from RCT and RWD. The composite likelihood approach is implemented as a down-weighing strategy to account for the impact of high variability in RWS population. An optimal sample size determination procedure for RCT and RWS is introduced, aiming to achieve the minimal expected sample size. Through extensive numerical study, the proposed design demonstrates the ability to control type I error inflation in most cases and consistently maintain statistical power above the desired level. In general, this RCT/RWS hybrid two-stage adaptive design is beneficial for effectiveness evaluations in drug development, especially for oncology and rare diseases.
Collapse
Affiliation(s)
- Peijin Wang
- Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, North Carolina, USA
| | - Shein-Chung Chow
- Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, North Carolina, USA
| |
Collapse
|
4
|
Glimm E, Robertson DS. Familywise error rate control for block response-adaptive randomization. Stat Methods Med Res 2023; 32:1193-1202. [PMID: 37021480 PMCID: PMC10394397 DOI: 10.1177/09622802231167437] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/07/2023]
Abstract
Response-adaptive randomization allows the probabilities of allocating patients to treatments in a clinical trial to change based on the previously observed response data, in order to achieve different experimental goals. One concern over the use of such designs in practice, particularly from a regulatory viewpoint, is controlling the type I error rate. To address this, Robertson and Wason (Biometrics, 2019) proposed methodology that guarantees familywise error rate control for a large class of response-adaptive designs by re-weighting the usual z -test statistic. In this article, we propose an improvement of their method that is conceptually simpler, in the context where patients are allocated to the experimental treatment arms in a trial in blocks (i.e. groups) using response-adaptive randomization. We show the modified method guarantees that there will never be negative weights for the contribution of each block of data to the adjusted test statistics, and can also provide a substantial power advantage in practice.
Collapse
Affiliation(s)
- Ekkehard Glimm
- Novartis Pharma AG, Basel, Switzerland
- Medical Faculty, Institute of Biometry and Medical Informatics, University of Magdeburg, Magdeburg, Germany
| | | |
Collapse
|
5
|
Stallard N. Adaptive enrichment designs with a continuous biomarker. Biometrics 2023; 79:9-19. [PMID: 35174875 DOI: 10.1111/biom.13644] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2021] [Accepted: 09/23/2021] [Indexed: 12/01/2022]
Abstract
A popular design for clinical trials assessing targeted therapies is the two-stage adaptive enrichment design with recruitment in stage 2 limited to a biomarker-defined subgroup chosen based on data from stage 1. The data-dependent selection leads to statistical challenges if data from both stages are used to draw inference on treatment effects in the selected subgroup. If subgroups considered are nested, as when defined by a continuous biomarker, treatment effect estimates in different subgroups follow the same distribution as estimates in a group-sequential trial. This result is used to obtain tests controlling the familywise type I error rate (FWER) for six simple subgroup selection rules, one of which also controls the FWER for any selection rule. Two approaches are proposed: one based on multivariate normal distributions suitable if the number of possible subgroups, k, is small, and one based on Brownian motion approximations suitable for large k. The methods, applicable in the wide range of settings with asymptotically normal test statistics, are illustrated using survival data from a breast cancer trial.
Collapse
Affiliation(s)
- Nigel Stallard
- Statistics and Epidemiology, Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK
| |
Collapse
|
6
|
Xu H, Liu Y, Beckman RA. Adaptive Endpoints Selection with Application in Rare Disease. Stat Biopharm Res 2023. [DOI: 10.1080/19466315.2023.2183252] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/25/2023]
Affiliation(s)
- Heng Xu
- Nektar Therapeutics, San Francisco, USA
| | - Yi Liu
- Nektar Therapeutics, San Francisco, USA
| | - Robert A. Beckman
- Departments of Oncology and of Biostatistics, Bioinformatics, and Biomathematics, Lombardi Comprehensive Cancer Center and Innovation Center for Biomedical Informatics, Georgetown University Medical Center
| |
Collapse
|
7
|
Robertson DS, Choodari‐Oskooei B, Dimairo M, Flight L, Pallmann P, Jaki T. Point estimation for adaptive trial designs I: A methodological review. Stat Med 2023; 42:122-145. [PMID: 36451173 PMCID: PMC7613995 DOI: 10.1002/sim.9605] [Citation(s) in RCA: 13] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2021] [Revised: 10/21/2022] [Accepted: 11/01/2022] [Indexed: 12/02/2022]
Abstract
Recent FDA guidance on adaptive clinical trial designs defines bias as "a systematic tendency for the estimate of treatment effect to deviate from its true value," and states that it is desirable to obtain and report estimates of treatment effects that reduce or remove this bias. The conventional end-of-trial point estimates of the treatment effects are prone to bias in many adaptive designs, because they do not take into account the potential and realized trial adaptations. While much of the methodological developments on adaptive designs have tended to focus on control of type I error rates and power considerations, in contrast the question of biased estimation has received relatively less attention. This article is the first in a two-part series that studies the issue of potential bias in point estimation for adaptive trials. Part I provides a comprehensive review of the methods to remove or reduce the potential bias in point estimation of treatment effects for adaptive designs, while part II illustrates how to implement these in practice and proposes a set of guidelines for trial statisticians. The methods reviewed in this article can be broadly classified into unbiased and bias-reduced estimation, and we also provide a classification of estimators by the type of adaptive design. We compare the proposed methods, highlight available software and code, and discuss potential methodological gaps in the literature.
Collapse
Affiliation(s)
| | | | - Munya Dimairo
- School of Health and Related Research (ScHARR)University of SheffieldSheffieldUK
| | - Laura Flight
- School of Health and Related Research (ScHARR)University of SheffieldSheffieldUK
| | | | - Thomas Jaki
- MRC Biostatistics UnitUniversity of CambridgeCambridgeUK
- Faculty of Informatics and Data ScienceUniversity of RegensburgRegensburgGermany
| |
Collapse
|
8
|
Johnston SE, Lipkovich I, Dmitrienko A, Zhao YD. A two-stage adaptive clinical trial design with data-driven subgroup identification at interim analysis. Pharm Stat 2022; 21:1090-1108. [PMID: 35322520 PMCID: PMC10429034 DOI: 10.1002/pst.2208] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2021] [Revised: 02/14/2022] [Accepted: 03/05/2022] [Indexed: 11/08/2022]
Abstract
In this paper, we consider randomized controlled clinical trials comparing two treatments in efficacy assessment using a time to event outcome. We assume a relatively small number of candidate biomarkers available in the beginning of the trial, which may help define an efficacy subgroup which shows differential treatment effect. The efficacy subgroup is to be defined by one or two biomarkers and cut-offs that are unknown to the investigator and must be learned from the data. We propose a two-stage adaptive design with a pre-planned interim analysis and a final analysis. At the interim, several subgroup-finding algorithms are evaluated to search for a subgroup with enhanced survival for treated versus placebo. Conditional powers computed based on the subgroup and the overall population are used to make decision at the interim to terminate the study for futility, continue the study as planned, or conduct sample size recalculation for the subgroup or the overall population. At the final analysis, combination tests together with closed testing procedures are used to determine efficacy in the subgroup or the overall population. We conducted simulation studies to compare our proposed procedures with several subgroup-identification methods in terms of a novel utility function and several other measures. This research demonstrated the benefit of incorporating data-driven subgroup selection into adaptive clinical trial designs.
Collapse
Affiliation(s)
- Sarah E. Johnston
- Global Biostatistics and Data Science, Bristol Myers Squibb, Berkeley Heights, New Jersey, USA
| | | | | | - Yan Daniel Zhao
- Department of Biostatistics and Epidemiology, University of Oklahoma Health Sciences Center, Oklahoma City, Oklahoma, USA
| |
Collapse
|
9
|
Heindel W, Weigel S, Gerß J, Hense HW, Sommer A, Krischke M, Kerschke L. Digital breast tomosynthesis plus synthesised mammography versus digital screening mammography for the detection of invasive breast cancer (TOSYMA): a multicentre, open-label, randomised, controlled, superiority trial. Lancet Oncol 2022; 23:601-611. [PMID: 35427470 DOI: 10.1016/s1470-2045(22)00194-2] [Citation(s) in RCA: 26] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2022] [Revised: 03/11/2022] [Accepted: 03/18/2022] [Indexed: 12/22/2022]
Abstract
BACKGROUND Two dimensional (2D) full-field digital mammography is the current standard of breast cancer screening. Digital breast tomosynthesis generates pseudo-three dimensional datasets of the breast from which synthesised 2D (s2D) mammograms can be reconstructed. This innovative approach reduces the likelihood of overlapping breast tissues that can conceal features of malignancy. We aimed to compare digital breast tomosynthesis plus s2D mammography with digital screening mammography for the detection of invasive breast cancer. METHODS TOSYMA was a randomised, open-label, superiority trial done at 17 screening units in two federal states of Germany. Eligible participants were women aged 50-69 years who had been invited to participate in a population-wide, quality-controlled mammography screening programme. Women were randomly assigned (1:1) to digital breast tomosynthesis plus s2D mammography or digital mammography alone using block randomisation (block size of 32), stratified by site. The primary endpoints were the detection rate of invasive breast cancer and invasive interval cancer rate at 24 months, analysed in the modified full analysis set, which included all randomly assigned participants who underwent either type of screening examination. Ten examinations, corresponding to a second study participation, were excluded. Analyses were done according to the intention-to-treat principle. Interval cancer rates will be reported in the follow-up study. Safety was assessed in the as-treated population, which included all participants who were randomly assigned. This trial is registered with ClinicalTrials.gov, NCT03377036, and is closed to accrual. FINDINGS Between July 5, 2018, and Dec 30, 2020, 99 689 women were randomly assigned to digital breast tomosynthesis plus s2D mammography (n=49 804) or digital mammography (n=49 830). Invasive breast cancers were detected in 354 of 49 715 women with evaluable primary endpoint data in the digital breast tomosynthesis plus s2D group (detection rate 7·1 cases per 1000 women screened) and in 240 of 49 762 women in the digital mammography group (4·8 cases per 1000 women screened; odds ratio 1·48 [95% CI 1·25-1·75]; p<0·0001). Adverse events and device deficiencies were rare (six adverse events in each group; 23 device deficiencies in the digital breast tomosynthesis plus s2D group vs five device deficiencies in the digital mammography group) and no serious adverse events were reported. INTERPRETATION The results from this study indicate that the detection rate for invasive breast cancer was significantly higher with digital breast tomosynthesis plus s2D mammography than digital mammography alone. Evaluation of interval cancer rates in the follow-up study will further help to investigate incremental long-term benefits of digital breast tomosynthesis screening. FUNDING Deutsche Forschungsgemeinschaft (German Research Foundation).
Collapse
Affiliation(s)
- Walter Heindel
- Clinic for Radiology and Reference Center for Mammography Münster, University of Münster and University Hospital Münster, Münster, Germany.
| | - Stefanie Weigel
- Clinic for Radiology and Reference Center for Mammography Münster, University of Münster and University Hospital Münster, Münster, Germany
| | - Joachim Gerß
- Institute of Biostatistics and Clinical Research, University of Münster and University Hospital Münster, Münster, Germany
| | - Hans-Werner Hense
- Institute of Epidemiology and Social Medicine, University of Münster and University Hospital Münster, Münster, Germany
| | - Alexander Sommer
- Clinic for Radiology and Reference Center for Mammography Münster, University of Münster and University Hospital Münster, Münster, Germany
| | - Miriam Krischke
- Centre for Clinical Trials Münster, University of Münster and University Hospital Münster, Münster, Germany
| | - Laura Kerschke
- Institute of Biostatistics and Clinical Research, University of Münster and University Hospital Münster, Münster, Germany
| |
Collapse
|
10
|
Ma S, McDermott MP. Adaptive dose-response studies to establish proof-of-concept in learning-phase clinical trials. Biom J 2021; 64:146-164. [PMID: 34605043 DOI: 10.1002/bimj.202100044] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2021] [Revised: 07/29/2021] [Accepted: 08/31/2021] [Indexed: 11/07/2022]
Abstract
In learning-phase clinical trials in drug development, adaptive designs can be efficient and highly informative when used appropriately. In this article, we extend the multiple comparison procedures with modeling techniques (MCP-Mod) procedure with generalized multiple contrast tests (GMCTs) to two-stage adaptive designs for establishing proof-of-concept. The results of an interim analysis of first-stage data are used to adapt the candidate dose-response models and the dosages studied in the second stage. GMCTs are used in both stages to obtain stage-wise p -values, which are then combined to determine an overall p -value. An alternative approach is also considered that combines the t -statistics across stages, employing the conditional rejection probability principle to preserve the Type I error probability. Simulation studies demonstrate that the adaptive designs are advantageous compared to the corresponding tests in a nonadaptive design if the selection of the candidate set of dose-response models is not well informed by evidence from preclinical and early-phase studies.
Collapse
Affiliation(s)
- Shiyang Ma
- Department of Biostatistics, Columbia University, New York, NY, USA
| | - Michael P McDermott
- Department of Biostatistics and Computational Biology, University of Rochester, Rochester, NY, USA
| |
Collapse
|
11
|
Zhan T, Cui L, Geng Z, Zhang L, Gu Y, Chan ISF. A practical response adaptive block randomization (RABR) design with analytic type I error protection. Stat Med 2021; 40:4947-4960. [PMID: 34111902 DOI: 10.1002/sim.9104] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2021] [Revised: 05/26/2021] [Accepted: 05/27/2021] [Indexed: 11/12/2022]
Abstract
Response adaptive randomization (RAR) is appealing from methodological, ethical, and pragmatic perspectives in the sense that subjects are more likely to be randomized to better performing treatment groups based on accumulating data. However, applications of RAR in confirmatory drug clinical trials with multiple active arms are limited largely due to its complexity, and lack of control of randomization ratios to different treatment groups. To address the aforementioned issues, we propose a Response Adaptive Block Randomization (RABR) design allowing arbitrarily prespecified randomization ratios for the control and high-performing groups to meet clinical trial objectives. We show the validity of the conventional unweighted test in RABR with a controlled type I error rate based on the weighted combination test for sample size adaptive design invoking no large sample approximation. The advantages of the proposed RABR in terms of robustly reaching target final sample size to meet regulatory requirements and increasing statistical power as compared with the popular Doubly Adaptive Biased Coin Design are demonstrated by statistical simulations and a practical clinical trial design example.
Collapse
Affiliation(s)
- Tianyu Zhan
- Data and Statistical Sciences, AbbVie Inc., North Chicago, Illinois, USA
| | - Lu Cui
- Statistical Sciences and Innovation, UCB Biosciences Inc., Raleigh, North Carolina, USA
| | - Ziqian Geng
- Data and Statistical Sciences, AbbVie Inc., North Chicago, Illinois, USA
| | - Lanju Zhang
- Biometrics, Vertex Pharmaceuticals, Boston, Massachusetts, USA
| | - Yihua Gu
- Data and Statistical Sciences, AbbVie Inc., North Chicago, Illinois, USA
| | - Ivan S F Chan
- Global Biometrics and Data Sciences, Bristol Myers Squibb, Berkeley Heights, New Jersey, USA
| |
Collapse
|
12
|
Lawrence J. Sample size adaptations and evaluation of pediatric study interpretability. Stat Med 2021; 40:4809-4814. [PMID: 34102704 DOI: 10.1002/sim.9096] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2020] [Revised: 04/21/2021] [Accepted: 05/27/2021] [Indexed: 11/11/2022]
Abstract
To extend marketing exclusivity, a drug manufacture is required by the Food and Drug Administration to conduct an interpretable study of approved drugs that may benefit pediatric patients. In return for conducting an interpretable study, the drug manufacturer is rewarded with an additional 6 months of marketing exclusivity. If the study design uses a fixed sample size and nothing is changed during the study, it is relatively easy to determine whether an interpretable study was conducted. However, if the sample size can be changed based on an unblinded interim analysis, it is difficult to answer the question of whether the study was interpretable. We investigate the optimal design from the manufacturers perspective and find the strategy for sample size adaptation that optimizes the utility including the chance of an interpretable study.
Collapse
Affiliation(s)
- John Lawrence
- Office of Biostatistics, US Food and Drug Administration, Silver Spring, Maryland, USA
| |
Collapse
|
13
|
Abstract
In Learn-As-you-GO (LAGO) adaptive studies, the intervention is a complex multicomponent package, and is adapted in stages during the study based on past outcome data. This design formalizes standard practice in public health intervention studies. An effective intervention package is sought, while minimizing intervention package cost. In LAGO study data, the interventions in later stages depend upon the outcomes in the previous stages, violating standard statistical theory. We develop an estimator for the intervention effects, and prove consistency and asymptotic normality using a novel coupling argument, ensuring the validity of the test for the hypothesis of no overall intervention effect. We develop a confidence set for the optimal intervention package and confidence bands for the success probabilities under alternative package compositions. We illustrate our methods in the BetterBirth Study, which aimed to improve maternal and neonatal outcomes among 157,689 births in Uttar Pradesh, India through a multicomponent intervention package.
Collapse
Affiliation(s)
- Daniel Nevo
- Department of Statistics and Operations Research, Tel Aviv University
| | - Judith J Lok
- Department of Mathematics and Statistics, Boston University
| | - Donna Spiegelman
- Department of Biostatistics and Center for Methods on Implementation and Prevention Science (CMIPS), Yale School of Public Health
| |
Collapse
|
14
|
Neuberger M, Kowalewski KF, Simon V, Wessels F, Siegel F, Worst TS, Westhoff N, von Hardenberg J, Kriegmair M, Michel MS, Honeck P, Nuhn P. Peritoneal flap for lymphocele prophylaxis following robotic-assisted laparoscopic radical prostatectomy with pelvic lymph node dissection: study protocol and trial update for the randomized controlled PELYCAN study. Trials 2021; 22:236. [PMID: 33781339 PMCID: PMC8008541 DOI: 10.1186/s13063-021-05168-x] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2020] [Accepted: 03/02/2021] [Indexed: 11/17/2022] Open
Abstract
Background Data from interventional studies suggest that a peritoneal flap after pelvic lymph node dissection (LND) during laparoscopic, robotic-assisted radical prostatectomy (RARP) may reduce the rate of symptomatic lymphoceles in transperitoneal approach. However, most of these studies are not conducted in a randomized controlled fashion, thus limiting their scientific value. A recent prospective, randomized, controlled trial (RCT) did not show superiority of a peritoneal flap while further trials are lacking. Therefore, the aim of the presented RCT will be to show that creating a peritoneal flap decreases the rate of symptomatic lymphoceles compared to the current standard procedure without creation of a flap. Methods/design PELYCAN is a parallel-group, patient- and assessor-blinded, phase III, adaptive randomized controlled superiority trial. Men with histologically confirmed prostate cancer who undergo transperitoneal RARP with pelvic LND will be randomly assigned in a 1:1 ratio to two groups—either with creating a peritoneal flap (PELYCAN) or without creating a peritoneal flap (control). Sample size calculation yielded a sample size of 300 with a planned interim analysis after 120 patients, which will be performed by an independent statistician. This provides a possibility for early stopping or sample size recalculation. Patients will be stratified for contributing factors for the development of postoperative lymphoceles. The primary outcome measure will be the rate of symptomatic lymphoceles in both groups within 6 months postoperatively. Patients and assessors will be blinded for the intervention until the end of the follow-up period of 6 months. The surgeon will be informed about the randomization result after performance of vesicourethral anastomosis. Secondary outcome measures include asymptomatic lymphoceles at the time of discharge and within 6 months of follow-up, postoperative complications, mortality, re-admission rate, and quality of life assessed by the EORTC QLQ-C30 questionnaire. Discussion The PELYCAN study is designed to assess whether the application of a peritoneal flap during RARP reduces the rate of symptomatic lymphoceles, as compared with the standard operation technique. In case of superiority of the intervention, this peritoneal flap may be suggested as a new standard of care. Trial registration German Clinical Trials Register DRKS00016794. Registered on 14 May 2019. Supplementary Information The online version contains supplementary material available at 10.1186/s13063-021-05168-x.
Collapse
Affiliation(s)
- M Neuberger
- Department of Urology and Urologic Surgery, University Medical Center Mannheim, Medical Faculty Mannheim, University of Heidelberg, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
| | - K F Kowalewski
- Department of Urology and Urologic Surgery, University Medical Center Mannheim, Medical Faculty Mannheim, University of Heidelberg, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
| | - V Simon
- Department of Urology and Urologic Surgery, University Medical Center Mannheim, Medical Faculty Mannheim, University of Heidelberg, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
| | - F Wessels
- Department of Urology and Urologic Surgery, University Medical Center Mannheim, Medical Faculty Mannheim, University of Heidelberg, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
| | - F Siegel
- Department of Urology and Urologic Surgery, University Medical Center Mannheim, Medical Faculty Mannheim, University of Heidelberg, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany.,Heinrich Lanz Centre for Digital Health, University Medical Center Mannheim, Medical Faculty Mannheim, University of Heidelberg, Mannheim, Germany
| | - T S Worst
- Department of Urology and Urologic Surgery, University Medical Center Mannheim, Medical Faculty Mannheim, University of Heidelberg, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
| | - N Westhoff
- Department of Urology and Urologic Surgery, University Medical Center Mannheim, Medical Faculty Mannheim, University of Heidelberg, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
| | - J von Hardenberg
- Department of Urology and Urologic Surgery, University Medical Center Mannheim, Medical Faculty Mannheim, University of Heidelberg, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
| | - M Kriegmair
- Department of Urology and Urologic Surgery, University Medical Center Mannheim, Medical Faculty Mannheim, University of Heidelberg, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
| | - M S Michel
- Department of Urology and Urologic Surgery, University Medical Center Mannheim, Medical Faculty Mannheim, University of Heidelberg, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
| | - P Honeck
- Department of Urology and Urologic Surgery, University Medical Center Mannheim, Medical Faculty Mannheim, University of Heidelberg, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
| | - P Nuhn
- Department of Urology and Urologic Surgery, University Medical Center Mannheim, Medical Faculty Mannheim, University of Heidelberg, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany.
| |
Collapse
|
15
|
Posch M, König F. Are p-values Useful to Judge the Evidence Against the Null Hypotheses in Complex Clinical Trials? A Comment on “The Role of p-values in Judging the Strength of Evidence and Realistic Replication Expectations”. Stat Biopharm Res 2020. [DOI: 10.1080/19466315.2020.1847182] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Affiliation(s)
- Martin Posch
- Center for Medical Statistics, Informatics, and Intelligent Systems, Medical University of Vienna, Vienna, Austria
| | - Franz König
- Center for Medical Statistics, Informatics, and Intelligent Systems, Medical University of Vienna, Vienna, Austria
| |
Collapse
|
16
|
Liu Y, Xu H. Sample size re-estimation for pivotal clinical trials. Contemp Clin Trials 2020; 102:106215. [PMID: 33217555 DOI: 10.1016/j.cct.2020.106215] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2020] [Revised: 10/13/2020] [Accepted: 11/10/2020] [Indexed: 10/22/2022]
Abstract
It is well known that if the hypothesis test is left unchanged, the Type I error rate may be inflated for sample size re-estimation (SSR) designs. To address this issue, three main approaches have been proposed in the literature: combination test, conditional error and conventional test with sample size increase in the allowable region (AR) only. These three seemingly different approaches are in fact connected. For each combination test, there is a corresponding conditional error function and AR. Designing adaptation rules in this AR with conventional test guarantees the Type I error rate control but at the same time always leads to smaller power comparing to the corresponding combination test (or conditional error) approach. In cases where conventional test is still preferable, step-wise type adaptation rules that do not fully reside in the AR can be alternatively considered. We believe controversies in the statistical community on the efficiency comparisons between group sequential (GS) and SSR design stem partially from the misalignment of performance metrics and conditional versus unconditional evaluations. We advocate summary metrics, such as median, variance or tail probabilities of the sample size in addition to expectation and personalizing efficiency definition for each trial sponsor. Conditional metrics by favorable, promising and unfavorable zones of the interim results provide additional insights and should always be incorporated into the decision-making process.
Collapse
Affiliation(s)
- Yi Liu
- Nektar Therapeutics, San Francisco, CA 94107, USA.
| | - Heng Xu
- Nektar Therapeutics, San Francisco, CA 94107, USA
| |
Collapse
|
17
|
Quan H, Luo X, Zhou T, Zhao PL. Seamless phase II/III/IIIb clinical trial designs with different endpoints for different phases. COMMUN STAT-THEOR M 2020. [DOI: 10.1080/03610926.2019.1618871] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Affiliation(s)
- Hui Quan
- Biostatistics and Programming, Sanofi, 55C-335A, Bridgewater, New Jersey, USA
| | - Xiaodong Luo
- Biostatistics and Programming, Sanofi, 55C-335A, Bridgewater, New Jersey, USA
| | - Tianyue Zhou
- Biostatistics and Programming, Sanofi, 55C-335A, Bridgewater, New Jersey, USA
| | - Peng-Liang Zhao
- Biostatistics and Programming, Sanofi, 55C-335A, Bridgewater, New Jersey, USA
| |
Collapse
|
18
|
Ellenberger D, Lausen B, Friede T. Exact change point detection with improved power in small-sample binomial sequences. Biom J 2020; 63:558-574. [PMID: 33151005 DOI: 10.1002/bimj.201900273] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2019] [Revised: 06/30/2020] [Accepted: 07/21/2020] [Indexed: 11/08/2022]
Abstract
To detect a change in the probability of a sequence of independent binomial random variables, a variety of asymptotic and exact testing procedures have been proposed. Whenever the sample size or the event rate is small, asymptotic approximations of maximally selected test statistics have been shown to be inaccurate. Although exact methods control the type I error rate, they can be overly conservative due to the discreteness of the test statistics in these situations. We extend approaches by Worsley and Halpern to develop a test that is less discrete to increase the power. Building on ideas from binary segmentation, the proposed test utilizes unused information in the binomial sequences to add a new ordering to test statistics that are of equal value. The exact distributions are derived under side conditions that arise in hypothetical segmentation steps and do not depend on the type of test statistic used (e.g., log likelihood ratio, cumulative sum, or Fisher's exact test). Using the proposed exact segmentation procedure, we construct a change point test and prove that it controls the type-I-error rate at any given nominal level. Furthermore, we prove that the new test is uniformly at least as powerful as Worsley's exact test. In a Monte Carlo simulation study, the gain in power can be remarkable, especially in scenarios with small sample size. Giving a clinical database example about pin site infections and an example assessing publication bias in neuropsychiatric drug research, we demonstrate the wide-ranging applicability of the test.
Collapse
Affiliation(s)
- David Ellenberger
- Department of Medical Statistics, University Medical Center Göttingen, Göttingen, Germany
| | - Berthold Lausen
- Department of Mathematical Sciences, University of Essex, Colchester, UK
| | - Tim Friede
- Department of Medical Statistics, University Medical Center Göttingen, Göttingen, Germany
| |
Collapse
|
19
|
Kunz CU, Jörgens S, Bretz F, Stallard N, Van Lancker K, Xi D, Zohar S, Gerlinger C, Friede T. Clinical Trials Impacted by the COVID-19 Pandemic: Adaptive Designs to the Rescue? Stat Biopharm Res 2020; 12:461-477. [PMID: 34191979 PMCID: PMC8011492 DOI: 10.1080/19466315.2020.1799857] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2020] [Revised: 07/17/2020] [Accepted: 07/18/2020] [Indexed: 01/09/2023]
Abstract
Very recently the new pathogen severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) was identified and the coronavirus disease 2019 (COVID-19) declared a pandemic by the World Health Organization. The pandemic has a number of consequences for ongoing clinical trials in non-COVID-19 conditions. Motivated by four current clinical trials in a variety of disease areas we illustrate the challenges faced by the pandemic and sketch out possible solutions including adaptive designs. Guidance is provided on (i) where blinded adaptations can help; (ii) how to achieve Type I error rate control, if required; (iii) how to deal with potential treatment effect heterogeneity; (iv) how to use early read-outs; and (v) how to use Bayesian techniques. In more detail approaches to resizing a trial affected by the pandemic are developed including considerations to stop a trial early, the use of group-sequential designs or sample size adjustment. All methods considered are implemented in a freely available R shiny app. Furthermore, regulatory and operational issues including the role of data monitoring committees are discussed.
Collapse
Affiliation(s)
| | | | - Frank Bretz
- Novartis Pharma AG, Basel, Switzerland
- Section for Medical Statistics, Medical University of Vienna, Vienna, Austria
| | - Nigel Stallard
- Division of Health Sciences, Warwick Medical School, The University of Warwick, Coventry, UK
| | - Kelly Van Lancker
- Department of Applied Mathematics, Computer Science and Statistics, Ghent University, Ghent, Belgium
| | - Dong Xi
- Novartis Pharmaceuticals, East Hanover, NJ
| | - Sarah Zohar
- INSERM, Centre de Recherche des Cordeliers, Sorbonne Université, Université de Paris, Paris, France
| | - Christoph Gerlinger
- Statistics and Data Insights, Bayer AG, Berlin, Germany
- Department of Gynecology, Obstetrics and Reproductive Medicine, University Medical School of Saarland, Homburg/Saar, Germany
| | - Tim Friede
- Department of Medical Statistics, University Medical Center Göttingen, Göttingen, Germany
- DZHK (German Center for Cardiovascular Research), Partner Site Göttingen, Göttingen, Germany
| |
Collapse
|
20
|
Friede T, Stallard N, Parsons N. Adaptive seamless clinical trials using early outcomes for treatment or subgroup selection: Methods, simulation model and their implementation in R. Biom J 2020; 62:1264-1283. [PMID: 32118317 PMCID: PMC8614126 DOI: 10.1002/bimj.201900020] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2019] [Revised: 01/10/2020] [Accepted: 01/12/2020] [Indexed: 11/12/2022]
Abstract
Adaptive seamless designs combine confirmatory testing, a domain of phase III trials, with features such as treatment or subgroup selection, typically associated with phase II trials. They promise to increase the efficiency of development programmes of new drugs, for example, in terms of sample size and/or development time. It is well acknowledged that adaptive designs are more involved from a logistical perspective and require more upfront planning, often in the form of extensive simulation studies, than conventional approaches. Here, we present a framework for adaptive treatment and subgroup selection using the same notation, which links the somewhat disparate literature on treatment selection on one side and on subgroup selection on the other. Furthermore, we introduce a flexible and efficient simulation model that serves both designs. As primary endpoints often take a long time to observe, interim analyses are frequently informed by early outcomes. Therefore, all methods presented accommodate interim analyses informed by either the primary outcome or an early outcome. The R package asd, previously developed to simulate designs with treatment selection, was extended to include subgroup selection (so‐called adaptive enrichment designs). Here, we describe the functionality of the R package asd and use it to present some worked‐up examples motivated by clinical trials in chronic obstructive pulmonary disease and oncology. The examples both illustrate various features of the R package and provide insights into the operating characteristics of adaptive seamless studies.
Collapse
Affiliation(s)
- Tim Friede
- Department of Medical StatisticsUniversity Medical Center GöttingenGöttingen Germany
| | - Nigel Stallard
- Division of Health SciencesWarwick Medical SchoolUniversity of WarwickCoventry UK
| | - Nicholas Parsons
- Division of Health SciencesWarwick Medical SchoolUniversity of WarwickCoventry UK
| |
Collapse
|
21
|
Xi D, Bretz F. Symmetric graphs for equally weighted tests, with application to the Hochberg procedure. Stat Med 2019; 38:5268-5282. [PMID: 31657025 DOI: 10.1002/sim.8375] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2019] [Revised: 05/02/2019] [Accepted: 09/02/2019] [Indexed: 01/21/2023]
Abstract
The graphical approach to multiple testing provides a convenient tool for designing, visualizing, and performing multiplicity adjustments in confirmatory clinical trials while controlling the familywise error rate. It assigns a set of weights to each intersection null hypothesis within the closed test framework. These weights form the basis for intersection tests using weighted individual p-values, such as the weighted Bonferroni test. In this paper, we extend the graphical approach to intersection tests that assume equal weights for the elementary null hypotheses associated with any intersection hypothesis, including the Hochberg procedure as well as omnibus tests such as Fisher's combination, O'Brien's, and F tests. More specifically, we introduce symmetric graphs that generate sets of equal weights so that the aforementioned tests can be applied with the graphical approach. In addition, we visualize the Hochberg and the truncated Hochberg procedures in serial and parallel gatekeeping settings using symmetric component graphs. We illustrate the method with two clinical trial examples.
Collapse
Affiliation(s)
- Dong Xi
- Statistical Methodology, Novartis Pharmaceuticals, East Hanover, New Jersey
| | - Frank Bretz
- Statistical Methodology, Novartis Pharma AG, Basel, Switzerland.,Section for Medical Statistics, Medical University of Vienna, Vienna, Austria
| |
Collapse
|
22
|
Zapf A, Stark M, Gerke O, Ehret C, Benda N, Bossuyt P, Deeks J, Reitsma J, Alonzo T, Friede T. Adaptive trial designs in diagnostic accuracy research. Stat Med 2019; 39:591-601. [PMID: 31773788 DOI: 10.1002/sim.8430] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2019] [Revised: 10/23/2019] [Accepted: 10/26/2019] [Indexed: 11/10/2022]
Abstract
The aim of diagnostic accuracy studies is to evaluate how accurately a diagnostic test can distinguish diseased from nondiseased individuals. Depending on the research question, different study designs and accuracy measures are appropriate. As the prior knowledge in the planning phase is often very limited, modifications of design aspects such as the sample size during the ongoing trial could increase the efficiency of diagnostic trials. In intervention studies, group sequential and adaptive designs are well established. Such designs are characterized by preplanned interim analyses, giving the opportunity to stop early for efficacy or futility or to modify elements of the study design. In contrast, in diagnostic accuracy studies, such flexible designs are less common, even if they are as important as for intervention studies. However, diagnostic accuracy studies have specific features, which may require adaptations of the statistical methods or may lead to specific advantages or limitations of sequential and adaptive designs. In this article, we summarize the current status of methodological research and applications of flexible designs in diagnostic accuracy research. Furthermore, we indicate and advocate future development of adaptive design methodology and their use in diagnostic accuracy trials from an interdisciplinary viewpoint. The term "interdisciplinary viewpoint" describes the collaboration of experts of the academic and nonacademic research.
Collapse
Affiliation(s)
- Antonia Zapf
- Department of Medical Biometry and Epidemiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Maria Stark
- Department of Medical Biometry and Epidemiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Oke Gerke
- Department of Nuclear Medicine, Odense University Hospital, Odense, Denmark
| | | | - Norbert Benda
- Federal Institute for Drugs and Medical Devices (BfArM), Bonn, Germany.,Department of Medical Statistics, University Medical Center Göttingen, Göttingen, Germany
| | - Patrick Bossuyt
- Department of Clinical Epidemiology and Biostatistics, University of Amsterdam, Amsterdam, The Netherlands
| | - Jon Deeks
- Institute of Applied Health Research, University of Birmingham, Birmingham, UK.,NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Trust and the University of Birmingham, Birmingham, UK
| | - Johannes Reitsma
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht & University Utrecht, Utrecht, The Netherlands
| | - Todd Alonzo
- Keck School of Medicine, University of Southern California, Los Angeles, California
| | - Tim Friede
- Department of Medical Statistics, University Medical Center Göttingen, Göttingen, Germany
| |
Collapse
|
23
|
Placzek M, Friede T. A conditional error function approach for adaptive enrichment designs with continuous endpoints. Stat Med 2019; 38:3105-3122. [PMID: 31066093 DOI: 10.1002/sim.8154] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2018] [Revised: 02/22/2019] [Accepted: 03/09/2019] [Indexed: 12/15/2022]
Abstract
Adaptive enrichment designs offer an efficient and flexible way to demonstrate the efficacy of a treatment in a clinically defined full population or in, eg, biomarker-defined subpopulations while controlling the family-wise Type I error rate in the strong sense. Frequently used testing strategies in designs with two or more stages include the combination test and the conditional error function approach. Here, we focus on the latter and present some extensions. In contrast to previous work, we allow for multiple subgroups rather than one subgroup only. For nested as well as nonoverlapping subgroups with normally distributed endpoints, we explore the effect of estimating the variances in the subpopulations. Instead of using a normal approximation, we derive new t-distribution-based methods for two different scenarios. First, in the case of equal variances across the subpopulations, we present exact results using a multivariate t-distribution. Second, in the case of potentially varying variances across subgroups, we provide some improved approximations compared to the normal approximation. The performance of the proposed conditional error function approaches is assessed and compared to the combination test in a simulation study. The proposed methods are motivated by an example in pulmonary arterial hypertension.
Collapse
Affiliation(s)
- Marius Placzek
- Department of Medical Statistics, University Medical Center Göttingen, Göttingen, Germany
| | - Tim Friede
- Department of Medical Statistics, University Medical Center Göttingen, Göttingen, Germany.,DZHK (German Center for Cardiovascular Research), Partner Site Göttingen, Göttingen, Germany
| |
Collapse
|
24
|
Bayar MA, Le Teuff G, Koenig F, Le Deley MC, Michiels S. Group sequential adaptive designs in series of time-to-event randomised trials in rare diseases: A simulation study. Stat Methods Med Res 2019; 29:1483-1498. [PMID: 31354106 DOI: 10.1177/0962280219862313] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
In rare diseases, fully powered large trials may not be doable in a reasonable time frame even with international collaborations. In a previous work, we proposed an approach based on a series of smaller parallel group two-arm randomised controlled trials (RCT) performed over a long research horizon. Within the series of trials, the treatment selected after each trial becomes the control treatment of the next one. We concluded that running more trials with smaller sample sizes and relaxed α-levels leads in the long term and under reasonable assumptions to larger survival benefits with a moderate increase of risk as compared to traditional designs based on larger but fewer trials designed to meet stringent evidence criteria. We now extend this quantitative framework with more 'flexible' designs including interim analyses for futility and/or efficacy, and three-arm adaptive designs with treatment selection at interim. In the simulation study, we considered different disease severities, accrual rates, and hypotheses of how treatments improve over time. For each design, we estimated the long-term survival benefit as the relative difference in hazard rates between the end and the start of the research horizon, and the risk defined as the probability of selecting at the end of the research horizon a treatment inferior to the initial control. We assessed the impact of the α-level and the choice of the stopping rule on the operating characteristics. We also compared the performance of series based on two- vs. three-arm trials. We show that relaxing α-levels within the limit of 0.1 is associated with larger survival gains and moderate increase of risk which remains within acceptable ranges. Including an interim analysis with a futility rule is associated with an additional survival gain and a better risk control as compared to series with no interim analysis, when the α-level is below or equal to 0.1, whereas the benefit of including an interim analysis is rather small for higher α-levels. Including an interim analysis for efficacy yields almost no additional gain. Series based on three-arm trials are associated with a systematic improvement in terms of survival gain and risk control as compared to series of two-arm trials.
Collapse
Affiliation(s)
- Mohamed Amine Bayar
- Service de Biostatistique et d'Épidémiologie, Gustave Roussy, Villejuif, France.,CESP, Faculté de médecine - Université Paris-Sud, Faculté de médecine - INSERM, Université Paris-Saclay, Villejuif, France
| | - Gwénaël Le Teuff
- Service de Biostatistique et d'Épidémiologie, Gustave Roussy, Villejuif, France.,CESP, Faculté de médecine - Université Paris-Sud, Faculté de médecine - INSERM, Université Paris-Saclay, Villejuif, France
| | - Franz Koenig
- Center for Medical Statistics, Informatics and Intelligent Systems, Medical University of Vienna, Spitalgasse 23, 1090 Vienna, Austria
| | - Marie-Cécile Le Deley
- Service de Biostatistique et d'Épidémiologie, Gustave Roussy, Villejuif, France.,CESP, Faculté de médecine - Université Paris-Sud, Faculté de médecine - INSERM, Université Paris-Saclay, Villejuif, France.,Unité de Méthodologie et Biostatistique, Centre Oscar Lambret, Lille, France
| | - Stefan Michiels
- Service de Biostatistique et d'Épidémiologie, Gustave Roussy, Villejuif, France.,CESP, Faculté de médecine - Université Paris-Sud, Faculté de médecine - INSERM, Université Paris-Saclay, Villejuif, France
| |
Collapse
|
25
|
Abstract
Objective: To detect the auditory brainstem response (ABR) automatically using an innovative sequentially applied Hotelling's T 2 test, with the overall goal of optimising test time whilst controlling the false-positive rate (FPR). Design: The stage-wise critical decision boundaries for accepting or rejecting the null hypothesis were found using a new approach called the Convolutional Group Sequential Test (CGST). Specificity, sensitivity, and test time were evaluated using simulations and subject recorded data. Study sample: Data consists of click-evoked ABR threshold series from 12 normal hearing adults, and recordings of EEG background activity from 17 normal hearing adults. Results: Reductions in mean test time of up to 40-45% were observed for the sequential test, relative to a conventional "single shot" test where the statistical test is applied to the data just once. To obtain these results, it will occasionally be necessary to run the test to a higher number of stimuli, i.e. the maximum test time needs to be increased. Conclusions: The CGST can be used to control the specificity of a sequentially applied ABR detection method. Doing so can reduce test time, relative to the "single shot" test, when considered across a cohort of test subjects.
Collapse
Affiliation(s)
- M A Chesnaye
- Faculty of Engineering and the Environment, Institute of Sound and Vibration Research, University of Southampton , UK
| | - S L Bell
- Faculty of Engineering and the Environment, Institute of Sound and Vibration Research, University of Southampton , UK
| | - J M Harte
- Faculty of Engineering and the Environment, Institute of Sound and Vibration Research, University of Southampton , UK
| | - D M Simpson
- Interacoustics Research Unit, c/o Technical University of Denmark , Denmark
| |
Collapse
|
26
|
Chesnaye MA, Bell SL, Harte JM, Simpson DM. The Convolutional Group Sequential Test: Reducing Test Time for Evoked Potentials. IEEE Trans Biomed Eng 2019; 67:697-705. [PMID: 31150332 DOI: 10.1109/tbme.2019.2919696] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
When using a statistical test for automatically detecting evoked potentials, the number of stimuli presented to the subject (the sample size for the statistical test) should be specified at the outset. For evoked response detection, this may be inefficient, i.e., because the signal-to-noise ratio (SNR) of the response is not known in advance, the user would usually err on the cautious side and use a relatively high number of stimuli to ensure adequate statistical power. A more efficient approach is to apply the statistical test repeatedly to the accumulating data over time, as this allows the test to be stopped early for the high SNR responses (thus reducing test time), or later for the low SNR responses. The caveat is that the critical decision boundaries for rejecting the null hypothesis need to be adjusted if the intended type-I error rate is to be obtained. This study presents an intuitive and flexible method for controlling the type-I error rate for sequentially applied statistical tests. The method is built around the discrete convolution of truncated probability density functions, which allows the null distribution for the test statistic to be constructed at each stage of the sequential analysis. Because the null distribution remains tractable, the procedure for finding the stage-wise critical decision boundaries is greatly simplified. The method also permits data-driven adaptations (using data from previous stages) to both the sample size and the statistical test, which offers new opportunities to speed up testing for evoked response detection.
Collapse
|
27
|
Ding AA, Wu SS, Dean NE, Zahigian RS. Two-stage adaptive enrichment design for testing an active factor. J Biopharm Stat 2019; 30:18-30. [PMID: 31135263 DOI: 10.1080/10543406.2019.1609015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Abstract
We propose an adaptive enrichment approach to test an active factor, which is a factor whose effect is non-zero in at least one subpopulation. We implement a two-stage play-the-winner design where all subjects in the second stage are enrolled from the subpopulation that has the highest observed effect in the first stage. We recommend a weighted Fisher's combination of the most powerful test for each stage, respectively: the first stage Hotelling's test and the second stage noncentral chi-square test. The test is further extended to cover binary outcomes and time-to-event outcomes.
Collapse
Affiliation(s)
- A Adam Ding
- Department of Mathematics, Northeastern University, Boston, Massachusetts, USA
| | - Samuel S Wu
- Department of Biostatistics, University of Florida, Gainesville, Florida, USA
| | - Natalie E Dean
- Department of Biostatistics, University of Florida, Gainesville, Florida, USA
| | - Rachel S Zahigian
- Department of Biostatistics, University of Florida, Gainesville, Florida, USA
| |
Collapse
|
28
|
Lee KM, Wason J, Stallard N. To add or not to add a new treatment arm to a multiarm study: A decision-theoretic framework. Stat Med 2019; 38:3305-3321. [PMID: 31115078 PMCID: PMC6619445 DOI: 10.1002/sim.8194] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2018] [Revised: 03/05/2019] [Accepted: 04/17/2019] [Indexed: 12/16/2022]
Abstract
Multiarm clinical trials, which compare several experimental treatments against control, are frequently recommended due to their efficiency gain. In practise, all potential treatments may not be ready to be tested in a phase II/III trial at the same time. It has become appealing to allow new treatment arms to be added into on‐going clinical trials using a “platform” trial approach. To the best of our knowledge, many aspects of when to add arms to an existing trial have not been explored in the literature. Most works on adding arm(s) assume that a new arm is opened whenever a new treatment becomes available. This strategy may prolong the overall duration of a study or cause reduction in marginal power for each hypothesis if the adaptation is not well accommodated. Within a two‐stage trial setting, we propose a decision‐theoretic framework to investigate when to add or not to add a new treatment arm based on the observed stage one treatment responses. To account for different prospect of multiarm studies, we define utility in two different ways; one for a trial that aims to maximise the number of rejected hypotheses; the other for a trial that would declare a success when at least one hypothesis is rejected from the study. Our framework shows that it is not always optimal to add a new treatment arm to an existing trial. We illustrate a case study by considering a completed trial on knee osteoarthritis.
Collapse
Affiliation(s)
- Kim May Lee
- MRC Biostatistics Unit, University of Cambridge, Cambridge, UK
| | - James Wason
- MRC Biostatistics Unit, University of Cambridge, Cambridge, UK.,Institute of Health and Society, Newcastle University, Newcastle upon Tyne, UK
| | - Nigel Stallard
- WMS - Statistics and Epidemiology, University of Warwick, Coventry, UK
| |
Collapse
|
29
|
Sarkar SK, Chen A, He L, Guo W. Group sequential BH and its adaptive versions controlling the FDR. J Stat Plan Inference 2019. [DOI: 10.1016/j.jspi.2018.07.001] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
30
|
Glimm E, Bezuidenhoudt M, Caputo A, Maurer W. A Testing Strategy With Adaptive Dose Selection and Two Endpoints. Stat Biopharm Res 2018. [DOI: 10.1080/19466315.2018.1497531] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/28/2022]
Affiliation(s)
- Ekkehard Glimm
- Novartis Pharma AG, Basel, Switzerland
- Otto-von-Guericke-University Magdeburg, Germany
| | | | | | | |
Collapse
|
31
|
Brückner M, Burger HU, Brannath W. Nonparametric adaptive enrichment designs using categorical surrogate data. Stat Med 2018; 37:4507-4524. [PMID: 30191578 DOI: 10.1002/sim.7936] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2017] [Revised: 07/14/2018] [Accepted: 07/16/2018] [Indexed: 11/06/2022]
Abstract
Adaptive survival trials are particularly important for enrichment designs in oncology and other life-threatening diseases. Current statistical methodology for adaptive survival trials provide type I error rate control only under restrictions. For instance, if we use stage-wise P values based on increments of the log-rank test, then the information used for the interim decisions need to be restricted to the primary survival endpoint. However, it is often desirable to base interim decisions also on correlated short-term endpoints like tumor response. Alternative statistical approaches based on a patient-wise splitting of the data require unnatural restrictions on the follow-up times and do not permit to efficiently account for an early rejection of the primary null hypothesis. We therefore suggest new approaches that enable us to use discrete surrogate endpoints (like tumor response status) and also to incorporate interim rejection boundaries. The new approaches are based on weighted Kaplan-Meier estimates and thereby have additional advantages. They permit us to account for nonproportional hazards and are robust against informative censoring based on the surrogate endpoint. We will show that nonproportionality is an intrinsic and relevant issue in enrichment designs. Moreover, informative censoring based on the surrogate endpoint is likely because of withdrawals and treatment switches after insufficient treatment response. It is shown and illustrated how nonparametric tests based on weighted Kaplan-Meier estimates can be used in closed combination tests for adaptive enrichment designs, such that type I error rate control is achieved and justified asymptotically.
Collapse
Affiliation(s)
- Matthias Brückner
- Competence Center for Clinical Trials and Institute for Statistics, University of Bremen, Bremen, Germany
| | | | - Werner Brannath
- Competence Center for Clinical Trials and Institute for Statistics, University of Bremen, Bremen, Germany
| |
Collapse
|
32
|
Biomarker-Defined Subgroup Selection Adaptive Design for Phase III Confirmatory Trial with Time-to-Event Data: Comparing Group Sequential and Various Adaptive Enrichment Designs. STATISTICS IN BIOSCIENCES 2018. [DOI: 10.1007/s12561-017-9198-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
|
33
|
Abstract
Blinded sample size reassessment is a popular means to control the power in clinical trials if no reliable information on nuisance parameters is available in the planning phase. We investigate how sample size reassessment based on blinded interim data affects the properties of point estimates and confidence intervals for parallel group superiority trials comparing the means of a normal endpoint. We evaluate the properties of two standard reassessment rules that are based on the sample size formula of the z-test, derive the worst case reassessment rule that maximizes the absolute mean bias and obtain an upper bound for the mean bias of the treatment effect estimate.
Collapse
Affiliation(s)
- Martin Posch
- Section for Medical Statistics, Center
for Medical Statistics, Informatics, and Intelligent Systems, Medical University of
Vienna, Vienna, Austria
| | - Florian Klinglmueller
- Section for Medical Statistics, Center
for Medical Statistics, Informatics, and Intelligent Systems, Medical University of
Vienna, Vienna, Austria
- Department of Statistical Sciences,
University of Padua, Padua, Italy
| | - Franz König
- Section for Medical Statistics, Center
for Medical Statistics, Informatics, and Intelligent Systems, Medical University of
Vienna, Vienna, Austria
| | - Frank Miller
- Department of Statistics, Stockholm
University, Stockholm, Sweden Martin Posch and Florian Klinglmueller share first
authorship
| |
Collapse
|
34
|
Abery JE, Todd S. Comparing the MAMS framework with the combination method in multi-arm adaptive trials with binary outcomes. Stat Methods Med Res 2018; 28:1716-1730. [PMID: 29734869 DOI: 10.1177/0962280218773546] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
In multi-arm adaptive trials, several treatments are assessed simultaneously and accumulating data are used to inform decisions about the trial, such as whether treatments are dropped or continued. Different methodological approaches have been developed for such trials and research has compared the performance of different subsets of these. One particular approach, for which we use the acronym MAMS(R), has generally not been included in these comparisons because control of the family-wise error rate (FWER) could not be guaranteed. Recently, the MAMS(R) approach has been extended to facilitate the generation of efficient designs which strongly control the FWER. We consider multi-arm two-stage trials with binary outcomes and propose parameterising treatment effects using the log odds ratio. We conduct a simulation study comparing the extended MAMS(R) framework with the well-established combination method both for trials where a different outcome is used for mid-trial analysis and for trials where the same outcome is used throughout. We show how the MAMS(R) framework compares favourably only in scenarios where the same outcome is used. We propose a hybrid selection rule within MAMS(R) methodology and demonstrate that this makes it possible to use the MAMS(R) framework in trials incorporating comparative treatment selection.
Collapse
Affiliation(s)
- Julia E Abery
- Department of Mathematics and Statistics, University of Reading, Reading, UK
| | - Susan Todd
- Department of Mathematics and Statistics, University of Reading, Reading, UK
| |
Collapse
|
35
|
Maurer W, Jones B, Chen Y. Controlling the type I error rate in two-stage sequential adaptive designs when testing for average bioequivalence. Stat Med 2018; 37:1587-1607. [PMID: 29462835 DOI: 10.1002/sim.7614] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2016] [Revised: 10/04/2017] [Accepted: 01/01/2018] [Indexed: 11/09/2022]
Abstract
In a 2×2 crossover trial for establishing average bioequivalence (ABE) of a generic agent and a currently marketed drug, the recommended approach to hypothesis testing is the two one-sided test (TOST) procedure, which depends, among other things, on the estimated within-subject variability. The power of this procedure, and therefore the sample size required to achieve a minimum power, depends on having a good estimate of this variability. When there is uncertainty, it is advisable to plan the design in two stages, with an interim sample size reestimation after the first stage, using an interim estimate of the within-subject variability. One method and 3 variations of doing this were proposed by Potvin et al. Using simulation, the operating characteristics, including the empirical type I error rate, of the 4 variations (called Methods A, B, C, and D) were assessed by Potvin et al and Methods B and C were recommended. However, none of these 4 variations formally controls the type I error rate of falsely claiming ABE, even though the amount of inflation produced by Method C was considered acceptable. A major disadvantage of assessing type I error rate inflation using simulation is that unless all possible scenarios for the intended design and analysis are investigated, it is impossible to be sure that the type I error rate is controlled. Here, we propose an alternative, principled method of sample size reestimation that is guaranteed to control the type I error rate at any given significance level. This method uses a new version of the inverse-normal combination of p-values test, in conjunction with standard group sequential techniques, that is more robust to large deviations in initial assumptions regarding the variability of the pharmacokinetic endpoints. The sample size reestimation step is based on significance levels and power requirements that are conditional on the first-stage results. This necessitates a discussion and exploitation of the peculiar properties of the power curve of the TOST testing procedure. We illustrate our approach with an example based on a real ABE study and compare the operating characteristics of our proposed method with those of Method B of Povin et al.
Collapse
Affiliation(s)
- Willi Maurer
- Statistical Methodology and Consulting Center, Novartis Pharma AG, Basel, Switzerland
| | - Byron Jones
- Statistical Methodology and Consulting Center, Novartis Pharma AG, Basel, Switzerland
| | - Ying Chen
- Shanghai University of Finance and Economics, Shanghai, China
| |
Collapse
|
36
|
Burkhardt B, Faldum A, Schmidt R. Adaptive Designs with Discrete Test Statistics and Consideration of Overrunning. Methods Inf Med 2018; 54:434-46. [DOI: 10.3414/me14-02-0023] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2014] [Accepted: 09/09/2015] [Indexed: 01/08/2023]
Abstract
Summary
Background: Interim analyses are used in clinical trials in order to enable early decisions for medical, ethical, and economic reasons. However, it appears unfeasible to stop a trial during such an interim analysis. New patients will thus enter the trial while the interim analysis is ongoing. Moreover, depending on the event kinetics of the specific disease, the trial design, and the corresponding endpoints, some patients might still be unevaluable at the interim analysis due to not yet completed follow-up. Occur-rence of these types of patients is characteristic for sequentially analyzed trials. Such patients are referred to as interim patients. In trials with multiple primary endpoints, another type of interim patients occurs. If some but not all null hypotheses can be rejected at the interim analysis, the trial might be continued to a second stage in order to answer the remaining questions. These second stage patients, however, provide new data to all trial questions including the already rejected ones and thus formally act as interim patients regarding the already rejected null hypotheses. Although all kinds of interim patients are not part of the interim analysis, the data collected on those patients have to be sent to the office of regulatory affairs and will be analyzed. If a smaller or contrasting treatment effect is observed in interim patients, this might lead to a withdrawal of an earlier superiority proof.
Objectives: Presently, interim patients and their data are usually not considered in the confirmatory test. We offer a strategy to deal with interim patients in sequentially analyzed trials with discrete test statistics. The method covers sequentially analyzed single-and multi-arm trials with one or multiple primary endpoints.
Methods: When planning adaptive designs, it is common practice to assume that the stage-wise p-values are independent and standard uniformly distributed under the null hypothesis. In the context of discrete test statistics, this implies conservative tests. We provide an algorithm which iteratively optimizes an initially given design while adjusting for both discreteness of test statistics and interim patients. The algorithm is described verbally, graphically and formally to facilitate immediate implementation in computer software.
Results: The optimized design exploits the aspired significance level better and is more powerful than the initial one. The algorithm applies to fixed sample and planned flexible adaptive designs for single- and multi-arm trials with one or multiple primary endpoints. The benefit increases with the number of interim patients.
Conclusions: When planning a trial with interim analyses, the rules for decisions must be adjusted to interim patients. Otherwise, the test procedure is conservative resulting in loss of power. This is essential in situations where the number of interim patients is important compared to the first stage, particularly in trials with multiple primary endpoints.
Collapse
|
37
|
Kordzakhia G, Dmitrienko A, Ishida E. Mixture-based gatekeeping procedures in adaptive clinical trials. J Biopharm Stat 2017; 28:129-145. [PMID: 29283310 DOI: 10.1080/10543406.2017.1399901] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Abstract
Clinical trials with data-driven decision rules often pursue multiple clinical objectives such as the evaluation of several endpoints or several doses of an experimental treatment. These complex analysis strategies give rise to "multivariate" multiplicity problems with several components or sources of multiplicity. A general framework for defining gatekeeping procedures in clinical trials with adaptive multistage designs is proposed in this paper. The mixture method is applied to build a gatekeeping procedure at each stage and inferences at each decision point (interim or final analysis) are performed using the combination function approach. An advantage of utilizing the mixture method is that it enables powerful gatekeeping procedures applicable to a broad class of settings with complex logical relationships among the hypotheses of interest. Further, the combination function approach supports flexible data-driven decisions such as a decision to increase the sample size or remove a treatment arm. The paper concludes with a clinical trial example that illustrates the methodology by applying it to develop an adaptive two-stage design with a mixture-based gatekeeping procedure.
Collapse
Affiliation(s)
- George Kordzakhia
- a U.S. Food and Drug Administration , Silver Spring , Maryland , USA
| | | | - Eiji Ishida
- a U.S. Food and Drug Administration , Silver Spring , Maryland , USA
| |
Collapse
|
38
|
|
39
|
Schmidt R, Faldum A, Kwiecien R. Adaptive designs for the one-sample log-rank test. Biometrics 2017; 74:529-537. [DOI: 10.1111/biom.12776] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2016] [Revised: 08/01/2017] [Accepted: 08/01/2017] [Indexed: 12/01/2022]
Affiliation(s)
- Rene Schmidt
- Institute of Biostatistics and Clinical Research; University of Muenster; 48149 Muenster Germany
| | - Andreas Faldum
- Institute of Biostatistics and Clinical Research; University of Muenster; 48149 Muenster Germany
| | - Robert Kwiecien
- Institute of Biostatistics and Clinical Research; University of Muenster; 48149 Muenster Germany
| |
Collapse
|
40
|
Heritier S, Lloyd CJ, Lô SN. Accurate p-values for adaptive designs with binary endpoints. Stat Med 2017; 36:2643-2655. [PMID: 28470713 DOI: 10.1002/sim.7324] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2016] [Revised: 01/27/2017] [Accepted: 04/02/2017] [Indexed: 11/06/2022]
Abstract
Adaptive designs encompass all trials allowing various types of design modifications over the course of the trial. A key requirement for confirmatory adaptive designs to be accepted by regulators is the strong control of the family-wise error rate. This can be achieved by combining the p-values for each arm and stage to account for adaptations (including but not limited to treatment selection), sample size adaptation and multiple stages. While the theory for this is novel and well-established, in practice, these methods can perform poorly, especially for unbalanced designs and for small to moderate sample sizes. The problem is that standard stagewise tests have inflated type I error rate, especially but not only when the baseline success rate is close to the boundary and this is carried over to the adaptive tests, seriously inflating the family-wise error rate. We propose to fix this problem by feeding the adaptive test with second-order accurate p-values, in particular bootstrap p-values. Secondly, an adjusted version of the Simes procedure for testing intersection hypotheses that reduces the built-in conservatism is suggested. Numerical work and simulations show that unlike their standard counterparts the new approach preserves the overall error rate, at or below the nominal level across the board, irrespective of the baseline rate, stagewise sample sizes or allocation ratio. Copyright © 2017 John Wiley & Sons, Ltd.
Collapse
Affiliation(s)
- Stephane Heritier
- School of Public Health and Preventive Medicine, Monash University, Melbourne, VIC, Australia
| | - Chris J Lloyd
- Melbourne Business School, Melbourne University, Carlton, VIC, Australia
| | - Serigne N Lô
- Melanoma Institute Australia, North Sydney, NSW, Australia.,Sydney Medical School, The University of Sydney, Sydney, NSW, Australia
| |
Collapse
|
41
|
Dong G, Vandemeulebroecke M. A modified varying-stage adaptive phase II/III clinical trial design. Pharm Stat 2016; 15:368-78. [DOI: 10.1002/pst.1753] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2015] [Indexed: 11/08/2022]
|
42
|
Bauer P, Bretz F, Dragalin V, König F, Wassmer G. Twenty-five years of confirmatory adaptive designs: opportunities and pitfalls. Stat Med 2016; 35:325-47. [PMID: 25778935 PMCID: PMC6680191 DOI: 10.1002/sim.6472] [Citation(s) in RCA: 135] [Impact Index Per Article: 16.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2014] [Revised: 02/03/2015] [Accepted: 02/19/2015] [Indexed: 12/26/2022]
Abstract
'Multistage testing with adaptive designs' was the title of an article by Peter Bauer that appeared 1989 in the German journal Biometrie und Informatik in Medizin und Biologie. The journal does not exist anymore but the methodology found widespread interest in the scientific community over the past 25 years. The use of such multistage adaptive designs raised many controversial discussions from the beginning on, especially after the publication by Bauer and Köhne 1994 in Biometrics: Broad enthusiasm about potential applications of such designs faced critical positions regarding their statistical efficiency. Despite, or possibly because of, this controversy, the methodology and its areas of applications grew steadily over the years, with significant contributions from statisticians working in academia, industry and agencies around the world. In the meantime, such type of adaptive designs have become the subject of two major regulatory guidance documents in the US and Europe and the field is still evolving. Developments are particularly noteworthy in the most important applications of adaptive designs, including sample size reassessment, treatment selection procedures, and population enrichment designs. In this article, we summarize the developments over the past 25 years from different perspectives. We provide a historical overview of the early days, review the key methodological concepts and summarize regulatory and industry perspectives on such designs. Then, we illustrate the application of adaptive designs with three case studies, including unblinded sample size reassessment, adaptive treatment selection, and adaptive endpoint selection. We also discuss the availability of software for evaluating and performing such designs. We conclude with a critical review of how expectations from the beginning were fulfilled, and - if not - discuss potential reasons why this did not happen.
Collapse
Affiliation(s)
- Peter Bauer
- Section of Medical StatisticsMedical University of ViennaSpitalgasse 231090 WienAustria
| | - Frank Bretz
- Novartis Pharma AGLichtstrasse 354002BaselSwitzerland
- Shanghai University of Finance and EconomicsChina
| | | | - Franz König
- Section of Medical StatisticsMedical University of ViennaSpitalgasse 231090 WienAustria
| | - Gernot Wassmer
- Aptiv Solutions, an ICON plc companyRobert‐Perthel‐Str. 77a50739KölnGermany
- Institute for Medical Statistics, Informatics and EpidemiologyUniversity of Cologne50924KölnGermany
| |
Collapse
|
43
|
Żebrowska M, Posch M, Magirr D. Maximum type I error rate inflation from sample size reassessment when investigators are blind to treatment labels. Stat Med 2015; 35:1972-84. [PMID: 26694878 PMCID: PMC4851240 DOI: 10.1002/sim.6848] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2014] [Revised: 11/23/2015] [Accepted: 11/24/2015] [Indexed: 12/23/2022]
Abstract
Consider a parallel group trial for the comparison of an experimental treatment to a control, where the second‐stage sample size may depend on the blinded primary endpoint data as well as on additional blinded data from a secondary endpoint. For the setting of normally distributed endpoints, we demonstrate that this may lead to an inflation of the type I error rate if the null hypothesis holds for the primary but not the secondary endpoint. We derive upper bounds for the inflation of the type I error rate, both for trials that employ random allocation and for those that use block randomization. We illustrate the worst‐case sample size reassessment rule in a case study. For both randomization strategies, the maximum type I error rate increases with the effect size in the secondary endpoint and the correlation between endpoints. The maximum inflation increases with smaller block sizes if information on the block size is used in the reassessment rule. Based on our findings, we do not question the well‐established use of blinded sample size reassessment methods with nuisance parameter estimates computed from the blinded interim data of the primary endpoint. However, we demonstrate that the type I error rate control of these methods relies on the application of specific, binding, pre‐planned and fully algorithmic sample size reassessment rules and does not extend to general or unplanned sample size adjustments based on blinded data. © 2015 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.
Collapse
Affiliation(s)
- Magdalena Żebrowska
- Center for Medical Statistics, Informatics and Intelligent Systems, Medical University of Vienna, Vienna, 1090, Austria
| | - Martin Posch
- Center for Medical Statistics, Informatics and Intelligent Systems, Medical University of Vienna, Vienna, 1090, Austria
| | - Dominic Magirr
- Center for Medical Statistics, Informatics and Intelligent Systems, Medical University of Vienna, Vienna, 1090, Austria
| |
Collapse
|
44
|
Zehetmayer S, Graf AC, Posch M. Sample size reassessment for a two-stage design controlling the false discovery rate. Stat Appl Genet Mol Biol 2015; 14:429-42. [PMID: 26461844 PMCID: PMC4789494 DOI: 10.1515/sagmb-2014-0025] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Sample size calculations for gene expression microarray and NGS-RNA-Seq experiments are challenging because the overall power depends on unknown quantities as the proportion of true null hypotheses and the distribution of the effect sizes under the alternative. We propose a two-stage design with an adaptive interim analysis where these quantities are estimated from the interim data. The second stage sample size is chosen based on these estimates to achieve a specific overall power. The proposed procedure controls the power in all considered scenarios except for very low first stage sample sizes. The false discovery rate (FDR) is controlled despite of the data dependent choice of sample size. The two-stage design can be a useful tool to determine the sample size of high-dimensional studies if in the planning phase there is high uncertainty regarding the expected effect sizes and variability.
Collapse
Affiliation(s)
- Sonja Zehetmayer
- Center for Medical Statistics, Informatics, and Intelligent Systems, Medical University of Vienna, Spitalgasse 23, Vienna, Austria
| | - Alexandra C. Graf
- Center for Medical Statistics, Informatics, and Intelligent Systems, Medical University of Vienna, Spitalgasse 23, Vienna, Austria
| | - Martin Posch
- Center for Medical Statistics, Informatics, and Intelligent Systems, Medical University of Vienna, Spitalgasse 23, Vienna, Austria
| |
Collapse
|
45
|
Stallard N, Kunz CU, Todd S, Parsons N, Friede T. Flexible selection of a single treatment incorporating short-term endpoint information in a phase II/III clinical trial. Stat Med 2015; 34:3104-15. [PMID: 26112909 PMCID: PMC4745001 DOI: 10.1002/sim.6567] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2014] [Revised: 05/11/2015] [Accepted: 06/01/2015] [Indexed: 11/07/2022]
Abstract
Seamless phase II/III clinical trials in which an experimental treatment is selected at an interim analysis have been the focus of much recent research interest. Many of the methods proposed are based on the group sequential approach. This paper considers designs of this type in which the treatment selection can be based on short-term endpoint information for more patients than have primary endpoint data available. We show that in such a case, the familywise type I error rate may be inflated if previously proposed group sequential methods are used and the treatment selection rule is not specified in advance. A method is proposed to avoid this inflation by considering the treatment selection that maximises the conditional error given the data available at the interim analysis. A simulation study is reported that illustrates the type I error rate inflation and compares the power of the new approach with two other methods: a combination testing approach and a group sequential method that does not use the short-term endpoint data, both of which also strongly control the type I error rate. The new method is also illustrated through application to a study in Alzheimer's disease.
Collapse
Affiliation(s)
- Nigel Stallard
- Statistics and Epidemiology, Division of Health SciencesWarwick Medical School, University of WarwickCoventryU.K.
| | - Cornelia Ursula Kunz
- Statistics and Epidemiology, Division of Health SciencesWarwick Medical School, University of WarwickCoventryU.K.
| | - Susan Todd
- Department of Mathematics and StatisticsUniversity of ReadingReadingU.K.
| | - Nicholas Parsons
- Statistics and Epidemiology, Division of Health SciencesWarwick Medical School, University of WarwickCoventryU.K.
| | - Tim Friede
- Department of Medical StatisticsUniversity Medical CenterGöttingenGermany
| |
Collapse
|
46
|
Kunz CU, Friede T, Parsons N, Todd S, Stallard N. A comparison of methods for treatment selection in seamless phase II/III clinical trials incorporating information on short-term endpoints. J Biopharm Stat 2015; 25:170-89. [PMID: 24697322 PMCID: PMC4339952 DOI: 10.1080/10543406.2013.840646] [Citation(s) in RCA: 23] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
In an adaptive seamless phase II/III clinical trial interim analysis, data are used for treatment selection, enabling resources to be focused on comparison of more effective treatment(s) with a control. In this paper, we compare two methods recently proposed to enable use of short-term endpoint data for decision-making at the interim analysis. The comparison focuses on the power and the probability of correctly identifying the most promising treatment. We show that the choice of method depends on how well short-term data predict the best treatment, which may be measured by the correlation between treatment effects on short- and long-term endpoints.
Collapse
|
47
|
Abstract
Adaptive designs have generated a great deal of attention to clinical trial communities. The literature contains many statistical methods to deal with added statistical uncertainties concerning the adaptations. Increasingly encountered in regulatory applications are adaptive statistical information designs that allow modification of sample size or related statistical information and adaptive selection designs that allow selection of doses or patient populations during the course of a clinical trial. For adaptive statistical information designs, a few statistical testing methods are mathematically equivalent, as a number of articles have stipulated, but arguably there are large differences in their practical ramifications. We pinpoint some undesirable features of these methods in this work. For adaptive selection designs, the selection based on biomarker data for testing the correlated clinical endpoints may increase statistical uncertainty in terms of type I error probability, and most importantly the increased statistical uncertainty may be impossible to assess.
Collapse
Affiliation(s)
- H M James Hung
- a Division of Biometrics I , Office of Biostatistics, Office of Translational Sciences, Center for Drug Evaluation and Research, Food and Drug Administration , Silver Spring , Maryland , USA
| | | | | |
Collapse
|
48
|
Henning KSS, Westfall PH. Closed Testing in Pharmaceutical Research: Historical and Recent Developments. Stat Biopharm Res 2015; 7:126-147. [PMID: 26366251 DOI: 10.1080/19466315.2015.1004270] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Abstract
In pharmaceutical research, making multiple statistical inferences is standard practice. Unless adjustments are made for multiple testing, the probability of making erroneous determinations of significance increases with the number of inferences. Closed testing is a flexible and easily explained approach to controlling the overall error rate that has seen wide use in pharmaceutical research, particularly in clinical trials settings. In this article, we first give a general review of the uses of multiple testing in pharmaceutical research, with particular emphasis on the benefits and pitfalls of closed testing procedures. We then provide a more technical examination of a class of closed tests that use additive-combination-based and minimum-based p-value statistics, both of which are commonly used in pharmaceutical research. We show that, while the additive combination tests are generally far superior to minimum p-value tests for composite hypotheses, the reverse is true for multiple comparisons using closure-based testing. The loss of power of additive combination tests is explained in terms worst-case "hurdles" that must be cleared before significance can be determined via closed testing. We prove mathematically that this problem can result in the power of a closure-based minimum p-value test approaching 1, while the power of an closure-based additive combination test approaches 0. Finally, implications of these results to pharmaceutical researchers are given.
Collapse
Affiliation(s)
- Kevin S S Henning
- Department of Economics and International Business, Sam Houston State University, Huntsville, TX 77341
| | - Peter H Westfall
- Area of Information Systems and Quantitative Sciences, Texas Tech University, Lubbock, TX 79409-2101 USA
| |
Collapse
|
49
|
Gerß J, Eveslage M, Faldum A, Schmidt R. [Modern study designs and analysis methods in clinical research]. Z Rheumatol 2015; 74:106-12. [PMID: 25805512 DOI: 10.1007/s00393-014-1450-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
BACKGROUND Progress in the field of medical research requires further development of clinical trial methodology to overcome the challenges resulting from small patient populations and restricted resources. METHODS Classical single-stage designs with fixed sample sizes do not allow for interim analyses or design modifications. In contrast, adaptive designs adhere to established quality criteria while providing flexibility when conducting a clinical trial. In the face of new discoveries or information collected in the course of a trial, sample size adjustment, the selection of the target population and further design modifications can be performed. This enhances the chance of success of a clinical trial. Besides adaptive designs, classical approaches may be replaced or complemented by Bayesian methods. In a Bayesian approach prior knowledge can be efficiently included and hence the amount of information utilized in statistical analyses is increased. Furthermore, Bayes procedures allow the results of a statistical evaluation to be displayed very clearly. CONCLUSION Modern approaches, such as adaptive designs and Bayesian designs overcome the challenges in clinical research due to enhanced flexibility and efficiency. In addition, both approaches can be combined.
Collapse
Affiliation(s)
- J Gerß
- Institut für Biometrie und Klinische Forschung (IBKF), Westfälische Wilhelms-Universität Münster, Schmeddingstr. 56, 48149, Münster, Deutschland,
| | | | | | | |
Collapse
|
50
|
Schmidt R, Faldum A, Gerß J. Adaptive designs with arbitrary dependence structure based on Fisher’s combination test. STAT METHOD APPL-GER 2014. [DOI: 10.1007/s10260-014-0291-6] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|