1
|
Novelli M, Rosenberger WF. Exploring Stratification Strategies for Population- Versus Randomization-Based Inference. Pharm Stat 2024. [PMID: 38986522 DOI: 10.1002/pst.2419] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2023] [Revised: 04/08/2024] [Accepted: 06/18/2024] [Indexed: 07/12/2024]
Abstract
Stratification on important variables is a common practice in clinical trials, since ensuring cosmetic balance on known baseline covariates is often deemed to be a crucial requirement for the credibility of the experimental results. However, the actual benefits of stratification are still debated in the literature. Other authors have shown that it does not improve efficiency in large samples and improves it only negligibly in smaller samples. This paper investigates different subgroup analysis strategies, with a particular focus on the potential benefits in terms of inferential precision of prestratification versus both poststratification and post hoc regression adjustment. For each of these approaches, the pros and cons of population-based versus randomization-based inference are discussed. The effects of the presence of a treatment-by-covariate interaction and the variability in the patient responses are also taken into account. Our results show that, in general, prestratifying does not provide substantial benefit. On the contrary, it may be deleterious, in particular for randomization-based procedures in the presence of a chronological bias. Even when there is treatment-by-covariate interaction, prestratification may backfire by considerably reducing the inferential precision.
Collapse
Affiliation(s)
- Marco Novelli
- Department of Statistics Bologna, University of Bologna, Bologna, Italy
| | | |
Collapse
|
2
|
Fiksel J. On exact randomization-based covariate-adjusted confidence intervals. Biometrics 2024; 80:ujae051. [PMID: 38837900 DOI: 10.1093/biomtc/ujae051] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2023] [Revised: 12/20/2023] [Accepted: 05/21/2024] [Indexed: 06/07/2024]
Abstract
Randomization-based inference using the Fisher randomization test allows for the computation of Fisher-exact P-values, making it an attractive option for the analysis of small, randomized experiments with non-normal outcomes. Two common test statistics used to perform Fisher randomization tests are the difference-in-means between the treatment and control groups and the covariate-adjusted version of the difference-in-means using analysis of covariance. Modern computing allows for fast computation of the Fisher-exact P-value, but confidence intervals have typically been obtained by inverting the Fisher randomization test over a range of possible effect sizes. The test inversion procedure is computationally expensive, limiting the usage of randomization-based inference in applied work. A recent paper by Zhu and Liu developed a closed form expression for the randomization-based confidence interval using the difference-in-means statistic. We develop an important extension of Zhu and Liu to obtain a closed form expression for the randomization-based covariate-adjusted confidence interval and give practitioners a sufficiency condition that can be checked using observed data and that guarantees that these confidence intervals have correct coverage. Simulations show that our procedure generates randomization-based covariate-adjusted confidence intervals that are robust to non-normality and that can be calculated in nearly the same time as it takes to calculate the Fisher-exact P-value, thus removing the computational barrier to performing randomization-based inference when adjusting for covariates. We also demonstrate our method on a re-analysis of phase I clinical trial data.
Collapse
Affiliation(s)
- Jacob Fiksel
- Department of Data & Computational Sciences, Vertex Pharmaceuticals, Boston 02210, United States
| |
Collapse
|
3
|
Coart E, Bamps P, Quinaux E, Sturbois G, Saad ED, Burzykowski T, Buyse M. Minimization in randomized clinical trials. Stat Med 2023; 42:5285-5311. [PMID: 37867447 DOI: 10.1002/sim.9916] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2022] [Revised: 08/23/2023] [Accepted: 09/13/2023] [Indexed: 10/24/2023]
Abstract
In randomized trials, comparability of the treatment groups is ensured through allocation of treatments using a mechanism that involves some random element, thus controlling for confounding of the treatment effect. Completely random allocation ensures comparability between the treatment groups for all known and unknown prognostic factors. For a specific trial, however, imbalances in prognostic factors among the treatment groups may occur. Although accidental bias can be avoided in the presence of such imbalances by stratifying the analysis, most trialists, regulatory agencies, and other stakeholders prefer a balanced distribution of prognostic factors across the treatment groups. Some procedures attempt to achieve balance in baseline covariates, by stratifying the allocation for these covariates, or by dynamically adapting the allocation using covariate information during the trial (covariate-adaptive procedures). In this Tutorial, the performance of minimization, a popular covariate-adaptive procedure, is compared with two other commonly used procedures, completely random allocation and stratified blocked designs. Using individual patient data of 2 clinical trials (in advanced ovarian cancer and age-related macular degeneration), the procedures are compared in terms of operating characteristics (using asymptotic and randomization tests), predictability of treatment allocation, and achieved balance. Fifty actual trials of various sizes that applied minimization for treatment allocation are used to investigate the achieved balance. Implementation issues of minimization are described. Minimization procedures are useful in all trials but especially when (1) many major prognostic factors are known, (2) many centers of different sizes accrue patients, or (3) the trial sample size is moderate.
Collapse
Affiliation(s)
| | | | | | | | | | - Tomasz Burzykowski
- IDDI, Louvain-la-Neuve, Belgium
- Data Science Institute, Hasselt University, Hasselt, Belgium
| | - Marc Buyse
- IDDI, Louvain-la-Neuve, Belgium
- Data Science Institute, Hasselt University, Hasselt, Belgium
| |
Collapse
|
4
|
Zhu K, Liu H. Pair-switching rerandomization. Biometrics 2023; 79:2127-2142. [PMID: 35758335 DOI: 10.1111/biom.13712] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2021] [Accepted: 06/21/2022] [Indexed: 11/29/2022]
Abstract
Rerandomization discards assignments with covariates unbalanced in the treatment and control groups to improve estimation and inference efficiency. However, the acceptance-rejection sampling method used in rerandomization is computationally inefficient. As a result, it is time-consuming for rerandomization to draw numerous independent assignments, which are necessary for performing Fisher randomization tests and constructing randomization-based confidence intervals. To address this problem, we propose a pair-switching rerandomization (PSRR) method to draw balanced assignments efficiently. We obtain the unbiasedness and variance reduction of the difference-in-means estimator and show that the Fisher randomization tests are valid under PSRR. Moreover, we propose an exact approach to invert Fisher randomization tests to confidence intervals, which is faster than the existing methods. In addition, our method is applicable to both nonsequentially and sequentially randomized experiments. We conduct comprehensive simulation studies to compare the finite-sample performance of the proposed method with that of classical rerandomization. Simulation results indicate that PSRR leads to comparable power of Fisher randomization tests and is 3-23 times faster than classical rerandomization. Finally, we apply the PSRR method to analyze two clinical trial datasets, both of which demonstrate the advantages of our method.
Collapse
Affiliation(s)
- Ke Zhu
- Center for Statistical Science, Department of Industrial Engineering, Tsinghua University, Beijing, China
| | - Hanzhong Liu
- Center for Statistical Science, Department of Industrial Engineering, Tsinghua University, Beijing, China
| |
Collapse
|
5
|
Alexandria SJ, Hudgens MG, Aiello AE. Assessing intervention effects in a randomized trial within a social network. Biometrics 2023; 79:1409-1419. [PMID: 34825368 PMCID: PMC9133268 DOI: 10.1111/biom.13606] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2021] [Revised: 11/15/2021] [Accepted: 11/18/2021] [Indexed: 11/29/2022]
Abstract
Studies of social networks provide unique opportunities to assess the causal effects of interventions that may impact more of the population than just those intervened on directly. Such effects are sometimes called peer or spillover effects, and may exist in the presence of interference, that is, when one individual's treatment affects another individual's outcome. Randomization-based inference (RI) methods provide a theoretical basis for causal inference in randomized studies, even in the presence of interference. In this article, we consider RI of the intervention effect in the eX-FLU trial, a randomized study designed to assess the effect of a social distancing intervention on influenza-like-illness transmission in a connected network of college students. The approach considered enables inference about the effect of the social distancing intervention on the per-contact probability of influenza-like-illness transmission in the observed network. The methods allow for interference between connected individuals and for heterogeneous treatment effects. The proposed methods are evaluated empirically via simulation studies, and then applied to data from the eX-FLU trial.
Collapse
Affiliation(s)
- Shaina J. Alexandria
- Department of Preventive Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois, U.S.A
| | - Michael G. Hudgens
- Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, U.S.A
| | - Allison E. Aiello
- Department of Epidemiology, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, U.S.A
| |
Collapse
|
6
|
Robertson DS, Lee KM, López-Kolkovska BC, Villar SS. Response-adaptive randomization in clinical trials: from myths to practical considerations. Stat Sci 2023; 38:185-208. [PMID: 37324576 PMCID: PMC7614644 DOI: 10.1214/22-sts865] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/17/2023]
Abstract
Response-Adaptive Randomization (RAR) is part of a wider class of data-dependent sampling algorithms, for which clinical trials are typically used as a motivating application. In that context, patient allocation to treatments is determined by randomization probabilities that change based on the accrued response data in order to achieve experimental goals. RAR has received abundant theoretical attention from the biostatistical literature since the 1930's and has been the subject of numerous debates. In the last decade, it has received renewed consideration from the applied and methodological communities, driven by well-known practical examples and its widespread use in machine learning. Papers on the subject present different views on its usefulness, and these are not easy to reconcile. This work aims to address this gap by providing a unified, broad and fresh review of methodological and practical issues to consider when debating the use of RAR in clinical trials.
Collapse
Affiliation(s)
- David S. Robertson
- MRC Biostatistics Unit, University of Cambridge, Forvie Site, Robinson Way, Cambridge CB2 0SR, United Kingdom
| | | | | | | |
Collapse
|
7
|
Grassano L, Ranzato G, Pellegrini M, Costantini M. Re-randomization tests as sensitivity analyses to confirm immunological noninferiority of an investigational vaccine: Case study. Pharm Stat 2023; 22:570-576. [PMID: 36707656 DOI: 10.1002/pst.2290] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2022] [Revised: 11/30/2022] [Accepted: 01/09/2023] [Indexed: 01/29/2023]
Abstract
Here we present as case study how re-randomization tests were performed in two randomized, controlled clinical trials as sensitivity analyses, as recommended by the United States Food and Drug Administration in the context of adaptive randomization. This was done to confirm primary conclusions on immunological noninferiority of an investigational new fully liquid presentation of a quadrivalent cross-reacting material conjugate meningococcal vaccine (MenACWY-CRM), over its licensed lyophilized/liquid presentation. In two phase 2b studies (Study #1: NCT03652610; Study #2: NCT03433482), noninferiority of the fully liquid presentation of MenACWY-CRM to the licensed presentation was assessed and demonstrated for immune responses against meningococcal serogroup A (MenA), the only vaccine component modified from lyophilized to liquid in the new presentation. The original vaccine assignment algorithm, with a minimization procedure accounting for center or center within age strata, was used to re-randomize participants belonging to the fully liquid and licensed vaccine groups while keeping antibody responses, covariates and entry order as observed. Test statistics under re-randomization were generated according to the ANCOVA model used in the primary analysis. To confirm immunological noninferiority following re-randomization, the corresponding p-values had to be <0.025. For both studies and all primary objective evaluations, the re-randomization p-values were well below 0.025 (0.0004 for Study #1; 0.0001 for the two co-primary endpoints in Study #2). Re-randomization tests performed to comply with a regulatory request confirmed the primary conclusions of immunological noninferiority for the MenA of the fully liquid compared to the licensed vaccine presentation.
Collapse
|
8
|
Hawila N, Berg A. Exact‐corrected confidence interval for risk difference in noninferiority binomial trials. Biometrics 2022. [DOI: 10.1111/biom.13688] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2021] [Accepted: 04/25/2022] [Indexed: 11/27/2022]
Affiliation(s)
- Nour Hawila
- Division of Biostatistics & Bioinformatics Penn State University
| | - Arthur Berg
- Division of Biostatistics & Bioinformatics Penn State University
| |
Collapse
|
9
|
Proschan MA. Discussion on "Improving precision and power in randomized trials for COVID-19 treatments using covariate adjustment for binary, ordinal, and time-to-event outcomes". Biometrics 2021; 77:1482-1484. [PMID: 34105763 PMCID: PMC8239582 DOI: 10.1111/biom.13493] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2021] [Accepted: 02/03/2021] [Indexed: 11/28/2022]
Abstract
Benkeser et al. present a very informative paper evaluating the efficiency gains of covariate adjustment in settings with binary, ordinal, and time‐to‐event outcomes. The adjustment method focuses on estimating the marginal treatment effect averaged over the covariate distribution in both arms combined. The authors show that covariate adjustment can achieve power gains that could find answers more quickly. The suggested approach is an important weapon in the armamentarium against epidemics like COVID‐19. I recommend evaluating the procedure against more traditional approaches for conditional analyses (e.g., logistic regression) and against blinded methods of building prediction models followed by randomization‐based inference.
Collapse
Affiliation(s)
- Michael A Proschan
- Biostatistics Research Branch, National Institute of Allergy and Infectious Diseases
| |
Collapse
|
10
|
A simple solution to the inadequacy of asymptotic likelihood-based inference for response-adaptive clinical trials. Stat Pap (Berl) 2021. [DOI: 10.1007/s00362-021-01234-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Abstract
AbstractThe present paper discusses drawbacks and limitations of likelihood-based inference in sequential clinical trials for treatment comparisons managed via Response-Adaptive Randomization. Taking into account the most common statistical models for the primary outcome—namely binary, Poisson, exponential and normal data—we derive the conditions under which (i) the classical confidence intervals degenerate and (ii) the Wald test becomes inconsistent and strongly affected by the nuisance parameters, also displaying a non monotonic power. To overcome these drawbacks, we provide a very simple solution that could preserve the fundamental properties of likelihood-based inference. Several illustrative examples and simulation studies are presented in order to confirm the relevance of our results and provide some practical recommendations.
Collapse
|