1
|
Wang B, Du Y. Improving the mixed model for repeated measures to robustly increase precision in randomized trials. Int J Biostat 2023; 0:ijb-2022-0101. [PMID: 38016707 DOI: 10.1515/ijb-2022-0101] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2022] [Accepted: 08/12/2023] [Indexed: 11/30/2023]
Abstract
In randomized trials, repeated measures of the outcome are routinely collected. The mixed model for repeated measures (MMRM) leverages the information from these repeated outcome measures, and is often used for the primary analysis to estimate the average treatment effect at the primary endpoint. MMRM, however, can suffer from bias and precision loss when it models intermediate outcomes incorrectly, and hence fails to use the post-randomization information harmlessly. This paper proposes an extension of the commonly used MMRM, called IMMRM, that improves the robustness and optimizes the precision gain from covariate adjustment, stratified randomization, and adjustment for intermediate outcome measures. Under regularity conditions and missing completely at random, we prove that the IMMRM estimator for the average treatment effect is robust to arbitrary model misspecification and is asymptotically equal or more precise than the analysis of covariance (ANCOVA) estimator and the MMRM estimator. Under missing at random, IMMRM is less likely to be misspecified than MMRM, and we demonstrate via simulation studies that IMMRM continues to have less bias and smaller variance. Our results are further supported by a re-analysis of a randomized trial for the treatment of diabetes.
Collapse
Affiliation(s)
- Bingkai Wang
- The Statistics and Data Science Department of the Wharton School, University of Pennsylvania, Philadelphia, PA, USA
| | - Yu Du
- Statistics, Data and Analytics, Eli Lilly and Company, Indianapolis, IN, USA
| |
Collapse
|
2
|
Tsiatis AA, Davidian M. Group sequential methods for interim monitoring of randomized clinical trials with time-lagged outcome. Stat Med 2022; 41:5517-5536. [PMID: 36117235 PMCID: PMC9825950 DOI: 10.1002/sim.9580] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2022] [Revised: 08/17/2022] [Accepted: 08/29/2022] [Indexed: 01/11/2023]
Abstract
The primary analysis in two-arm clinical trials usually involves inference on a scalar treatment effect parameter; for example, depending on the outcome, the difference of treatment-specific means, risk difference, risk ratio, or odds ratio. Most clinical trials are monitored for the possibility of early stopping. Because ordinarily the outcome on any given subject can be ascertained only after some time lag, at the time of an interim analysis, among the subjects already enrolled, the outcome is known for only a subset and is effectively censored for those who have not been enrolled sufficiently long for it to be observed. Typically, the interim analysis is based only on the data from subjects for whom the outcome has been ascertained. A goal of an interim analysis is to stop the trial as soon as the evidence is strong enough to do so, suggesting that the analysis ideally should make the most efficient use of all available data, thus including information on censoring as well as other baseline and time-dependent covariates in a principled way. A general group sequential framework is proposed for clinical trials with a time-lagged outcome. Treatment effect estimators that take account of censoring and incorporate covariate information at an interim analysis are derived using semiparametric theory and are demonstrated to lead to stronger evidence for early stopping than standard approaches. The associated test statistics are shown to have the independent increments structure, so that standard software can be used to obtain stopping boundaries.
Collapse
Affiliation(s)
| | - Marie Davidian
- Department of StatisticsNorth Carolina State UniversityRaleighNorth Carolina
| |
Collapse
|
3
|
Williams N, Rosenblum M, Díaz I. Optimising precision and power by machine learning in randomised trials with ordinal and time-to-event outcomes with an application to COVID-19. JOURNAL OF THE ROYAL STATISTICAL SOCIETY. SERIES A, (STATISTICS IN SOCIETY) 2022; 185:RSSA12915. [PMID: 36246572 PMCID: PMC9539267 DOI: 10.1111/rssa.12915] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/05/2021] [Revised: 05/23/2022] [Accepted: 07/05/2022] [Indexed: 05/23/2023]
Abstract
The rapid finding of effective therapeutics requires efficient use of available resources in clinical trials. Covariate adjustment can yield statistical estimates with improved precision, resulting in a reduction in the number of participants required to draw futility or efficacy conclusions. We focus on time-to-event and ordinal outcomes. When more than a few baseline covariates are available, a key question for covariate adjustment in randomised studies is how to fit a model relating the outcome and the baseline covariates to maximise precision. We present a novel theoretical result establishing conditions for asymptotic normality of a variety of covariate-adjusted estimators that rely on machine learning (e.g.,ℓ 1 -regularisation, Random Forests, XGBoost, and Multivariate Adaptive Regression Splines [MARS]), under the assumption that outcome data are missing completely at random. We further present a consistent estimator of the asymptotic variance. Importantly, the conditions do not require the machine learning methods to converge to the true outcome distribution conditional on baseline variables, as long as they converge to some (possibly incorrect) limit. We conducted a simulation study to evaluate the performance of the aforementioned prediction methods in COVID-19 trials. Our simulation is based on resampling longitudinal data from over 1500 patients hospitalised with COVID-19 at Weill Cornell Medicine New York Presbyterian Hospital. We found that usingℓ 1 -regularisation led to estimators and corresponding hypothesis tests that control type 1 error and are more precise than an unadjusted estimator across all sample sizes tested. We also show that when covariates are not prognostic of the outcome,ℓ 1 -regularisation remains as precise as the unadjusted estimator, even at small sample sizes (n = 100 ). We give an R package adjrct that performs model-robust covariate adjustment for ordinal and time-to-event outcomes.
Collapse
Affiliation(s)
- Nicholas Williams
- Department of EpidemiologyColumbia University Mailman School of Public HealthNew York CityNew YorkUSA
| | - Michael Rosenblum
- Department of BiostatisticsJohns Hopkins Bloomberg School of Public HealthBaltimoreMarylandUSA
| | - Iván Díaz
- Division of Biostatistics, Department of Population HealthNew York University Grossman School of MedicineNew York CityNew YorkUSA
| |
Collapse
|
4
|
Benkeser D, Díaz I, Luedtke A, Segal J, Scharfstein D, Rosenblum M. Improving precision and power in randomized trials for COVID-19 treatments using covariate adjustment, for binary, ordinal, and time-to-event outcomes. Biometrics 2021; 77:1467-1481. [PMID: 32978962 PMCID: PMC7537316 DOI: 10.1111/biom.13377] [Citation(s) in RCA: 33] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2020] [Revised: 09/01/2020] [Accepted: 09/15/2020] [Indexed: 12/31/2022]
Abstract
Time is of the essence in evaluating potential drugs and biologics for the treatment and prevention of COVID-19. There are currently 876 randomized clinical trials (phase 2 and 3) of treatments for COVID-19 registered on clinicaltrials.gov. Covariate adjustment is a statistical analysis method with potential to improve precision and reduce the required sample size for a substantial number of these trials. Though covariate adjustment is recommended by the U.S. Food and Drug Administration and the European Medicines Agency, it is underutilized, especially for the types of outcomes (binary, ordinal, and time-to-event) that are common in COVID-19 trials. To demonstrate the potential value added by covariate adjustment in this context, we simulated two-arm, randomized trials comparing a hypothetical COVID-19 treatment versus standard of care, where the primary outcome is binary, ordinal, or time-to-event. Our simulated distributions are derived from two sources: longitudinal data on over 500 patients hospitalized at Weill Cornell Medicine New York Presbyterian Hospital and a Centers for Disease Control and Prevention preliminary description of 2449 cases. In simulated trials with sample sizes ranging from 100 to 1000 participants, we found substantial precision gains from using covariate adjustment-equivalent to 4-18% reductions in the required sample size to achieve a desired power. This was the case for a variety of estimands (targets of inference). From these simulations, we conclude that covariate adjustment is a low-risk, high-reward approach to streamlining COVID-19 treatment trials. We provide an R package and practical recommendations for implementation.
Collapse
Affiliation(s)
- David Benkeser
- Department of Biostatistics and BioinformaticsEmory UniversityAtlantaGeorgiaUSA
| | - Iván Díaz
- Division of BiostatisticsDepartment of Population Health SciencesWeill Cornell MedicineNew YorkNew YorkUSA
| | - Alex Luedtke
- Department of StatisticsUniversity of WashingtonSeattleWashingtonUSA
- Vaccine and Infectious Disease DivisionFred Hutchinson Cancer Research Center, University of WashingtonSeattleWashingtonUSA
| | - Jodi Segal
- Department of MedicineSchool of MedicineJohns Hopkins UniversityBaltimoreMarylandUSA
| | - Daniel Scharfstein
- Division of BiostatisticsDepartment of Population Health SciencesUniversity of Utah School of MedicineSalt Lake CityUtahUSA
| | - Michael Rosenblum
- Department of BiostatisticsJohns Hopkins Bloomberg School of Public HealthJohns Hopkins UniversityBaltimoreMarylandUSA
| |
Collapse
|
5
|
Tsiatis AA, Davidian M, Holloway ST. Estimation of the odds ratio in a proportional odds model with censored time-lagged outcome in a randomized clinical trial. Biometrics 2021. [PMID: 34825704 DOI: 10.1111/biom.13603] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2021] [Revised: 11/05/2021] [Accepted: 11/18/2021] [Indexed: 12/01/2022]
Abstract
In many randomized clinical trials of therapeutics for COVID-19, the primary outcome is an ordinal categorical variable, and interest focuses on the odds ratio (OR; active agent vs control) under the assumption of a proportional odds model. Although at the final analysis the outcome will be determined for all subjects, at an interim analysis, the status of some participants may not yet be determined, for example, because ascertainment of the outcome may not be possible until some prespecified follow-up time. Accordingly, the outcome from these subjects can be viewed as censored. A valid interim analysis can be based on data only from those subjects with full follow-up; however, this approach is inefficient, as it does not exploit additional information that may be available on those for whom the outcome is not yet available at the time of the interim analysis. Appealing to the theory of semiparametrics, we propose an estimator for the OR in a proportional odds model with censored, time-lagged categorical outcome that incorporates additional baseline and time-dependent covariate information and demonstrate that it can result in considerable gains in efficiency relative to simpler approaches. A byproduct of the approach is a covariate-adjusted estimator for the OR based on the full data that would be available at a final analysis.
Collapse
Affiliation(s)
- Anastasios A Tsiatis
- Department of Statistics, North Carolina State University, Raleigh, North Carolina
| | - Marie Davidian
- Department of Statistics, North Carolina State University, Raleigh, North Carolina
| | - Shannon T Holloway
- Department of Statistics, North Carolina State University, Raleigh, North Carolina
| |
Collapse
|
6
|
Wang B, Susukida R, Mojtabai R, Amin-Esmaeili M, Rosenblum M. Model-Robust Inference for Clinical Trials that Improve Precision by Stratified Randomization and Covariate Adjustment. J Am Stat Assoc 2021. [DOI: 10.1080/01621459.2021.1981338] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Affiliation(s)
- Bingkai Wang
- Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, MD
| | - Ryoko Susukida
- Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, MD
| | - Ramin Mojtabai
- Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, MD
| | - Masoumeh Amin-Esmaeili
- Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, MD
- Iranian National Center for Addiction Studies (INCAS), Tehran University of Medical Sciences, Tehran, Iran
| | - Michael Rosenblum
- Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, MD
| |
Collapse
|
7
|
Benkeser D, Díaz I, Luedtke A, Segal J, Scharfstein D, Rosenblum M. Improving Precision and Power in Randomized Trials for COVID-19 Treatments Using Covariate Adjustment, for Binary, Ordinal, and Time-to-Event Outcomes. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2020:2020.04.19.20069922. [PMID: 32577668 PMCID: PMC7302221 DOI: 10.1101/2020.04.19.20069922] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Time is of the essence in evaluating potential drugs and biologics for the treatment and prevention of COVID-19. There are currently over 400 clinical trials (phase 2 and 3) of treatments for COVID-19 registered on clinicaltrials.gov. Covariate adjustment is a statistical analysis method with potential to improve precision and reduce the required sample size for a substantial number of these trials. Though covariate adjustment is recommended by the U.S. Food and Drug Administration and the European Medicines Agency, it is underutilized, especially for the types of outcomes (binary, ordinal and time-to-event) that are common in COVID-19 trials. To demonstrate the potential value added by covariate adjustment in this context, we simulated two-arm, randomized trials comparing a hypothetical COVID-19 treatment versus standard of care, where the primary outcome is binary, ordinal, or time-to-event. Our simulated distributions are derived from two sources: longitudinal data on over 500 patients hospitalized at Weill Cornell Medicine New York Presbyterian Hospital, and a Centers for Disease Control and Prevention (CDC) preliminary description of 2449 cases. We found substantial precision gains from using covariate adjustment--equivalent to 9-21% reductions in the required sample size to achieve a desired power--for a variety of estimands (targets of inference) when the trial sample size was at least 200. We provide an R package and practical recommendations for implementing covariate adjustment. The estimators that we consider are robust to model misspecification.
Collapse
Affiliation(s)
- David Benkeser
- Department of Biostatistics and Bioinformatics, Emory University
| | - Iván Díaz
- Division of Biostatistics, Department of Population Health Sciences, Weill Cornell Medicine
| | - Alex Luedtke
- Department of Statistics, University of Washington, Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center
| | - Jodi Segal
- Department of Medicine, School of Medicine, Johns Hopkins University
| | - Daniel Scharfstein
- Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Johns Hopkins University
| | - Michael Rosenblum
- Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Johns Hopkins University
| |
Collapse
|
8
|
Díaz I, Colantuoni E, Hanley DF, Rosenblum M. Improved precision in the analysis of randomized trials with survival outcomes, without assuming proportional hazards. LIFETIME DATA ANALYSIS 2019; 25:439-468. [PMID: 29492746 DOI: 10.1007/s10985-018-9428-5] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/01/2017] [Accepted: 02/18/2018] [Indexed: 06/08/2023]
Abstract
We present a new estimator of the restricted mean survival time in randomized trials where there is right censoring that may depend on treatment and baseline variables. The proposed estimator leverages prognostic baseline variables to obtain equal or better asymptotic precision compared to traditional estimators. Under regularity conditions and random censoring within strata of treatment and baseline variables, the proposed estimator has the following features: (i) it is interpretable under violations of the proportional hazards assumption; (ii) it is consistent and at least as precise as the Kaplan-Meier and inverse probability weighted estimators, under identifiability conditions; (iii) it remains consistent under violations of independent censoring (unlike the Kaplan-Meier estimator) when either the censoring or survival distributions, conditional on covariates, are estimated consistently; and (iv) it achieves the nonparametric efficiency bound when both of these distributions are consistently estimated. We illustrate the performance of our method using simulations based on resampling data from a completed, phase 3 randomized clinical trial of a new surgical treatment for stroke; the proposed estimator achieves a 12% gain in relative efficiency compared to the Kaplan-Meier estimator. The proposed estimator has potential advantages over existing approaches for randomized trials with time-to-event outcomes, since existing methods either rely on model assumptions that are untenable in many applications, or lack some of the efficiency and consistency properties (i)-(iv). We focus on estimation of the restricted mean survival time, but our methods may be adapted to estimate any treatment effect measure defined as a smooth contrast between the survival curves for each study arm. We provide R code to implement the estimator.
Collapse
Affiliation(s)
- Iván Díaz
- Division of Biostatistics and Epidemiology, Weill Cornell Medicine, New York, NY, USA.
| | - Elizabeth Colantuoni
- Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
| | - Daniel F Hanley
- Division of Brain Injury Outcomes, Johns Hopkins Medical Institutions, Baltimore, MD, USA
| | - Michael Rosenblum
- Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
| |
Collapse
|
9
|
Wang B, Ogburn EL, Rosenblum M. Analysis of covariance in randomized trials: More precision and valid confidence intervals, without model assumptions. Biometrics 2019; 75:1391-1400. [DOI: 10.1111/biom.13062] [Citation(s) in RCA: 26] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2018] [Accepted: 03/14/2019] [Indexed: 01/28/2023]
Affiliation(s)
- Bingkai Wang
- Department of Biostatistics Johns Hopkins Bloomberg School of Public Health Baltimore Maryland
| | - Elizabeth L. Ogburn
- Department of Biostatistics Johns Hopkins Bloomberg School of Public Health Baltimore Maryland
| | - Michael Rosenblum
- Department of Biostatistics Johns Hopkins Bloomberg School of Public Health Baltimore Maryland
| |
Collapse
|
10
|
Rosenblum M, Wang B. The Critical Role of Statistical Analyses in Maximizing Power Gains From Covariate-Adaptive Trial Designs. JAMA Netw Open 2019; 2:e190789. [PMID: 30977839 DOI: 10.1001/jamanetworkopen.2019.0789] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Affiliation(s)
- Michael Rosenblum
- Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland
| | - Bingkai Wang
- Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland
| |
Collapse
|
11
|
Sensitivity of adaptive enrichment trial designs to accrual rates, time to outcome measurement, and prognostic variables. Contemp Clin Trials Commun 2017; 8:39-48. [PMID: 29696195 PMCID: PMC5898543 DOI: 10.1016/j.conctc.2017.08.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2016] [Revised: 04/19/2017] [Accepted: 08/11/2017] [Indexed: 11/21/2022] Open
|