Optimal design of clinical trials with computer simulation based on results of earlier trials, illustrated with a lipodystrophy trial in HIV patients

Using simulation modeling to inform intervention and implementation selection in a rapid stakeholder-engaged hybrid effectiveness-implementation randomized trial

BACKGROUND

Implementation research generally assumes established evidence-based practices and prior piloting of implementation strategies, which may not be feasible during a public health emergency. We describe the use of a simulation model of the effectiveness of COVID-19 mitigation strategies to inform a stakeholder-engaged process of rapidly designing a tailored intervention and implementation strategy for individuals with serious mental illness (SMI) and intellectual/developmental disabilities (ID/DD) in group homes in a hybrid effectiveness-implementation randomized trial.

METHODS

We used a validated dynamic microsimulation model of COVID-19 transmission and disease in late 2020/early 2021 to determine the most effective strategies to mitigate infections among Massachusetts group home staff and residents. Model inputs were informed by data from stakeholders, public records, and published literature. We assessed different prevention strategies, iterated over time with input from multidisciplinary stakeholders and pandemic evolution, including varying symptom screening, testing frequency, isolation, contact-time, use of personal protective equipment, and vaccination. Model outcomes included new infections in group home residents, new infections in group home staff, and resident hospital days. Sensitivity analyses were performed to account for parameter uncertainty. Results of the simulations informed a stakeholder-engaged process to select components of a tailored best practice intervention and implementation strategy.

RESULTS

The largest projected decrease in infections was with initial vaccination, with minimal benefit for additional routine testing. The initial level of actual vaccination in the group homes was estimated to reduce resident infections by 72.4% and staff infections by 55.9% over the 90-day time horizon. Increasing resident and staff vaccination uptake to a target goal of 90% further decreased resident infections by 45.2% and staff infections by 51.3%. Subsequent simulated removal of masking led to a 6.5% increase in infections among residents and 3.2% among staff. The simulation model results were presented to multidisciplinary stakeholders and policymakers to inform the "Tailored Best Practice" package for the hybrid effectiveness-implementation trial.

CONCLUSIONS

Vaccination and decreasing vaccine hesitancy among staff were predicted to have the greatest impact in mitigating COVID-19 risk in vulnerable populations of group home residents and staff. Simulation modeling was effective in rapidly informing the selection of the prevention and implementation strategy in a hybrid effectiveness-implementation trial. Future implementation may benefit from this approach when rapid deployment is necessary in the absence of data on tailored interventions.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04726371.

Combining Model-Based Clinical Trial Simulation, Pharmacoeconomics, and Value of Information to Optimize Trial Design

The Bayesian decision‐analytic approach to trial design uses prior distributions for treatment effects, updated with likelihoods for proposed trial data. Prior distributions for treatment effects based on previous trial results risks sample selection bias and difficulties when a proposed trial differs in terms of patient characteristics, medication adherence, or treatment doses and regimens. The aim of this study was to demonstrate the utility of using pharmacometric‐based clinical trial simulation (CTS) to generate prior distributions for use in Bayesian decision‐theoretic trial design. The methods consisted of four principal stages: a CTS to predict the distribution of treatment response for a range of trial designs; Bayesian updating for a proposed sample size; a pharmacoeconomic model to represent the perspective of a reimbursement authority in which price is contingent on trial outcome; and a model of the pharmaceutical company return on investment linking drug prices to sales revenue. We used a case study of febuxostat versus allopurinol for the treatment of hyperuricemia in patients with gout. Trial design scenarios studied included alternative treatment doses, inclusion criteria, input uncertainty, and sample size. Optimal trial sample sizes varied depending on the uncertainty of model inputs, trial inclusion criteria, and treatment doses. This interdisciplinary framework for trial design and sample size calculation may have value in supporting decisions during later phases of drug development and in identifying costly sources of uncertainty, and thus inform future research and development strategies.

Simulation methods to estimate design power: an overview for applied research

BACKGROUND

Estimating the required sample size and statistical power for a study is an integral part of study design. For standard designs, power equations provide an efficient solution to the problem, but they are unavailable for many complex study designs that arise in practice. For such complex study designs, computer simulation is a useful alternative for estimating study power. Although this approach is well known among statisticians, in our experience many epidemiologists and social scientists are unfamiliar with the technique. This article aims to address this knowledge gap.

METHODS

We review an approach to estimate study power for individual- or cluster-randomized designs using computer simulation. This flexible approach arises naturally from the model used to derive conventional power equations, but extends those methods to accommodate arbitrarily complex designs. The method is universally applicable to a broad range of designs and outcomes, and we present the material in a way that is approachable for quantitative, applied researchers. We illustrate the method using two examples (one simple, one complex) based on sanitation and nutritional interventions to improve child growth.

RESULTS

We first show how simulation reproduces conventional power estimates for simple randomized designs over a broad range of sample scenarios to familiarize the reader with the approach. We then demonstrate how to extend the simulation approach to more complex designs. Finally, we discuss extensions to the examples in the article, and provide computer code to efficiently run the example simulations in both R and Stata.

CONCLUSIONS

Simulation methods offer a flexible option to estimate statistical power for standard and non-traditional study designs and parameters of interest. The approach we have described is universally applicable for evaluating study designs used in epidemiologic and social science research.