Use of multiple period, cluster randomised, crossover trial designs for comparative effectiveness research

Behavioral carry-over effect and power consideration in crossover trials

A crossover trial is an efficient trial design when there is no carry-over effect. To reduce the impact of the biological carry-over effect, a washout period is often designed. However, the carry-over effect remains an outstanding concern when a washout period is unethical or cannot sufficiently diminish the impact of the carry-over effect. The latter can occur in comparative effectiveness research, where the carry-over effect is often non-biological but behavioral. In this paper, we investigate the crossover design under a potential outcomes framework with and without the carry-over effect. We find that when the carry-over effect exists and satisfies a sign condition, the basic estimator underestimates the treatment effect, which does not inflate the type I error of one-sided tests but negatively impacts the power. This leads to a power trade-off between the crossover design and the parallel-group design, and we derive the condition under which the crossover design does not lead to type I error inflation and is still more powerful than the parallel-group design. We also develop covariate adjustment methods for crossover trials. We evaluate the performance of cross-over design and covariate adjustment using data from the MTN-034/REACH study.

Skin Antisepsis before Surgical Fixation of Extremity Fractures

BACKGROUND

Studies evaluating surgical-site infection have had conflicting results with respect to the use of alcohol solutions containing iodine povacrylex or chlorhexidine gluconate as skin antisepsis before surgery to repair a fractured limb (i.e., an extremity fracture).

METHODS

In a cluster-randomized, crossover trial at 25 hospitals in the United States and Canada, we randomly assigned hospitals to use a solution of 0.7% iodine povacrylex in 74% isopropyl alcohol (iodine group) or 2% chlorhexidine gluconate in 70% isopropyl alcohol (chlorhexidine group) as preoperative antisepsis for surgical procedures to repair extremity fractures. Every 2 months, the hospitals alternated interventions. Separate populations of patients with either open or closed fractures were enrolled and included in the analysis. The primary outcome was surgical-site infection, which included superficial incisional infection within 30 days or deep incisional or organ-space infection within 90 days. The secondary outcome was unplanned reoperation for fracture-healing complications.

RESULTS

A total of 6785 patients with a closed fracture and 1700 patients with an open fracture were included in the trial. In the closed-fracture population, surgical-site infection occurred in 77 patients (2.4%) in the iodine group and in 108 patients (3.3%) in the chlorhexidine group (odds ratio, 0.74; 95% confidence interval [CI], 0.55 to 1.00; P = 0.049). In the open-fracture population, surgical-site infection occurred in 54 patients (6.5%) in the iodine group and in 60 patients (7.3%) in the chlorhexidine group (odd ratio, 0.86; 95% CI, 0.58 to 1.27; P = 0.45). The frequencies of unplanned reoperation, 1-year outcomes, and serious adverse events were similar in the two groups.

CONCLUSIONS

Among patients with closed extremity fractures, skin antisepsis with iodine povacrylex in alcohol resulted in fewer surgical-site infections than antisepsis with chlorhexidine gluconate in alcohol. In patients with open fractures, the results were similar in the two groups. (Funded by the Patient-Centered Outcomes Research Institute and the Canadian Institutes of Health Research; PREPARE ClinicalTrials.gov number, NCT03523962.).

Innovative designs for trials informing the care of cardiac surgical patients: Part II

PURPOSE OF REVIEW

This review examines advances in clinical trial methodologies relevant to cardiac anesthesia. It focuses on innovative approaches, including factorial randomization, composite outcomes, and cluster randomized trials, which enhance the efficiency, practical relevance, and generalizability of trial outcomes.

RECENT FINDINGS

Factorial randomization is becoming popular because the approach allows investigators to simultaneously evaluate two or more interventions. Furthermore, factorial designs can evaluate interactions among treatments which is highly relevant information that cannot be obtained from separate trials. Composite outcomes are also increasingly utilized, combining multiple individual outcomes into a single measure, which increases statistical power and can better represent relevant physiology. Designing valid composites requires careful consideration of component outcome severity and incidence. Cluster randomized trials, including stepped wedge and multiple crossover designs, address the challenges of group-level effects and shared environments.

SUMMARY

The evolution of clinical trial designs is marked by a shift towards methodologies that enhance efficiency and provide more nuanced insights into treatment effects. These include factorial designs for simultaneous intervention assessment, composite outcomes for comprehensive physiological representation, and cluster trials for group-level effect analysis. Such advancements are shaping the future of clinical research, making it more relevant, efficient, and broadly applicable.

Is a preoperative multidisciplinary team meeting (cost)effective to improve outcome for high-risk adult patients undergoing noncardiac surgery: the PREPARATION study-a multicenter stepped-wedge cluster randomized trial

BACKGROUND

As a result of increased life expectancy and improved care for patients suffering from chronic disease, the number of patients with multimorbidity requiring surgical intervention is increasing. For complex surgical patients, it is essential to balance the potential benefits of surgical treatment against the risk of permanent loss of functional capacity and quality of life due to complications. European and US guidelines on perioperative care recommend preoperative multidisciplinary team (MDT) discussions for high-risk noncardiac surgical patients. However, the evidence underlying benefits from preoperative MDT meetings with all relevant perioperative specialties present is limited. The current study aims to investigate the effect of implementation of preoperative MDT discussions for high-risk patients undergoing noncardiac surgery on serious adverse events.

METHODS/DESIGN

PREPARATION is a stepped-wedge cluster randomized trial in 14 Dutch hospitals without currently established preoperative MDT meeting. The intervention, preoperative MDT meetings, will be implemented sequentially with seven blocks of 2 hospitals switching from control (preoperative screening as usual) to the intervention every 3 months. Each hospital will be randomized to one of seven blocks. We aim to include 1200 patients. The primary outcome is the incidence of serious adverse events at 6 months. Secondary outcomes include (cost)effectiveness, functional outcome, and quality of life for up to 12 months.

DISCUSSION

PREPARATION is the first study to assess the effectiveness of a preoperative MDT meeting for high-risk noncardiac surgical patients in the presence of an anesthesiologist. If the results suggest that preoperative MDT discussions for high-risk patients are (cost)-effective, the current study facilitates implementation of preoperative MDT meetings in clinical practice.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05703230. Registered on 11/09/2022.

Estimating intra-cluster correlation coefficients for planning longitudinal cluster randomized trials: a tutorial

It is well-known that designing a cluster randomized trial (CRT) requires an advance estimate of the intra-cluster correlation coefficient (ICC). In the case of longitudinal CRTs, where outcomes are assessed repeatedly in each cluster over time, estimates for more complex correlation structures are required. Three common types of correlation structures for longitudinal CRTs are exchangeable, nested/block exchangeable and exponential decay correlations-the latter two allow the strength of the correlation to weaken over time. Determining sample sizes under these latter two structures requires advance specification of the within-period ICC and cluster autocorrelation coefficient as well as the intra-individual autocorrelation coefficient in the case of a cohort design. How to estimate these coefficients is a common challenge for investigators. When appropriate estimates from previously published longitudinal CRTs are not available, one possibility is to re-analyse data from an available trial dataset or to access observational data to estimate these parameters in advance of a trial. In this tutorial, we demonstrate how to estimate correlation parameters under these correlation structures for continuous and binary outcomes. We first introduce the correlation structures and their underlying model assumptions under a mixed-effects regression framework. With practical advice for implementation, we then demonstrate how the correlation parameters can be estimated using examples and we provide programming code in R, SAS, and Stata. An Rshiny app is available that allows investigators to upload an existing dataset and obtain the estimated correlation parameters. We conclude by identifying some gaps in the literature.

FLUID trial: a hospital-wide open-label cluster cross-over pragmatic comparative effectiveness randomised pilot trial comparing normal saline to Ringer's lactate

OBJECTIVES

Normal saline (NS) and Ringer's lactate (RL) are the most common crystalloids used for fluid therapy. Despite evidence of possible harm associated with NS (eg, hyperchloremic metabolic acidosis, impaired kidney function and death), few large multi-centre randomised trials have evaluated the effect of these fluids on clinically important outcomes. We conducted a pilot trial to explore the feasibility of a large trial powered for clinically important outcomes.

DESIGN

FLUID was a pragmatic pilot cluster randomised cross-over trial.

SETTING

Four hospitals in the province of Ontario, Canada PARTICIPANTS: All hospitalised adult and paediatric patients with an incident admission to the hospital over the course of each study period.

INTERVENTIONS

A hospital wide policy/strategy which stocked either NS or RL throughout the hospital for 12 weeks before crossing over to the alternate fluid for the subsequent 12 weeks.

PRIMARY AND SECONDARY OUTCOME MEASURES

The primary feasibility outcome was study fluid protocol adherence. Secondary feasibility outcomes included time to Research Ethics Board (REB) approval and trial initiation. Primary (composite of death or re-admission to hospital in first 90 days of index hospitalisation) and secondary clinical outcomes were analysed descriptively.

RESULTS

Among 24 905 included patients, mean age 59.1 (SD 20.5); 13 977 (56.1%) were female and 21 150 (85.0%) had medical or surgical admitting diagnoses. Overall, 96 821 L were administered in the NS arm, and 78 348 L in the RL arm. Study fluid adherence to NS and RL was 93.7% (site range: 91.6%-98.0%) and 79.8% (site range: 72.5%-83.9%), respectively. Time to REB approval ranged from 2 to 48 days and readiness for trial initiation from 51 to 331 days. 5544 (22.3%) patients died or required hospital re-admission in the first 90 days.

CONCLUSIONS

The future large trial is feasible. Anticipating and addressing logistical challenges during the planning stages will be imperative.

TRIAL REGISTRATION NUMBER

NCT02721485.

Analysis of multiple-period group randomized trials: random coefficients model or repeated measures ANOVA?

BACKGROUND

Multiple-period parallel group randomized trials (GRTs) analyzed with linear mixed models can represent time in mean models as continuous or categorical. If time is continuous, random effects are traditionally group- and member-level deviations from condition-specific slopes and intercepts and are referred to as random coefficients (RC) analytic models. If time is categorical, random effects are traditionally group- and member-level deviations from time-specific condition means and are referred to as repeated measures ANOVA (RM-ANOVA) analytic models. Longstanding guidance recommends the use of RC over RM-ANOVA for parallel GRTs with more than two periods because RC exhibited nominal type I error rates for both time parameterizations while RM-ANOVA exhibited inflated type I error rates when applied to data generated using the RC model. However, this recommendation was developed assuming a variance components covariance matrix for the RM-ANOVA, using only cross-sectional data, and explicitly modeling time × group variation. Left unanswered were how well RM-ANOVA with an unstructured covariance would perform on data generated according to the RC mechanism, if similar patterns would be observed in cohort data, and the impact of not modeling time × group variation if such variation was present in the data-generating model.

METHODS

Continuous outcomes for cohort and cross-sectional parallel GRT data were simulated according to RM-ANOVA and RC mechanisms at five total time periods. All simulations assumed time × group variation. We varied the number of groups, group size, and intra-cluster correlation. Analytic models using RC, RM-ANOVA, RM-ANOVA with unstructured covariance, and a Saturated random effects structure were applied to the data. All analytic models specified time × group random effects. The analytic models were then reapplied without specifying random effects for time × group.

RESULTS

Results indicated the RC and saturated analytic models maintained the nominal type I error rate in all data sets, RM-ANOVA with an unstructured covariance did not avoid type I error rate inflation when applied to cohort RC data, and analytic models omitting time-varying group random effects when such variation exists in the data were prone to substantial type I error inflation unless the residual error variance is high relative to the time × group variance.

CONCLUSION

The time × group RC and saturated analytic models are recommended as the default for multiple period parallel GRTs.

A pragmatic randomized trial evaluating pre-operative aqueous antiseptic skin solutions in open fractures (Aqueous-PREP): statistical analysis plan

Background

Approximately 1 in 10 patients with a surgically treated open fracture will develop a surgical site infection. The Aqueous-PREP trial will investigate the effect of 10% povidone-iodine versus 4% chlorhexidine in aqueous antiseptic solutions in reducing infections after open fracture surgery. The study protocol was published in April 2020.

Methods and design

The Aqueous-PREP trial is a pragmatic, multicenter, open-label, randomized multiple period cluster crossover trial. Each participating cluster is randomly assigned in a 1:1 ratio to provide 1 of the 2 study interventions on all eligible patients during a study period. The intervention periods are 2 months in length. After completing a 2-month period, the participating cluster crosses over to the alternative intervention. We plan to enroll a minimum of 1540 patients at 14 sites.

Results

The primary outcome is surgical site infection guided by the Centers for Disease Control and Prevention’s National Healthcare Safety Network reporting criteria (2017). All participants’ surgical site infection surveillance period will end 30 days after definitive fracture management surgery for superficial infections and 90 days after definitive fracture management surgery for deep incisional or organ/space infections [1]. The secondary outcome is an unplanned fracture-related reoperation within 12 months of the fracture.

Conclusion

This manuscript serves as the formal statistical analysis plan (version 1.0) for the Aqueous-PREP trial. The statistical analysis plan was completed on February 28, 2022.

Treatment Patterns and Characteristics of Dialysis Facilities Randomly Assigned to the Medicare End-Stage Renal Disease Treatment Choices Model

IMPORTANCE

In 2021, Medicare launched the End-Stage Renal Disease Treatment Choices (ETC) model, which randomly assigned approximately 30% of dialysis facilities to new financial incentives to increase use of transplantation and home dialysis; these financial bonuses and penalties are calculated by comparing living-donor transplantation, transplant wait-listing, and home dialysis use in ETC-assigned facilities vs benchmarks from non-ETC-assigned (ie, control) facilities. Because model participation is randomly assigned, evaluators may attribute any downstream differences in outcomes to facility performance rather than any imbalance in baseline characteristics.

OBJECTIVE

To identify preintervention imbalances in dialysis facility characteristics that should be recognized in any ETC model evaluations.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional study compared ETC-assigned and control dialysis facility characteristics in the United States from 2017 to 2018. A total of 6062 facilities were included. Data were analyzed from February 2021 to May 2022.

EXPOSURES

Assignment to the ETC model.

MAIN OUTCOMES AND MEASURES

Dialysis facilities' preintervention transplantations and home dialysis use, facility characteristics (notably, profit and chain status), patient demographic characteristics, and community socioeconomic characteristics.

RESULTS

Among 316 927 patients, with 6 178 855 attributed patient-months, the mean (SD) age in January 2017 was 59 (11) years, and 132 462 (42%) were female. Patients in ETC-assigned facilities had 9% (0.2 [95% CI, 0.1-0.2] percentage points) lower prevalence of living donor transplantation, 12% (3.2 [95% CI, 3.0-3.3] percentage points) lower prevalence of transplantation wait-listing, and 4% (0.4 [95% CI, 0.3-0.4] percentage points) lower prevalence of peritoneal dialysis use compared with control facilities. ETC-assigned facilities were 14% (5.1 [95% CI, 0.9-9.4] percentage points) more likely than control facilities to be owned by the second largest dialysis organization. Relative to control facilities, ETC-assigned facilities also treated 34% (6.6 [95% CI, 6.5-6.7] percentage point) fewer patients with Hispanic ethnicity and were located in communities with median household incomes that were 4% ($2500; 95% CI, $500-$4500) lower on average.

CONCLUSIONS AND RELEVANCE

In this study, dialysis facilities in ETC-assigned regions had lower preintervention prevalence of transplantation wait-listing, living donor transplantation, and peritoneal dialysis use, relative to control facilities. ETC-assigned and control facilities also differed with respect to other facility, patient, and community characteristics. Evaluators should account for these preintervention imbalances to minimize bias in their inferences about the model's association with postintervention outcomes.

Methods for dealing with unequal cluster sizes in cluster randomized trials: A scoping review

In a cluster-randomized trial (CRT), the number of participants enrolled often varies across clusters. This variation should be considered during both trial design and data analysis to ensure statistical performance goals are achieved. Most methodological literature on the CRT design has assumed equal cluster sizes. This scoping review focuses on methodology for unequal cluster size CRTs. EMBASE, Medline, Google Scholar, MathSciNet and Web of Science databases were searched to identify English-language articles reporting on methodology for unequal cluster size CRTs published until March 2021. We extracted data on the focus of the paper (power calculation, Type I error etc.), the type of CRT, the type and the range of parameter values investigated (number of clusters, mean cluster size, cluster size coefficient of variation, intra-cluster correlation coefficient, etc.), and the main conclusions. Seventy-nine of 5032 identified papers met the inclusion criteria. Papers primarily focused on the parallel-arm CRT (p-CRT, n = 60, 76%) and the stepped-wedge CRT (n = 14, 18%). Roughly 75% of the papers addressed trial design issues (sample size/power calculation) while 25% focused on analysis considerations (Type I error, bias, etc.). The ranges of parameter values explored varied substantially across different studies. Methods for accounting for unequal cluster sizes in the p-CRT have been investigated extensively for Gaussian and binary outcomes. Synthesizing the findings of these works is difficult as the magnitude of impact of the unequal cluster sizes varies substantially across the combinations and ranges of input parameters. Limited investigations have been done for other combinations of a CRT design by outcome type, particularly methodology involving binary outcomes-the most commonly used type of primary outcome in trials. The paucity of methodological papers outside of the p-CRT with Gaussian or binary outcomes highlights the need for further methodological development to fill the gaps.

Pilot and feasibility studies for pragmatic trials have unique considerations and areas of uncertainty

BACKGROUND AND OBJECTIVE

Feasibility studies are increasingly being used to support the development of, and investigate uncertainties around, future large-scale trials. The future trial can be designed with either a pragmatic or explanatory mindset. Whereas pragmatic trials aim to inform the choice between different care options and thus, are designed to resemble conditions outside of a clinical trial environment, explanatory trials examine the benefit of a treatment under more controlled conditions. There is existing guidance for designing feasibility studies, but none that explicitly considers the goals of pragmatic designs. We aimed to identify unique areas of uncertainty that are relevant to planning a pragmatic trial.

RESULTS

We identified ten relevant domains, partly based on the pragmatic-explanatory continuum indicator summary-2 (PRECIS-2) framework, and describe potential questions of uncertainty within each: intervention development, research ethics, participant identification and eligibility, recruitment of individuals, setting, organization, flexibility of delivery, flexibility of adherence, follow-up, and importance of primary outcome to patients and decision-makers. We present examples to illustrate how uncertainty in these domains might be addressed within a feasibility study.

CONCLUSION

Researchers planning a feasibility study in advance of a pragmatic trial should consider feasibility objectives specifically relevant to areas of uncertainty for pragmatic trials.