Contamination: How much can an individually randomized trial tolerate?

Optimal design of cluster randomized crossover trials with a continuous outcome: Optimal number of time periods and treatment switches under a fixed number of clusters or fixed budget

In the cluster randomized crossover trial, a sequence of treatment conditions, rather than just one treatment condition, is assigned to each cluster. This contribution studies the optimal number of time periods in studies with a treatment switch at the end of each time period, and the optimal number of treatment switches in a trial with a fixed number of time periods. This is done for trials with a fixed number of clusters, and for trials in which the costs per cluster, subject, and treatment switch are taken into account using a budgetary constraint. The focus is on trials with a cross-sectional design where a continuous outcome variable is measured at the end of each time period. An exponential decay correlation structure is used to model dependencies among subjects within the same cluster. A linear multilevel mixed model is used to estimate the treatment effect and its associated variance. The optimal design minimizes this variance. Matrix algebra is used to identify the optimal design and other highly efficient designs. For a fixed number of clusters, a design with the maximum number of time periods is optimal and treatment switches should occur at each time period. However, when a budgetary constraint is taken into account, the optimal design may have fewer time periods and fewer treatment switches. The Shiny app was developed to facilitate the use of the methodology in this contribution.

Evaluating analytic models for individually randomized group treatment trials with complex clustering in nested and crossed designs

Many individually randomized group treatment (IRGT) trials randomly assign individuals to study arms but deliver treatments via shared agents, such as therapists, surgeons, or trainers. Post-randomization interactions induce correlations in outcome measures between participants sharing the same agent. Agents can be nested in or crossed with trial arm, and participants may interact with a single agent or with multiple agents. These complications have led to ambiguity in choice of models but there have been no systematic efforts to identify appropriate analytic models for these study designs. To address this gap, we undertook a simulation study to examine the performance of candidate analytic models in the presence of complex clustering arising from multiple membership, single membership, and single agent settings, in both nested and crossed designs and for a continuous outcome. With nested designs, substantial type I error rate inflation was observed when analytic models did not account for multiple membership and when analytic model weights characterizing the association with multiple agents did not match the data generating mechanism. Conversely, analytic models for crossed designs generally maintained nominal type I error rates unless there was notable imbalance in the number of participants that interact with each agent.

Multilevel Intervention Stepped Wedge Designs (MLI-SWDs)

Multilevel interventions (MLIs) hold promise for reducing health inequities by intervening at multiple types of social determinants of health consistent with the socioecological model of health. In spite of their potential, methodological challenges related to study design compounded by a lack of tools for sample size calculation inhibit their development. We help address this gap by proposing the Multilevel Intervention Stepped Wedge Design (MLI-SWD), a hybrid experimental design which combines cluster-level (CL) randomization using a Stepped Wedge design (SWD) with independent individual-level (IL) randomization. The MLI-SWD is suitable for MLIs where the IL intervention has a low risk of interference between individuals in the same cluster, and it enables estimation of the component IL and CL treatment effects, their interaction, and the combined intervention effect. The MLI-SWD accommodates cross-sectional and cohort designs as well as both incomplete (clusters are not observed in every study period) and complete observation patterns. We adapt recent work using generalized estimating equations for SWD sample size calculation to the multilevel setting and provide an R package for power and sample size calculation. Furthermore, motivated by our experiences with the ongoing NC Works 4 Health study, we consider how to apply the MLI-SWD when individuals join clusters over the course of the study. This situation arises when unemployment MLIs include IL interventions that are delivered while the individual is unemployed. This extension requires carefully considering whether the study interventions will satisfy additional causal assumptions but could permit randomization in new settings.

Challenging assumptions underlying physical activity promotion for health care professionals in Australia: A data-prompted interview study

ISSUE ADDRESSED

Interventions targeting health care professionals' behaviours are assumed to support them in learning how to give behavioural advice to patients, but such assumptions are rarely examined. This study investigated whether key assumptions were held regarding the design and delivery of physical activity interventions among health care professionals in applied health care settings. This study was part of the 'Physical Activity Tailored intervention in Hospital Staff' randomised controlled trial of three variants of a web-based intervention.

METHODS

We used data-prompted interviews to explore whether the interventions were delivered and operated as intended in health care professionals working in four hospitals in Western Australia (N = 25). Data were analysed using codebook thematic analysis.

RESULTS

Five themes were constructed: (1) health care professionals' perceived role in changing patients' health behaviours; (2) work-related barriers to physical activity intervention adherence; (3) health care professionals' use of behaviour change techniques; (4) contamination between groups; and (5) perceptions of intervention tailoring.

CONCLUSIONS

The intervention was not experienced by participants, nor did they implement the intervention guidance, in the way we expected. For example, not all health care professionals felt responsible for providing behaviour change advice, time and shift constraints were key barriers to intervention participation, and contamination effects were difficult to avoid. SO WHAT?: Our study challenges assumptions about how health care professionals respond to behaviour change advice and possible knock-on benefits for patients. Applying our learnings may improve the implementation of health promotion interventions in health care settings.

Cluster Randomized Trials with a Pretest and Posttest: Equivalence of Three-, Two- and One-Level Analyses, and Sample Size Calculation

In a cluster randomized trial clusters of persons, for instance, schools or health centers, are assigned to treatments, and all persons in the same cluster get the same treatment. Although less powerful than individual randomization, cluster randomization is a good alternative if individual randomization is impossible or leads to severe treatment contamination (carry-over). Focusing on cluster randomized trials with a pretest and post-test of a quantitative outcome, this paper shows the equivalence of four methods of analysis: a three-level mixed (multilevel) regression for repeated measures with as levels cluster, person, and time, and allowing for unstructured between-cluster and within-cluster covariance matrices; a two-level mixed regression with as levels cluster and person, using change from baseline as outcome; a two-level mixed regression with as levels cluster and time, using cluster means as data; a one-level analysis of cluster means of change from baseline. Subsequently, similar equivalences are shown between a constrained mixed model and methods using the pretest as covariate. All methods are also compared on a cluster randomized trial on mental health in children. From these equivalences follows a simple method to calculate the sample size for a cluster randomized trial with baseline measurement, which is demonstrated step-by-step.

Why and when should we cluster randomize?

A cluster randomized trial is defined as a randomized trial in which intact social units of individuals are randomized rather than individuals themselves. Outcomes are observed on individual participants within clusters (such as patients). Such a design allows assessing interventions targeting cluster-level participants (such as physicians), individual participants or both. Indeed, many interventions assessed in cluster randomized trials are actually complex ones, with distinct components targeting different levels. For a cluster-level intervention, cluster randomization is an obvious choice: the intervention is not divisible at the individual-level. For individual-level interventions, cluster randomization may nevertheless be suitable to prevent group contamination, for logistical reasons, to enhance participants' adherence, or when objectives pertain to the cluster level. An unacceptable reason for cluster randomization would be to avoid obtaining individual consent. Indeed, participants in cluster randomized trials have to be protected as in any type of trial design. Participants may be people from whom data are collected, but they may also be people who are intervened upon, and this includes both patients and physicians (for example, physicians receiving training interventions). Consent should be sought as soon as possible, although there may exist situations where participants may consent only for data collection, not for being exposed to the intervention (because, for instance, they cannot opt-out). There may even be situations where participants are not able to consent at all. In this latter situation a waiver of consent must be granted by a research ethics committee.

Key considerations for designing, conducting and analysing a cluster randomized trial

Not only do cluster randomized trials require a larger sample size than individually randomized trials, they also face many additional complexities. The potential for contamination is the most commonly used justification for using cluster randomization, but the risk of contamination should be carefully weighed against the more serious problem of questionable scientific validity in settings with post-randomization identification or recruitment of participants unblinded to the treatment allocation. In this paper we provide some simple guidelines to help researchers conduct cluster trials in a way that minimizes potential biases and maximizes statistical efficiency. The overarching theme of this guidance is that methods that apply to individually randomized trials rarely apply to cluster randomized trials. We recommend that cluster randomization be only used when necessary-balancing the benefits of cluster randomization with its increased risks of bias and increased sample size. Researchers should also randomize at the lowest possible level-balancing the risks of contamination with ensuring an adequate number of randomization units-as well as exploring other options for statistically efficient designs. Clustering should always be allowed for in the sample size calculation; and the use of restricted randomization (and adjustment in the analysis for covariates used in the randomization) should be considered. Where possible, participants should be recruited before randomizing clusters and, when recruiting (or identifying) participants post-randomization, recruiters should be masked to the allocation. In the analysis, the target of inference should align with the research question, and adjustment for clustering and small sample corrections should be used when the trial includes less than about 40 clusters.

Optimal allocation of clusters in stepped wedge designs with a decaying correlation structure

The cluster randomized stepped wedge design is a multi-period uni-directional switch design in which all clusters start in the control condition and at the beginning of each new period a random sample of clusters crosses over to the intervention condition. Such designs often use uniform allocation, with an equal number of clusters at each treatment switch. However, the uniform allocation is not necessarily the most efficient. This study derives the optimal allocation of clusters to treatment sequences in the cluster randomized stepped wedge design, for both cohort and cross-sectional designs. The correlation structure is exponential decay, meaning the correlation decreases with the time lag between two measurements. The optimal allocation is shown to depend on the intraclass correlation coefficient, the number of subjects per cluster-period and the cluster and (in the case of a cohort design) individual autocorrelation coefficients. For small to medium values of these autocorrelations those sequences that have their treatment switch earlier or later in the study are allocated a larger proportion of clusters than those clusters that have their treatment switch halfway the study. When the autocorrelation coefficients increase, the clusters become more equally distributed across the treatment sequences. For the cohort design, the optimal allocation is almost equal to the uniform allocation when both autocorrelations approach the value 1. For almost all scenarios that were studied, the efficiency of the uniform allocation is 0.8 or higher. R code to derive the optimal allocation is available online.

A randomized clinical trial assessing the effect of automated medication-targeted alerts on acute kidney injury outcomes

Acute kidney injury is common among hospitalized individuals, particularly those exposed to certain medications, and is associated with substantial morbidity and mortality. In a pragmatic, open-label, National Institutes of Health-funded, parallel group randomized controlled trial (clinicaltrials.gov NCT02771977), we investigate whether an automated clinical decision support system affects discontinuation rates of potentially nephrotoxic medications and improves outcomes in patients with AKI. Participants included 5060 hospitalized adults with AKI and an active order for any of three classes of medications of interest: non-steroidal anti-inflammatory drugs, renin-angiotensin-aldosterone system inhibitors, or proton pump inhibitors. Within 24 hours of randomization, a medication of interest was discontinued in 61.1% of the alert group versus 55.9% of the usual care group (relative risk 1.08, 1.04 - 1.14, p = 0.0003). The primary outcome - a composite of progression of acute kidney injury, dialysis, or death within 14 days - occurred in 585 (23.1%) of individuals in the alert group and 639 (25.3%) of patients in the usual care group (RR 0.92, 0.83 - 1.01, p = 0.09). Trial Registration Clinicaltrials.gov NCT02771977.

Randomization, design and analysis for interdependency in aging research: no person or mouse is an island

Investigators traditionally use randomized designs and corresponding analysis procedures to make causal inferences about the effects of interventions, assuming independence between an individual's outcome and treatment assignment and the outcomes of other individuals in the study. Often, such independence may not hold. We provide examples of interdependency in model organism studies and human trials and group effects in aging research and then discuss methodologic issues and solutions. We group methodologic issues as they pertain to (1) single-stage individually randomized trials; (2) cluster-randomized controlled trials; (3) pseudo-cluster-randomized trials; (4) individually randomized group treatment; and (5) two-stage randomized designs. Although we present possible strategies for design and analysis to improve the rigor, accuracy and reproducibility of the science, we also acknowledge real-world constraints. Consequences of nonadherence, differential attrition or missing data, unintended exposure to multiple treatments and other practical realities can be reduced with careful planning, proper study designs and best practices.

A Review of the Ring Trial Design for Evaluating Ring Interventions for Infectious Diseases

In trials of infectious disease interventions, rare outcomes and unpredictable spatiotemporal variation can introduce bias, reduce statistical power, and prevent conclusive inferences. Spillover effects can complicate inference if individual randomization is used to gain efficiency. Ring trials are a type of cluster-randomized trial that may increase efficiency and minimize bias, particularly in emergency and elimination settings with strong clustering of infection. They can be used to evaluate ring interventions, which are delivered to individuals in proximity to or contact with index cases. We conducted a systematic review of ring trials, compare them with other trial designs for evaluating ring interventions, and describe strengths and weaknesses of each design. Of 849 articles and 322 protocols screened, we identified 26 ring trials, 15 cluster-randomized trials, 5 trials that randomized households or individuals within rings, and 1 individually randomized trial. The most common interventions were postexposure prophylaxis (n = 23) and focal mass drug administration and screening and treatment (n = 7). Ring trials require robust surveillance systems and contact tracing for directly transmitted diseases. For rare diseases with strong spatiotemporal clustering, they may have higher efficiency and internal validity than cluster-randomized designs, in part because they ensure that no clusters are excluded from analysis due to zero cluster incidence. Though more research is needed to compare them with other types of trials, ring trials hold promise as a design that can increase trial speed and efficiency while reducing bias.

Cluster randomized trials of individual-level interventions were at high risk of bias

OBJECTIVES

To describe the prevalence of risks of bias in cluster-randomized trials of individual-level interventions, according to the Cochrane Risk of Bias tool.

STUDY DESIGN AND SETTING

Review undertaken in duplicate of a random sample of 40 primary reports of cluster-randomized trials of individual-level interventions.

RESULTS

The most common reported reasons for adopting cluster randomization were the need to avoid contamination (17, 42.5%) and practical considerations (14, 35%). Of the 40 trials all but one was assessed as being at risk of bias. A majority (27, 67.5%) were assessed as at risk due to the timing of identification and recruitment of participants; many (21, 52.5%) due to an apparent lack of adequate allocation concealment; and many due to selectively reported results (22, 55%), arising from a mixture of reasons including lack of documentation of primary outcome. Other risks mostly occurred infrequently.

CONCLUSION

Many cluster-randomized trials evaluating individual-level interventions appear to be at risk of bias, mostly due to identification and recruitment biases. We recommend that investigators carefully consider the need for cluster randomization; follow recommended procedures to mitigate risks of identification and recruitment bias; and adhere to good reporting practices including clear documentation of primary outcome and allocation concealment methods.