Stepped wedge cluster randomized controlled trial designs: a review of reporting quality and design features

Efficient designs for three-sequence stepped wedge trials with continuous recruitment

BACKGROUND/AIMS

The standard approach to designing stepped wedge trials that recruit participants in a continuous stream is to divide time into periods of equal length. But the choice of design in such cases is infinitely more flexible: each cluster could cross from the control to the intervention at any point on the continuous time-scale. We consider the case of a stepped wedge design with clusters randomised to just three sequences (designs with small numbers of sequences may be preferred for their simplicity and practicality) and investigate the choice of design that minimises the variance of the treatment effect estimator under different assumptions about the intra-cluster correlation.

METHODS

We make some simplifying assumptions in order to calculate the variance: in particular that we recruit the same number of participants, m , from each cluster over the course of the trial, and that participants present at regularly spaced intervals. We consider an intra-cluster correlation that decays exponentially with separation in time between the presentation of two individuals from the same cluster, from a value of ρ for two individuals who present at the same time, to a value of ρ τ for individuals presenting at the start and end of the trial recruitment interval. We restrict attention to three-sequence designs with centrosymmetry - the property that if we reverse time and swap the intervention and control conditions then the design looks the same. We obtain an expression for the variance of the treatment effect estimator adjusted for effects of time, using methods for generalised least squares estimation, and we evaluate this expression numerically for different designs, and for different parameter values.

RESULTS

There is a two-dimensional space of possible three-sequence, centrosymmetric stepped wedge designs with continuous recruitment. The variance of the treatment effect estimator for given ρ and τ can be plotted as a contour map over this space. The shape of this variance surface depends on τ and on the parameter m ρ / ( 1 - ρ ) , but typically indicates a broad, flat region of close-to-optimal designs. The 'standard' design with equally spaced periods and 1:1:1 allocation rarely performs well, however.

CONCLUSIONS

In many different settings, a relatively simple design can be found (e.g. one based on simple fractions) that offers close-to-optimal efficiency in that setting. There may also be designs that are robustly efficient over a wide range of settings. Contour maps of the kind we illustrate can help guide this choice. If efficiency is offered as one of the justifications for using a stepped wedge design, then it is worth designing with optimal efficiency in mind.

Using Implementation Mapping to develop an intervention program to support veterinarians' adherence to the guideline on Streptococcus suis clinical practice in weaned pigs

Streptococcus suis (S. suis) infections in weaned pigs are common and responsible for a high consumption of antimicrobials, and their presence is assumed to be multi-factorial. A specific evidence-based veterinary guideline to support the control of S. suis in weaned pigs was developed for veterinary practitioners in the Netherlands in 2014. Adherence to the S. suis clinical practice guideline helps veterinary practitioners to prevent and control the disease in a systematical approach and thereby improve antimicrobial stewardship and contribute to the prevention of antimicrobial resistance in animals and humans. The impact of such a clinical practice guideline on (animal) disease management depends not only on its content, but also largely on the extent to which practitioners adhere to the clinical guideline in practice. When the S. suis guideline was published, no specific activities were undertaken to support veterinarians' uptake and implementation, thereby contributing to suboptimal adherence in clinical practice. As the S. suis guideline was comprehensively written by veterinary experts following an evidence-based approach, our aim was not to judge the (scientific) quality of the guideline but to study the possibility to improve the currently low adherence of this guideline in veterinary practice. This paper describes the systematic development, using Implementation Mapping, of a theory-based intervention program to support swine veterinarians' adherence to the S. suis guideline. The knowledge, skills, beliefs about capabilities, and beliefs about consequences domains are addressed in the program, which includes seven evidence-based methods (modelling, tailoring, feedback, discussion, persuasive communication, active learning, and self-monitoring) for use in program activities such as a peer-learning meeting and an e-learning module. The intervention program has been developed for practicing swine veterinarians, lasts eight months, and is evaluated through a stepped-wedge design. The Implementation Mapping approach ensured that all relevant adopters and implementers were involved, and that outcomes, determinants (influencing factors), and objectives were systematically discussed.

Adherence to key recommendations for design and analysis of stepped-wedge cluster randomized trials: A review of trials published 2016-2022

BACKGROUND/AIMS

The stepped-wedge cluster randomized trial (SW-CRT), in which clusters are randomized to a time at which they will transition to the intervention condition - rather than a trial arm - is a relatively new design. SW-CRTs have additional design and analytical considerations compared to conventional parallel arm trials. To inform future methodological development, including guidance for trialists and the selection of parameters for statistical simulation studies, we conducted a review of recently published SW-CRTs. Specific objectives were to describe (1) the types of designs used in practice, (2) adherence to key requirements for statistical analysis, and (3) practices around covariate adjustment. We also examined changes in adherence over time and by journal impact factor.

METHODS

We used electronic searches to identify primary reports of SW-CRTs published 2016-2022. Two reviewers extracted information from each trial report and its protocol, if available, and resolved disagreements through discussion.

RESULTS

We identified 160 eligible trials, randomizing a median (Q1-Q3) of 11 (8-18) clusters to 5 (4-7) sequences. The majority (122, 76%) were cross-sectional (almost all with continuous recruitment), 23 (14%) were closed cohorts and 15 (9%) open cohorts. Many trials had complex design features such as multiple or multivariate primary outcomes (50, 31%) or time-dependent repeated measures (27, 22%). The most common type of primary outcome was binary (51%); continuous outcomes were less common (26%). The most frequently used method of analysis was a generalized linear mixed model (112, 70%); generalized estimating equations were used less frequently (12, 8%). Among 142 trials with fewer than 40 clusters, only 9 (6%) reported using methods appropriate for a small number of clusters. Statistical analyses clearly adjusted for time effects in 119 (74%), for within-cluster correlations in 132 (83%), and for distinct between-period correlations in 13 (8%). Covariates were included in the primary analysis of the primary outcome in 82 (51%) and were most often individual-level covariates; however, clear and complete pre-specification of covariates was uncommon. Adherence to some key methodological requirements (adjusting for time effects, accounting for within-period correlation) was higher among trials published in higher versus lower impact factor journals. Substantial improvements over time were not observed although a slight improvement was observed in the proportion accounting for a distinct between-period correlation.

CONCLUSIONS

Future methods development should prioritize methods for SW-CRTs with binary or time-to-event outcomes, small numbers of clusters, continuous recruitment designs, multivariate outcomes, or time-dependent repeated measures. Trialists, journal editors, and peer reviewers should be aware that SW-CRTs have additional methodological requirements over parallel arm designs including the need to account for period effects as well as complex intracluster correlations.

A Bayesian adaptive design approach for stepped-wedge cluster randomized trials

BACKGROUND

The Bayesian group sequential design has been applied widely in clinical studies, especially in Phase II and III studies. It allows early termination based on accumulating interim data. However, to date, there lacks development in its application to stepped-wedge cluster randomized trials, which are gaining popularity in pragmatic trials conducted by clinical and health care delivery researchers.

METHODS

We propose a Bayesian adaptive design approach for stepped-wedge cluster randomized trials, which makes adaptive decisions based on the predictive probability of declaring the intervention effective at the end of study given interim data. The Bayesian models and the algorithms for posterior inference and trial conduct are presented.

RESULTS

We present how to determine design parameters through extensive simulations to achieve desired operational characteristics. We further evaluate how various design factors, such as the number of steps, cluster size, random variability in cluster size, and correlation structures, impact trial properties, including power, type I error, and the probability of early stopping. An application example is presented.

CONCLUSION

This study presents the incorporation of Bayesian adaptive strategies into stepped-wedge cluster randomized trials design. The proposed approach provides the flexibility to stop the trial early if substantial evidence of efficacy or futility is observed, improving the flexibility and efficiency of stepped-wedge cluster randomized trials.

Implementation of Electronic Health Record Integration and Clinical Decision Support to Improve Emergency Department Prescription Drug Monitoring Program Use

STUDY OBJECTIVE(S)

To evaluate the implementation of 3 electronic health record (EHR)-based interventions to increase prescription drug monitoring program (PDMP) use in the emergency department (ED): EHR-PDMP integration, addition of a PDMP risk score, and addition of EHR-based clinical decision support alert to review the PDMP when prescribing an opioid.

METHODS

Three intervention stages were implemented using a prospective stepped-wedge design at 5 university-affiliated EDs split into 3 practice groups. The PDMP use and prescribing rates during the 3 stages were compared with baseline before EHR integration and a sustainability stage where the clinical decision support alert was removed, but EHR integration and risk score remained. Generalized linear mixed model with logit link function and a random intercept for clinicians was analyzed.

RESULTS

The ED provider PDMP review before opioid prescribing was low in all stages. The highest review rate occurred during interruptive clinical decision support alerts, 23.8% (interquartile range 10.6 to 37.5). Overall, opioid prescribing declined, and PDMP review was not associated with a decrease in opioid prescribing. PDMP review was associated with a reduction in the probability of prescribing an opioid as the number of prior opioid prescriptions increased (odds ratio: 0.92 [95% confidence interval: 0.91 to 0.94] for every additional prescription).

CONCLUSION

The EHR-PDMP integration did not increase PDMP use in the ED, but a PDMP risk score and a clinical decision support alert were associated with modest increases in the probability of PDMP review. When the PDMP is reviewed, ED clinicians are less likely to prescribe opioids to patients with a high number of prior opioid prescriptions.

Assessing exposure-time treatment effect heterogeneity in stepped-wedge cluster randomized trials

A stepped-wedge cluster randomized trial (CRT) is a unidirectional crossover study in which timings of treatment initiation for clusters are randomized. Because the timing of treatment initiation is different for each cluster, an emerging question is whether the treatment effect depends on the exposure time, namely, the time duration since the initiation of treatment. Existing approaches for assessing exposure-time treatment effect heterogeneity either assume a parametric functional form of exposure time or model the exposure time as a categorical variable, in which case the number of parameters increases with the number of exposure-time periods, leading to a potential loss in efficiency. In this article, we propose a new model formulation for assessing treatment effect heterogeneity over exposure time. Rather than a categorical term for each level of exposure time, the proposed model includes a random effect to represent varying treatment effects by exposure time. This allows for pooling information across exposure-time periods and may result in more precise average and exposure-time-specific treatment effect estimates. In addition, we develop an accompanying permutation test for the variance component of the heterogeneous treatment effect parameters. We conduct simulation studies to compare the proposed model and permutation test to alternative methods to elucidate their finite-sample operating characteristics, and to generate practical guidance on model choices for assessing exposure-time treatment effect heterogeneity in stepped-wedge CRTs.

Integrated Early Childhood Development in Cambodia: Protocol of a Cluster Stepped-Wedge Trial

OBJECTIVES

Limited evidence is available on mechanisms linking integrated, multisector interventions with early childhood development. The Integrated Early Childhood Development program aims to improve children's development by promoting targeted caregiving behaviors beginning prenatally through age 5 years, in partnership with the Royal Government of Cambodia.

METHODS

This cluster stepped-wedge trial is being conducted in Cambodia among 3 cohorts, encompassing 339 villages and 1790 caregivers who are pregnant or caring for a child aged <5 years. The 12- to 15-month intervention is delivered to each cohort using a staggered stepped-wedge design. Among all cohorts, enrollment evaluations will be followed by 3 data collection waves. Targeted caregiving interventions are provided through community, group, and home-visiting platforms. Child development is measured using the Caregiver Reported Early Development Instrument and the Early Childhood Development Index 2030. The evaluation assesses mediation through targeted caregiving behaviors: responsive caregiving, nutrition, health and hygiene, and household stability and support; moderation by household wealth, caregiver education, and child birth weight; and sustainability after the intervention concludes.

CONCLUSIONS

This protocol article describes the plans for a cluster randomized controlled trial to measure the impact of an integrated, multisector intervention on children's development. By partnering with the Royal Government of Cambodia and addressing intervention pathways and moderators, this trial will provide guidance for policies and programs to promote early childhood development using principles of implementation science and equity, including increased investment for vulnerable families.

Sample size considerations for stepped wedge designs with subclusters

The stepped wedge cluster randomized trial (SW-CRT) is an increasingly popular design for evaluating health service delivery or policy interventions. An essential consideration of this design is the need to account for both within-period and between-period correlations in sample size calculations. Especially when embedded in health care delivery systems, many SW-CRTs may have subclusters nested in clusters, within which outcomes are collected longitudinally. However, existing sample size methods that account for between-period correlations have not allowed for multiple levels of clustering. We present computationally efficient sample size procedures that properly differentiate within-period and between-period intracluster correlation coefficients in SW-CRTs in the presence of subclusters. We introduce an extended block exchangeable correlation matrix to characterize the complex dependencies of outcomes within clusters. For Gaussian outcomes, we derive a closed-form sample size expression that depends on the correlation structure only through two eigenvalues of the extended block exchangeable correlation structure. For non-Gaussian outcomes, we present a generic sample size algorithm based on linearization and elucidate simplifications under canonical link functions. For example, we show that the approximate sample size formula under a logistic linear mixed model depends on three eigenvalues of the extended block exchangeable correlation matrix. We provide an extension to accommodate unequal cluster sizes and validate the proposed methods via simulations. Finally, we illustrate our methods in two real SW-CRTs with subclusters.

Power analyses for stepped wedge designs with multivariate continuous outcomes

Multivariate outcomes are common in pragmatic cluster randomized trials. While sample size calculation procedures for multivariate outcomes exist under parallel assignment, none have been developed for a stepped wedge design. In this article, we present computationally efficient power and sample size procedures for stepped wedge cluster randomized trials (SW-CRTs) with multivariate outcomes that differentiate the within-period and between-period intracluster correlation coefficients (ICCs). Under a multivariate linear mixed model, we derive the joint distribution of the intervention test statistics which can be used for determining power under different hypotheses and provide an example using the commonly utilized intersection-union test for co-primary outcomes. Simplifications under a common treatment effect and common ICCs across endpoints and an extension to closed-cohort designs are also provided. Finally, under the common ICC across endpoints assumption, we formally prove that the multivariate linear mixed model leads to a more efficient treatment effect estimator compared to the univariate linear mixed model, providing a rigorous justification on the use of the former with multivariate outcomes. We illustrate application of the proposed methods using data from an existing SW-CRT and present extensive simulations to validate the methods.

A general method for calculating power for GEE analysis of complete and incomplete stepped wedge cluster randomized trials

Stepped wedge designs have uni-directional crossovers at randomly assigned time points (steps) where clusters switch from control to intervention condition. Incomplete stepped wedge designs are increasingly used in cluster randomized trials of health care interventions and have periods without data collection due to logistical, resource and patient-centered considerations. The development of sample size formulae for stepped wedge trials has primarily focused on complete designs and continuous responses. Addressing this gap, a general, fast, non-simulation based power procedure is proposed for generalized estimating equations analysis of complete and incomplete stepped wedge designs and its predicted power is compared to simulated power for binary and continuous responses. An extensive set of simulations for six and twelve clusters is based upon the Connect-Home trial with an incomplete stepped wedge design. Results show that empirical test size is well controlled using a t-test with bias-corrected sandwich variance estimator for as few as six clusters. Analytical power agrees well with a simulated power in scenarios with twelve clusters. For six clusters, analytical power is similar to simulated power with estimation using the correctly specified model-based variance estimator. To explore the impact of study design choice on power, the proposed fast GEE power method is applied to the Connect-Home trial design, four alternative incomplete stepped wedge designs and one complete design.

power swgee: GEE-based power calculations in stepped wedge cluster randomized trials

Stepped wedge cluster randomized trials are increasingly being used to evaluate interventions in medical, public health, educational, and social science contexts. With the longitudinal and crossover nature of a SW-CRT, complex analysis techniques are often needed which makes appropriately powering SW-CRTs challenging. In this paper, we introduce a newly-developed SW-CRT power calculator, embedded within the power command in Stata. The power calculator assumes a marginal model (i.e., generalized estimating equations [GEE]) for the primary analysis of SW-CRTs, for which other currently available SW-CRT power calculators may not be suitable. The program accommodates complete cross-sectional and closed-cohort designs, and includes multilevel correlation structures appropriate for such designs. We discuss the methods and formulae underlying our SW-CRT calculator, and provide illustrative examples of the use of power swgee. We provide suggestions about the choice of parameters in power swgee, and conclude by discussing areas of future research which may improve the program.

Effectiveness of interprofessional shared decision-making training: A mixed-method study

OBJECTIVE

This study evaluated the learning effects and examined the participants' perceptions of an interprofessional shared decision-making (IP-SDM) training program.

METHODS

This mixed-method study used a quasi-experimental pretest-posttest design in the quantitative phase and semi-structured interviews in the qualitative phase. The 6-week curriculum design, based on Kolb's experiential learning cycle, consisted of two simulated objective structured clinical examinations with standardized patients and blended teaching methods through various course modules.

RESULTS

A total of 39 multidisciplinary healthcare personnel completed the 6-week training program, and 32 of them participated in qualitative interviews. The IP-SDM training program effectively improved the SDM process competency of the participants from the perspectives of the participants, standardized patients, and clinical teachers. The interviews illustrated how the curriculum design enhanced learning; the effectiveness results indicated improvements in learners' attitude, knowledge, skills, and teamwork.

CONCLUSION

This IP-SDM training program improved multidisciplinary healthcare personnel's competency, self-efficacy, and intention to engage in IP-SDM.

PRACTICE IMPLICATIONS

Applying Kolb's experiential learning cycle and blended teaching methods to develop and implement the IP-SDM training program can improve multidisciplinary healthcare personnel's knowledge, attitude, skills, and teamwork in IP-SDM.

A randomized controlled trial evaluating the effectiveness of a self-management program for adolescents with a chronic condition: a study protocol

BACKGROUND

Self-management support is increasingly viewed as an integral part of chronic condition management in adolescence. It is well recognized that markers of chronic illness control deteriorate during adolescence. Due to the increasing prevalence of long-term chronic health conditions in childhood and improved survival rates of previously life-limiting conditions in children and adolescents, significant numbers of adolescents are having to manage their chronic condition effectively as they transition to adult health care. Therapy adherence has been identified as a major challenge for young people living with a chronic condition such as cystic fibrosis, diabetes, or asthma requiring long-term pharmacological therapy and/or lifestyle modifications. Most systematic reviews on self-management interventions address adult populations. Very few intervention studies are directed at adolescents with a chronic condition who are transitioning to adult health services. This protocol describes a prospective randomized controlled trial of a standardized self-management intervention program delivered to adolescents aged 15-18 years prior to their transfer to adult care. This study has been designed to provide evidence regarding self-management programs for adolescents and is the first study to use the Flinders Program with this important, under-researched age group.

METHODS

A randomized controlled trial is used to investigate the effectiveness of a modified adolescent-friendly version of an adult self-management program. This program is directed at improving self-management in an adolescent cohort 15-18 years of age with a chronic condition being treated in a specialist pediatric hospital. Participants will be randomized to either usual care or the modified Flinders Program plus usual care. Data collection will include measures of specific illness control, unscheduled hospital admissions, and questionnaires to record self-management competencies, quality of life, self-efficacy, and outcome measures specific to the chronic condition at baseline, 3 months, 6 months, and 12 months after delivery.

DISCUSSION

This study will provide a better understanding of the elements required for effective self-management programs in adolescents with a chronic condition and address some important knowledge gaps in current literature. The study will be carried out in collaboration with the Discipline of Behavioural Health at Flinders University, Adelaide, Australia, in order to inform the development of an adolescent version of the successful and validated Flinders Program™.

TRIAL REGISTRATION

Australian and New Zealand Clinical Trials Registry (ACTRN12621000390886). Registered on April 8, 2021.

Marginal modeling of cluster-period means and intraclass correlations in stepped wedge designs with binary outcomes

Stepped wedge cluster randomized trials (SW-CRTs) with binary outcomes are increasingly used in prevention and implementation studies. Marginal models represent a flexible tool for analyzing SW-CRTs with population-averaged interpretations, but the joint estimation of the mean and intraclass correlation coefficients (ICCs) can be computationally intensive due to large cluster-period sizes. Motivated by the need for marginal inference in SW-CRTs, we propose a simple and efficient estimating equations approach to analyze cluster-period means. We show that the quasi-score for the marginal mean defined from individual-level observations can be reformulated as the quasi-score for the same marginal mean defined from the cluster-period means. An additional mapping of the individual-level ICCs into correlations for the cluster-period means further provides a rigorous justification for the cluster-period approach. The proposed approach addresses a long-recognized computational burden associated with estimating equations defined based on individual-level observations, and enables fast point and interval estimation of the intervention effect and correlations. We further propose matrix-adjusted estimating equations to improve the finite-sample inference for ICCs. By providing a valid approach to estimate ICCs within the class of generalized linear models for correlated binary outcomes, this article operationalizes key recommendations from the CONSORT extension to SW-CRTs, including the reporting of ICCs.

Stepped Wedge Cluster Randomized Trials: A Methodological Overview

BACKGROUND

Stepped wedge cluster randomized trials enable rigorous evaluations of health intervention programs in pragmatic settings. In the present study, we aimed to update neurosurgeon scientists on the design of stepped wedge randomized trials.

METHODS

We have presented an overview of recent methodological developments for stepped wedge designs and included an update on the newer associated methodological tools to aid with future study designs.

RESULTS

We defined the stepped wedge trial design and reviewed the indications for the design in depth. In addition, key considerations, including mainstream methods of analysis and sample size determination, were discussed.

CONCLUSIONS

Stepped wedge designs can be attractive for study intervention programs aiming to improve the delivery of patient care, especially when examining a small number of heterogeneous clusters.

Evaluating the performance of Bayesian and restricted maximum likelihood estimation for stepped wedge cluster randomized trials with a small number of clusters

Background

Stepped wedge trials are an appealing and potentially powerful cluster randomized trial design. However, they are frequently implemented with a small number of clusters. Standard analysis methods for these trials such as a linear mixed model with estimation via maximum likelihood or restricted maximum likelihood (REML) rely on asymptotic properties and have been shown to yield inflated type I error when applied to studies with a small number of clusters. Small-sample methods such as the Kenward-Roger approximation in combination with REML can potentially improve estimation of the fixed effects such as the treatment effect. A Bayesian approach may also be promising for such multilevel models but has not yet seen much application in cluster randomized trials.

Methods

We conducted a simulation study comparing the performance of REML with and without a Kenward-Roger approximation to a Bayesian approach using weakly informative prior distributions on the intracluster correlation parameters. We considered a continuous outcome and a range of stepped wedge trial configurations with between 4 and 40 clusters. To assess method performance we calculated bias and mean squared error for the treatment effect and correlation parameters and the coverage of 95% confidence/credible intervals and relative percent error in model-based standard error for the treatment effect.

Results

Both REML with a Kenward-Roger standard error and degrees of freedom correction and the Bayesian method performed similarly well for the estimation of the treatment effect, while intracluster correlation parameter estimates obtained via the Bayesian method were less variable than REML estimates with different relative levels of bias.

Conclusions

The use of REML with a Kenward-Roger approximation may be sufficient for the analysis of stepped wedge cluster randomized trials with a small number of clusters. However, a Bayesian approach with weakly informative prior distributions on the intracluster correlation parameters offers a viable alternative, particularly when there is interest in the probability-based inferences permitted within this paradigm.

Supplementary Information

The online version contains supplementary material available at (10.1186/s12874-022-01550-8).

Ethical and epistemic issues in the design and conduct of pragmatic stepped-wedge cluster randomized clinical trials

Stepped-wedge cluster randomized trial (SW-CRT) designs are increasingly employed in pragmatic research; they differ from traditional parallel cluster randomized trials in which an intervention is delivered to a subset of clusters, but not to all. In a SW-CRT, all clusters receive the intervention under investigation by the end of the study. This approach is thought to avoid ethical concerns about the denial of a desired intervention to participants in control groups. Such concerns have been cited in the literature as a primary motivation for choosing SW-CRT design, however SW-CRTs raise additional ethical concerns related to the delayed implementation of an intervention and consent. Yet, PCT investigators may choose SW-CRT designs simply because they are concerned that other study designs are infeasible. In this paper, we examine justifications for the use of SW-CRT study design, over other designs, by drawing on the experience of the National Institutes of Health's Health Care Systems Research Collaboratory (NIH Collaboratory) with five pragmatic SW-CRTs. We found that decisions to use SW-CRT design were justified by practical and epistemic reasons rather than ethical ones. These include concerns about feasibility, the heterogeneity of cluster characteristics, and the desire for simultaneous clinical evaluation and implementation. In this paper we compare the potential benefits of SW-CRTs against the ethical and epistemic challenges brought forth by the design and suggest that the choice of SW-CRT design must balance epistemic, feasibility and ethical justifications. Moreover, given their complexity, such studies need rigorous and informed ethical oversight.

Response adaptive intervention allocation in stepped-wedge cluster randomized trials

BACKGROUND

Stepped-wedge cluster randomized trial (SW-CRT) designs are often used when there is a desire to provide an intervention to all enrolled clusters, because of a belief that it will be effective. However, given there should be equipoise at trial commencement, there has been discussion around whether a pre-trial decision to provide the intervention to all clusters is appropriate. In pharmaceutical drug development, a solution to a similar desire to provide more patients with an effective treatment is to use a response adaptive (RA) design.

METHODS

We introduce a way in which RA design could be incorporated in an SW-CRT, permitting modification of the intervention allocation during the trial. The proposed framework explicitly permits a balance to be sought between power and patient benefit considerations. A simulation study evaluates the methodology.

RESULTS

In one scenario, for one particular RA design, the proportion of cluster-periods spent in the intervention condition was observed to increase from 32.2% to 67.9% as the intervention effect was increased. A cost of this was a 6.2% power drop compared to a design that maximized power by fixing the proportion of time in the intervention condition at 45.0%, regardless of the intervention effect.

CONCLUSIONS

An RA approach may be most applicable to settings for which the intervention has substantial individual or societal benefit considerations, potentially in combination with notable safety concerns. In such a setting, the proposed methodology may routinely provide the desired adaptability of the roll-out speed, with only a small cost to the study's power.

Influential methods reports for group-randomized trials and related designs

BACKGROUND

This article identifies the most influential methods reports for group-randomized trials and related designs published through 2020. Many interventions are delivered to participants in real or virtual groups or in groups defined by a shared interventionist so that there is an expectation for positive correlation among observations taken on participants in the same group. These interventions are typically evaluated using a group- or cluster-randomized trial, an individually randomized group treatment trial, or a stepped wedge group- or cluster-randomized trial. These trials face methodological issues beyond those encountered in the more familiar individually randomized controlled trial.

METHODS

PubMed was searched to identify candidate methods reports; that search was supplemented by reports known to the author. Candidate reports were reviewed by the author to include only those focused on the designs of interest. Citation counts and the relative citation ratio, a new bibliometric tool developed at the National Institutes of Health, were used to identify influential reports. The relative citation ratio measures influence at the article level by comparing the citation rate of the reference article to the citation rates of the articles cited by other articles that also cite the reference article.

RESULTS

In total, 1043 reports were identified that were published through 2020. However, 55 were deemed to be the most influential based on their relative citation ratio or their citation count using criteria specific to each of the three designs, with 32 group-randomized trial reports, 7 individually randomized group treatment trial reports, and 16 stepped wedge group-randomized trial reports. Many of the influential reports were early publications that drew attention to the issues that distinguish these designs from the more familiar individually randomized controlled trial. Others were textbooks that covered a wide range of issues for these designs. Others were "first reports" on analytic methods appropriate for a specific type of data (e.g. binary data, ordinal data), for features commonly encountered in these studies (e.g. unequal cluster size, attrition), or for important variations in study design (e.g. repeated measures, cohort versus cross-section). Many presented methods for sample size calculations. Others described how these designs could be applied to a new area (e.g. dissemination and implementation research). Among the reports with the highest relative citation ratios were the CONSORT statements for each design.

CONCLUSIONS

Collectively, the influential reports address topics of great interest to investigators who might consider using one of these designs and need guidance on selecting the most appropriate design for their research question and on the best methods for design, analysis, and sample size.

Power and sample size calculation for stepped-wedge designs with discrete outcomes

BACKGROUND

Stepped-wedge designs (SWD) are increasingly used to evaluate the impact of changes to the process of care within health care systems. However, to generate definitive evidence, a correct sample size calculation is crucial to ensure such studies are properly powered. The seminal work of Hussey and Hughes (Contemp Clin Trials 28(2):182-91, 2004) provides an analytical formula for power calculations with normal outcomes using a linear model and simple random effects. However, minimal development and evaluation have been done for power calculation with non-normal outcomes on their natural scale (e.g., logit, log). For example, binary endpoints are common, and logistic regression is the natural multilevel model for such clustered data.

METHODS

We propose a power calculation formula for SWD with either normal or non-normal outcomes in the context of generalized linear mixed models by adopting the Laplace approximation detailed in Breslow and Clayton (J Am Stat Assoc 88(421):9-25, 1993) to obtain the covariance matrix of the estimated parameters.

RESULTS

We compare the performance of our proposed method with simulation-based sample size calculation and demonstrate its use on a study of patient-delivered partner therapy for STI treatment and a study that assesses the impact of providing additional benchmark prevalence information in a radiologic imaging report. To facilitate adoption of our methods we also provide a function embedded in the R package "swCRTdesign" for sample size and power calculation for multilevel stepped-wedge designs.

CONCLUSIONS

Our method requires minimal computational power. Therefore, the proposed procedure facilitates rapid dynamic updates of sample size calculations and can be used to explore a wide range of design options or assumptions.

Mixed-effects models for the design and analysis of stepped wedge cluster randomized trials: An overview

The stepped wedge cluster randomized design has received increasing attention in pragmatic clinical trials and implementation science research. The key feature of the design is the unidirectional crossover of clusters from the control to intervention conditions on a staggered schedule, which induces confounding of the intervention effect by time. The stepped wedge design first appeared in the Gambia hepatitis study in the 1980s. However, the statistical model used for the design and analysis was not formally introduced until 2007 in an article by Hussey and Hughes. Since then, a variety of mixed-effects model extensions have been proposed for the design and analysis of these trials. In this article, we explore these extensions under a unified perspective. We provide a general model representation and regard various model extensions as alternative ways to characterize the secular trend, intervention effect, as well as sources of heterogeneity. We review the key model ingredients and clarify their implications for the design and analysis. The article serves as an entry point to the evolving statistical literatures on stepped wedge designs.

Variance formulae for multiphase stepped wedge cluster randomized trial

In a multiphase stepped wedge cluster randomized trial (MSW-CRT), more than one intervention will be initiated on each sequence in a fixed order. Hence, with the MSW-CRT design, the effect of the first intervention can be evaluated when compared to control, as well as the added-on effects of the subsequent interventions. Studies that use MSW-CRT have been proposed, but properties of this design have not been explicitly studied. We derive closed-form variance formulae to test the interventions' effects, which can be readily used for sample size and power calculation. Additionally, we provide relationships between variances to test the interventions' effects and design parameters. Under special conditions, some important properties include: (i) the variances to test different interventions' effects (ie, the first intervention effect and the second intervention effect) may be same; (ii) as the cluster-period mean autocorrelation increases, the variance to test an intervention effect may first increase and then decrease; (iii) as the amount of periods between the initiations of two interventions (ie, lag) increases, the variance to test an intervention effect may remain unchanged. We illustrate the relationships between power and design parameters using the variance formulae. From a few illustrative examples, we observe that the statistical test that uses data only relevant to a specific intervention has inferior power (relative power loss <15%) compared to the test when using all the study data. Also, power is reduced when both the total number of periods and lag are decreased simultaneously (relative power loss <20%).

High-Sensitivity Cardiac Troponin-Optimizing the Diagnosis of Acute Myocardial Infarction/Injury in Women (CODE-MI): Rationale and design for a multicenter, stepped-wedge, cluster-randomized trial

Despite evidence that high-sensitivity cardiac troponin (hs-cTn) levels in women are lower than in men, a single threshold based on the 99th percentile upper reference limit of the overall reference population is commonly used to diagnose myocardial infarction in clinical practice. This trial aims to determine whether the use of a lower female-specific hs-cTn threshold would improve the diagnosis, treatment, and outcomes of women presenting to the emergency department with symptoms suggestive of myocardial ischemia. METHODS/DESIGN: CODE-MI (hs-cTn-Optimizing the Diagnosis of Acute Myocardial Infarction/Injury in Women) is a multicenter, stepped-wedge, cluster-randomized trial of 30 secondary and tertiary care hospitals across 8 Canadian provinces, with the unit of randomization being the hospital. All adults (≥20 years of age) presenting to the emergency department with symptoms suggestive of myocardial ischemia and at least 1 hs-cTn test are eligible for inclusion. Over five, 5-month intervals, hospitals will be randomized to implement lower female hs-cTn thresholds according to the assay being used at each site. Men will continue to be assessed using the overall thresholds throughout. Women with a peak hs-cTn value between the female-specific and the overall thresholds will form our primary cohort. The primary outcome, a 1-year composite of all-cause mortality or readmission for nonfatal myocardial infarction, incident heart failure, or emergent/urgent coronary revascularization, will be compared before and after the implementation of female thresholds using mixed-effects logistic regression models. The cohort and outcomes will be obtained from routinely collected administrative data. The trial is designed to detect a 20% relative risk difference in the primary outcome, or a 2.2% absolute difference, with 82% power. CONCLUSIONS: This pragmatic trial will assess whether adopting lower female hs-cTn thresholds leads to appropriate assessment of women with symptoms suggestive of myocardial infarction, thereby improving treatment and outcomes.

IDeA States Pediatric Clinical Trials Network for Underserved and Rural Communities

The National Institutes of Health's Environmental Influences on Child Health Outcomes (ECHO) program aims to study high-priority and high-impact pediatric conditions. This broad-based health initiative is unique in the National Institutes of Health research portfolio and involves 2 research components: (1) a large group of established centers with pediatric cohorts combining data to support longitudinal studies (ECHO cohorts) and (2) pediatric trials program for institutions within Institutional Development Awards states, known as the ECHO Institutional Development Awards States Pediatric Clinical Trials Network (ISPCTN). In the current presentation, we provide a broad overview of the ISPCTN and, particularly, its importance in enhancing clinical trials capabilities of pediatrician scientists through the support of research infrastructure, while at the same time implementing clinical trials that inform future health care for children. The ISPCTN research mission is aligned with the health priority conditions emphasized in the ECHO program, with a commitment to bringing state-of-the-science trials to children residing in underserved and rural communities. ISPCTN site infrastructure is critical to successful trial implementation and includes research training for pediatric faculty and coordinators. Network sites exist in settings that have historically had limited National Institutes of Health funding success and lacked pediatric research infrastructure, with the initial funding directed to considerable efforts in professional development, implementation of regulatory procedures, and engagement of communities and families. The Network has made considerable headway with these objectives, opening two large research studies during its initial 18 months as well as producing findings that serve as markers of success that will optimize sustainability.

Protect MSM from HIV and other sexually transmitted diseases by providing mobile health services of partner notification: protocol for a pragmatic stepped wedge cluster randomized controlled trial

Background

Recently, more and more men who have sex with men (MSM) look for casual partners through online dating platforms in China. However, most are unable to know their partners’ HIV and other sexually transmitted diseases (STD) statuses, leading to the rapid increase in HIV infection among Chinese MSM. Effective partner notification is urgently needed to increase the risk awareness of MSM and prevent HIV and other STDs transmission. However, the traditional intervention mainly targets to the HIV-positive MSM and the effect is not promising. Our study aims to provide Internet-based partner notification, along with a series of health services for HIV-negative MSM to protect them from HIV and other STDs.

Methods

A pragmatic stepped wedge cluster randomized controlled trial design is used to evaluate the effectiveness of a new intervention paradigm, which aims to reduce HIV and other STDs incidences among MSM in China. Through integrating a mobile health (mHealth) service application (app) to the current HIV and other STDs prevention and control methods, the new paradigm provides partner notification of HIV, syphilis, hepatitis B, and hepatitis C statuses. A total of 6172 MSM in 16 districts of Beijing, China will be recruited and randomized to sequentially receive partner notification intervention through the app at 6-month intervals. The primary outcomes are HIV incidence and the additional cost of the intervention. The secondary outcomes include incidences of syphilis, hepatitis B, and hepatitis C, disease transmission social networks, testing adherence, knowledge of HIV and STDs control, health self-responsibility awareness, changes of high risk behaviors and other related outcomes. The generalized linear mixed models (GLMM) will be used to analyze the differences of outcomes in the control period and in the intervention period.

Discussion

We expect that the HIV incidence will be significantly lower and the secondary outcomes will also be improved with providing health service of partner notification through mhealth intervention. The feasible and affordable public health intervention paradigm will have implications for HIV and STDs prevention and control among MSM and other key populations.

Trial registration

ClinicalTrials.gov, NCT04349748. Registered on 16 April 2020.

Explaining the variation in the attained power of a stepped-wedge trial with unequal cluster sizes

Background

In a cross-sectional stepped-wedge trial with unequal cluster sizes, attained power in the trial depends on the realized allocation of the clusters. This attained power may differ from the expected power calculated using standard formulae by averaging the attained powers over all allocations the randomization algorithm can generate. We investigated the effect of design factors and allocation characteristics on attained power and developed models to predict attained power based on allocation characteristics.

Method

Based on data simulated and analyzed using linear mixed-effects models, we evaluated the distribution of attained powers under different scenarios with varying intraclass correlation coefficient (ICC) of the responses, coefficient of variation (CV) of the cluster sizes, number of cluster-size groups, distributions of group sizes, and number of clusters. We explored the relationship between attained power and two allocation characteristics: the individual-level correlation between treatment status and time period, and the absolute treatment group imbalance. When computational time was excessive due to a scenario having a large number of possible allocations, we developed regression models to predict attained power using the treatment-vs-time period correlation and absolute treatment group imbalance as predictors.

Results

The risk of attained power falling more than 5% below the expected or nominal power decreased as the ICC or number of clusters increased and as the CV decreased. Attained power was strongly affected by the treatment-vs-time period correlation. The absolute treatment group imbalance had much less impact on attained power. The attained power for any allocation was predicted accurately using a logistic regression model with the treatment-vs-time period correlation and the absolute treatment group imbalance as predictors.

Conclusion

In a stepped-wedge trial with unequal cluster sizes, the risk that randomization yields an allocation with inadequate attained power depends on the ICC, the CV of the cluster sizes, and number of clusters. To reduce the computational burden of simulating attained power for allocations, the attained power can be predicted via regression modeling. Trial designers can reduce the risk of low attained power by restricting the randomization algorithm to avoid allocations with large treatment-vs-time period correlations.

Reflection on modern methods: when is a stepped-wedge cluster randomized trial a good study design choice?

The stepped-wedge cluster randomized trial (SW-CRT) involves the sequential transition of clusters (such as hospitals, public health units or communities) from control to intervention conditions in a randomized order. The use of the SW-CRT is growing rapidly. Yet the SW-CRT is at greater risks of bias compared with the conventional parallel cluster randomized trial (parallel-CRT). For this reason, the CONSORT extension for SW-CRTs requires that investigators provide a clear justification for the choice of study design. In this paper, we argue that all other things being equal, the SW-CRT is at greater risk of bias due to misspecification of the secular trends at the analysis stage. This is particularly problematic for studies randomizing a small number of heterogeneous clusters. We outline the potential conditions under which an SW-CRT might be an appropriate choice. Potentially appropriate and often overlapping justifications for conducting an SW-CRT include: (i) the SW-CRT provides a means to conduct a randomized evaluation which otherwise would not be possible; (ii) the SW-CRT facilitates cluster recruitment as it enhances the acceptability of a randomized evaluation either to cluster gatekeepers or other stakeholders; (iii) the SW-CRT is the only feasible design due to pragmatic and logistical constraints (for example the roll-out of a scare resource); and (iv) the SW-CRT has increased statistical power over other study designs (which will include situations with a limited number of clusters). As the number of arguments in favour of an SW-CRT increases, the likelihood that the benefits of using the SW-CRT, as opposed to a parallel-CRT, outweigh its risks also increases. We argue that the mere popularity and novelty of the SW-CRT should not be a factor in its adoption. In situations when a conventional parallel-CRT is feasible, it is likely to be the preferred design.

How to Ease the Pain of Taking a Diagnostic Point of Care Test to the Market: A Framework for Evidence Development

Bringing a diagnostic point of care test (POCT) to a healthcare market can be a painful experience as it requires the manufacturer to meet considerable technical, financial, managerial, and regulatory challenges. In this opinion article we propose a framework for developing the evidence needed to support product development, marketing, and adoption. We discuss each step in the evidence development pathway from the invention phase to the implementation of a new POCT in the healthcare system. We highlight the importance of articulating the value propositions and documenting the care pathway. We provide guidance on how to conduct care pathway analysis as little has been published on this. We summarize the clinical, economic and qualitative studies to be considered for developing evidence, and provide useful links to relevant software, on-line applications, websites, and give practical advice. We also provide advice on patient and public involvement and engagement (PPIE), and on product management. Our aim is to help device manufacturers to understand the concepts and terminology used in evaluation of in vitro diagnostics (IVDs) so that they can communicate effectively with evaluation methodologists, statisticians, and health economists. Manufacturers of medical tests and devices can use the proposed framework to plan their evidence development strategy in alignment with device development, applications for regulatory approval, and publication.

Information growth for sequential monitoring of clinical trials with a stepped wedge cluster randomized design and unknown intracluster correlation

Background/aims

In a stepped wedge study design, study clusters usually start with the baseline treatment and then cross over to the intervention at randomly determined times. Such designs are useful when the intervention must be delivered at the cluster level and are becoming increasingly common in practice. In these trials, if the outcome is death or serious morbidity, one may have an ethical imperative to monitor the trial and stop before maximum enrollment if the new therapy is proven to be beneficial. In addition, because formal monitoring allows for the stoppage of trials when a significant benefit for new therapy has been ruled out, their use can make a research program more efficient. However, use of the stepped wedge cluster randomized study design complicates the implementation of standard group sequential monitoring methods. Both the correlation of observations introduced by the clustered randomization and the timing of crossover from one treatment to the other impact the rate of information growth, an important component of an interim analysis.

Methods

We simulated cross-sectional stepped wedge study data in order to evaluate the impact of sequential monitoring on the Type I error and power when the true intracluster correlation is unknown. We studied the impact of varying intracluster correlations, treatment effects, methods of estimating the information growth, and boundary shapes.

Results

While misspecified information growth can impact both the Type I error and power of a study in some settings, we observed little inflation of the Type I error and only moderate reductions in power across a range of misspecified information growth patterns in our simulations.

Conclusion

Taking the study design into account and using either an estimate of the intracluster correlation from the ongoing study or other data in the same clusters should allow for easy implementation of group sequential methods in future stepped wedge designs.

Essential Ingredients and Innovations in the Design and Analysis of Group-Randomized Trials

This article reviews the essential ingredients and innovations in the design and analysis of group-randomized trials. The methods literature for these trials has grown steadily since they were introduced to the biomedical research community in the late 1970s, and we summarize those developments. We review, in addition to the group-randomized trial, methods for two closely related designs, the individually randomized group treatment trial and the stepped-wedge group-randomized trial. After describing the essential ingredients for these designs, we review the most important developments in the evolution of their methods using a new bibliometric tool developed at the National Institutes of Health. We then discuss the questions to be considered when selecting from among these designs or selecting the traditional randomized controlled trial. We close with a review of current methods for the analysis of data from these designs, a case study to illustrate each design, and a brief summary.

Stepped-wedge trials should be classified as research for the purpose of ethical review

BACKGROUND

All studies classified as research involving human participants require research ethics review. Most regulation and guidance on ethical oversight of research involving human participants was written for pharmacotherapy interventions. Interpretation of such guidance for cluster-randomized trials and stepped-wedge trials, which commonly evaluate complex non-therapeutic interventions such as knowledge translation, public health, or health service delivery interventions, can pose challenges to researchers and regulators.

CURRENT GUIDANCE

The Ottawa Statement on the Ethical Design and Conduct of Cluster-Randomized Trials provides guidance on the ethical oversight and consent procedures for cluster-randomized trials, and while not explicit, this includes stepped-wedge trials. Yet, stepped-wedge trials have unique characteristics that differentiate them from standard cluster-randomized trials. In particular, they can be used to evaluate knowledge translation interventions within the context of a routine health system rollout; they may have a non-randomized design; and the decision to implement the intervention is not always made by the researcher. Many stepped-wedge trials do not undergo ethical review and do not report trial registration. This suggests that those undertaking these studies and research ethics committees perceive them as non-research activities.

RECOMMENDATIONS

Through an ethical analysis of two case studies, we argue that stepped-wedge trials, like parallel arm cluster trials, are systematic investigations designed to produce generalizable knowledge. We contend that stepped-wedge trials usually include human research participants, which may be patients, health care providers, or both. Stepped-wedge trials are therefore research involving human participants for the purpose of ethical review. Nevertheless, the use of a waiver or alteration of consent may be appropriate in many stepped-wedge trials due to the infeasibility of obtaining informed consent and the low-risk nature of the interventions. To ensure that traditional ethical principles such as respect for persons are upheld, these studies must undergo research ethics review.

Design and analysis considerations for cohort stepped wedge cluster randomized trials with a decay correlation structure

A stepped wedge cluster randomized trial is a type of longitudinal cluster design that sequentially switches clusters to intervention over time until all clusters are treated. While the traditional posttest-only parallel design requires adjustment for a single intraclass correlation coefficient, the stepped wedge design allows multiple outcome measurements from the same cluster and so additional correlation parameters are necessary to characterize the within-cluster correlation structure. Although a number of studies have differentiated between the concepts of within-period and between-period correlations, only a few studies have allowed the between-period correlation to decay over time. In this article, we consider the proportional decay correlation structure for a cohort stepped wedge design, and provide a matrix-adjusted quasi-least squares approach to accurately estimate the correlation parameters along with the marginal intervention effect. We further develop the sample size and power procedures accounting for the correlation decay, and investigate the accuracy of the power procedure with continuous outcomes in a simulation study. We show that the empirical power agrees well with the prediction even with as few as nine clusters, when data are analyzed with matrix-adjusted quasi-least squares concurrently with a suitable bias-corrected sandwich variance. Two trial examples are provided to illustrate the new sample size procedure.

Delaying and withholding interventions: ethics and the stepped wedge trial

Ethics has been identified as a central reason for choosing the stepped wedge trial over other kinds of trial designs. The potential advantage of the stepped wedge design is that it provides all arms of the trial with the active intervention over the course of the study. Some groups receive it later than others, but the study intervention is not withheld from any group. This feature of the stepped wedge design seems particularly ethically advantageous in two instances: (1) when the study intervention appears especially likely to be effective and (2) when the consequences of not receiving the intervention may be dire. But despite an increase in the use of the stepped wedge design and appeals to its ethical superiority as the motivation for its selection, there has been limited attention to the stepped wedge trial in the ethics literature. In the following, I examine whether there are persuasive ethical reasons to prefer or to require a stepped wedge trial. I argue that while the stepped wedge design is ethically permissible, it is not morally superior to other kinds of trials. To this end, I examine the ethical justification for providing, withholding, and delaying interventions in research.

A minimum evaluation protocol and stepped-wedge cluster randomized trial of ACCESS Open Minds, a large Canadian youth mental health services transformation project

BACKGROUND

Many Canadian adolescents and young adults with mental health problems face delayed detection, long waiting lists, poorly accessible services, care of inconsistent quality and abrupt or absent inter-service transitions. To address these issues, ACCESS Open Minds, a multi-stakeholder network, is implementing and systematically evaluating a transformation of mental health services for youth aged 11 to 25 at 14 sites across Canada. The transformation plan has five key foci: early identification, rapid access, appropriate care, the elimination of age-based transitions between services, and the engagement of youth and families.

METHODS

The ACCESS Open Minds Research Protocol has multiple components including a minimum evaluation protocol and a stepped-wedge cluster randomized trial, that are detailed in this paper. Additional components include qualitative methods and cost-effectiveness analyses. The services transformation is being evaluated at all sites via a minimum evaluation protocol. Six sites are participating in the stepped-wedge trial whereby the intervention (a service transformation along the key foci) was rolled out in three waves, each commencing six months apart. Two sites, one high-population and one low-population, were randomly assigned to each of the three waves, i.e., randomization was stratified by population size. Our primary hypotheses pertain to increased referral numbers, and reduced wait times to initial assessment and to the commencement of appropriate care. Secondary hypotheses pertain to simplified pathways to care; improved clinical, functional and subjective outcomes; and increased satisfaction among youth and families. Quantitative measures addressing these hypotheses are being used to determine the effectiveness of the intervention.

DISCUSSION

Data from our overall research strategy will help test the effectiveness of the ACCESS Open Minds transformation, refine it further, and inform its scale-up. The process by which our research strategy was developed has implications for the practice of research itself in that it highlights the need to actively engage all stakeholder groups and address unique considerations in designing evaluations of complex healthcare interventions in multiple, diverse contexts. Our approach will generate both concrete evidence and nuanced insights, including about the challenges of conducting research in real-world settings. More such innovative approaches are needed to advance youth mental health services research.

TRIAL REGISTRATION NUMBER

Clinicaltrials.gov, ISRCTN23349893 (Retrospectively registered: 16/02/2017).

Power calculation in stepped-wedge cluster randomized trial with reduced intervention sustainability effect

The stepped-wedge design for pragmatic clinical trials has received increased attention in health service-related research seeking to evaluate the effect of interventions. Compared with the parallel design, the stepped-wedge design is preferred when there is prior knowledge supporting the effectiveness and harmlessness of the intervention, and/or when practical or financial constraints exist such that the intervention can only be implemented sequentially on a fraction of clusters. In some health service studies, the study period may consist of two parts: an active implementation followed by a sustainability phase, where the intervention effect is possibly reduced. There is a gap in current literature of the stepped-wedge design for cluster randomization trials for dealing with this specific scenario. We aim to provide an analytical formula for power analysis under this situation to aid the stepped-wedge design of an ongoing PREVENT trial.

Stepped wedge trials with continuous recruitment require new ways of thinking

OBJECTIVES

There is substantial variation in the design of stepped wedge trials. Many recruit participants continuously over time, although the methodological literature has tended not to differentiate closely between continuous recruitment and discrete sampling. We argue for a deeper understanding of the special features of stepped wedge trials with continuous recruitment.

STUDY DESIGN AND SETTING

This is a commentary and informal review.

RESULTS

We discuss the scheduling of recruitment and implementation in continuous time and how contamination might be avoided. We also offer some suggestions on reporting and terminology for stepped wedge trials with continuous recruitment and comment on issues for analysis.

CONCLUSION

Repeated cross-section and continuous recruitment stepped wedge trials are not the same thing. More work is needed to develop the theory and practice of stepped wedge designs with continuous recruitment. Thoughtful approaches to design and clarity of reporting are vital.

Feasibility and acceptability of Indigenous Counselling and Nicotine (ICAN) QUIT in Pregnancy multicomponent implementation intervention and study design for Australian Indigenous pregnant women: A pilot cluster randomised step-wedge trial

BACKGROUND

Many health providers (HPs) lack knowledge, confidence, optimism and skills in addressing smoking with pregnant women. This study aimed to explore the feasibility and acceptability of a) a co-designed multi-component intervention for HPs at Aboriginal Medical Services (AMSs) in culturally-targeted pregnancy-specific smoking cessation care and b) the study design.

METHODS

Using a randomised step-wedge cluster design, the Indigenous Counselling And Nicotine (ICAN) QUIT in Pregnancy Trial was evaluated across six AMSs in three Australian states. HPs were provided educational resource packages including live interactive webinars, treatment manuals, patient resources, carbon monoxide (CO) meters, and oral Nicotine Replacement Therapy (NRT). Feasibility was assessed through recruitment and retention rates of both pregnant women (12-weeks) and HPs (end of study) as well as the potential to improve women's quit rates. Qualitative interviews with staff post-trial explored acceptability of the intervention and study, based on capability, opportunity and motivation from the Behaviour Change Wheel.

RESULTS

Pregnant women (n = 22; 47% (95% CI: 32%, 63%) eligible) and HPs (n = 50; 54% (95% CI: 44%, 64%) eligible) were recruited over 6 months with retention rates of 77% (95% CI: 57%, 90%) and 40% (95% CI: 28%, 54%) respectively. Self-reported 12-week 7-day point-prevalence abstinence was 13.6% (n = 3) and validated abstinent with CO readings ≤6 ppm. Staff interviewed regarding intervention implementation highlighted the importance of provision and use of resources, including training materials, patient resources, CO meters and oral NRT. Resources helped increase capability and opportunity, restructure the environment, and provided social comparison and modelling. Staff were motivated by greater engagement with pregnant women and seeing the women's reductions in CO readings. Having the intervention at the AMSs improved organisational capacity to engage with pregnant women. Staff reported changes to their routine practice that were potentially sustainable. Recommendations for improvement to the implementation of the intervention and research included reducing training length and the tasks related to conducting the study.

CONCLUSION

ICAN QUIT in Pregnancy was a pilot study with the ability to enrol Indigenous women. It was feasible to implement and acceptable to most staff of the AMSs in three states, with modifications recommended. Smoking in pregnancy is a key challenge for Indigenous health. The intervention needs to be evaluated through a methodologically rigorous fully-powered study to determine the efficacy of outcomes for women.

TRIAL REGISTRATION

Australian and New Zealand Clinical Trials Registry, ACTRN12616001603404. Registered 21 November 2016 - retrospectively registered, https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=371778.

Quality of stepped-wedge trial reporting can be reliably assessed using an updated CONSORT: crowd-sourcing systematic review

OBJECTIVES

The Consolidated Standards of Reporting Trials extension for the stepped-wedge cluster randomized trial (SW-CRT) is a recently published reporting guideline for SW-CRTs. We assess the quality of reporting of a recent sample of SW-CRTs.

STUDY DESIGN AND SETTING

Quality of reporting was asssessed according to the 26 items in the new guideline using a novel crowd sourcing methodology conducted independently and in duplicate, with random assignment, by 50 reviewers. We assessed reliability of the quality assessments, proposing this as a novel way to assess robustness of items in reporting guidelines.

RESULTS

Several items were well reported. Some items were very poorly reported, including several items that have unique requirements for the SW-CRT, such as the rationale for use of the design, description of the design, identification and recruitment of participants within clusters, and concealment of cluster allocation (not reported in more than 50% of the reports). Agreement across items was moderate (median percentage agreement was 76% [IQR 64 to 86]). Agreement was low for several items including the description of the trial design and why trial ended or stopped for example.

CONCLUSIONS

When reporting SW-CRTs, authors should pay particular attention to ensure clear reporting on the exact format of the design with justification, as well as how clusters and individuals were identified for inclusion in the study, and whether this was done before or after randomization of the clusters, which are crucial for risk of bias assessments. Some items, including why the trial ended, might either not be relevant to SW-CRTs or might be unclearly described in the statement.

The iPROMOS protocol: a stepped-wedge study to implement routine patient-reported outcomes in a medical oncology outpatient setting

INTRODUCTION

Patient-reported outcome measures (PROMs) are data capture tools that collect information directly from patients. Several large research studies provide evidence that the use of PROMs in routine care provides benefits to mortality and morbidity outcomes in medical oncology patients. Despite this, implementation of PROMs in daily clinical routine is slow and challenging.

METHODS AND ANALYSIS

This study will use a stepped-wedge design to assess the implementation of a PROM intervention in highly frequented medical oncology outpatient clinics. During a lead-in period of 4 weeks, control data will be collected. The intervention will then be implemented for 4 weeks in Clinic 1 initially, then in Clinic 2 for another 4 weeks. 500 patient encounters will be measured over the 12 weeks in total. The process of implementation will be informed and evaluated using the Medical Research Council Guidelines for Implementing Complex Interventions. The study will be guided by the Promoting Action Research in Health Services framework approach for implementation. The intervention and implementation outcomes will be measured using qualitative and quantitative data.

ETHICS AND DISSEMINATION

Ethical approval has been obtained, approval number HREC/16/QRBW/100 by the Royal Brisbane and Women's Hospital Human Research Ethics Committee. Results will be disseminated in peer-reviewed journals and at scientific meetings.

TRIAL REGISTRATION

ACTRN12618000398202. Trial Status: Opened on 25 March 2018 and will continue until 12 months after the last PROMs reporting encounter.

Patient safety, cost-effectiveness, and quality of life: reduction of delirium risk and postoperative cognitive dysfunction after elective procedures in older adults-study protocol for a stepped-wedge cluster randomized trial (PAWEL Study)

Background

Postoperative delirium is a common disorder in older adults that is associated with higher morbidity and mortality, prolonged cognitive impairment, development of dementia, higher institutionalization rates, and rising healthcare costs. The probability of delirium after surgery increases with patients’ age, with pre-existing cognitive impairment, and with comorbidities, and its diagnosis and treatment is dependent on the knowledge of diagnostic criteria, risk factors, and treatment options of the medical staff. In this study, we will investigate whether a cross-sectoral and multimodal intervention for preventing delirium can reduce the prevalence of delirium and postoperative cognitive decline (POCD) in patients older than 70 years undergoing elective surgery. Additionally, we will analyze whether the intervention is cost-effective.

Methods

The study will be conducted at five medical centers (with two or three surgical departments each) in the southwest of Germany. The study employs a stepped-wedge design with cluster randomization of the medical centers. Measurements are performed at six consecutive points: preadmission, preoperative, and postoperative with daily delirium screening up to day 7 and POCD evaluations at 2, 6, and 12 months after surgery. Recruitment goals are to enroll 1500 patients older than 70 years undergoing elective operative procedures (cardiac, thoracic, vascular, proximal big joints and spine, genitourinary, gastrointestinal, and general elective surgery procedures).

Discussion

Results of the trial should form the basis of future standards for preventing delirium and POCD in surgical wards. Key aims are the improvement of patient safety and quality of life, as well as the reduction of the long-term risk of conversion to dementia. Furthermore, from an economic perspective, we expect benefits and decreased costs for hospitals, patients, and healthcare insurances.

Trial registration

German Clinical Trials Register, DRKS00013311. Registered on 10 November 2017.

Electronic supplementary material

The online version of this article (10.1186/s13063-018-3148-8) contains supplementary material, which is available to authorized users.

Introducing the new CONSORT extension for stepped-wedge cluster randomised trials

The use of the stepped-wedge cluster randomised trial (SW-CRT) is on the increase, and although there are still relatively few SW-CRTs currently published its use is bound to show an increase in the near future. An extension of the CONSORT reporting guideline for SW-CRTs has recently been developed. By making reporting guidelines for this innovative design available relatively early in its development, it is possible that the methodological conduct and reporting of future SW-CRTs will not be at the same risk of low-quality of reporting as is the case with many other study designs. We provide a brief overview of this reporting guideline and encourage authors to use it appropriately; and for journal editors to endorse its use.

The current use of feasibility studies in the assessment of feasibility for stepped-wedge cluster randomised trials: a systematic review

Background

Stepped-wedge cluster randomised trials (SW-CRTs) are a pragmatic trial design, providing an unprecedented opportunity to increase the robustness of evidence underpinning implementation and quality improvement interventions. Given the complexity of the SW-CRT, the likelihood of trials not delivering on their objectives will be mitigated if a feasibility study precedes the definitive trial. It is not currently known if feasibility studies are being conducted for SW-CRTs nor what the objectives of these studies are.

Methods

Searches were conducted of several databases to identify published feasibility studies which were designed to inform a future SW-CRT. For each eligible study, data were extracted on the characteristics of and rationale for the feasibility study; the process for determining progression to the main trial; how the feasibility study informed the main trial; and whether the main trial went ahead. A narrative synthesis and descriptive analysis are presented.

Results

Eleven feasibility studies were identified, which included eight completed study reports and three protocols. Three studies used a stepped-wedge design and these were the only studies to be randomised. Studies were predominantly of a mixed-methods design. Only one study assessed specific features related to the feasibility of using a SW-CRT and one investigated the time taken to complete the study procedures. The other studies were mostly assessing the feasibility and acceptability of the intervention.

Conclusion

Published feasibility studies for SW-CRTs are scarce and those that are being reported do not investigate issues specific to the complexities of the trial design. When conducting feasibility studies in advance of a definitive SW-CRT, researchers should consider assessing the feasibility of study procedures, particularly those specific to the SW-CRT design, and ensure that the findings are published for the benefit of other researchers.

Electronic supplementary material

The online version of this article (10.1186/s12874-019-0658-3) contains supplementary material, which is available to authorized users.

Quality of Reporting in Oncology Randomized Controlled Trials: From 2011 to 2015

Randomized controlled trials (RCTs) are important for evidence-based medicine; however, their quality of reporting remains to be evaluated. The aim of this study was to assess the quality of the report concerning solid tumor medication. Articles were searched in PubMed to identify all oncology phase III RCTs published from 2011 to 2015, and the results were classified manually through Endnote X7.0 software. Registration rate, primary end point (PEP) consistency, positive result rate, enrollment time point, outcome feedback in the registry, and publish time zone were extracted and assessed. The overall registration rate was higher than years before; nevertheless, a portion of trials showed PEP discrepancies and enrolled patients before registration in either journal formats. Trials published in top 5 general medical journals paid more attention to results feedback on registration websites and were more prompt with publication after study accomplishment. Our data suggested general medical journals may be more rigorous compared to oncology journals but identified a preference for positive results. On the whole, RCTs published between 2011 and 2015 seemed fairly standardized. Surveillance in registry and outcome feedback still needs to be strengthened for the stringency and reliability of clinical trials in solid tumor medication territory.

Blinded and unblinded sample size reestimation procedures for stepped-wedge cluster randomized trials

The ability to accurately estimate the sample size required by a stepped-wedge (SW) cluster randomized trial (CRT) routinely depends upon the specification of several nuisance parameters. If these parameters are misspecified, the trial could be overpowered, leading to increased cost, or underpowered, enhancing the likelihood of a false negative. We address this issue here for cross-sectional SW-CRTs, analyzed with a particular linear-mixed model, by proposing methods for blinded and unblinded sample size reestimation (SSRE). First, blinded estimators for the variance parameters of a SW-CRT analyzed using the Hussey and Hughes model are derived. Following this, procedures for blinded and unblinded SSRE after any time period in a SW-CRT are detailed. The performance of these procedures is then examined and contrasted using two example trial design scenarios. We find that if the two key variance parameters were underspecified by 50%, the SSRE procedures were able to increase power over the conventional SW-CRT design by up to 41%, resulting in an empirical power above the desired level. Thus, though there are practical issues to consider, the performance of the procedures means researchers should consider incorporating SSRE in to future SW-CRTs.

Multicolour versus monocolour inking specimens after pancreaticoduodenectomy for periampullary cancer: A single centre prospective randomised clinical trial

BACKGROUND

R status represents an important prognostic factors in periampullary cancers. Thus, it is useful to verify if it can be influenced by different techniques of margination.

METHODS

Single-centre, randomised clinical trial of patients affected by periampullary cancer who underwent pancreaticoduodenectomies which included two different types of margination: arm A (multicolour inking) and arm B (monocolour inking). The primary endpoint was the overall R1 resection rate and its difference between the two arms. The secondary endpoints were the R1 resection rate in each margin and its difference between the two arms, and the impact of margin status on survival.

RESULTS

Fifty patients were randomised, 41 analysed: 22 in arm A, 19 arm B. The overall R1 status was 61%, without significant differences between the two arms. The margin most commonly involved was the superior mesenteric artery (SMA) (36.6%). A trend in favour of arm B was shown for the superior mesenteric artery margin (arm A = 22.7% versus arm B = 52.6%; P = 0.060). The anterior surface (P = 0.015), SMA (P = 0.047) and pancreatic remnant (P = 0.018) margins significantly influenced disease-free survival.

CONCLUSIONS

The R status was not influenced by different techniques of margination using a standardised pathological protocol. The SMA margin seemed to be the most important margin for evaluating both R status and disease-free survival.

Unequal cluster sizes in stepped-wedge cluster randomised trials: a systematic review

OBJECTIVES

To investigate the extent to which cluster sizes vary in stepped-wedge cluster randomised trials (SW-CRT) and whether any variability is accounted for during the sample size calculation and analysis of these trials.

SETTING

Any, not limited to healthcare settings.

PARTICIPANTS

Any taking part in an SW-CRT published up to March 2016.

PRIMARY AND SECONDARY OUTCOME MEASURES

The primary outcome is the variability in cluster sizes, measured by the coefficient of variation (CV) in cluster size. Secondary outcomes include the difference between the cluster sizes assumed during the sample size calculation and those observed during the trial, any reported variability in cluster sizes and whether the methods of sample size calculation and methods of analysis accounted for any variability in cluster sizes.

RESULTS

Of the 101 included SW-CRTs, 48% mentioned that the included clusters were known to vary in size, yet only 13% of these accounted for this during the calculation of the sample size. However, 69% of the trials did use a method of analysis appropriate for when clusters vary in size. Full trial reports were available for 53 trials. The CV was calculated for 23 of these: the median CV was 0.41 (IQR: 0.22-0.52). Actual cluster sizes could be compared with those assumed during the sample size calculation for 14 (26%) of the trial reports; the cluster sizes were between 29% and 480% of that which had been assumed.

CONCLUSIONS

Cluster sizes often vary in SW-CRTs. Reporting of SW-CRTs also remains suboptimal. The effect of unequal cluster sizes on the statistical power of SW-CRTs needs further exploration and methods appropriate to studies with unequal cluster sizes need to be employed.

Group sequential designs for stepped-wedge cluster randomised trials

BACKGROUND/AIMS

The stepped-wedge cluster randomised trial design has received substantial attention in recent years. Although various extensions to the original design have been proposed, no guidance is available on the design of stepped-wedge cluster randomised trials with interim analyses. In an individually randomised trial setting, group sequential methods can provide notable efficiency gains and ethical benefits. We address this by discussing how established group sequential methodology can be adapted for stepped-wedge designs.

METHODS

Utilising the error spending approach to group sequential trial design, we detail the assumptions required for the determination of stepped-wedge cluster randomised trials with interim analyses. We consider early stopping for efficacy, futility, or efficacy and futility. We describe first how this can be done for any specified linear mixed model for data analysis. We then focus on one particular commonly utilised model and, using a recently completed stepped-wedge cluster randomised trial, compare the performance of several designs with interim analyses to the classical stepped-wedge design. Finally, the performance of a quantile substitution procedure for dealing with the case of unknown variance is explored.

RESULTS

We demonstrate that the incorporation of early stopping in stepped-wedge cluster randomised trial designs could reduce the expected sample size under the null and alternative hypotheses by up to 31% and 22%, respectively, with no cost to the trial's type-I and type-II error rates. The use of restricted error maximum likelihood estimation was found to be more important than quantile substitution for controlling the type-I error rate.

CONCLUSION

The addition of interim analyses into stepped-wedge cluster randomised trials could help guard against time-consuming trials conducted on poor performing treatments and also help expedite the implementation of efficacious treatments. In future, trialists should consider incorporating early stopping of some kind into stepped-wedge cluster randomised trials according to the needs of the particular trial.