Impact of CONSORT extension for cluster randomised trials on quality of reporting and study methodology: review of random sample of 300 trials, 2000-8

Estimates of intra-cluster correlation coefficients from 2018 USA Medicare data to inform the design of cluster randomized trials in Alzheimer's and related dementias

BACKGROUND

Cluster randomized trials (CRTs) are increasingly important for evaluating interventions embedded in health care systems. An essential parameter in sample size calculation to detect both overall and heterogeneous treatment effects for CRTs is the intra-cluster correlation coefficient (ICC) of both outcome and covariates of interest. However, obtaining advance estimates for the ICC can be challenging. When trial outcomes will be obtained from routinely collected data sources, there is an opportunity to obtain reliable ICC estimates in advance of the trial. Using USA national Medicare data, we estimated ICCs for a range of outcomes to inform the design of CRTs for people living with Alzheimer's and related dementias (ADRD).

METHOD

Data from 2018 Medicare Fee-for-Service beneficiaries, specifically, 1,898,812 individuals (≥ 65 years) with diagnosis of ADRD within 3436 hospital service areas (treated as clusters) and 306 hospital referral regions (treated as fixed strata), were used to calculate unadjusted and adjusted ICC estimates for three outcomes: death, any hospitalizations, and any emergency department (ED) visits and three covariates: age, race and sex. We present both overall and stratum-specific ICC estimates. We illustrate their use in sample size calculations for overall treatment effects as well as detecting treatment effect heterogeneity.

RESULTS

The unadjusted overall ICCs for death, hospitalizations, and ED visits were 0.001, 0.010, and 0.017 respectively. Stratum-specific ICCs varied widely across the 306 HRRs: median 0.001, 0.010 and 0.025 for death, hospitalizations, and ED visits respectively and 0.007, 0.001, and 0.080 for age, sex and race. An interactive R Shiny app is provided that allows users to retrieve estimates overlayed on a map of the USA.

CONCLUSIONS

We presented both adjusted and unadjusted ICCs for outcomes as well as unadjusted ICCs for covariates of potential interest from population-level data in the USA and demonstrated how the estimates may be used in sample size calculations for CRTs in ADRD.

Bayesian semi-parametric inference for clustered recurrent events with zero inflation and a terminal event

Recurrent events are common in clinical studies and are often subject to terminal events. In pragmatic trials, participants are often nested in clinics and can be susceptible or structurally unsusceptible to the recurrent events. We develop a Bayesian shared random effects model to accommodate this complex data structure. To achieve robustness, we consider the Dirichlet processes to model the residual of the accelerated failure time model for the survival process as well as the cluster-specific shared frailty distribution, along with an efficient sampling algorithm for posterior inference. Our method is applied to a recent cluster randomized trial on fall injury prevention.

Multiply robust generalized estimating equations for cluster randomized trials with missing outcomes

Generalized estimating equations (GEEs) provide a useful framework for estimating marginal regression parameters based on data from cluster randomized trials (CRTs), but they can result in inaccurate parameter estimates when some outcomes are informatively missing. Existing techniques to handle missing outcomes in CRTs rely on correct specification of a propensity score model, a covariate-conditional mean outcome model, or require at least one of these two models to be correct, which can be challenging in practice. In this article, we develop new weighted GEEs to simultaneously estimate the marginal mean, scale, and correlation parameters in CRTs with missing outcomes, allowing for multiple propensity score models and multiple covariate-conditional mean models to be specified. The resulting estimators are consistent provided that any one of these models is correct. An iterative algorithm is provided for implementing this more robust estimator and practical considerations for specifying multiple models are discussed. We evaluate the performance of the proposed method through Monte Carlo simulations and apply the proposed multiply robust estimator to analyze the Botswana Combination Prevention Project, a large HIV prevention CRT designed to evaluate whether a combination of HIV-prevention measures can reduce HIV incidence.

Estimating average treatment effects for clustered RCTs with recruitment bias

In clustered randomized controlled trials (RCTs), sample recruitment is often conducted after cluster randomization. This timing can lead to recruitment bias if access to the intervention affects the composition of study-eligible cluster entrants and study consenters. This article develops a potential outcomes framework in such settings that yields a causal estimand that pertains to the always-recruited in either research condition. A consistent inverse probability weighting (IPW) estimator is developed using data on recruits only, and a generalized estimating equations approach is used to obtain robust clustered SE estimators that adjust for estimation error in the IPW weights. A simple data collection strategy is discussed to improve the predictive accuracy of the logit propensity score models. Simulations show that the IPW estimator achieves nominal confidence interval coverage under the assumed identification conditions. An empirical application demonstrates the methods using data from an RCT testing the effects of a behavioral health intervention in schools. An R program for estimation is available for download.

Why and when should we cluster randomize?

A cluster randomized trial is defined as a randomized trial in which intact social units of individuals are randomized rather than individuals themselves. Outcomes are observed on individual participants within clusters (such as patients). Such a design allows assessing interventions targeting cluster-level participants (such as physicians), individual participants or both. Indeed, many interventions assessed in cluster randomized trials are actually complex ones, with distinct components targeting different levels. For a cluster-level intervention, cluster randomization is an obvious choice: the intervention is not divisible at the individual-level. For individual-level interventions, cluster randomization may nevertheless be suitable to prevent group contamination, for logistical reasons, to enhance participants' adherence, or when objectives pertain to the cluster level. An unacceptable reason for cluster randomization would be to avoid obtaining individual consent. Indeed, participants in cluster randomized trials have to be protected as in any type of trial design. Participants may be people from whom data are collected, but they may also be people who are intervened upon, and this includes both patients and physicians (for example, physicians receiving training interventions). Consent should be sought as soon as possible, although there may exist situations where participants may consent only for data collection, not for being exposed to the intervention (because, for instance, they cannot opt-out). There may even be situations where participants are not able to consent at all. In this latter situation a waiver of consent must be granted by a research ethics committee.

A mixed model approach to estimate the survivor average causal effect in cluster-randomized trials

In many medical studies, the outcome measure (such as quality of life, QOL) for some study participants becomes informatively truncated (censored, missing, or unobserved) due to death or other forms of dropout, creating a nonignorable missing data problem. In such cases, the use of a composite outcome or imputation methods that fill in unmeasurable QOL values for those who died rely on strong and untestable assumptions and may be conceptually unappealing to certain stakeholders when estimating a treatment effect. The survivor average causal effect (SACE) is an alternative causal estimand that surmounts some of these issues. While principal stratification has been applied to estimate the SACE in individually randomized trials, methods for estimating the SACE in cluster-randomized trials are currently limited. To address this gap, we develop a mixed model approach along with an expectation-maximization algorithm to estimate the SACE in cluster-randomized trials. We model the continuous outcome measure with a random intercept to account for intracluster correlations due to cluster-level randomization, and model the principal strata membership both with and without a random intercept. In simulations, we compare the performance of our approaches with an existing fixed-effects approach to illustrate the importance of accounting for clustering in cluster-randomized trials. The methodology is then illustrated using a cluster-randomized trial of telecare and assistive technology on health-related QOL in the elderly.

Leveraging baseline covariates to analyze small cluster-randomized trials with a rare binary outcome

Cluster-randomized trials (CRTs) involve randomizing entire groups of participants-called clusters-to treatment arms but are often comprised of a limited or fixed number of available clusters. While covariate adjustment can account for chance imbalances between treatment arms and increase statistical efficiency in individually randomized trials, analytical methods for individual-level covariate adjustment in small CRTs have received little attention to date. In this paper, we systematically investigate, through extensive simulations, the operating characteristics of propensity score weighting and multivariable regression as two individual-level covariate adjustment strategies for estimating the participant-average causal effect in small CRTs with a rare binary outcome and identify scenarios where each adjustment strategy has a relative efficiency advantage over the other to make practical recommendations. We also examine the finite-sample performance of the bias-corrected sandwich variance estimators associated with propensity score weighting and multivariable regression for quantifying the uncertainty in estimating the participant-average treatment effect. To illustrate the methods for individual-level covariate adjustment, we reanalyze a recent CRT testing a sedation protocol in 31 pediatric intensive care units.

Analysing cluster randomised controlled trials using GLMM, GEE1, GEE2, and QIF: results from four case studies

BACKGROUND

Using four case studies, we aim to provide practical guidance and recommendations for the analysis of cluster randomised controlled trials.

METHODS

Four modelling approaches (Generalized Linear Mixed Models with parameters estimated by maximum likelihood/restricted maximum likelihood; Generalized Linear Models with parameters estimated by Generalized Estimating Equations (1st order or second order) and Quadratic Inference Function, for analysing correlated individual participant level outcomes in cluster randomised controlled trials were identified after we reviewed the literature. We systematically searched the online bibliography databases of MEDLINE, EMBASE, PsycINFO (via OVID), CINAHL (via EBSCO), and SCOPUS. We identified the above-mentioned four statistical analytical approaches and applied them to four case studies of cluster randomised controlled trials with the number of clusters ranging from 10 to 100, and individual participants ranging from 748 to 9,207. Results were obtained for both continuous and binary outcomes using R and SAS statistical packages.

RESULTS

The intracluster correlation coefficient (ICC) estimates for the case studies were less than 0.05 and are consistent with the observed ICC values commonly reported in primary care and community-based cluster randomised controlled trials. In most cases, the four methods produced similar results. However, in a few analyses, quadratic inference function produced different results compared to the generalized linear mixed model, first-order generalized estimating equations, and second-order generalized estimating equations, especially in trials with small to moderate numbers of clusters.

CONCLUSION

This paper demonstrates the analysis of cluster randomised controlled trials with four modelling approaches. The results obtained were similar in most cases, however, for trials with few clusters we do recommend that the quadratic inference function should be used with caution, and where possible a small sample correction should be used. The generalisability of our results is limited to studies with similar features to our case studies, for example, studies with a similar-sized ICC. It is important to conduct simulation studies to comprehensively evaluate the performance of the four modelling approaches.

Estimating intra-cluster correlation coefficients for planning longitudinal cluster randomized trials: a tutorial

It is well-known that designing a cluster randomized trial (CRT) requires an advance estimate of the intra-cluster correlation coefficient (ICC). In the case of longitudinal CRTs, where outcomes are assessed repeatedly in each cluster over time, estimates for more complex correlation structures are required. Three common types of correlation structures for longitudinal CRTs are exchangeable, nested/block exchangeable and exponential decay correlations-the latter two allow the strength of the correlation to weaken over time. Determining sample sizes under these latter two structures requires advance specification of the within-period ICC and cluster autocorrelation coefficient as well as the intra-individual autocorrelation coefficient in the case of a cohort design. How to estimate these coefficients is a common challenge for investigators. When appropriate estimates from previously published longitudinal CRTs are not available, one possibility is to re-analyse data from an available trial dataset or to access observational data to estimate these parameters in advance of a trial. In this tutorial, we demonstrate how to estimate correlation parameters under these correlation structures for continuous and binary outcomes. We first introduce the correlation structures and their underlying model assumptions under a mixed-effects regression framework. With practical advice for implementation, we then demonstrate how the correlation parameters can be estimated using examples and we provide programming code in R, SAS, and Stata. An Rshiny app is available that allows investigators to upload an existing dataset and obtain the estimated correlation parameters. We conclude by identifying some gaps in the literature.

Restricted mean survival time to estimate an intervention effect in a cluster randomized trial

For time-to-event outcomes, the difference in restricted mean survival time is a measure of the intervention effect, an alternative to the hazard ratio, corresponding to the expected survival duration gain due to the intervention up to a predefined time t*. We extended two existing approaches of restricted mean survival time estimation for independent data to clustered data in the framework of cluster randomized trials: one based on the direct integration of Kaplan-Meier curves and the other based on pseudo-values regression. Then, we conducted a simulation study to assess and compare the statistical performance of the proposed methods, varying the number and size of clusters, the degree of clustering, and the magnitude of the intervention effect under proportional and non-proportional hazards assumption. We found that the extended methods well estimated the variance and controlled the type I error if there was a sufficient number of clusters (≥ 50) under both proportional and non-proportional hazards assumption. For cluster randomized trials with a limited number of clusters (< 50), a permutation test for pseudo-values regression was implemented and corrected the type I error. We also provided a procedure to estimate permutation-based confidence intervals which produced adequate coverage. All the extended methods performed similarly, but the pseudo-values regression offered the possibility to adjust for covariates. Finally, we illustrated each considered method with a cluster randomized trial evaluating the effectiveness of an asthma-control education program.

Use of guidelines, checklists, frameworks, and recommendations in behavioral intervention preliminary studies and associations with reporting comprehensiveness: a scoping bibliometric review

BACKGROUND

Guidelines, checklists, frameworks, and recommendations (GCFRs) related to preliminary studies serve as essential resources to assist behavioral intervention researchers in reporting findings from preliminary studies, but their impact on preliminary study reporting comprehensiveness is unknown. The purpose of this study was to conduct a scoping bibliometric review of recently published preliminary behavioral-focused intervention studies to (1) examine the prevalence of GCFR usage and (2) determine the associations between GCFR usage and reporting feasibility-related characteristics.

METHODS

A systematic search was conducted for preliminary studies of behavioral-focused interventions published between 2018 and 2020. Studies were limited to the top 25 journals publishing behavioral-focused interventions, text mined to identify usage of GCFRs, and categorized as either not citing GCFRs or citing ≥ 2 GCFRs (Citers). A random sample of non-Citers was text mined to identify studies which cited other preliminary studies that cited GCFRs (Indirect Citers) and those that did not (Never Citers). The presence/absence of feasibility-related characteristics was compared between Citers, Indirect Citers, and Never Citers via univariate logistic regression.

RESULTS

Studies (n = 4143) were identified, and 1316 were text mined to identify GCFR usage (n = 167 Citers). A random sample of 200 studies not citing a GCFR were selected and categorized into Indirect Citers (n = 71) and Never Citers (n = 129). Compared to Never Citers, Citers had higher odds of reporting retention, acceptability, adverse events, compliance, cost, data collection feasibility, and treatment fidelity (ORrange = 2.62-14.15, p < 0.005). Citers also had higher odds of mentioning feasibility in purpose statements, providing progression criteria, framing feasibility as the primary outcome, and mentioning feasibility in conclusions (ORrange = 6.31-17.04, p < 0.005) and lower odds of mentioning efficacy in purpose statements, testing for efficacy, mentioning efficacy in conclusions, and suggesting future testing (ORrange = 0.13-0.54, p < 0.05). Indirect Citers had higher odds of reporting acceptability and treatment fidelity (ORrange = 2.12-2.39, p < 0.05) but lower odds of testing for efficacy (OR = 0.36, p < 0.05) compared to Never Citers.

CONCLUSION

The citation of GCFRs is associated with greater reporting of feasibility-related characteristics in preliminary studies of behavioral-focused interventions. Researchers are encouraged to use and cite literature that provides guidance on design, implementation, analysis, and reporting to improve the comprehensiveness of reporting for preliminary studies.

Comparing analytical strategies for balancing site-level characteristics in stepped-wedge cluster randomized trials: a simulation study

BACKGROUND

Stepped-wedge cluster randomized trials (SWCRTs) are a type of cluster-randomized trial in which clusters are randomized to cross-over to the active intervention sequentially at regular intervals during the study period. For SWCRTs, sequential imbalances of cluster-level characteristics across the random sequence of clusters may lead to biased estimation. Our study aims to examine the effects of balancing cluster-level characteristics in SWCRTs.

METHODS

To quantify the level of cluster-level imbalance, a novel imbalance index was developed based on the Spearman correlation and rank regression of the cluster-level characteristic with the cross-over timepoints. A simulation study was conducted to assess the impact of sequential cluster-level imbalances across different scenarios varying the: number of sites (clusters), sample size, number of cross-over timepoints, site-level intra-cluster correlation coefficient (ICC), and effect sizes. SWCRTs assumed either an immediate "constant" treatment effect, or a gradual "learning" treatment effect which increases over time after crossing over to the active intervention. Key performance metrics included the relative root mean square error (RRMSE) and relative mean bias.

RESULTS

Fully-balanced designs almost always had the highest efficiency, as measured by the RRMSE, regardless of the number of sites, ICC, effect size, or sample sizes at each time for SWCRTs with learning effect. A consistent decreasing trend of efficiency was observed by increasing RRMSE as imbalance increased. For example, for a 12-site study with 20 participants per site/timepoint and ICC of 0.10, between the most balanced and least balanced designs, the RRMSE efficiency loss ranged from 52.5% to 191.9%. In addition, the RRMSE was decreased for larger sample sizes, larger number of sites, smaller ICC, and larger effect sizes. The impact of pre-balancing diminished when there was no learning effect.

CONCLUSION

The impact of pre-balancing on preventing efficiency loss was easily observed when there was a learning effect. This suggests benefit of pre-balancing with respect to impacting factors of treatment effects.

Power analysis for cluster randomized trials with continuous coprimary endpoints

Pragmatic trials evaluating health care interventions often adopt cluster randomization due to scientific or logistical considerations. Systematic reviews have shown that coprimary endpoints are not uncommon in pragmatic trials but are seldom recognized in sample size or power calculations. While methods for power analysis based on K (K ≥ 2 $K\ge 2$ ) binary coprimary endpoints are available for cluster randomized trials (CRTs), to our knowledge, methods for continuous coprimary endpoints are not yet available. Assuming a multivariate linear mixed model (MLMM) that accounts for multiple types of intraclass correlation coefficients among the observations in each cluster, we derive the closed-form joint distribution of K treatment effect estimators to facilitate sample size and power determination with different types of null hypotheses under equal cluster sizes. We characterize the relationship between the power of each test and different types of correlation parameters. We further relax the equal cluster size assumption and approximate the joint distribution of the K treatment effect estimators through the mean and coefficient of variation of cluster sizes. Our simulation studies with a finite number of clusters indicate that the predicted power by our method agrees well with the empirical power, when the parameters in the MLMM are estimated via the expectation-maximization algorithm. An application to a real CRT is presented to illustrate the proposed method.

Improving social justice in observational studies: protocol for the development of a global and Indigenous STROBE-equity reporting guideline

BACKGROUND

Addressing persistent and pervasive health inequities is a global moral imperative, which has been highlighted and magnified by the societal and health impacts of the COVID-19 pandemic. Observational studies can aid our understanding of the impact of health and structural oppression based on the intersection of gender, race, ethnicity, age and other factors, as they frequently collect this data. However, the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guideline, does not provide guidance related to reporting of health equity. The goal of this project is to develop a STROBE-Equity reporting guideline extension.

METHODS

We assembled a diverse team across multiple domains, including gender, age, ethnicity, Indigenous background, disciplines, geographies, lived experience of health inequity and decision-making organizations. Using an inclusive, integrated knowledge translation approach, we will implement a five-phase plan which will include: (1) assessing the reporting of health equity in published observational studies, (2) seeking wide international feedback on items to improve reporting of health equity, (3) establishing consensus amongst knowledge users and researchers, (4) evaluating in partnership with Indigenous contributors the relevance to Indigenous peoples who have globally experienced the oppressive legacy of colonization, and (5) widely disseminating and seeking endorsement from relevant knowledge users. We will seek input from external collaborators using social media, mailing lists and other communication channels.

DISCUSSION

Achieving global imperatives such as the Sustainable Development Goals (e.g., SDG 10 Reduced inequalities, SDG 3 Good health and wellbeing) requires advancing health equity in research. The implementation of the STROBE-Equity guidelines will enable a better awareness and understanding of health inequities through better reporting. We will broadly disseminate the reporting guideline with tools to enable adoption and use by journal editors, authors, and funding agencies, using diverse strategies tailored to specific audiences.

Improving sandwich variance estimation for marginal Cox analysis of cluster randomized trials

Cluster randomized trials (CRTs) frequently recruit a small number of clusters, therefore necessitating the application of small-sample corrections for valid inference. A recent systematic review indicated that CRTs reporting right-censored, time-to-event outcomes are not uncommon and that the marginal Cox proportional hazards model is one of the common approaches used for primary analysis. While small-sample corrections have been studied under marginal models with continuous, binary, and count outcomes, no prior research has been devoted to the development and evaluation of bias-corrected sandwich variance estimators when clustered time-to-event outcomes are analyzed by the marginal Cox model. To improve current practice, we propose nine bias-corrected sandwich variance estimators for the analysis of CRTs using the marginal Cox model and report on a simulation study to evaluate their small-sample properties. Our results indicate that the optimal choice of bias-corrected sandwich variance estimator for CRTs with survival outcomes can depend on the variability of cluster sizes and can also slightly differ whether it is evaluated according to relative bias or type I error rate. Finally, we illustrate the new variance estimators in a real-world CRT where the conclusion about intervention effectiveness differs depending on the use of small-sample bias corrections. The proposed sandwich variance estimators are implemented in an R package CoxBcv.

Fostering Resilience in Adolescents at Risk: Study protocol for a cluster randomized controlled trial within the resilience school-based intervention

Introduction

Resilience is considered of high relevance when developing interventions to cope with stressful situations. Schools are one of the key settings to promote resilience among adolescents. The purpose of this cluster randomized controlled trial is to assess the effectiveness of an intervention in adolescents at risk, aged 12-to-15 years old, to increase resilience and emotional regulation strategies.

Methods

The recruitment period started in January 2022. Schools will be randomly allocated to control and intervention groups by an external researcher using computer-generated random numbers. The minimum sample size was estimated to be 70 participants per group. Primary health care nurses will carry out the intervention during the school period (January to June 2022). Students will follow a specific training consisting of six 55-min sessions, for 6 weeks. Each session will consist of 5 min of mindfulness, followed by 45 min of the corresponding activity: introducing resilience, self-esteem, emotional regulation strategies, social skills, problem-solving, community resources, social and peer support, and 5 min to explain the activity to do at home. Data will be collected at baseline, 6 weeks, and 24 weeks after the intervention. The child youth resilience measure-32 (CYRM-32) scale will be used to assess the effectiveness of the intervention. This study received a grant in June 2021.

Discussion

The intervention is intended to improve mental health through resilience. Different factors related to resilience will be promoted, such as self-esteem, emotional regulation, social and communication skills, problem-solving and peer support, among others. As it has been designed as a cluster-randomized school-based intervention, we will directly ameliorate the participation and engagement of the target population. With the present intervention, we expect to improve coping skills in adolescents by enhancing resilience capacities.

Cross-sectional study of the quality of randomized control trials on problem-based learning in medical education

Problem-based learning (PBL) is increasingly being used in medical education globally, but its effectiveness in teaching remains controversial. A randomized controlled trial (RCT) is the method of choice for evaluating its effectiveness. The quality of an RCT has a significant effect on this evaluation, but to date we have not seen an assessment of the quality of RCTs for PBL. Two researchers searched MEDLINE and EMBASE for RCTs addressing PBL in medical education. The overall quality of each report was measured on a 28-point overall quality score (OQS) based on the 2010 revised Comprehensive Standards for Reporting and Testing (CONSORT) Statement. Furthermore, to study the key factors affecting OQS more effectively, a linear regression model of those factors was established using SPSS. After literature screening, 30 RCTs were eventually included and analyzed. The median OQS was 15 (range, 7-20), which meant that half of the items in the revised 2010 CONSORT statement were poorly reported in at least 40% of the RCTs analyzed. The regression model showed that the year of publication of RCTs and the impact factors of the journals in which they were published were the main factors affecting OQS. The overall quality of reporting of RCTs on PBL teaching in medical education was not satisfactory. Some RCTs were subjectively selective in reporting certain items, leading to heterogeneity in quality. It is expected that statisticians will develop new standards more suitable for evaluating RCTs related to teaching research and that editors and peer reviewers will be required to review the relevant RCTs more strictly.

The reporting and methodological quality of split-mouth trials in oral implantology: A methodological study

OBJECTIVES

The objective of this study is to assess the reporting and methodological quality of split-mouth trials (SMTs) in oral implantology published during the past 10 years, and to investigate whether there was any improvement over time.

MATERIALS AND METHODS

We searched PubMed for SMTs in oral implantology published during 2011-20. We used CONSORT 2010, its extension for within-person trial (WPT), and an SMT-specific methodological checklist to assess trial reporting quality (TRQ), WPT-specific reporting quality (WRQ), and SMT-specific methodological quality (SMQ), respectively. Binary scores were given to each item, and total scores of TRQ (range 0-32), WRQ (0-15), and SMQ (0-3) were calculated for each study. Multivariable regression analyses were performed to compare the quality of SMTs published before (2011-17) and after (2018-20) the release of CONSORT for WPT.

RESULTS

Seventy-nine SMTs were included. The mean TRQ, WRQ, and SMQ were 16.4, 6.7, and 1.3, respectively. Less than one-third (n = 25, 31.6%) reported the rationale for using split-mouth designs. Only 4 (5.1%) trials adequately conducted sample size calculation, and 40 (50.6%) used appropriate statistical methods that considered dependency and clustering of data. In multivariable analyses, compared with 2011-17, studies published in 2018-20 had significantly higher TRQ (p = .044), while WRQ and SMQ did not show improvement.

CONCLUSIONS

The reporting and methodological quality of SMTs in oral implantology need to be improved. Joint efforts are needed to improve the reporting and methodology of SMTs in this field.

Reporting quality for abstracts of randomised trials on child and adolescent depression prevention: a meta-epidemiological study on adherence to CONSORT for abstracts

OBJECTIVES

This study aimed to investigate adherence to Consolidated Standards of Reporting Trials (CONSORT) for abstracts in reports of randomised trials on child and adolescent depression prevention. Secondary objective was to examine factors associated with overall reporting quality.

DESIGN

Meta-epidemiological study.

DATA SOURCES

We searched MEDLINE, EMBASE, PsycINFO, PsycArticles and CENTRAL.

ELIGIBILITY CRITERIA

Trials were eligible if the sample consisted of children and adolescents under 18 years with or without an increased risk for depression or subthreshold depression. We included reports published from 1 January 2003 to 8 August 2020 on randomised controlled trials (RCTs) and cluster randomised trials (CRTs) assessing universal, selective and indicated interventions aiming to prevent the onset of depression or reducing depressive symptoms.

DATA EXTRACTION AND SYNTHESIS

As the primary outcome measure, we assessed for each trial abstract whether information recommended by CONSORT was adequately reported, inadequately reported or not reported. Moreover, we calculated a summative score of overall reporting quality and analysed associations with trial and journal characteristics.

RESULTS

We identified 169 eligible studies, 103 (61%) RCTs and 66 (39%) CRTs. Adequate reporting varied considerably across CONSORT items: while 9 out of 10 abstracts adequately reported the study objective, no abstract adequately provided information on blinding. Important adverse events or side effects were only adequately reported in one out of 169 abstracts. Summative scores for the abstracts' overall reporting quality ranged from 17% to 83%, with a median of 40%. Scores were associated with the number of authors, abstract word count, journal impact factor, year of publication and abstract structure.

CONCLUSIONS

Reporting quality for abstracts of trials on child and adolescent depression prevention is suboptimal. To help health professionals make informed judgements, efforts for improving adherence to reporting guidelines for abstracts are needed.

Influential methods reports for group-randomized trials and related designs

BACKGROUND

This article identifies the most influential methods reports for group-randomized trials and related designs published through 2020. Many interventions are delivered to participants in real or virtual groups or in groups defined by a shared interventionist so that there is an expectation for positive correlation among observations taken on participants in the same group. These interventions are typically evaluated using a group- or cluster-randomized trial, an individually randomized group treatment trial, or a stepped wedge group- or cluster-randomized trial. These trials face methodological issues beyond those encountered in the more familiar individually randomized controlled trial.

METHODS

PubMed was searched to identify candidate methods reports; that search was supplemented by reports known to the author. Candidate reports were reviewed by the author to include only those focused on the designs of interest. Citation counts and the relative citation ratio, a new bibliometric tool developed at the National Institutes of Health, were used to identify influential reports. The relative citation ratio measures influence at the article level by comparing the citation rate of the reference article to the citation rates of the articles cited by other articles that also cite the reference article.

RESULTS

In total, 1043 reports were identified that were published through 2020. However, 55 were deemed to be the most influential based on their relative citation ratio or their citation count using criteria specific to each of the three designs, with 32 group-randomized trial reports, 7 individually randomized group treatment trial reports, and 16 stepped wedge group-randomized trial reports. Many of the influential reports were early publications that drew attention to the issues that distinguish these designs from the more familiar individually randomized controlled trial. Others were textbooks that covered a wide range of issues for these designs. Others were "first reports" on analytic methods appropriate for a specific type of data (e.g. binary data, ordinal data), for features commonly encountered in these studies (e.g. unequal cluster size, attrition), or for important variations in study design (e.g. repeated measures, cohort versus cross-section). Many presented methods for sample size calculations. Others described how these designs could be applied to a new area (e.g. dissemination and implementation research). Among the reports with the highest relative citation ratios were the CONSORT statements for each design.

CONCLUSIONS

Collectively, the influential reports address topics of great interest to investigators who might consider using one of these designs and need guidance on selecting the most appropriate design for their research question and on the best methods for design, analysis, and sample size.

A comparison of analytical strategies for cluster randomized trials with survival outcomes in the presence of competing risks

While statistical methods for analyzing cluster randomized trials with continuous and binary outcomes have been extensively studied and compared, little comparative evidence has been provided for analyzing cluster randomized trials with survival outcomes in the presence of competing risks. Motivated by the Strategies to Reduce Injuries and Develop Confidence in Elders trial, we carried out a simulation study to compare the operating characteristics of several existing population-averaged survival models, including the marginal Cox, marginal Fine and Gray, and marginal multi-state models. For each model, we found that adjusting for the intraclass correlations through the sandwich variance estimator effectively maintained the type I error rate when the number of clusters is large. With no more than 30 clusters, however, the sandwich variance estimator can exhibit notable negative bias, and a permutation test provides better control of type I error inflation. Under the alternative, the power for each model is differentially affected by two types of intraclass correlations-the within-individual and between-individual correlations. Furthermore, the marginal Fine and Gray model occasionally leads to higher power than the marginal Cox model or the marginal multi-state model, especially when the competing event rate is high. Finally, we provide an illustrative analysis of Strategies to Reduce Injuries and Develop Confidence in Elders trial using each analytical strategy considered.

Clustered restricted mean survival time regression

For multicenter randomized trials or multilevel observational studies, the Cox regression model has long been the primary approach to study the effects of covariates on time-to-event outcomes. A critical assumption of the Cox model is the proportionality of the hazard functions for modeled covariates, violations of which can result in ambiguous interpretations of the hazard ratio estimates. To address this issue, the restricted mean survival time (RMST), defined as the mean survival time up to a fixed time in a target population, has been recommended as a model-free target parameter. In this article, we generalize the RMST regression model to clustered data by directly modeling the RMST as a continuous function of restriction times with covariates while properly accounting for within-cluster correlations to achieve valid inference. The proposed method estimates regression coefficients via weighted generalized estimating equations, coupled with a cluster-robust sandwich variance estimator to achieve asymptotically valid inference with a sufficient number of clusters. In small-sample scenarios where a limited number of clusters are available, however, the proposed sandwich variance estimator can exhibit negative bias in capturing the variability of regression coefficient estimates. To overcome this limitation, we further propose and examine bias-corrected sandwich variance estimators to reduce the negative bias of the cluster-robust sandwich variance estimator. We study the finite-sample operating characteristics of proposed methods through simulations and reanalyze two multicenter randomized trials.

A review identified challenges distinguishing primary reports of randomized trials for meta-research: A proposal for improved reporting

Meta-research is the discipline of studying research itself. A core investigative tool in meta-research is the use of systematic or scoping reviews to study the characteristics, methods and reporting of primary research studies. In the context of identifying eligible publications for methodological reviews of randomized controlled trials (RCTs), a challenge is to efficiently distinguish the primary trial report - which reports results for the primary outcome - from other types of reports, including design papers and secondary or supplementary analyses, or what we collectively refer to as non-primary reports. This may not be a straightforward task and may contribute to inefficiencies in the review process. Here, we draw on our recent methodological review of over 13,000 records to identify primary reports of pragmatic RCTs. We offer recommendations to improve the reporting of RCTs to facilitate more efficient identification of primary trial reports. We suggest that future updates to existing CONSORT guidelines include consideration of multiple trial reports and recommendations to clarify the primary or non-primary nature of each report. Our recommendations, together with improved adherence to inclusion of the trial registration number in the abstract and citation of a protocol or previously published primary report, would facilitate the conduct of methodological reviews.

Finite-sample adjustments in variance estimators for clustered competing risks regression

The marginal Fine-Gray proportional subdistribution hazards model is a popular approach to directly study the association between covariates and the cumulative incidence function with clustered competing risks data, which often arise in multicenter randomized trials or multilevel observational studies. To account for the within-cluster correlations between failure times, the uncertainty of the regression parameters estimators is quantified by the robust sandwich variance estimator, which may have unsatisfactory performance with a limited number of clusters. To overcome this limitation, we propose four bias-corrected variance estimators to reduce the negative bias of the usual sandwich variance estimator, extending the bias-correction techniques from generalized estimating equations with noncensored exponential family outcomes to clustered competing risks outcomes. We further compare their finite-sample operating characteristics through simulations and two real data examples. In particular, we found the Mancl and DeRouen (MD) type sandwich variance estimator generally has the smallest bias. Furthermore, with a small number of clusters, the Wald t -confidence interval with the MD sandwich variance estimator carries close to nominal coverage for the cluster-level effect parameter. The t -confidence intervals based on the sandwich variance estimator with any one of the three types of multiplicative bias correction or the z -confidence interval with the Morel, Bokossa and Neerchal (MBN) type sandwich variance estimator have close to nominal coverage for the individual-level effect parameter. Finally, we develop a user-friendly R package crrcbcv implementing the proposed sandwich variance estimators to assist practical applications.

Sample size calculation in hierarchical 2×2 factorial trials with unequal cluster sizes

Motivated by a suicide prevention trial with hierarchical treatment allocation (cluster-level and individual-level treatments), we address the sample size requirements for testing the treatment effects as well as their interaction. We assume a linear mixed model, within which two types of treatment effect estimands (controlled effect and marginal effect) are defined. For each null hypothesis corresponding to an estimand, we derive sample size formulas based on large-sample z-approximation, and provide finite-sample modifications based on a t-approximation. We relax the equal cluster size assumption and express the sample size formulas as functions of the mean and coefficient of variation of cluster sizes. We show that the sample size requirement for testing the controlled effect of the cluster-level treatment is more sensitive to cluster size variability than that for testing the controlled effect of the individual-level treatment; the same observation holds for testing the marginal effects. In addition, we show that the sample size for testing the interaction effect is proportional to that for testing the controlled or the marginal effect of the individual-level treatment. We conduct extensive simulations to validate the proposed sample size formulas, and find the empirical power agrees well with the predicted power for each test. Furthermore, the t-approximations often provide better control of type I error rate with a small number of clusters. Finally, we illustrate our sample size formulas to design the motivating suicide prevention factorial trial. The proposed methods are implemented in the R package H2x2Factorial.

Sample size calculation for cluster randomized trials with zero-inflated count outcomes

Cluster randomized trials (CRT) have been widely employed in medical and public health research. Many clinical count outcomes, such as the number of falls in nursing homes, exhibit excessive zero values. In the presence of zero inflation, traditional power analysis methods for count data based on Poisson or negative binomial distribution may be inadequate. In this study, we present a sample size method for CRTs with zero-inflated count outcomes. It is developed based on GEE regression directly modeling the marginal mean of a zero-inflated Poisson outcome, which avoids the challenge of testing two intervention effects under traditional modeling approaches. A closed-form sample size formula is derived which properly accounts for zero inflation, ICCs due to clustering, unbalanced randomization, and variability in cluster size. Robust approaches, including t-distribution-based approximation and Jackknife re-sampling variance estimator, are employed to enhance trial properties under small sample sizes. Extensive simulations are conducted to evaluate the performance of the proposed method. An application example is presented in a real clinical trial setting.

Statistical analysis of publicly funded cluster randomised controlled trials: a review of the National Institute for Health Research Journals Library

BACKGROUND

In cluster randomised controlled trials (cRCTs), groups of individuals (rather than individuals) are randomised to minimise the risk of contamination and/or efficiently use limited resources or solve logistic and administrative problems. A major concern in the primary analysis of cRCT is the use of appropriate statistical methods to account for correlation among outcomes from a particular group/cluster. This review aimed to investigate the statistical methods used in practice for analysing the primary outcomes in publicly funded cluster randomised controlled trials, adherence to the CONSORT (Consolidated Standards of Reporting Trials) reporting guidelines for cRCTs and the recruitment abilities of the cluster trials design.

METHODS

We manually searched the United Kingdom's National Institute for Health Research (NIHR) online Journals Library, from 1 January 1997 to 15 July 2021 chronologically for reports of cRCTs. Information on the statistical methods used in the primary analyses was extracted. One reviewer conducted the search and extraction while the two other independent reviewers supervised and validated 25% of the total trials reviewed.

RESULTS

A total of 1942 reports, published online in the NIHR Journals Library were screened for eligibility, 118 reports of cRCTs met the initial inclusion criteria, of these 79 reports containing the results of 86 trials with 100 primary outcomes analysed were finally included. Two primary outcomes were analysed at the cluster-level using a generalized linear model. At the individual-level, the generalized linear mixed model was the most used statistical method (80%, 80/100), followed by regression with robust standard errors (7%) then generalized estimating equations (6%). Ninety-five percent (95/100) of the primary outcomes in the trials were analysed with appropriate statistical methods that accounted for clustering while 5% were not. The mean observed intracluster correlation coefficient (ICC) was 0.06 (SD, 0.12; range, - 0.02 to 0.63), and the median value was 0.02 (IQR, 0.001-0.060), although 42% of the observed ICCs for the analysed primary outcomes were not reported.

CONCLUSIONS

In practice, most of the publicly funded cluster trials adjusted for clustering using appropriate statistical method(s), with most of the primary analyses done at the individual level using generalized linear mixed models. However, the inadequate analysis and poor reporting of cluster trials published in the UK is still happening in recent times, despite the availability of the CONSORT reporting guidelines for cluster trials published over a decade ago.

Simple compared to covariate-constrained randomization methods in balancing baseline characteristics: a case study of randomly allocating 72 hemodialysis centers in a cluster trial

Background and aim

Some parallel-group cluster-randomized trials use covariate-constrained rather than simple randomization. This is done to increase the chance of balancing the groups on cluster- and patient-level baseline characteristics. This study assessed how well two covariate-constrained randomization methods balanced baseline characteristics compared with simple randomization.

Methods

We conducted a mock 3-year cluster-randomized trial, with no active intervention, that started April 1, 2014, and ended March 31, 2017. We included a total of 11,832 patients from 72 hemodialysis centers (clusters) in Ontario, Canada. We randomly allocated the 72 clusters into two groups in a 1:1 ratio on a single date using individual- and cluster-level data available until April 1, 2013. Initially, we generated 1000 allocation schemes using simple randomization. Then, as an alternative, we performed covariate-constrained randomization based on historical data from these centers. In one analysis, we restricted on a set of 11 individual-level prognostic variables; in the other, we restricted on principal components generated using 29 baseline historical variables.

We created 300,000 different allocations for the covariate-constrained randomizations, and we restricted our analysis to the 30,000 best allocations based on the smallest sum of the penalized standardized differences. We then randomly sampled 1000 schemes from the 30,000 best allocations. We summarized our results with each randomization approach as the median (25th and 75th percentile) number of balanced baseline characteristics. There were 156 baseline characteristics, and a variable was balanced when the between-group standardized difference was ≤ 10%.

Results

The three randomization techniques had at least 125 of 156 balanced baseline characteristics in 90% of sampled allocations. The median number of balanced baseline characteristics using simple randomization was 147 (142, 150). The corresponding value for covariate-constrained randomization using 11 prognostic characteristics was 149 (146, 151), while for principal components, the value was 150 (147, 151).

Conclusion

In this setting with 72 clusters, constraining the randomization using historical information achieved better balance on baseline characteristics compared with simple randomization; however, the magnitude of benefit was modest.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-021-05590-1.

Using cluster-robust standard errors when analyzing group-randomized trials with few clusters

Accounting for dependent observations in cluster-randomized trials (CRTs) using nested data is necessary in order to avoid misestimated standard errors resulting in questionable inferential statistics. Cluster-robust standard errors (CRSEs) are often used to address this issue. However, CRSEs are still well-known to underestimate standard errors for group-level variables when the number of clusters is low (e.g., < 50) and with CRTs, a small number of clusters, due to logistical or financial considerations, is the norm rather than the exception. Using a simulation with various conditions, we investigate the use of a small sample correction (i.e., CR2 estimator) proposed by Bell and McCaffrey (2002) together with empirically derived degrees of freedom estimates (dof_BM). Findings indicate that even with as few as 10 clusters, the CR2 estimator used with dof_BM yields generally unbiased results with acceptable type I error and coverage rates. Results show that coverage and type I error rates can be largely influenced by the choice of dof, not just the standard error adjustments. An applied example is provided together with R syntax to conduct the analysis. To facilitate the use of different CRSEs, a free graphical, menu-driven SPSS add-on to compute the various cluster-robust variance estimates can be downloaded from https://github.com/flh3/CR2/tree/master/SPSS .

Completeness of reporting and risks of overstating impact in cluster randomised trials: a systematic review

Overstating the impact of interventions through incomplete or inaccurate reporting can lead to inappropriate scale-up of interventions with low impact. Accurate reporting of the impact of interventions is of great importance in global health research to protect scarce resources. In global health, the cluster randomised trial design is commonly used to evaluate complex, multicomponent interventions, and outcomes are often binary. Complete reporting of impact for binary outcomes means reporting both relative and absolute measures. We did a systematic review to assess reporting practices and potential to overstate impact in contemporary cluster randomised trials with binary primary outcome. We included all reports registered in the Cochrane Central Register of Controlled Trials of two-arm parallel cluster randomised trials with at least one binary primary outcome that were published in 2017. Of 73 cluster randomised trials, most (60 [82%]) showed incomplete reporting. Of 64 cluster randomised trials for which it was possible to evaluate, most (40 [63%]) reported results in such a way that impact could be overstated. Care is needed to report complete evidence of impact for the many interventions evaluated using the cluster randomised trial design worldwide.

Characteristics and practices of school-based cluster randomised controlled trials for improving health outcomes in pupils in the United Kingdom: a methodological systematic review

Background

Cluster randomised trials (CRTs) are increasingly used to evaluate non-pharmacological interventions for improving child health. Although methodological challenges of CRTs are well documented, the characteristics of school-based CRTs with pupil health outcomes have not been systematically described. Our objective was to describe methodological characteristics of these studies in the United Kingdom (UK).

Methods

MEDLINE was systematically searched from inception to 30^th June 2020. Included studies used the CRT design in schools and measured primary outcomes on pupils. Study characteristics were described using descriptive statistics.

Results

Of 3138 articles identified, 64 were included. CRTs with pupil health outcomes have been increasingly used in the UK school setting since the earliest included paper was published in 1993; 37 (58%) studies were published after 2010. Of the 44 studies that reported information, 93% included state-funded schools. Thirty six (56%) were exclusively in primary schools and 24 (38%) exclusively in secondary schools. Schools were randomised in 56 studies, classrooms in 6 studies, and year groups in 2 studies. Eighty percent of studies used restricted randomisation to balance cluster-level characteristics between trial arms, but few provided justification for their choice of balancing factors. Interventions covered 11 different health areas; 53 (83%) included components that were necessarily administered to entire clusters. The median (interquartile range) number of clusters and pupils recruited was 31.5 (21 to 50) and 1308 (604 to 3201), respectively. In half the studies, at least one cluster dropped out. Only 26 (41%) studies reported the intra-cluster correlation coefficient (ICC) of the primary outcome from the analysis; this was often markedly different to the assumed ICC in the sample size calculation. The median (range) ICC for school clusters was 0.028 (0.0005 to 0.21).

Conclusions

The increasing pool of school-based CRTs examining pupil health outcomes provides methodological knowledge and highlights design challenges. Data from these studies should be used to identify the best school-level characteristics for balancing the randomisation. Better information on the ICC of pupil health outcomes is required to aid the planning of future CRTs. Improved reporting of the recruitment process will help to identify barriers to obtaining representative samples of schools.

Guidance for biostatisticians on their essential contributions to clinical and translational research protocol review

Rigorous scientific review of research protocols is critical to making funding decisions, and to the protection of both human and non-human research participants. Given the increasing complexity of research designs and data analysis methods, quantitative experts, such as biostatisticians, play an essential role in evaluating the rigor and reproducibility of proposed methods. However, there is a common misconception that a statistician's input is relevant only to sample size/power and statistical analysis sections of a protocol. The comprehensive nature of a biostatistical review coupled with limited guidance on key components of protocol review motived this work. Members of the Biostatistics, Epidemiology, and Research Design Special Interest Group of the Association for Clinical and Translational Science used a consensus approach to identify the elements of research protocols that a biostatistician should consider in a review, and provide specific guidance on how each element should be reviewed. We present the resulting review framework as an educational tool and guideline for biostatisticians navigating review boards and panels. We briefly describe the approach to developing the framework, and we provide a comprehensive checklist and guidance on review of each protocol element. We posit that the biostatistical reviewer, through their breadth of engagement across multiple disciplines and experience with a range of research designs, can and should contribute significantly beyond review of the statistical analysis plan and sample size justification. Through careful scientific review, we hope to prevent excess resource expenditure and risk to humans and animals on poorly planned studies.

Imputing intracluster correlation coefficients from a posterior predictive distribution is a feasible method of dealing with unit of analysis errors in a meta-analysis of cluster RCTs

BACKGROUND

Incorporating cluster randomized trials (CRTs) into meta-analyses is challenging because appropriate standard errors of study estimates accounting for clustering are not always reported. Systematic reviews of CRTs often use a single constant external estimate of the intraclass correlation coefficient (ICC) to adjust study estimate standard errors and facilitate meta-analyses; an approach that fails to account for possible variation of ICCs among studies and the imprecision with which they are estimated. Using a large systematic review of the effects of diabetes quality improvement interventions, we investigated whether we could better account for ICC variation and uncertainty in meta-analyzed effect estimates by imputing missing ICCs from a posterior predictive distribution constructed from a database of relevant ICCs.

METHODS

We constructed a dataset of ICC estimates from applicable studies. For outcomes with two or more available ICC estimates, we constructed posterior predictive ICC distributions in a Bayesian framework. For a selected continuous outcome, glycosylated hemoglobin (HbA1c), we compared the impact of incorporating a single constant ICC versus imputing ICCs drawn from the posterior predictive distribution when estimating the effect of intervention components on post treatment mean in a case study of diabetes quality improvement trials.

RESULTS

Using internal and external ICC estimates, we were able to construct a database of 59 ICCs for 12 of the 13 review outcomes (range 1-10 per outcome) and estimate the posterior predictive ICC distribution for 11 review outcomes. Synthesized results were not markedly changed by our approach for HbA1c.

CONCLUSION

Building posterior predictive distributions to impute missing ICCs is a feasible approach to facilitate principled meta-analyses of cluster randomized trials using prior data. Further work is needed to establish whether the application of these methods leads to improved review inferences for different reviews based on different factors (e.g., proportion of CRTs and CRTs with missing ICCs, different outcomes, variation and precision of ICCs).

Alternatives to Logistic Regression Models when Analyzing Cluster Randomized Trials with Binary Outcomes

Binary outcomes are often encountered when analyzing cluster randomized trials (CRTs). A common approach to obtaining the average treatment effect of an intervention may involve using a logistic regression model. We outline some interpretive and statistical challenges associated with using logistic regression and discuss two alternative/supplementary approaches for analyzing clustered data with binary outcomes: the linear probability model (LPM) and the modified Poisson regression model. In our simulation and applied example, all models use a standard error adjustment that is effective even if a low number of clusters is present. Simulation results show that both the LPM and modified Poisson regression models can provide unbiased point estimates with acceptable coverage and type I error rates even with as little as 20 clusters.

Bayesian statistics in the design and analysis of cluster randomised controlled trials and their reporting quality: a methodological systematic review

BACKGROUND

In a cluster randomised controlled trial (CRCT), randomisation units are "clusters" such as schools or GP practices. This has methodological implications for study design and statistical analysis, since clustering often leads to correlation between observations which, if not accounted for, can lead to spurious conclusions of efficacy/effectiveness. Bayesian methodology offers a flexible, intuitive framework to deal with such issues, but its use within CRCT design and analysis appears limited. This review aims to explore and quantify the use of Bayesian methodology in the design and analysis of CRCTs, and appraise the quality of reporting against CONSORT guidelines.

METHODS

We sought to identify all reported/published CRCTs that incorporated Bayesian methodology and papers reporting development of new Bayesian methodology in this context, without restriction on publication date or location. We searched Medline and Embase and the Cochrane Central Register of Controlled Trials (CENTRAL). Reporting quality metrics according to the CONSORT extension for CRCTs were collected, as well as demographic data, type and nature of Bayesian methodology used, journal endorsement of CONSORT guidelines, and statistician involvement.

RESULTS

Twenty-seven publications were included, six from an additional hand search. Eleven (40.7%) were reports of CRCT results: seven (25.9%) were primary results papers and four (14.8%) reported secondary results. Thirteen papers (48.1%) reported Bayesian methodological developments, the remaining three (11.1%) compared different methods. Four (57.1%) of the primary results papers described the method of sample size calculation; none clearly accounted for clustering. Six (85.7%) clearly accounted for clustering in the analysis. All results papers reported use of Bayesian methods in the analysis but none in the design or sample size calculation.

CONCLUSIONS

The popularity of the CRCT design has increased rapidly in the last twenty years but this has not been mirrored by an uptake of Bayesian methodology in this context. Of studies using Bayesian methodology, there were some differences in reporting quality compared to CRCTs in general, but this study provided insufficient data to draw firm conclusions. There is an opportunity to further develop Bayesian methodology for the design and analysis of CRCTs in order to expand the accessibility, availability, and, ultimately, use of this approach.

Sample size and power considerations for cluster randomized trials with count outcomes subject to right truncation

Cluster randomized trials (CRTs) are widely used in epidemiological and public health studies assessing population-level effect of group-based interventions. One important application of CRTs is the control of vector-borne disease, such as malaria. However, a particular challenge for designing these trials is that the primary outcome involves counts of episodes that are subject to right truncation. While sample size formulas have been developed for CRTs with clustered counts, they are not directly applicable when the counts are right truncated. To address this limitation, we discuss two marginal modeling approaches for the analysis of CRTs with truncated counts and develop two corresponding closed-form sample size formulas to facilitate the design of such trials. The proposed sample size formulas allow investigators to explore the power under a large number of scenarios without computationally intensive simulations. The proposed formulas are validated in extensive simulations. We further explore the implication of right truncation on power and apply the proposed formulas to illustrate the power calculation for a malaria control CRT where the primary outcome is subject to right truncation.

Characteristics and practices of school-based cluster randomised controlled trials for improving health outcomes in pupils in the UK: a systematic review protocol

INTRODUCTION

Cluster randomised trials (CRTs) are studies in which groups (clusters) of participants rather than the individuals themselves are randomised to trial arms. CRTs are becoming increasingly relevant for evaluating interventions delivered in school settings for improving the health of children. Schools are a convenient setting for health interventions targeted at children and the CRT design respects the clustered structure in schools (ie, pupils within classrooms/teachers within schools). Some of the methodological challenges of CRTs, such as ethical considerations for enrolment of children into trials and how best to handle the analysis of data from participants (pupils) that change clusters (schools), may be more salient for the school setting. A better understanding of the characteristics and methodological considerations of school-based CRTs of health interventions would inform the design of future similar studies. To our knowledge, this is the only systematic review to focus specifically on the characteristics and methodological practices of CRTs delivered in schools to evaluate interventions for improving health outcomes in pupils in the UK.

METHODS AND ANALYSIS

We will search for CRTs published from inception to 30 June 2020 inclusively indexed in MEDLINE (Ovid). We will identify relevant articles through title and abstract screening, and subsequent full-text screening for eligibility against predefined inclusion criteria. Disagreements will be resolved through discussion. Two independent reviewers will extract data for each study using a prepiloted data extraction form. Findings will be summarised using descriptive statistics and graphs.

ETHICS AND DISSEMINATION

This methodological systematic review does not require ethical approval as only secondary data extracted from papers will be analysed and the data are not linked to individual participants. After completion of the systematic review, the data will be analysed, and the findings disseminated through peer-reviewed publications and scientific meetings.

PROSPERO REGISTRATION NUMBER

CRD42020201792.

Rationale, Methodological Quality, and Reporting of Cluster-Randomized Controlled Trials in Critical Care Medicine: A Systematic Review

OBJECTIVE

Compared with individual-patient randomized controlled trials, cluster randomized controlled trials have unique methodological and ethical considerations. We evaluated the rationale, methodological quality, and reporting of cluster randomized controlled trials in critical care studies.

DATA SOURCES

Systematic searches of Medline, Embase, and Cochrane Central Register were performed.

STUDY SELECTION

We included all cluster randomized controlled trials conducted in adult, pediatric, or neonatal critical care units from January 2005 to September 2019.

DATA EXTRACTION

Two reviewers independently screened citations, reviewed full texts, protocols, and supplements of potentially eligible studies, abstracted data, and assessed methodology of included studies.

DATA SYNTHESIS

From 1,902 citations, 59 cluster randomized controlled trials met criteria. Most focused on quality improvement (24, 41%), antimicrobial therapy (9, 15%), or infection control (9, 15%) interventions. Designs included parallel-group (25, 42%), crossover (21, 36%), and stepped-wedge (13, 22%). Concealment of allocation was reported in 21 studies (36%). Thirteen studies (22%) reported at least one method of blinding. The median total sample size was 1,660 patients (interquartile range, 813-4,295); the median number of clusters was 12 (interquartile range, 5-24); and the median patients per cluster was 141 (interquartile range, 54-452). Sample size calculations were reported in 90% of trials, but only 54% met Consolidated Standards of Reporting Trials guidance for sample size reporting. Twenty-seven of the studies (46%) identified a fixed number of available clusters prior to trial commencement, and only nine (15%) prespecified both the number of clusters and patients required to detect the expected effect size. Overall, 36 trials (68%) achieved the total prespecified sample size. When analyzing data, 44 studies (75%) appropriately adjusted for clustering when analyzing the primary outcome. Only 12 (20%) reported an intracluster coefficient (median 0.047 [interquartile range, 0.01-0.13]).

CONCLUSIONS

Cluster randomized controlled trials in critical care typically involve a small and fixed number of relatively large clusters. The reporting of key methodological aspects of these trials is often inadequate.

Methodological review showed that time-to-event outcomes are often inadequately handled in cluster randomized trials

OBJECTIVES

To estimate the prevalence of time-to-event (TTE) outcomes in cluster randomized trials (CRTs) and to examine their statistical management.

STUDY DESIGN AND SETTING

We searched PubMed to identify primary reports of CRTs published in six major general medical journals (2013-2018). Nature of outcomes and, for TTE outcomes, statistical methods for sample size, analysis, and measures of intracluster correlation were extracted.

RESULTS

A TTE analysis was used in 17% of the CRTs (32/184) either as a primary or secondary outcome analysis, or in a sensitivity analysis. Among the five CRTs with a TTE primary outcome, two accounted for both intracluster correlation and the TTE nature of the outcome in sample size calculation; one reported a measure of intracluster correlation in the analysis. Among the 32 CRTs with a least one TTE analysis, 44% (14/32) accounted for clustering in all TTE analyses. We identified 12 additional CRTs in which there was at least one outcome not analyzed as TTE for which a TTE analysis might have been preferred.

CONCLUSION

TTE outcomes are not uncommon in CRTs but appropriate statistical methods are infrequently used. Our results suggest that further methodological development and explicit recommendations for TTE outcomes in CRTs are needed.

Receipt of infant HIV DNA PCR test results is associated with a reduction in retention of HIV-exposed infants in integrated HIV care and healthcare services: a quantitative sub-study nested within a cluster randomised trial in rural Malawi

Background

Retention of HIV-infected mothers in integrated HIV and healthcare facilities is effective at reducing mother-to-child-transmission (MTCT) of HIV. In the context of Option B+, we examined maternal and HIV-exposed infant retention across three study arms to 18 months postpartum: mother-and-infant clinics (MIP), MIP with short-messaging service (MIP + SMS) and standard of care (SOC). In particular, we focused on the impact of mothers receiving an infant’s HIV PCR test result on maternal and infant study retention.

Methods

A quantitative sub-study nested within a cluster randomised trial undertaken between May 2013 and August 2016 across 30 healthcare facilities in rural Malawi enrolling HIV-infected pregnant mothers and HIV-exposed infants on delivery, was performed. Survival probabilities of maternal and HIV-exposed infant study retention was estimated using Kaplan-Meier curves. Associations between mother’s receiving an infant’s HIV test result and in particular, an infant’s HIV-positive result on maternal and infant study retention were modelled using time-varying multivariate Cox regression.

Results

Four hundred sixty-one, 493, and 396 HIV-infected women and 386, 399, and 300 HIV-exposed infants were enrolled across study arms; MIP, MIP + SMS and SOC, respectively. A total of 47.5% of mothers received their infant’s HIV test results < 5 months postpartum. Receiving an infant’s HIV result by mothers was associated with a 70% increase in infant non-retention in the study compared with not receiving an infant’s result (HR = 1.70; P-value< 0.001). Receiving a HIV-positive result was associated with 3.12 times reduced infant retention compared with a HIV-negative result (P-value< 0.001). Of the infants with a HIV-negative test result, 87% were breastfed at their final study follow-up.

Conclusions

Receiving an infant’s HIV test result was a driving factor for reduced infant study retention, especially an infant’s HIV-positive test result. As most HIV-negative infants were still breastfed at their last follow-up, this indicates a large proportion of HIV-exposed infants were potentially at future risk of MTCT of HIV via breastfeeding but were unlikely to undergo follow-up HIV testing after breastfeeding cessation. Future studies to identify and address underlying factors associated with infant HIV testing and reduced infant retention could potentially improve infant retention in HIV/healthcare facilities.

Trial registration

Pan African Clinical Trial Registry: PACTR201312000678196.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-020-09973-y.

Analysis and reporting of stratified cluster randomized trials-a systematic survey

BACKGROUND

In order to correctly assess the effect of intervention from stratified cluster randomized trials (CRTs), it is necessary to adjust for both clustering and stratification, as failure to do so leads to misleading conclusions about the intervention effect. We have conducted a systematic survey to examine the current practices about analysis and reporting of stratified CRTs.

METHOD

We used the search terms to identify the stratified CRTs from MEDLINE since the inception to July 2019. In phase 1, we screened the title and abstract for English-only studies and selected, including the main results paper of the identified protocols, for the next phase. In phase 2, we screened the full text and selected studies for data abstraction. The data abstraction form was piloted and developed using the REDCap. We abstracted data on multiple design and methodological aspects of the study including whether the primary method adjusted for both clustering and stratification, reporting of sample size, randomization, and results.

RESULTS

We screened 2686 studies in the phase 1 and selected 286 studies for phase 2-among them 185 studies were selected for data abstraction. Most of the selected studies were two-arm 140/185 (76%) and parallel-group 165/185 (89%) trials. Among these 185 studies, 27 (15%) of them did not provide any sample size or power calculation, while 105 (57%) studies did not mention any method used for randomization within each stratum. Further, 43 (23%) and 150 (81%) of 185 studies did not provide the definition of all the strata, while more than 60% of the studies did not include all the stratification variable(s) in the flow chart or baseline characteristics table. More than half 114/185 (62%) of the studies did not adjust the primary method for both clustering and stratification.

CONCLUSION

Stratification helps to achieve the balance among intervention groups. However, to correctly assess the intervention effect from stratified CRTs, it is important to adjust the primary analysis for both stratification and clustering. There are significant deficiencies in the reporting of methodological aspects of stratified CRTs, which require substantial improvements in several areas including definition of strata, inclusion of stratification variable(s) in the flow chart or baseline characteristics table, and reporting the stratum-specific number of clusters and individuals in the intervention groups.

Reporting of key methodological and ethical aspects of cluster trials in hemodialysis require improvement: a systematic review

Background

The hemodialysis setting is suitable for trials that use cluster randomization, where intact groups of individuals are randomized. However, cluster randomized trials (CRTs) are complicated in their design, analysis, and reporting and can pose ethical challenges. We reviewed CRTs in the hemodialysis setting with respect to reporting of key methodological and ethical issues.

Methods

We conducted a systematic review of CRTs in the hemodialysis setting, published in English, between 2000 and 2019, and indexed in MEDLINE or Embase. Two reviewers extracted data, and study results were summarized using descriptive statistics.

Results

We identified 26 completed CRTs and five study protocols of CRTs. These studies randomized hemodialysis centers (n = 17, 55%), hemodialysis shifts (n = 12, 39%), healthcare providers (n = 1, 3%), and nephrology units (n = 1, 3%). Trials included a median of 28 clusters with a median cluster size of 20 patients. Justification for using a clustered design was provided by 15 trials (48%). Methods that accounted for clustering were used during sample size calculation in 14 (45%), during analyses in 22 (71%), and during both sample size calculation and analyses in 13 trials (42%). Among all CRTs, 26 (84%) reported receiving research ethics committee approval; patient consent was reported in 22 trials: 10 (32%) reported the method of consent for trial participation and 12 (39%) reported no details about how consent was obtained or its purpose. Four trials (13%) reported receiving waivers of consent, and the remaining 5 (16%) provided no or unclear information about the consent process.

Conclusion

There is an opportunity to improve the conduct and reporting of essential methodological and ethical issues in future CRTs in hemodialysis.

Review Registration

We conducted this systematic review using a pre-specified protocol that was not registered.

Incorporating pragmatic features into power analysis for cluster randomized trials with a count outcome

Cluster randomized designs are frequently employed in pragmatic clinical trials which test interventions in the full spectrum of everyday clinical settings in order to maximize applicability and generalizability. In this study, we propose to directly incorporate pragmatic features into power analysis for cluster randomized trials with count outcomes. The pragmatic features considered include arbitrary randomization ratio, overdispersion, random variability in cluster size, and unequal lengths of follow-up over which the count outcome is measured. The proposed method is developed based on generalized estimating equation (GEE) and it is advantageous in that the sample size formula retains a closed form, facilitating its implementation in pragmatic trials. We theoretically explore the impact of various pragmatic features on sample size requirements. An efficient Jackknife algorithm is presented to address the problem of underestimated variance by the GEE sandwich estimator when the number of clusters is small. We assess the performance of the proposed sample size method through extensive simulation and an application example to a real clinical trial is presented.

The adaptive designs CONSORT extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design

Adaptive designs (ADs) allow pre-planned changes to an ongoing trial without compromising the validity of conclusions and it is essential to distinguish pre-planned from unplanned changes that may also occur. The reporting of ADs in randomised trials is inconsistent and needs improving. Incompletely reported AD randomised trials are difficult to reproduce and are hard to interpret and synthesise. This consequently hampers their ability to inform practice as well as future research and contributes to research waste. Better transparency and adequate reporting will enable the potential benefits of ADs to be realised.This extension to the Consolidated Standards Of Reporting Trials (CONSORT) 2010 statement was developed to enhance the reporting of randomised AD clinical trials. We developed an Adaptive designs CONSORT Extension (ACE) guideline through a two-stage Delphi process with input from multidisciplinary key stakeholders in clinical trials research in the public and private sectors from 21 countries, followed by a consensus meeting. Members of the CONSORT Group were involved during the development process.The paper presents the ACE checklists for AD randomised trial reports and abstracts, as well as an explanation with examples to aid the application of the guideline. The ACE checklist comprises seven new items, nine modified items, six unchanged items for which additional explanatory text clarifies further considerations for ADs, and 20 unchanged items not requiring further explanatory text. The ACE abstract checklist has one new item, one modified item, one unchanged item with additional explanatory text for ADs, and 15 unchanged items not requiring further explanatory text.The intention is to enhance transparency and improve reporting of AD randomised trials to improve the interpretability of their results and reproducibility of their methods, results and inference. We also hope indirectly to facilitate the much-needed knowledge transfer of innovative trial designs to maximise their potential benefits. In order to encourage its wide dissemination this article is freely accessible on the BMJ and Trials journal websites."To maximise the benefit to society, you need to not just do research but do it well" Douglas G Altman.

The Adaptive designs CONSORT Extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design

Adaptive designs (ADs) allow pre-planned changes to an ongoing trial without compromising the validity of conclusions and it is essential to distinguish pre-planned from unplanned changes that may also occur. The reporting of ADs in randomised trials is inconsistent and needs improving. Incompletely reported AD randomised trials are difficult to reproduce and are hard to interpret and synthesise. This consequently hampers their ability to inform practice as well as future research and contributes to research waste. Better transparency and adequate reporting will enable the potential benefits of ADs to be realised.This extension to the Consolidated Standards Of Reporting Trials (CONSORT) 2010 statement was developed to enhance the reporting of randomised AD clinical trials. We developed an Adaptive designs CONSORT Extension (ACE) guideline through a two-stage Delphi process with input from multidisciplinary key stakeholders in clinical trials research in the public and private sectors from 21 countries, followed by a consensus meeting. Members of the CONSORT Group were involved during the development process.The paper presents the ACE checklists for AD randomised trial reports and abstracts, as well as an explanation with examples to aid the application of the guideline. The ACE checklist comprises seven new items, nine modified items, six unchanged items for which additional explanatory text clarifies further considerations for ADs, and 20 unchanged items not requiring further explanatory text. The ACE abstract checklist has one new item, one modified item, one unchanged item with additional explanatory text for ADs, and 15 unchanged items not requiring further explanatory text.The intention is to enhance transparency and improve reporting of AD randomised trials to improve the interpretability of their results and reproducibility of their methods, results and inference. We also hope indirectly to facilitate the much-needed knowledge transfer of innovative trial designs to maximise their potential benefits.

xtgeebcv: A command for bias-corrected sandwich variance estimation for GEE analyses of cluster randomized trials

Cluster randomized trials, where clusters (for example, schools or clinics) are randomized to comparison arms but measurements are taken on individuals, are commonly used to evaluate interventions in public health, education, and the social sciences. Analysis is often conducted on individual-level outcomes, and such analysis methods must consider that outcomes for members of the same cluster tend to be more similar than outcomes for members of other clusters. A popular individual-level analysis technique is generalized estimating equations (GEE). However, it is common to randomize a small number of clusters (for example, 30 or fewer), and in this case, the GEE standard errors obtained from the sandwich variance estimator will be biased, leading to inflated type I errors. Some bias-corrected standard errors have been proposed and studied to account for this finite-sample bias, but none has yet been implemented in Stata. In this article, we describe several popular bias corrections to the robust sandwich variance. We then introduce our newly created command, xtgeebcv, which will allow Stata users to easily apply finite-sample corrections to standard errors obtained from GEE models. We then provide examples to demonstrate the use of xtgeebcv. Finally, we discuss suggestions about which finite-sample corrections to use in which situations and consider areas of future research that may improve xtgeebcv.

An empirical comparison of methods for analyzing over-dispersed zero-inflated count data from stratified cluster randomized trials

Background

The assessment of methods for analyzing over-dispersed zero inflated count outcome has received very little or no attention in stratified cluster randomized trials. In this study, we performed sensitivity analyses to empirically compare eight methods for analyzing zero inflated over-dispersed count outcome from the Vitamin D and Osteoporosis Study (ViDOS) – originally designed to assess the feasibility of a knowledge translation intervention in long-term care home setting.

Method

Forty long-term care (LTC) homes were stratified and then randomized into knowledge translation (KT) intervention (19 homes) and control (21 homes) groups. The homes/clusters were stratified by home size (<250/> = 250) and profit status (profit/non-profit). The outcome of this study was number of falls measured at 6-month post-intervention. The following methods were used to assess the effect of KT intervention on number of falls: i) standard Poisson and negative binomial regression; ii) mixed-effects method with Poisson and negative binomial distribution; iii) generalized estimating equation (GEE) with Poisson and negative binomial; iv) zero inflated Poisson and negative binomial — with the latter used as a primary approach. All these methods were compared with or without adjusting for stratification.

Results

A total of 5,478 older people from 40 LTC homes were included in this study. The mean (=1) of the number of falls was smaller than the variance (=6). Also 72% and 46% of the number of falls were zero in the control and intervention groups, respectively. The direction of the estimated incidence rate ratios (IRRs) was similar for all methods. The zero inflated negative binomial yielded the lowest IRRs and narrowest 95% confidence intervals when adjusted for stratification compared to GEE and mixed-effect methods. Further, the widths of the 95% confidence intervals were narrower when the methods adjusted for stratification compared to the same method not adjusted for stratification.

Conclusion

The overall conclusion from the GEE, mixed-effect and zero inflated methods were similar. However, these methods differ in terms of effect estimate and widths of the confidence interval.

Trial registration

ClinicalTrials.gov: NCT01398527. Registered: 19 July 2011.

Mapping the nomenclature, methodology, and reporting of studies that review methods: a pilot methodological review

BACKGROUND

A relatively novel method of appraisal, methodological reviews (MRs) are used to synthesize information on the methods used in health research. There are currently no guidelines available to inform the reporting of MRs.

OBJECTIVES

This pilot review aimed to determine the feasibility of a full review and the need for reporting guidance for methodological reviews.

METHODS

Search strategy: We conducted a search of PubMed, restricted to 2017 to include the most recently published studies, using different search terms often used to describe methodological reviews: "literature survey" OR "meta-epidemiologic* review" OR "meta-epidemiologic* survey" OR "methodologic* review" OR "methodologic* survey" OR "systematic survey."Data extraction: Study characteristics including country, nomenclature, number of included studies, search strategy, a priori protocol use, and sampling methods were extracted in duplicate and summarized.Outcomes: Primary feasibility outcomes were the sensitivity and specificity of the search terms (criteria for success of feasibility set at sensitivity and specificity of ≥ 70%).Analysis: The estimates are reported as a point estimate (95% confidence interval).

RESULTS

Two hundred thirty-six articles were retrieved and 31 were included in the final analysis. The most accurate search term was "meta-epidemiological" (sensitivity [Sn] 48.39; 95% CI 31.97-65.16; specificity [Sp] 97.56; 94.42-98.95). The majority of studies were published by authors from Canada (n = 12, 38.7%), and Japan and USA (n = 4, 12.9% each). The median (interquartile range [IQR]) number of included studies in the MRs was 77 (13-1127). Reporting of a search strategy was done in most studies (n = 23, 74.2%). The use of a pre-published protocol (n = 7, 22.6%) or a justifiable sampling method (n = 5, 16.1%) occurred rarely.

CONCLUSIONS

Using the MR nomenclature identified, it is feasible to build a comprehensive search strategy and conduct a full review. Given the variation in reporting practices and nomenclature attributed to MRs, there is a need for guidance on standardized and transparent reporting of MRs. Future guideline development would likely include stakeholders from Canada, USA, and Japan.

Power analysis for cluster randomized trials with multiple binary co-primary endpoints

Cluster randomized trials (CRTs) are widely used in different areas of medicine and public health. Recently, with increasing complexity of medical therapies and technological advances in monitoring multiple outcomes, many clinical trials attempt to evaluate multiple co-primary endpoints. In this study, we present a power analysis method for CRTs with K ≥ 2 binary co-primary endpoints. It is developed based on the GEE (generalized estimating equation) approach, and three types of correlations are considered: inter-subject correlation within each endpoint, intra-subject correlation across endpoints, and inter-subject correlation across endpoints. A closed-form joint distribution of the K test statistics is derived, which facilitates the evaluation of power and type I error for arbitrarily constructed hypotheses. We further present a theorem that characterizes the relationship between various correlations and testing power. We assess the performance of the proposed power analysis method based on extensive simulation studies. An application example to a real clinical trial is presented.