Statistical analysis and handling of missing data in cluster randomized trials: a systematic review

Blurring cluster randomized trials and observational studies: Two-Stage TMLE for subsampling, missingness, and few independent units

Cluster randomized trials (CRTs) often enroll large numbers of participants; yet due to resource constraints, only a subset of participants may be selected for outcome assessment, and those sampled may not be representative of all cluster members. Missing data also present a challenge: if sampled individuals with measured outcomes are dissimilar from those with missing outcomes, unadjusted estimates of arm-specific endpoints and the intervention effect may be biased. Further, CRTs often enroll and randomize few clusters, limiting statistical power and raising concerns about finite sample performance. Motivated by SEARCH-TB, a CRT aimed at reducing incident tuberculosis infection, we demonstrate interlocking methods to handle these challenges. First, we extend Two-Stage targeted minimum loss-based estimation to account for three sources of missingness: (i) subsampling; (ii) measurement of baseline status among those sampled; and (iii) measurement of final status among those in the incidence cohort (persons known to be at risk at baseline). Second, we critically evaluate the assumptions under which subunits of the cluster can be considered the conditionally independent unit, improving precision and statistical power but also causing the CRT to behave like an observational study. Our application to SEARCH-TB highlights the real-world impact of different assumptions on measurement and dependence; estimates relying on unrealistic assumptions suggested the intervention increased the incidence of TB infection by 18% (risk ratio [RR]=1.18, 95% confidence interval [CI]: 0.85-1.63), while estimates accounting for the sampling scheme, missingness, and within community dependence found the intervention decreased the incident TB by 27% (RR=0.73, 95% CI: 0.57-0.92).

Assessing treatment effect heterogeneity in the presence of missing effect modifier data in cluster-randomized trials

Understanding whether and how treatment effects vary across subgroups is crucial to inform clinical practice and recommendations. Accordingly, the assessment of heterogeneous treatment effects based on pre-specified potential effect modifiers has become a common goal in modern randomized trials. However, when one or more potential effect modifiers are missing, complete-case analysis may lead to bias and under-coverage. While statistical methods for handling missing data have been proposed and compared for individually randomized trials with missing effect modifier data, few guidelines exist for the cluster-randomized setting, where intracluster correlations in the effect modifiers, outcomes, or even missingness mechanisms may introduce further threats to accurate assessment of heterogeneous treatment effect. In this article, the performance of several missing data methods are compared through a simulation study of cluster-randomized trials with continuous outcome and missing binary effect modifier data, and further illustrated using real data from the Work, Family, and Health Study. Our results suggest that multilevel multiple imputation and Bayesian multilevel multiple imputation have better performance than other available methods, and that Bayesian multilevel multiple imputation has lower bias and closer to nominal coverage than standard multilevel multiple imputation when there are model specification or compatibility issues.

[Special issue PRO] A demonstration of estimands and sensitivity analyses for time-to-deterioration of patient reported outcomes

In oncology trials, health-related quality of life (HRQoL), specifically patient-reported symptom burden and functional status, can support the interpretation of survival endpoints, such as progression-free survival. However, applying time-to-event endpoints to patient-reported outcomes (PRO) data is challenging. For example, in time-to-deterioration analyses clinical events such as disease progression are common in many settings and are often handled through censoring the patient at the time of occurrence; however, disease progression and HRQoL are often related leading to informative censoring. Special consideration to the definition of events and intercurrent events (ICEs) is necessary. In this work, we demonstrate time-to-deterioration of PRO estimands and sensitivity analyses to answer research questions using composite, hypothetical, and treatment policy strategies applied to a single endpoint of disease-related symptoms. Multiple imputation methods under both the missing-at-random and missing-not-at-random assumptions are used as sensitivity analyses of primary estimands. Hazard ratios ranged from 0.52 to 0.66 over all the estimands and sensitivity analyses modeling a robust treatment effect favoring the treatment in time to disease symptom deterioration or death. Differences in the estimands include how people who experience disease progression or discontinue the randomized treatment due to AEs are accounted for in the analysis. We use the estimand framework to define interpretable and principled approaches for different time-to-deterioration research questions and provide practical recommendations. Reporting the proportions of patient events and patient censoring by reason helps understand the mechanisms that drive the results, allowing for optimal interpretation.

Multiply robust generalized estimating equations for cluster randomized trials with missing outcomes

Generalized estimating equations (GEEs) provide a useful framework for estimating marginal regression parameters based on data from cluster randomized trials (CRTs), but they can result in inaccurate parameter estimates when some outcomes are informatively missing. Existing techniques to handle missing outcomes in CRTs rely on correct specification of a propensity score model, a covariate-conditional mean outcome model, or require at least one of these two models to be correct, which can be challenging in practice. In this article, we develop new weighted GEEs to simultaneously estimate the marginal mean, scale, and correlation parameters in CRTs with missing outcomes, allowing for multiple propensity score models and multiple covariate-conditional mean models to be specified. The resulting estimators are consistent provided that any one of these models is correct. An iterative algorithm is provided for implementing this more robust estimator and practical considerations for specifying multiple models are discussed. We evaluate the performance of the proposed method through Monte Carlo simulations and apply the proposed multiply robust estimator to analyze the Botswana Combination Prevention Project, a large HIV prevention CRT designed to evaluate whether a combination of HIV-prevention measures can reduce HIV incidence.

Occupational Performance Coaching for Children With Neurodisability: A Randomized Controlled Trial Protocol

Background. Occupational Performance Coaching (OPC) is a goal-oriented approach in which client agency takes precedence in goal selection, analysis, choice of action, and evaluation of success. The intended outcomes of OPC are improved occupational performance and participation in clients' life situations. Randomized clinical trials are needed to determine the effectiveness of OPC. Purpose. This study protocol outlines a randomized controlled trial (RCT) of OPC compared to usual care with caregivers of children with neurodisability in improving child, caregiver, and family occupational performance. Method. A single-blind, 2-arm parallel-group, cluster RCT of OPC compared to usual care is planned. Therapists delivering the intervention (N = 14) are randomized to "OPC training" or "usual care" groups. The primary outcome is occupational performance improvement in caregiver (N = 84) identified goals. Implications. Findings will provide translational evidence of the effectiveness of OPC and clarify intervention processes. Areas of future OPC research and development will be indicated.

Missing values and inconclusive results in diagnostic studies - A scoping review of methods

Most diagnostic studies exclude missing values and inconclusive results from the analysis or apply simple methods resulting in biased accuracy estimates. This may be due to the lack of availability or awareness of appropriate methods. This scoping review aimed to provide an overview of strategies to handle missing values and inconclusive results in the reference standard or index test in diagnostic accuracy studies. Conducting a systematic literature search in MEDLINE, Cochrane Library, and Web of Science, we could identify many articles proposing methods for addressing missing values in the reference standard. There are also several articles describing methods regarding missing values or inconclusive results in the index test. The latter encompass imputation, frequentist and Bayesian likelihood, model-based, and latent class methods. While methods for missing values in the reference standard are regularly applied in practice, this is not true for methods addressing missing values and inconclusive results in the index test. Our comprehensive overview and description of available methods may raise further awareness of these methods and will enhance their application. Future research is needed to compare the performance of these methods under different conditions to give valid and robust recommendations for their usage in various diagnostic accuracy research scenarios.

Collaborating to Improve Neonatal Care: ParentAl Participation on the NEonatal Ward-Study Protocol of the neoPARTNER Study

Parents are often appointed a passive role in the care for their hospitalised child. In the family-integrated care (FICare) model, parental involvement in neonatal care is emulated. Parental participation in medical rounds, or family-centred rounds (FCR), forms a key element. A paucity remains of randomised trials assessing the outcomes of FCR (embedded in FICare) in families and neonates, and outcomes on an organisational level are relatively unexplored. Likewise, biological mechanisms through which a potential effect may be exerted are lacking robust evidence. Ten level two Dutch neonatal wards are involved in this stepped-wedge cluster-randomised trial FCR (embedded in FICare) by one common implementation strategy. Parents of infants hospitalised for at least 7 days are eligible for inclusion. The primary outcome is parental stress (PSS:NICU) at discharge. Secondary outcomes include parental, neonatal, healthcare professional and organisational outcomes. Biomarkers of stress will be analysed in parent-infant dyads. With a practical approach and broad outcome set, this study aims to obtain evidence on the possible (mechanistic) effect of FCR (as part of FICare) on parents, infants, healthcare professionals and organisations. The practical approach provides (experiences of) FICare material adjusted to the Dutch setting, available for other hospitals after the study.

Defining and estimating effects in cluster randomized trials: A methods comparison

Across research disciplines, cluster randomized trials (CRTs) are commonly implemented to evaluate interventions delivered to groups of participants, such as communities and clinics. Despite advances in the design and analysis of CRTs, several challenges remain. First, there are many possible ways to specify the causal effect of interest (eg, at the individual-level or at the cluster-level). Second, the theoretical and practical performance of common methods for CRT analysis remain poorly understood. Here, we present a general framework to formally define an array of causal effects in terms of summary measures of counterfactual outcomes. Next, we provide a comprehensive overview of CRT estimators, including the t-test, generalized estimating equations (GEE), augmented-GEE, and targeted maximum likelihood estimation (TMLE). Using finite sample simulations, we illustrate the practical performance of these estimators for different causal effects and when, as commonly occurs, there are limited numbers of clusters of different sizes. Finally, our application to data from the Preterm Birth Initiative (PTBi) study demonstrates the real-world impact of varying cluster sizes and targeting effects at the cluster-level or at the individual-level. Specifically, the relative effect of the PTBi intervention was 0.81 at the cluster-level, corresponding to a 19% reduction in outcome incidence, and was 0.66 at the individual-level, corresponding to a 34% reduction in outcome risk. Given its flexibility to estimate a variety of user-specified effects and ability to adaptively adjust for covariates for precision gains while maintaining Type-I error control, we conclude TMLE is a promising tool for CRT analysis.

Effectiveness of a comprehensive telemedicine intervention replacing standard care in gestational diabetes: a randomized controlled trial

AIMS

Telemedicine improves glycemic and perinatal outcomes when used as an adjunct to standard care in gestational diabetes (GDM). Little is known about its effectiveness when used instead of standard care. We aimed to compare the outcomes of telemedicine care and the standard care in women with GDM.

METHODS

In a single-center, parallel, randomized controlled trial, women were randomized to: (1) a telemedicine group, sending glucose readings via an application installed on a smartphone and monthly individual video calls replacing on-site visits or (2) standard care group with routine monthly on-site visits. The primary outcome was the effectiveness of glycemic control. The secondary outcomes were gestational weight gain (GWG) and perinatal data, including birth weight, gestational age, the incidence of the offspring large for gestational age, preterm birth, preeclampsia and cesarean section.

RESULTS

A total of 106 women were randomized to the telemedicine (n = 54) and the standard care group (n = 52). The telemedicine group demonstrated less postprandial measurements above the glycemic target (10.4% [3.9-17.9] vs. 14.6% [6.5-27.1]; p = 0.015), together with lower average postprandial glucose (5.6 ± 0.3 vs. 5.9 ± 0.4; p = 0.004). Percentage of cesarean section was lower in the telemedicine group (9 (17.3%) vs. 18 (35.3%); p = 0.038).

CONCLUSIONS

Telemedicine offers an effective alternative to delivering care to women with GDM. Trial registration NCT05521893, ClinicalTrials.gov Identifier URL: https://www.

CLINICALTRIALS

gov/ct2/show/NCT05521893?term=NCT05521893&draw=2&rank=1.

Accounting for expected attrition in the planning of cluster randomized trials for assessing treatment effect heterogeneity

BACKGROUND

Detecting treatment effect heterogeneity is an important objective in cluster randomized trials and implementation research. While sample size procedures for testing the average treatment effect accounting for participant attrition assuming missing completely at random or missing at random have been previously developed, the impact of attrition on the power for detecting heterogeneous treatment effects in cluster randomized trials remains unknown.

METHODS

We provide a sample size formula for testing for a heterogeneous treatment effect assuming the outcome is missing completely at random. We also propose an efficient Monte Carlo sample size procedure for assessing heterogeneous treatment effect assuming covariate-dependent outcome missingness (missing at random). We compare our sample size methods with the direct inflation method that divides the estimated sample size by the mean follow-up rate. We also evaluate our methods through simulation studies and illustrate them with a real-world example.

RESULTS

Simulation results show that our proposed sample size methods under both missing completely at random and missing at random provide sufficient power for assessing heterogeneous treatment effect. The proposed sample size methods lead to more accurate sample size estimates than the direct inflation method when the missingness rate is high (e.g., ≥ 30%). Moreover, sample size estimation under both missing completely at random and missing at random is sensitive to the missingness rate, but not sensitive to the intracluster correlation coefficient among the missingness indicators.

CONCLUSION

Our new sample size methods can assist in planning cluster randomized trials that plan to assess a heterogeneous treatment effect and participant attrition is expected to occur.

Keep It Up! 3.0: Study protocol for a type III hybrid implementation-effectiveness cluster-randomized trial

BACKGROUND

Despite evidence that eHealth approaches can be effective in reducing HIV risk, their implementation requirements for public health scale up are not well established, and effective strategies to bring these programs into practice are still unknown. Keep It Up! (KIU!) is an online program proven to reduce HIV risk among young men who have sex with men (YMSM) and ideal candidate to develop and evaluate novel strategies for implementing eHealth HIV prevention programs. KIU! 3.0 is a Type III Hybrid Effectiveness-Implementation cluster randomized trial designed to 1) compare two strategies for implementing KIU!: community-based organizations (CBO) versus centralized direct-to-consumer (DTC) recruitment; 2) examine the effect of strategies and determinants on variability in implementation success; and 3) develop materials for sustainment of KIU! after the trial concludes. In this article, we describe the approaches used to achieve these aims.

METHODS

Using county-level population estimates of YMSM, 66 counties were selected and randomized 2:1 to the CBO and DTC approaches. The RE-AIM model was used to drive outcome measurements, which were collected from CBO staff, YMSM, and technology providers. Mixed-methods research mapped onto the domains of the Consolidated Framework for Implementation Research will examine determinants and their relationship with implementation outcomes.

DISCUSSION

In comparing our implementation recruitment models, we are examining two strategies which have shown effectiveness in delivering health technology interventions in the past, yet little is known about their comparative advantages and disadvantages in implementation. The results of the trial will further the understanding of eHealth prevention intervention implementation.

Improving sandwich variance estimation for marginal Cox analysis of cluster randomized trials

Cluster randomized trials (CRTs) frequently recruit a small number of clusters, therefore necessitating the application of small-sample corrections for valid inference. A recent systematic review indicated that CRTs reporting right-censored, time-to-event outcomes are not uncommon and that the marginal Cox proportional hazards model is one of the common approaches used for primary analysis. While small-sample corrections have been studied under marginal models with continuous, binary, and count outcomes, no prior research has been devoted to the development and evaluation of bias-corrected sandwich variance estimators when clustered time-to-event outcomes are analyzed by the marginal Cox model. To improve current practice, we propose nine bias-corrected sandwich variance estimators for the analysis of CRTs using the marginal Cox model and report on a simulation study to evaluate their small-sample properties. Our results indicate that the optimal choice of bias-corrected sandwich variance estimator for CRTs with survival outcomes can depend on the variability of cluster sizes and can also slightly differ whether it is evaluated according to relative bias or type I error rate. Finally, we illustrate the new variance estimators in a real-world CRT where the conclusion about intervention effectiveness differs depending on the use of small-sample bias corrections. The proposed sandwich variance estimators are implemented in an R package CoxBcv.

Handling of missing data with multiple imputation in observational studies that address causal questions: protocol for a scoping review

INTRODUCTION

Observational studies in health-related research often aim to answer causal questions. Missing data are common in these studies and often occur in multiple variables, such as the exposure, outcome and/or variables used to control for confounding. The standard classification of missing data as missing completely at random, missing at random (MAR) or missing not at random does not allow for a clear assessment of missingness assumptions when missingness arises in more than one variable. This presents challenges for selecting an analytic approach and determining when a sensitivity analysis under plausible alternative missing data assumptions is required. This is particularly pertinent with multiple imputation (MI), which is often justified by assuming data are MAR. The objective of this scoping review is to examine the use of MI in observational studies that address causal questions, with a focus on if and how (a) missingness assumptions are expressed and assessed, (b) missingness assumptions are used to justify the choice of a complete case analysis and/or MI for handling missing data and (c) sensitivity analyses under alternative plausible assumptions about the missingness mechanism are conducted.

METHODS AND ANALYSIS

We will review observational studies that aim to answer causal questions and use MI, published between January 2019 and December 2021 in five top general epidemiology journals. Studies will be identified using a full text search for the term 'multiple imputation' and then assessed for eligibility. Information extracted will include details about the study characteristics, missing data, missingness assumptions and MI implementation. Data will be summarised using descriptive statistics.

ETHICS AND DISSEMINATION

Ethics approval is not required for this review because data will be collected only from published studies. The results will be disseminated through a peer reviewed publication and conference presentations.

TRIAL REGISTRATION NUMBER

This protocol is registered on figshare (https://doi.org/10.6084/m9.figshare.20010497.v1).

Sensitivity analyses for data missing at random versus missing not at random using latent growth modelling: a practical guide for randomised controlled trials

Background

Missing data are ubiquitous in randomised controlled trials. Although sensitivity analyses for different missing data mechanisms (missing at random vs. missing not at random) are widely recommended, they are rarely conducted in practice. The aim of the present study was to demonstrate sensitivity analyses for different assumptions regarding the missing data mechanism for randomised controlled trials using latent growth modelling (LGM).

Methods

Data from a randomised controlled brief alcohol intervention trial was used. The sample included 1646 adults (56% female; mean age = 31.0 years) from the general population who had received up to three individualized alcohol feedback letters or assessment-only. Follow-up interviews were conducted after 12 and 36 months via telephone. The main outcome for the analysis was change in alcohol use over time. A three-step LGM approach was used. First, evidence about the process that generated the missing data was accumulated by analysing the extent of missing values in both study conditions, missing data patterns, and baseline variables that predicted participation in the two follow-up assessments using logistic regression. Second, growth models were calculated to analyse intervention effects over time. These models assumed that data were missing at random and applied full-information maximum likelihood estimation. Third, the findings were safeguarded by incorporating model components to account for the possibility that data were missing not at random. For that purpose, Diggle-Kenward selection, Wu-Carroll shared parameter and pattern mixture models were implemented.

Results

Although the true data generating process remained unknown, the evidence was unequivocal: both the intervention and control group reduced their alcohol use over time, but no significant group differences emerged. There was no clear evidence for intervention efficacy, neither in the growth models that assumed the missing data to be at random nor those that assumed the missing data to be not at random.

Conclusion

The illustrated approach allows the assessment of how sensitive conclusions about the efficacy of an intervention are to different assumptions regarding the missing data mechanism. For researchers familiar with LGM, it is a valuable statistical supplement to safeguard their findings against the possibility of nonignorable missingness.

Trial registration

The PRINT trial was prospectively registered at the German Clinical Trials Register (DRKS00014274, date of registration: 12th March 2018).

A comparison of analytical strategies for cluster randomized trials with survival outcomes in the presence of competing risks

While statistical methods for analyzing cluster randomized trials with continuous and binary outcomes have been extensively studied and compared, little comparative evidence has been provided for analyzing cluster randomized trials with survival outcomes in the presence of competing risks. Motivated by the Strategies to Reduce Injuries and Develop Confidence in Elders trial, we carried out a simulation study to compare the operating characteristics of several existing population-averaged survival models, including the marginal Cox, marginal Fine and Gray, and marginal multi-state models. For each model, we found that adjusting for the intraclass correlations through the sandwich variance estimator effectively maintained the type I error rate when the number of clusters is large. With no more than 30 clusters, however, the sandwich variance estimator can exhibit notable negative bias, and a permutation test provides better control of type I error inflation. Under the alternative, the power for each model is differentially affected by two types of intraclass correlations-the within-individual and between-individual correlations. Furthermore, the marginal Fine and Gray model occasionally leads to higher power than the marginal Cox model or the marginal multi-state model, especially when the competing event rate is high. Finally, we provide an illustrative analysis of Strategies to Reduce Injuries and Develop Confidence in Elders trial using each analytical strategy considered.

Practical considerations in utilizing cluster randomized controlled trials conducted in biopharmaceutical industry

Cluster randomized controlled trials (cluster RCTs), also known as parallel-arm group-randomized trials, are trials in which the randomized units are groups of participants, as opposed to individual participants. These trials have largely been implemented to address broad public health issues, but with the growing interest in use of real-world data in the regulatory setting, this design may be increasingly considered for industry trials. The key difference between cluster RCTs and traditional RCTs is the intraclass correlation coefficient (ICC) that needs to be considered in cluster RCTs. In this article, we discuss some key practical considerations that are related to ICC in the design, conduct, analysis, and report stages of a cluster RCT. These key considerations related to ICC can lead to improvement in how we translate research findings from cluster RCTs into practices in the biopharmaceutical industry.

Sample size calculation in hierarchical 2×2 factorial trials with unequal cluster sizes

Motivated by a suicide prevention trial with hierarchical treatment allocation (cluster-level and individual-level treatments), we address the sample size requirements for testing the treatment effects as well as their interaction. We assume a linear mixed model, within which two types of treatment effect estimands (controlled effect and marginal effect) are defined. For each null hypothesis corresponding to an estimand, we derive sample size formulas based on large-sample z-approximation, and provide finite-sample modifications based on a t-approximation. We relax the equal cluster size assumption and express the sample size formulas as functions of the mean and coefficient of variation of cluster sizes. We show that the sample size requirement for testing the controlled effect of the cluster-level treatment is more sensitive to cluster size variability than that for testing the controlled effect of the individual-level treatment; the same observation holds for testing the marginal effects. In addition, we show that the sample size for testing the interaction effect is proportional to that for testing the controlled or the marginal effect of the individual-level treatment. We conduct extensive simulations to validate the proposed sample size formulas, and find the empirical power agrees well with the predicted power for each test. Furthermore, the t-approximations often provide better control of type I error rate with a small number of clusters. Finally, we illustrate our sample size formulas to design the motivating suicide prevention factorial trial. The proposed methods are implemented in the R package H2x2Factorial.

Statistical analysis of publicly funded cluster randomised controlled trials: a review of the National Institute for Health Research Journals Library

BACKGROUND

In cluster randomised controlled trials (cRCTs), groups of individuals (rather than individuals) are randomised to minimise the risk of contamination and/or efficiently use limited resources or solve logistic and administrative problems. A major concern in the primary analysis of cRCT is the use of appropriate statistical methods to account for correlation among outcomes from a particular group/cluster. This review aimed to investigate the statistical methods used in practice for analysing the primary outcomes in publicly funded cluster randomised controlled trials, adherence to the CONSORT (Consolidated Standards of Reporting Trials) reporting guidelines for cRCTs and the recruitment abilities of the cluster trials design.

METHODS

We manually searched the United Kingdom's National Institute for Health Research (NIHR) online Journals Library, from 1 January 1997 to 15 July 2021 chronologically for reports of cRCTs. Information on the statistical methods used in the primary analyses was extracted. One reviewer conducted the search and extraction while the two other independent reviewers supervised and validated 25% of the total trials reviewed.

RESULTS

A total of 1942 reports, published online in the NIHR Journals Library were screened for eligibility, 118 reports of cRCTs met the initial inclusion criteria, of these 79 reports containing the results of 86 trials with 100 primary outcomes analysed were finally included. Two primary outcomes were analysed at the cluster-level using a generalized linear model. At the individual-level, the generalized linear mixed model was the most used statistical method (80%, 80/100), followed by regression with robust standard errors (7%) then generalized estimating equations (6%). Ninety-five percent (95/100) of the primary outcomes in the trials were analysed with appropriate statistical methods that accounted for clustering while 5% were not. The mean observed intracluster correlation coefficient (ICC) was 0.06 (SD, 0.12; range, - 0.02 to 0.63), and the median value was 0.02 (IQR, 0.001-0.060), although 42% of the observed ICCs for the analysed primary outcomes were not reported.

CONCLUSIONS

In practice, most of the publicly funded cluster trials adjusted for clustering using appropriate statistical method(s), with most of the primary analyses done at the individual level using generalized linear mixed models. However, the inadequate analysis and poor reporting of cluster trials published in the UK is still happening in recent times, despite the availability of the CONSORT reporting guidelines for cluster trials published over a decade ago.

Improving mortality in older adult trauma patients: Are we doing better?

BACKGROUND

Older adult trauma is associated with high morbidity and mortality. Individuals older than 65 years are expected to make up more than 21% of the total population and almost 39% of trauma admissions by 2050. Our objective was to perform a national review of older adult trauma mortality and identify associated risk factors to highlight potential areas for improvement in quality of care.

MATERIALS AND METHODS

This is a retrospective cohort study of the National Trauma Data Bank including all patients age ≥65 years with at least one International Classification of Diseases, Ninth Revision, Clinical Modification trauma code admitted to a Level I or II US trauma center between 2007 and 2015. Variables examined included demographics, comorbidities, emergency department vitals, injury characteristics, and trauma center characteristics. Multilevel mixed-effect logistic regression was performed to identify independent risk factors of in-hospital mortality.

RESULTS

There were 1,492,759 patients included in this study. The number of older adult trauma patients increased from 88,056 in 2007 to 158,929 in 2015 (p > 0.001). Adjusted in-hospital mortality decreased in 2014 to 2015 (odds ratio [OR], 0.88; 95% confidence interval [CI], 0.86-0.91) when compared with 2007 to 2009. Admission to a university hospital was protective (OR, 0.83; 95% CI, 0.74-0.93) as compared with a community hospital admission. There was no difference in mortality risk between Level II and Level I admission (OR, 1.00; 95% CI, 0.92-1.08). The strongest trauma-related risk factor for in-patient mortality was pancreas/bowel injury (OR, 2.25; 95% CI, 2.04-2.49).

CONCLUSION

Mortality in older trauma patients is decreasing over time, indicating an improvement in the quality of trauma care. The outcomes of university based hospitals can be used as national benchmarks to guide quality metrics.

LEVEL OF EVIDENCE

Therapeutic/Care Management, Level IV.

Influential methods reports for group-randomized trials and related designs

BACKGROUND

This article identifies the most influential methods reports for group-randomized trials and related designs published through 2020. Many interventions are delivered to participants in real or virtual groups or in groups defined by a shared interventionist so that there is an expectation for positive correlation among observations taken on participants in the same group. These interventions are typically evaluated using a group- or cluster-randomized trial, an individually randomized group treatment trial, or a stepped wedge group- or cluster-randomized trial. These trials face methodological issues beyond those encountered in the more familiar individually randomized controlled trial.

METHODS

PubMed was searched to identify candidate methods reports; that search was supplemented by reports known to the author. Candidate reports were reviewed by the author to include only those focused on the designs of interest. Citation counts and the relative citation ratio, a new bibliometric tool developed at the National Institutes of Health, were used to identify influential reports. The relative citation ratio measures influence at the article level by comparing the citation rate of the reference article to the citation rates of the articles cited by other articles that also cite the reference article.

RESULTS

In total, 1043 reports were identified that were published through 2020. However, 55 were deemed to be the most influential based on their relative citation ratio or their citation count using criteria specific to each of the three designs, with 32 group-randomized trial reports, 7 individually randomized group treatment trial reports, and 16 stepped wedge group-randomized trial reports. Many of the influential reports were early publications that drew attention to the issues that distinguish these designs from the more familiar individually randomized controlled trial. Others were textbooks that covered a wide range of issues for these designs. Others were "first reports" on analytic methods appropriate for a specific type of data (e.g. binary data, ordinal data), for features commonly encountered in these studies (e.g. unequal cluster size, attrition), or for important variations in study design (e.g. repeated measures, cohort versus cross-section). Many presented methods for sample size calculations. Others described how these designs could be applied to a new area (e.g. dissemination and implementation research). Among the reports with the highest relative citation ratios were the CONSORT statements for each design.

CONCLUSIONS

Collectively, the influential reports address topics of great interest to investigators who might consider using one of these designs and need guidance on selecting the most appropriate design for their research question and on the best methods for design, analysis, and sample size.

Two-Stage TMLE to reduce bias and improve efficiency in cluster randomized trials

Cluster randomized trials (CRTs) randomly assign an intervention to groups of individuals (e.g., clinics or communities) and measure outcomes on individuals in those groups. While offering many advantages, this experimental design introduces challenges that are only partially addressed by existing analytic approaches. First, outcomes are often missing for some individuals within clusters. Failing to appropriately adjust for differential outcome measurement can result in biased estimates and inference. Second, CRTs often randomize limited numbers of clusters, resulting in chance imbalances on baseline outcome predictors between arms. Failing to adaptively adjust for these imbalances and other predictive covariates can result in efficiency losses. To address these methodological gaps, we propose and evaluate a novel two-stage targeted minimum loss-based estimator to adjust for baseline covariates in a manner that optimizes precision, after controlling for baseline and postbaseline causes of missing outcomes. Finite sample simulations illustrate that our approach can nearly eliminate bias due to differential outcome measurement, while existing CRT estimators yield misleading results and inferences. Application to real data from the SEARCH community randomized trial demonstrates the gains in efficiency afforded through adaptive adjustment for baseline covariates, after controlling for missingness on individual-level outcomes.

Simple compared to covariate-constrained randomization methods in balancing baseline characteristics: a case study of randomly allocating 72 hemodialysis centers in a cluster trial

Background and aim

Some parallel-group cluster-randomized trials use covariate-constrained rather than simple randomization. This is done to increase the chance of balancing the groups on cluster- and patient-level baseline characteristics. This study assessed how well two covariate-constrained randomization methods balanced baseline characteristics compared with simple randomization.

Methods

We conducted a mock 3-year cluster-randomized trial, with no active intervention, that started April 1, 2014, and ended March 31, 2017. We included a total of 11,832 patients from 72 hemodialysis centers (clusters) in Ontario, Canada. We randomly allocated the 72 clusters into two groups in a 1:1 ratio on a single date using individual- and cluster-level data available until April 1, 2013. Initially, we generated 1000 allocation schemes using simple randomization. Then, as an alternative, we performed covariate-constrained randomization based on historical data from these centers. In one analysis, we restricted on a set of 11 individual-level prognostic variables; in the other, we restricted on principal components generated using 29 baseline historical variables.

We created 300,000 different allocations for the covariate-constrained randomizations, and we restricted our analysis to the 30,000 best allocations based on the smallest sum of the penalized standardized differences. We then randomly sampled 1000 schemes from the 30,000 best allocations. We summarized our results with each randomization approach as the median (25th and 75th percentile) number of balanced baseline characteristics. There were 156 baseline characteristics, and a variable was balanced when the between-group standardized difference was ≤ 10%.

Results

The three randomization techniques had at least 125 of 156 balanced baseline characteristics in 90% of sampled allocations. The median number of balanced baseline characteristics using simple randomization was 147 (142, 150). The corresponding value for covariate-constrained randomization using 11 prognostic characteristics was 149 (146, 151), while for principal components, the value was 150 (147, 151).

Conclusion

In this setting with 72 clusters, constraining the randomization using historical information achieved better balance on baseline characteristics compared with simple randomization; however, the magnitude of benefit was modest.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-021-05590-1.

When should matching be used in the design of cluster randomized trials?

For cluster randomized trials (CRTs) with a small number of clusters, the matched-pair (MP) design, where clusters are paired before randomizing one to each trial arm, is often recommended to minimize imbalance on known prognostic factors, add face-validity to the study, and increase efficiency, provided the analysis recognizes the matching. Little evidence exists to guide decisions on when to use matching. We used simulation to compare the efficiency of the MP design with the stratified and simple designs, based on the mean confidence interval width of the estimated intervention effect. Matched and unmatched analyses were used for the MP design; a stratified analysis was used for the stratified design; and analyses without and with post-stratification adjustment for factors that would otherwise have been used for restricted allocation were used for the simple design. Results showed the MP design was generally the most efficient for CRTs with 10 or more pairs when the correlation between cluster-level outcomes within pairs (matching correlation) was moderate to strong (0.3-0.5). There was little gain in efficiency for the MP or stratified designs compared to simple randomization when the matching correlation was weak (0.05-0.1). For trials with four pairs of clusters, the simple and stratified designs were more efficient than the MP design because greater degrees of freedom were available for the analysis, although an unmatched analysis of the MP design recovered precision for weak matching correlations. Practical guidance on choosing between the MP, stratified, and simple designs is provided.

Completeness of reporting and risks of overstating impact in cluster randomised trials: a systematic review

Overstating the impact of interventions through incomplete or inaccurate reporting can lead to inappropriate scale-up of interventions with low impact. Accurate reporting of the impact of interventions is of great importance in global health research to protect scarce resources. In global health, the cluster randomised trial design is commonly used to evaluate complex, multicomponent interventions, and outcomes are often binary. Complete reporting of impact for binary outcomes means reporting both relative and absolute measures. We did a systematic review to assess reporting practices and potential to overstate impact in contemporary cluster randomised trials with binary primary outcome. We included all reports registered in the Cochrane Central Register of Controlled Trials of two-arm parallel cluster randomised trials with at least one binary primary outcome that were published in 2017. Of 73 cluster randomised trials, most (60 [82%]) showed incomplete reporting. Of 64 cluster randomised trials for which it was possible to evaluate, most (40 [63%]) reported results in such a way that impact could be overstated. Care is needed to report complete evidence of impact for the many interventions evaluated using the cluster randomised trial design worldwide.

Sample size considerations for matched-pair cluster randomization design with incomplete observations of binary outcomes

Multiple public health and medical research studies have applied matched-pair cluster randomization design to the evaluation of the intervention and/or prevention effects. One of the most common and severe problems faced by researchers when conducting cluster randomized trials (CRTs) is incomplete observations, which are associated with various reasons causing the individuals to discontinue participating in the trials. Although statistical methods to remedy the problems of missing data have already been proposed, there are still methodological gaps in research concerning the determination of sample size in matched-pair CRTs with incomplete binary outcomes. One conventional method for adjusting for missing data in the sample size determination is to divide the sample size under complete data by the expected follow-up rate. However, such crude adjustment ignores the impact of the structure and strength of correlations regarding both outcome data and missing data mechanism. This article provides a closed-form sample size formula for matched-pair CRTs with incomplete binary outcomes, which appropriately accounts for different missing patterns and magnitudes as well as the effects of matching and clustering on the outcome and missing data. The generalized estimating equation (GEE) approach treats incomplete observations as missing data in a marginal logistic regression model, which flexibly accommodates various types of intraclass correlation, missing patterns, and missing proportions. In the presence of missing data, the proposed GEE sample size method provides higher accuracy as compared with the conventional method. The performance of the proposed method is assessed by simulation studies. This article also illustrates how the proposed method can be used to design a real-world matched-pair CRT to examine the effect of a team-based approach on controlling blood pressure (BP).

A feasibility study for CODE-MI: High-sensitivity cardiac troponin-Optimizing the diagnosis of acute myocardial infarction/injury in women

BACKGROUND

CODE-MI is a pan-Canadian, multicentre, stepped-wedge, cluster randomized trial that evaluates the impact of using the female-specific 99th percentile threshold for high-sensitivity cardiac troponin (hs-cTn) on the diagnosis, treatment and outcomes of women presenting to the emergency department (ED) with symptoms suggestive for myocardial ischemia. A feasibility study was conducted to estimate the number of eligible patients, the rate of the study's primary outcome under control conditions, and the statistical power to detect a clinically important difference in the primary outcome.

METHODS

Using linked administrative data from 11 hospitals in Ontario, Canada, from October 2014 to September 2017, the following estimates were obtained: number of women presenting to the ED with symptoms suggestive of myocardial ischemia and a 24-hour peak hs-cTn value within the female-specific and overall thresholds (ie, primary cohort); the rate of the 1-year composite outcome of all-cause mortality, re-admission for nonfatal myocardial infarction, incident heart failure, or emergent/urgent coronary revascularization. Study power was evaluated via simulations.

RESULTS

Overall, 2,073,849 ED visits were assessed. Among women, chest pain (with or without cardiac features) and shortness of breath were the most common complaints associated with a diagnosis of acute coronary syndrome. An estimated 7.7% of women with these complaints are eligible for inclusion in the primary cohort. The rate of the 1-year outcome in the primary cohort varied significantly across hospitals with a median rate of 12.2% (95%CI: 7.9%-17.7%). With 30 hospitals, randomized at 5-month intervals in 5 steps, approximately 19,600 women are expected to be included in CODE-MI, resulting in >82% power to detect a 20% decrease in the odds of the primary outcome at a 0.05 significance level.

CONCLUSIONS

This feasibility study greatly enhanced the design of CODE-MI, allowed accurate evaluation of the study power, and demonstrated the strength of using linked administrative health data to guide the design of pragmatic clinical trials.

Sample size considerations for matched-pair cluster randomization design with incomplete observations of continuous outcomes

Matched-pair cluster randomization design is becoming increasingly used in clinical and health behavioral studies. Investigators often encounter incomplete observations in the data collected. Statistical inference for matched-pair cluster randomization design with incomplete observations has been extensively studied in literature. However, sample size method for such study design is sparsely available. We propose a closed-form sample size formula for matched-pair cluster randomization design with continuous outcomes, based on the generalized estimating equation approach by treating incomplete observations as missing data in a marginal linear model. The sample size formula is flexible to accommodate different correlation structures, missing patterns, and magnitude of missingness. In the presence of missing data, the proposed method would lead to a more accurate sample size estimation than the crude adjustment method. Simulation studies are conducted to evaluate the finite-sample performance of the proposed sample size method under various design configurations. We use bias-corrected variance estimators to address the issue of inflated type I error when the number of clusters per group is small. A real application example of physical fitness study in Ecuadorian adolescents is presented for illustration.

Characteristics and practices of school-based cluster randomised controlled trials for improving health outcomes in pupils in the UK: a systematic review protocol

INTRODUCTION

Cluster randomised trials (CRTs) are studies in which groups (clusters) of participants rather than the individuals themselves are randomised to trial arms. CRTs are becoming increasingly relevant for evaluating interventions delivered in school settings for improving the health of children. Schools are a convenient setting for health interventions targeted at children and the CRT design respects the clustered structure in schools (ie, pupils within classrooms/teachers within schools). Some of the methodological challenges of CRTs, such as ethical considerations for enrolment of children into trials and how best to handle the analysis of data from participants (pupils) that change clusters (schools), may be more salient for the school setting. A better understanding of the characteristics and methodological considerations of school-based CRTs of health interventions would inform the design of future similar studies. To our knowledge, this is the only systematic review to focus specifically on the characteristics and methodological practices of CRTs delivered in schools to evaluate interventions for improving health outcomes in pupils in the UK.

METHODS AND ANALYSIS

We will search for CRTs published from inception to 30 June 2020 inclusively indexed in MEDLINE (Ovid). We will identify relevant articles through title and abstract screening, and subsequent full-text screening for eligibility against predefined inclusion criteria. Disagreements will be resolved through discussion. Two independent reviewers will extract data for each study using a prepiloted data extraction form. Findings will be summarised using descriptive statistics and graphs.

ETHICS AND DISSEMINATION

This methodological systematic review does not require ethical approval as only secondary data extracted from papers will be analysed and the data are not linked to individual participants. After completion of the systematic review, the data will be analysed, and the findings disseminated through peer-reviewed publications and scientific meetings.

PROSPERO REGISTRATION NUMBER

CRD42020201792.

Methodological review showed that time-to-event outcomes are often inadequately handled in cluster randomized trials

OBJECTIVES

To estimate the prevalence of time-to-event (TTE) outcomes in cluster randomized trials (CRTs) and to examine their statistical management.

STUDY DESIGN AND SETTING

We searched PubMed to identify primary reports of CRTs published in six major general medical journals (2013-2018). Nature of outcomes and, for TTE outcomes, statistical methods for sample size, analysis, and measures of intracluster correlation were extracted.

RESULTS

A TTE analysis was used in 17% of the CRTs (32/184) either as a primary or secondary outcome analysis, or in a sensitivity analysis. Among the five CRTs with a TTE primary outcome, two accounted for both intracluster correlation and the TTE nature of the outcome in sample size calculation; one reported a measure of intracluster correlation in the analysis. Among the 32 CRTs with a least one TTE analysis, 44% (14/32) accounted for clustering in all TTE analyses. We identified 12 additional CRTs in which there was at least one outcome not analyzed as TTE for which a TTE analysis might have been preferred.

CONCLUSION

TTE outcomes are not uncommon in CRTs but appropriate statistical methods are infrequently used. Our results suggest that further methodological development and explicit recommendations for TTE outcomes in CRTs are needed.

Bias in natriuretic peptide-guided heart failure trials: time to improve guideline adherence using alternative approaches

Treatment of patients with heart failure with reduced ejection fraction (HFrEF) with currently available therapies reduces morbidity and mortality. However, implementation of these therapies is a problem with only few patients achieving guideline-recommended maximal doses of therapy. In an effort to improve guideline adherence and uptitration, several trials have investigated a biomarker-guided strategy (using natriuretic peptide targets in specific), but although conceptually promising, these trials failed to show a consistent beneficial effect on outcomes. In this review, we discuss different methodological issues that may explain the failure of these trials and offer potential solutions. Moreover, alternative approaches to increase heart failure guideline adherence are evaluated.

Meta-Analyses Proved Inconsistent in How Missing Data Were Handled Across Their Included Primary Trials: A Methodological Survey

Background

How systematic review authors address missing data among eligible primary studies remains uncertain.

Objective

To assess whether systematic review authors are consistent in the way they handle missing data, both across trials included in the same meta-analysis, and with their reported methods.

Methods

We first identified 100 eligible systematic reviews that included a statistically significant meta-analysis of a patient-important dichotomous efficacy outcome. Then, we successfully retrieved 638 of the 653 trials included in these systematic reviews’ meta-analyses. From each trial report, we extracted statistical data used in the analysis of the outcome of interest to compare with the data used in the meta-analysis. First, we used these comparisons to classify the “analytical method actually used” for handling missing data by the systematic review authors for each included trial. Second, we assessed whether systematic reviews explicitly reported their analytical method of handling missing data. Third, we calculated the proportion of systematic reviews that were consistent in their “analytical method actually used” across trials included in the same meta-analysis. Fourth, among systematic reviews that were consistent in the “analytical method actually used” across trials and explicitly reported on a method for handling missing data, we assessed whether the “analytical method actually used” and the reported methods were consistent.

Results

We were unable to determine the “analytical method reviews actually used” for handling missing outcome data among 397 trials. Among the remaining 241, systematic review authors most commonly conducted “complete case analysis” (n=128, 53%) or assumed “none of the participants with missing data had the event of interest” (n=58, 24%). Only eight of 100 systematic reviews were consistent in their approach to handling missing data across included trials, but none of these reported methods for handling missing data. Among seven reviews that did explicitly report their analytical method of handling missing data, only one was consistent in their approach across included trials (using complete case analysis), and their approach was inconsistent with their reported methods (assumed all participants with missing data had the event).

Conclusion

The majority of systematic review authors were inconsistent in their approach towards reporting and handling missing outcome data across eligible primary trials, and most did not explicitly report their methods to handle missing data. Systematic review authors should clearly identify missing outcome data among their eligible trials, specify an approach for handling missing data in their analyses, and apply their approach consistently across all primary trials.

IMPROVEjob - Participatory intervention to improve job satisfaction of general practice teams: a model for structural and behavioural prevention in small and medium-sized enterprises - a study protocol of a cluster-randomised controlled trial

Background

Perceived high chronic stress is twice as prevalent among German general practitioners (GPs) and non-physician medical staff compared to the general population. The reasons are multi-factorial and include patient, practice, healthcare system and societal factors, such as multi-morbidity, the diversity of populations and innovations in medical care. Also, practice-related factors, like stressful patient-staff interactions, poor process management of waiting times and lack of leadership, play a role. This publicly funded study evaluates the effectiveness of the newly developed participatory, interdisciplinary, and multimodal IMPROVEjob intervention on improving job satisfaction among general practice personnel. The intervention aims at structural stress prevention with regard to working conditions and behavioural stress prevention for leaders and other practice personnel.

Methods

In this cluster-randomised controlled trial, a total of 56 general practices will be assigned to either (1) participation in the IMPROVEjob intervention or (2) the waiting-list control group. The IMPROVEjob intervention consists of the following elements: three workshops, a toolbox with supplemental material and an implementation period with regular contact to so-called IMPROVEjob facilitators. The first workshop, addressing leadership issues, is designed for physicians with leadership responsibilities only. The two subsequent workshops target all GP and non-physician personnel; they address issues of communication (with patients and within the team), self-care and team-care and practice organisation. During the 9-month implementation period, practices will be contacted by IMPROVEjob facilitators to enhance motivation. Additionally, the practices will have access to the toolbox materials online. All participants will complete questionnaires at baseline and follow up. The primary outcome is the change in job satisfaction as measured by the respective scale of the validated German version of the Copenhagen Psychosocial Questionnaire (COPSOQ, version 2018). Secondary outcomes obtained by questionnaires and - qualitatively - by facilitators comprise psychosocial working conditions including leadership aspects, expectations and experiences of the workshops, team and individual efforts and organisational changes.

Discussion

It is hypothesised that participation in the IMPROVEjob intervention will improve job satisfaction and thus constitute a structural and behavioural prevention strategy for the promotion of psychological wellbeing of personnel in general practices and prospectively in other small and medium sized enterprises.

Trial registration

German Clinical Trials Register: DRKS00012677. Registered on 16 October 2019. Retrospectively, https://www.drks.de/drks_web/navigate.do?navigationId=trial. HTML&TRIAL_ID = DRKS00012677.

xtgeebcv: A command for bias-corrected sandwich variance estimation for GEE analyses of cluster randomized trials

Cluster randomized trials, where clusters (for example, schools or clinics) are randomized to comparison arms but measurements are taken on individuals, are commonly used to evaluate interventions in public health, education, and the social sciences. Analysis is often conducted on individual-level outcomes, and such analysis methods must consider that outcomes for members of the same cluster tend to be more similar than outcomes for members of other clusters. A popular individual-level analysis technique is generalized estimating equations (GEE). However, it is common to randomize a small number of clusters (for example, 30 or fewer), and in this case, the GEE standard errors obtained from the sandwich variance estimator will be biased, leading to inflated type I errors. Some bias-corrected standard errors have been proposed and studied to account for this finite-sample bias, but none has yet been implemented in Stata. In this article, we describe several popular bias corrections to the robust sandwich variance. We then introduce our newly created command, xtgeebcv, which will allow Stata users to easily apply finite-sample corrections to standard errors obtained from GEE models. We then provide examples to demonstrate the use of xtgeebcv. Finally, we discuss suggestions about which finite-sample corrections to use in which situations and consider areas of future research that may improve xtgeebcv.

How are missing data in covariates handled in observational time-to-event studies in oncology? A systematic review

Background

Missing data in covariates can result in biased estimates and loss of power to detect associations. It can also lead to other challenges in time-to-event analyses including the handling of time-varying effects of covariates, selection of covariates and their flexible modelling. This review aims to describe how researchers approach time-to-event analyses with missing data.

Methods

Medline and Embase were searched for observational time-to-event studies in oncology published from January 2012 to January 2018. The review focused on proportional hazards models or extended Cox models. We investigated the extent and reporting of missing data and how it was addressed in the analysis. Covariate modelling and selection, and assessment of the proportional hazards assumption were also investigated, alongside the treatment of missing data in these procedures.

Results

148 studies were included. The mean proportion of individuals with missingness in any covariate was 32%. 53% of studies used complete-case analysis, and 22% used multiple imputation. In total, 14% of studies stated an assumption concerning missing data and only 34% stated missingness as a limitation. The proportional hazards assumption was checked in 28% of studies, of which, 17% did not state the assessment method. 58% of 144 multivariable models stated their covariate selection procedure with use of a pre-selected set of covariates being the most popular followed by stepwise methods and univariable analyses. Of 69 studies that included continuous covariates, 81% did not assess the appropriateness of the functional form.

Conclusion

While guidelines for handling missing data in epidemiological studies are in place, this review indicates that few report implementing recommendations in practice. Although missing data are present in many studies, we found that few state clearly how they handled it or the assumptions they have made. Easy-to-implement but potentially biased approaches such as complete-case analysis are most commonly used despite these relying on strong assumptions and where often more appropriate methods should be employed. Authors should be encouraged to follow existing guidelines to address missing data, and increased levels of expectation from journals and editors could be used to improve practice.

Effectiveness of an Educational Intervention in Reducing New International Postgraduates' Acculturative Stress in Malaysian Public Universities: Protocol for a Cluster Randomized Controlled Trial

Background

Universities around the world, including Malaysia, have attracted many international students from different countries. Research has reported that acculturative stress resulting from international students’ attempts to adjust to the cultures of host countries is one of the most challenging issues that affects their lives in general and academic lives in particular.

Objective

This study aims to examine the effectiveness of an educational intervention on acculturative stress among new postgraduate international students joining Malaysian public universities.

Methods

A cluster randomized controlled trial design with Malaysian public universities as the unit of randomization will be used in this study. Public universities will be randomized in a 1:1 ratio to be either in the intervention (educational program) or control group (waiting list). Participants in the intervention group will receive 7 sessions in 9 hours delivered by an expert in psychology and the researcher. The control group will receive the intervention once the 3-month follow-up evaluation is completed.

Results

The data will be analyzed using the generalized estimation equation with a confidence interval value of 95%; significant differences between and within groups are determined as P<.05. The results of the study underlie the effectiveness of educational program in decreasing acculturative stress of new international students and enabling them to cope with a new environment. The results of this study will contribute to previous knowledge of acculturative stress, acculturation, and adjustment of international students. Furthermore, such results are expected to play a role in raising university policy makers’ awareness of their postgraduate international students’ acculturative stress issues and how they can help them avoid such stress and perform well in their academic life.

Conclusions

We expect that the intervention group will score significantly lower than the wait-list group on the immediate and 3-month postintervention evaluation of acculturative stress and achieve a higher level of adjustment. Results will have implications for international students, policy makers at universities, the Malaysian Ministry of Higher Education, and future research.

Trial Registration

Clinical Trials Registry India CTRI/2018/01/011223; http://ctri.nic.in/Clinicaltrials/showallp.php?mid1= 21978&amp;EncHid=&amp;userName=Muhamad%20Hanafiah%20Juni

International Registered Report Identifier (IRRID)

PRR1-10.2196/12950

The mixed model for repeated measures for cluster randomized trials: a simulation study investigating bias and type I error with missing continuous data

BACKGROUND

Cluster randomized trials (CRTs) are a design used to test interventions where individual randomization is not appropriate. The mixed model for repeated measures (MMRM) is a popular choice for individually randomized trials with longitudinal continuous outcomes. This model's appeal is due to avoidance of model misspecification and its unbiasedness for data missing completely at random or at random.

METHODS

We extended the MMRM to cluster randomized trials by adding a random intercept for the cluster and undertook a simulation experiment to investigate statistical properties when data are missing at random. We simulated cluster randomized trial data where the outcome was continuous and measured at baseline and three post-intervention time points. We varied the number of clusters, the cluster size, the intra-cluster correlation, missingness and the data-generation models. We demonstrate the MMRM-CRT with an example of a cluster randomized trial on cardiovascular disease prevention among diabetics.

RESULTS

When simulating a treatment effect at the final time point we found that estimates were unbiased when data were complete and when data were missing at random. Variance components were also largely unbiased. When simulating under the null, we found that type I error was largely nominal, although for a few specific cases it was as high as 0.081.

CONCLUSIONS

Although there have been assertions that this model is inappropriate when there are more than two repeated measures on subjects, we found evidence to the contrary. We conclude that the MMRM for CRTs is a good analytic choice for cluster randomized trials with a continuous outcome measured longitudinally.

TRIAL REGISTRATION

ClinicalTrials.gov, ID: NCT02804698.

Essential Ingredients and Innovations in the Design and Analysis of Group-Randomized Trials

This article reviews the essential ingredients and innovations in the design and analysis of group-randomized trials. The methods literature for these trials has grown steadily since they were introduced to the biomedical research community in the late 1970s, and we summarize those developments. We review, in addition to the group-randomized trial, methods for two closely related designs, the individually randomized group treatment trial and the stepped-wedge group-randomized trial. After describing the essential ingredients for these designs, we review the most important developments in the evolution of their methods using a new bibliometric tool developed at the National Institutes of Health. We then discuss the questions to be considered when selecting from among these designs or selecting the traditional randomized controlled trial. We close with a review of current methods for the analysis of data from these designs, a case study to illustrate each design, and a brief summary.

Properties and pitfalls of weighting as an alternative to multilevel multiple imputation in cluster randomized trials with missing binary outcomes under covariate-dependent missingness

The generalized estimating equation (GEE) approach can be used to analyze cluster randomized trial data to obtain population-averaged intervention effects. However, most cluster randomized trials have some missing outcome data and a GEE analysis of available data may be biased when outcome data are not missing completely at random. Although multilevel multiple imputation for GEE (MMI-GEE) has been widely used, alternative approaches such as weighted GEE are less common in practice. Using both simulations and a real data example, we evaluate the performance of inverse probability weighted GEE vs. MMI-GEE for binary outcomes. Simulated data are generated assuming a covariate-dependent missing data pattern across a range of missingness clustering (from none to high), where all covariates are measured at baseline and are fully observed (i.e. a type of missing-at-random mechanism). Two types of weights are estimated and used in the weighted GEE: (1) assuming no clustering of missingness (W-GEE) and (2) accounting for such clustering (CW-GEE). Results show that, even in settings with high missingness clustering, CW-GEE can lead to more bias and lower coverage than W-GEE, whereas W-GEE and MMI-GEE provide comparable results. W-GEE should be considered a viable strategy to account for missing outcomes in cluster randomized trials.

Analytical approaches and estimands to take account of missing patient-reported data in longitudinal studies

Patient-reported outcomes, such as quality of life, functioning, and symptoms, are used widely in therapeutic and behavioral trials and are increasingly used in drug development to represent the patient voice. Missing patient reported data is common and can undermine the validity of results reporting by reducing power, biasing estimates, and ultimately reducing confidence in the results. In this paper, we review statistically principled approaches for handling missing patient-reported outcome data and introduce the idea of estimands in the context of behavioral trials. Specifically, we outline a plan that considers missing data at each stage of research: design, data collection, analysis, and reporting. The design stage includes processes to prevent missing data, define the estimand, and specify primary and sensitivity analyses. The analytic strategy considering missing data depends on the estimand. Reviewed approaches include maximum likelihood-based models, multiple imputation, generalized estimating equations, and responder analysis. We outline sensitivity analyses to assess the robustness of the primary analysis results when data are missing. We also describe ad-hoc methods, including approaches to avoid. Last, we demonstrate methods using data from a behavioral intervention, where the primary outcome was self-reported cognition.

Feasibility and acceptability of brief behavioral therapy for cancer-related insomnia: effects on insomnia and circadian rhythm during chemotherapy: a phase II randomised multicentre controlled trial

BACKGROUND

This phase II RCT was conducted to determine the feasibility and acceptability of brief behavioral therapy for cancer-related insomnia (BBT-CI) in breast cancer patients undergoing chemotherapy. We also assessed the preliminary effects of BBT-CI on insomnia and circadian rhythm in comparison to a Healthy Eating Education Learning control condition (HEAL).

METHODS

Of the 71 participants recruited, 34 were randomised to receive BBT-CI and 37 to receive HEAL. Oncology staff was trained to deliver the intervention in four community clinics affiliated with the NCI. Insomnia was assessed with the Insomnia Severity Index (ISI), and circadian rhythm was assessed using a wrist-worn actiwatch.

RESULTS

Community staff interveners delivered 72% of the intervention components, with a recruitment rate of 77% and an adherence rate of 73%, meeting acceptability and feasibility benchmarks. Those randomised to BBT-CI improved their ISI scores by 6.3 points compared to a 2.5-point improvement in those randomised to HEAL (P = 0.041). Actigraphy data indicated that circadian functioning improved in the BBT-CI arm as compared to the HEAL arm at post-intervention (all P-values <0.05).

CONCLUSIONS

BBT-CI is an acceptable and feasible intervention that can be delivered directly in the community oncology setting by trained staff. The BBT-CI arm experienced significant improvements in insomnia and circadian rhythm as compared to the control condition.

cvcrand and cptest: Commands for efficient design and analysis of cluster randomized trials using constrained randomization and permutation tests

Cluster randomized trials (CRTs), where clusters (for example, schools or clinics) are randomized to comparison arms but measurements are taken on individuals, are commonly used to evaluate interventions in public health, education, and the social sciences. Because CRTs typically involve a small number of clusters (for example, fewer than 20), simple randomization frequently leads to baseline imbalance of cluster characteristics across study arms, threatening the internal validity of the trial. In CRTs with a small number of clusters, classic approaches to balancing baseline characteristics-such as matching and stratification-have several drawbacks, especially when the number of baseline characteristics the researcher desires to balance is large (Ivers et al., 2012, Trials 13: 120). An alternative design approach is covariate-constrained randomization, whereby a randomization scheme is randomly selected from a subset of all possible randomization schemes based on the value of a balancing criterion (Raab and Butcher, 2001, Statistics in Medicine 20: 351-365). Subsequently, a clustered permutation test can be used in the analysis, which provides increased power under constrained randomization compared with simple randomization (Li et al., 2016, Statistics in Medicine 35: 1565-1579). In this article, we describe covariate-constrained randomization and the permutation test for the design and analysis of CRTs and provide an example to demonstrate the use of our new commands cvcrand and cptest to implement constrained randomization and the permutation test.

Missing data handling in non-inferiority and equivalence trials: A systematic review

BACKGROUND

Non-inferiority (NI) and equivalence clinical trials test whether a new treatment is therapeutically no worse than, or equivalent to, an existing standard of care. Missing data in clinical trials have been shown to reduce statistical power and potentially bias estimates of effect size; however, in NI and equivalence trials, they present additional issues. For instance, they may decrease sensitivity to differences between treatment groups and bias toward the alternative hypothesis of NI (or equivalence).

AIMS

Our primary aim was to review the extent of and methods for handling missing data (model-based methods, single imputation, multiple imputation, complete case), the analysis sets used (Intention-To-Treat, Per-Protocol, or both), and whether sensitivity analyses were used to explore departures from assumptions about the missing data.

METHODS

We conducted a systematic review of NI and equivalence trials published between May 2015 and April 2016 by searching the PubMed database. Articles were reviewed primarily by 2 reviewers, with 6 articles reviewed by both reviewers to establish consensus.

RESULTS

Of 109 selected articles, 93% reported some missing data in the primary outcome. Among those, 50% reported complete case analysis, and 28% reported single imputation approaches for handling missing data. Only 32% reported conducting analyses of both intention-to-treat and per-protocol populations. Only 11% conducted any sensitivity analyses to test assumptions with respect to missing data.

CONCLUSION

Missing data are common in NI and equivalence trials, and they are often handled by methods which may bias estimates and lead to incorrect conclusions.

Interventions for implementation of thromboprophylaxis in hospitalized patients at risk for venous thromboembolism

BACKGROUND

Venous thromboembolism (VTE) is a leading cause of morbidity and mortality in hospitalized patients. While numerous randomized controlled trials (RCTs) have shown that the appropriate use of thromboprophylaxis in hospitalized patients at risk for VTE is safe, effective, and cost-effective, thromboprophylaxis remains underused or inappropriately used. Our previous review suggested that system-wide interventions, such as education, alerts, and multifaceted interventions were more effective at improving the prescribing of thromboprophylaxis than relying on individual providers' behaviors. However, 47 of the 55 included studies in our previous review were observational in design. Thus, an update to our systematic review, focused on the higher level of evidence of RCTs only, was warranted.

OBJECTIVES

To assess the effects of system-wide interventions designed to increase the implementation of thromboprophylaxis and decrease the incidence of VTE in hospitalized adult medical and surgical patients at risk for VTE, focusing on RCTs only.

SEARCH METHODS

Our research librarian conducted a systematic literature search of MEDLINE Ovid, and subsequently translated it to CENTRAL, PubMed, Embase Ovid, BIOSIS Previews Ovid, CINAHL, Web of Science, the Database of Abstracts of Reviews of Effects (DARE; in the Cochrane Library), NHS Economic Evaluation Database (EED; in the Cochrane Library), LILACS, and clinicaltrials.gov from inception to 7 January 2017. We also screened reference lists of relevant review articles. We identified 12,920 potentially relevant records.

SELECTION CRITERIA

We included all types of RCTs, with random or quasi-random methods of allocation of interventions, which either randomized individuals (e.g. parallel group, cross-over, or factorial design RCTs), or groups of individuals (cluster RCTs (CRTs)), which aimed to increase the use of prophylaxis or appropriate prophylaxis, or decrease the occurrence of VTE in hospitalized adult patients. We excluded observational studies, studies in which the intervention was simply distribution of published guidelines, and studies whose interventions were not clearly described. Studies could be in any language.

DATA COLLECTION AND ANALYSIS

We collected data on the following outcomes: the number of participants who received prophylaxis or appropriate prophylaxis (as defined by study authors), the occurrence of any VTE (symptomatic or asymptomatic), mortality, and safety outcomes, such as bleeding. We categorized the interventions into alerts (computer or human alerts), multifaceted interventions (combination of interventions that could include an alert component), educational interventions (e.g. grand rounds, courses), and preprinted orders (written predefined orders completed by the physician on paper or electronically). We meta-analyzed data across RCTs using a random-effects model. For CRTs, we pooled effect estimates (risk difference (RD) and risk ratio (RR), with 95% confidence interval (CI), adjusted for clustering, when possible. We pooled results if three or more trials were available for a particular intervention. We assessed the certainty of the evidence according to the GRADE approach.

MAIN RESULTS

From the 12,920 records identified by our search, we included 13 RCTs (N = 35,997 participants) in our qualitative analysis and 11 RCTs (N = 33,207 participants) in our meta-analyses.

PRIMARY OUTCOME

Alerts were associated with an increase in the proportion of participants who received prophylaxis (RD 21%, 95% CI 15% to 27%; three studies; 5057 participants; I² = 75%; low-certainty evidence). The substantial statistical heterogeneity may be in part explained by patient types, type of hospital, and type of alert. Subgroup analyses were not feasible due to the small number of studies included in the meta-analysis.Multifaceted interventions were associated with a small increase in the proportion of participants who received prophylaxis (cluster-adjusted RD 4%, 95% CI 2% to 6%; five studies; 9198 participants; I² = 0%; moderate-certainty evidence). Multifaceted interventions with an alert component were found to be more effective than multifaceted interventions that did not include an alert, although there were not enough studies to conduct a pooled analysis.

SECONDARY OUTCOMES

Alerts were associated with an increase in the proportion of participants who received appropriate prophylaxis (RD 16%, 95% CI 12% to 20%; three studies; 1820 participants; I² = 0; moderate-certainty evidence). Alerts were also associated with a reduction in the rate of symptomatic VTE at three months (RR 64%, 95% CI 47% to 86%; three studies; 5353 participants; I² = 15%; low-certainty evidence). Computer alerts were associated with a reduction in the rate of symptomatic VTE, although there were not enough studies to pool computer alerts and human alerts results separately.

AUTHORS' CONCLUSIONS

We reviewed RCTs that implemented a variety of system-wide strategies aimed at improving thromboprophylaxis in hospitalized patients. We found increased prescription of prophylaxis associated with alerts and multifaceted interventions, and increased prescription of appropriate prophylaxis associated with alerts. While multifaceted interventions were found to be less effective than alerts, a multifaceted intervention with an alert was more effective than one without an alert. Alerts, particularly computer alerts, were associated with a reduction in symptomatic VTE at three months, although there were not enough studies to pool computer alerts and human alerts results separately.Our analysis was underpowered to assess the effect on mortality and safety outcomes, such as bleeding.The incomplete reporting of relevant study design features did not allow complete assessment of the certainty of the evidence. However, the certainty of the evidence for improvement in outcomes was judged to be better than for our previous review (low- to moderate-certainty evidence, compared to very low-certainty evidence for most outcomes). The results of our updated review will help physicians, hospital administrators, and policy makers make practical decisions about adopting specific system-wide measures to improve prescription of thromboprophylaxis, and ultimately prevent VTE in hospitalized patients.

A pattern-mixture model approach for handling missing continuous outcome data in longitudinal cluster randomized trials

We extend the pattern-mixture approach to handle missing continuous outcome data in longitudinal cluster randomized trials, which randomize groups of individuals to treatment arms, rather than the individuals themselves. Individuals who drop out at the same time point are grouped into the same dropout pattern. We approach extrapolation of the pattern-mixture model by applying multilevel multiple imputation, which imputes missing values while appropriately accounting for the hierarchical data structure found in cluster randomized trials. To assess parameters of interest under various missing data assumptions, imputed values are multiplied by a sensitivity parameter, k, which increases or decreases imputed values. Using simulated data, we show that estimates of parameters of interest can vary widely under differing missing data assumptions. We conduct a sensitivity analysis using real data from a cluster randomized trial by increasing k until the treatment effect inference changes. By performing a sensitivity analysis for missing data, researchers can assess whether certain missing data assumptions are reasonable for their cluster randomized trial.

Quality of reporting of pilot and feasibility cluster randomised trials: a systematic review

OBJECTIVES

To systematically review the quality of reporting of pilot and feasibility of cluster randomised trials (CRTs). In particular, to assess (1) the number of pilot CRTs conducted between 1 January 2011 and 31 December 2014, (2) whether objectives and methods are appropriate and (3) reporting quality.

METHODS

We searched PubMed (2011-2014) for CRTs with 'pilot' or 'feasibility' in the title or abstract; that were assessing some element of feasibility and showing evidence the study was in preparation for a main effectiveness/efficacy trial. Quality assessment criteria were based on the Consolidated Standards of Reporting Trials (CONSORT) extensions for pilot trials and CRTs.

RESULTS

Eighteen pilot CRTs were identified. Forty-four per cent did not have feasibility as their primary objective, and many (50%) performed formal hypothesis testing for effectiveness/efficacy despite being underpowered. Most (83%) included 'pilot' or 'feasibility' in the title, and discussed implications for progression from the pilot to the future definitive trial (89%), but fewer reported reasons for the randomised pilot trial (39%), sample size rationale (44%) or progression criteria (17%). Most defined the cluster (100%), and number of clusters randomised (94%), but few reported how the cluster design affected sample size (17%), whether consent was sought from clusters (11%), or who enrolled clusters (17%).

CONCLUSIONS

That only 18 pilot CRTs were identified necessitates increased awareness of the importance of conducting and publishing pilot CRTs and improved reporting. Pilot CRTs should primarily be assessing feasibility, avoiding formal hypothesis testing for effectiveness/efficacy and reporting reasons for the pilot, sample size rationale and progression criteria, as well as enrolment of clusters, and how the cluster design affects design aspects. We recommend adherence to the CONSORT extensions for pilot trials and CRTs.