Comparison of population-averaged and cluster-specific models for the analysis of cluster randomized trials with missing binary outcomes: a simulation study

Demystifying estimands in cluster-randomised trials

Estimands can help clarify the interpretation of treatment effects and ensure that estimators are aligned with the study's objectives. Cluster-randomised trials require additional attributes to be defined within the estimand compared to individually randomised trials, including whether treatment effects are marginal or cluster-specific, and whether they are participant- or cluster-average. In this paper, we provide formal definitions of estimands encompassing both these attributes using potential outcomes notation and describe differences between them. We then provide an overview of estimators for each estimand, describe their assumptions, and show consistency (i.e. asymptotically unbiased estimation) for a series of analyses based on cluster-level summaries. Then, through a re-analysis of a published cluster-randomised trial, we demonstrate that the choice of both estimand and estimator can affect interpretation. For instance, the estimated odds ratio ranged from 1.38 (p = 0.17) to 1.83 (p = 0.03) depending on the target estimand, and for some estimands, the choice of estimator affected the conclusions by leading to smaller treatment effect estimates. We conclude that careful specification of the estimand, along with an appropriate choice of estimator, is essential to ensuring that cluster-randomised trials address the right question.

Analysing cluster randomised controlled trials using GLMM, GEE1, GEE2, and QIF: results from four case studies

BACKGROUND

Using four case studies, we aim to provide practical guidance and recommendations for the analysis of cluster randomised controlled trials.

METHODS

Four modelling approaches (Generalized Linear Mixed Models with parameters estimated by maximum likelihood/restricted maximum likelihood; Generalized Linear Models with parameters estimated by Generalized Estimating Equations (1st order or second order) and Quadratic Inference Function, for analysing correlated individual participant level outcomes in cluster randomised controlled trials were identified after we reviewed the literature. We systematically searched the online bibliography databases of MEDLINE, EMBASE, PsycINFO (via OVID), CINAHL (via EBSCO), and SCOPUS. We identified the above-mentioned four statistical analytical approaches and applied them to four case studies of cluster randomised controlled trials with the number of clusters ranging from 10 to 100, and individual participants ranging from 748 to 9,207. Results were obtained for both continuous and binary outcomes using R and SAS statistical packages.

RESULTS

The intracluster correlation coefficient (ICC) estimates for the case studies were less than 0.05 and are consistent with the observed ICC values commonly reported in primary care and community-based cluster randomised controlled trials. In most cases, the four methods produced similar results. However, in a few analyses, quadratic inference function produced different results compared to the generalized linear mixed model, first-order generalized estimating equations, and second-order generalized estimating equations, especially in trials with small to moderate numbers of clusters.

CONCLUSION

This paper demonstrates the analysis of cluster randomised controlled trials with four modelling approaches. The results obtained were similar in most cases, however, for trials with few clusters we do recommend that the quadratic inference function should be used with caution, and where possible a small sample correction should be used. The generalisability of our results is limited to studies with similar features to our case studies, for example, studies with a similar-sized ICC. It is important to conduct simulation studies to comprehensively evaluate the performance of the four modelling approaches.

Value assessment of medicinal products by the Italian Medicines Agency (AIFA) and French National Authority for Health (HAS): Similarities and discrepancies

The evaluation of pharmaceutical innovation and therapeutic value is an increasingly complex exercise for which different approaches are adopted at the national level, despite the need for standardisation of processes and harmonisation of public health decisions. The objective of our analysis was to compare the approaches of the AIFA (Agenzia Italiana del Farmaco) and the HAS (Haute Autorité de Santé) in assessing the same medicinal products. In Italy, the 1525/2017 AIFA Deliberation introduces a transparent scheme for the evaluation of innovative status (innovative, conditional, not innovative) based on the therapeutic added value (TAV), therapeutic need, and quality of evidence. In contrast, in France, the HAS makes judgements using the effective clinical benefit (Service Médical Rendu) and improvement of effective clinical benefit (Amélioration du Service Médical Rendu, ASMR). This analysis focused on medicinal products evaluated both by the AIFA and by the HAS from July 2017 to September 2021. Similarities between AIFA and HAS evaluations were investigated in terms of the TAV, recognition of innovativeness, and the ASMR. Both total and partial agreements were considered relevant. Therefore, raw agreement, Cohen's kappa (weighted and unweighted), and Bangdiwala's B-statistic were estimated. A total of 102 medicinal products were included in this study. Out of these, 38 (37.2%) were orphan drugs, while 56 (54.9%) had a clinical indication for the treatment of cancer. The AIFA and HAS reached a higher level of agreement on the innovativeness status compared with the TAV. A moderate total agreement emerged in the recognition of innovativeness (k = 0.463, p-value ≤0.0001), and partial agreement was substantial (equal weight k = 0.547, squared k = 0.638), while a lack of agreement resulted in a comparison of the TAV according to the AIFA and the ASMR recognised by the HAS. Indeed, whereas the AIFA determined the TAV to be important, the HAS considered it to be moderate. In addition, whereas the AIFA identified a bias towards a moderate TAV, the HAS identified a bias towards a minor ASMR. A higher level of agreement was reached, both on the TAV and on innovative status, for less critical medical products (non-cancer-related, or non-orphan, or with a standard European Medicines Agency approval). These results underline the importance of implementing European procedures that are more broadly aligned in terms of value definition criteria.

How to Improve Healthcare for Patients with Multimorbidity and Polypharmacy in Primary Care: A Pragmatic Cluster-Randomized Clinical Trial of the MULTIPAP Intervention

(1) Purpose: To investigate a complex MULTIPAP intervention that implements the Ariadne principles in a primary care population of young-elderly patients with multimorbidity and polypharmacy and to evaluate its effectiveness for improving the appropriateness of prescriptions. (2) Methods: A pragmatic cluster-randomized clinical trial was conducted involving 38 family practices in Spain. Patients aged 65–74 years with multimorbidity and polypharmacy were recruited. Family physicians (FPs) were randomly allocated to continue usual care or to provide the MULTIPAP intervention based on the Ariadne principles with two components: FP training (eMULTIPAP) and FP patient interviews. The primary outcome was the appropriateness of prescribing, measured as the between-group difference in the mean Medication Appropriateness Index (MAI) score change from the baseline to the 6-month follow-up. The secondary outcomes were quality of life (EQ-5D-5 L), patient perceptions of shared decision making (collaboRATE), use of health services, treatment adherence, and incidence of drug adverse events (all at 1 year), using multi-level regression models, with FP as a random effect. (3) Results: We recruited 117 FPs and 593 of their patients. In the intention-to-treat analysis, the between-group difference for the mean MAI score change after a 6-month follow-up was −2.42 (95% CI from −4.27 to −0.59) and, between baseline and a 12-month follow-up was −3.40 (95% CI from −5.45 to −1.34). There were no significant differences in any other secondary outcomes. (4) Conclusions: The MULTIPAP intervention improved medication appropriateness sustainably over the follow-up time. The small magnitude of the effect, however, advises caution in the interpretation of the results given the paucity of evidence for the clinical benefit of the observed change in the MAI. Trial registration: Clinicaltrials.gov NCT02866799.

Prevention of poisonings by educational intervention aimed at parents of preschool children

The aim of the study was to assess the effectiveness of the specific design of a poisoning prevention intervention. This controlled before-after study followed Solomon design for educational interventions using two groups (the educational intervention group and the control group). Participants comprised parents of children attending kindergartens under the jurisdiction of the City of Zagreb and in the vicinity of Zagreb. The intervention group (N = 336) underwent an educational intervention during parents' meetings comprising oral presentation by the Croatian Poison Control Centre (CPCC) and distribution of gift packages containing child-proof locks, flyers, and stickers with the CPCC contact number. After the intervention they more frequently started keeping the CPCC's number by their telephone or in the list of important numbers than parents in the control group, and this association remained significant when tested by generalized estimating equations for binary outcomes, after the adjustment for parents' characteristics (age, gender and educational level), and clustered by kindergartens (p < 0.001). This means parents acknowledged the CPCC as an adequate and accessible way for initial management of poisoning incidents.

Sample size considerations for matched-pair cluster randomization design with incomplete observations of binary outcomes

Multiple public health and medical research studies have applied matched-pair cluster randomization design to the evaluation of the intervention and/or prevention effects. One of the most common and severe problems faced by researchers when conducting cluster randomized trials (CRTs) is incomplete observations, which are associated with various reasons causing the individuals to discontinue participating in the trials. Although statistical methods to remedy the problems of missing data have already been proposed, there are still methodological gaps in research concerning the determination of sample size in matched-pair CRTs with incomplete binary outcomes. One conventional method for adjusting for missing data in the sample size determination is to divide the sample size under complete data by the expected follow-up rate. However, such crude adjustment ignores the impact of the structure and strength of correlations regarding both outcome data and missing data mechanism. This article provides a closed-form sample size formula for matched-pair CRTs with incomplete binary outcomes, which appropriately accounts for different missing patterns and magnitudes as well as the effects of matching and clustering on the outcome and missing data. The generalized estimating equation (GEE) approach treats incomplete observations as missing data in a marginal logistic regression model, which flexibly accommodates various types of intraclass correlation, missing patterns, and missing proportions. In the presence of missing data, the proposed GEE sample size method provides higher accuracy as compared with the conventional method. The performance of the proposed method is assessed by simulation studies. This article also illustrates how the proposed method can be used to design a real-world matched-pair CRT to examine the effect of a team-based approach on controlling blood pressure (BP).

Managing Missing Data in the Hospital Survey on Patient Safety Culture: A Simulation Study

BACKGROUND

Case-wise analysis is advocated for the Hospital Survey on Patient Safety culture (HSOPS).

OBJECTIVES

Through a computer-intensive simulation study, we aimed to evaluate the accuracy of various imputation methods in managing missing data in the HSOPS.

METHODS

Using the original data from a cross-sectional survey of 5064 employees at a single university hospital in France, we produced simulation data on two levels. First, we resampled 1000 completed data based on the original 3045 complete responses using a bootstrap procedure. Second, missing values were simulated in these 1000 completed case data for comparison purposes, using eight different missing data scenarios. Third, missing values were imputed using five different imputation methods (1, random imputation; 2, item mean; 3, individual mean; 4, multiple imputation, and 5, sparse nonnegative matrix factorization. The performance for each imputation method was assessed using the root mean square error and dimension score bias.

RESULTS

The five imputation methods yielded close root mean square errors, with an advantage for the multiple imputation. The bias differences were greater regarding the dimension scores, with a clear advantage for multiple imputation. The worst performance was achieved by the mean imputation methods.

DISCUSSION AND CONCLUSIONS

We recommend the use of multiple imputation to handle missing data in HSOPS-based surveys, whereas mean imputation methods should be avoided. Overall, these results suggest the possibility of optimizing the HSOPS instrument, which should be reduced without loss of overall information.

Properties and pitfalls of weighting as an alternative to multilevel multiple imputation in cluster randomized trials with missing binary outcomes under covariate-dependent missingness

The generalized estimating equation (GEE) approach can be used to analyze cluster randomized trial data to obtain population-averaged intervention effects. However, most cluster randomized trials have some missing outcome data and a GEE analysis of available data may be biased when outcome data are not missing completely at random. Although multilevel multiple imputation for GEE (MMI-GEE) has been widely used, alternative approaches such as weighted GEE are less common in practice. Using both simulations and a real data example, we evaluate the performance of inverse probability weighted GEE vs. MMI-GEE for binary outcomes. Simulated data are generated assuming a covariate-dependent missing data pattern across a range of missingness clustering (from none to high), where all covariates are measured at baseline and are fully observed (i.e. a type of missing-at-random mechanism). Two types of weights are estimated and used in the weighted GEE: (1) assuming no clustering of missingness (W-GEE) and (2) accounting for such clustering (CW-GEE). Results show that, even in settings with high missingness clustering, CW-GEE can lead to more bias and lower coverage than W-GEE, whereas W-GEE and MMI-GEE provide comparable results. W-GEE should be considered a viable strategy to account for missing outcomes in cluster randomized trials.

A Pragmatic Cluster Randomized Trial of an Electronic Clinical Decision Support System to Improve Chronic Kidney Disease Management in Primary Care: Design, Rationale, and Implementation Experience

BACKGROUND

The diagnosis of chronic kidney disease (CKD) is based on laboratory results easily extracted from electronic health records; therefore, CKD identification and management is an ideal area for targeted electronic decision support efforts. Early CKD management frequently occurs in primary care settings where primary care providers (PCPs) may not implement all the best practices to prevent CKD-related complications. Few previous studies have employed randomized trials to assess a CKD electronic clinical decision support system (eCDSS) that provided recommendations to PCPs tailored to each patient based on laboratory results.

OBJECTIVE

The aim of this study was to report the trial design and implementation experience of a CKD eCDSS in primary care.

METHODS

This was a 3-arm pragmatic cluster-randomized trial at an academic general internal medicine practice. Eligible patients had 2 previous estimated-glomerular-filtration-rates by serum creatinine (eGFRCr) <60 mL/min/1.73m2 at least 90 days apart. Randomization occurred at the PCP level. For patients of PCPs in either of the 2 intervention arms, the research team ordered triple-marker testing (serum creatinine, serum cystatin-c, and urine albumin-creatinine-ratio) at the beginning of the study period, to be completed when acquiring labs for regular clinical care. The eCDSS launched for PCPs and patients in the intervention arms during a regular PCP visit subsequent to completing the triple-marker testing. The eCDSS delivered individualized guidance on cardiovascular risk-reduction, potassium and proteinuria management, and patient education. Patients in the eCDSS+ arm also received a pharmacist phone call to reinforce CKD-related education. The primary clinical outcome is blood pressure change from baseline at 6 months after the end of the trial, and the main secondary outcome is provider awareness of CKD diagnosis. We also collected process, patient-centered, and implementation outcomes.

RESULTS

A multidisciplinary team (primary care internist, nephrologists, pharmacist, and informaticist) designed the eCDSS to integrate into the current clinical workflow. All 81 PCPs contacted agreed to participate and were randomized. Of 995 patients initially eligible by eGFRCr, 413 were excluded per protocol and 58 opted out or withdrew, resulting in 524 patient participants (188 usual care; 165 eCDSS; and 171 eCDSS+). During the 12-month intervention period, 53.0% (178/336) of intervention patient participants completed triple-marker labs. Among these, 138/178 (77.5%) had a PCP appointment after the triple-marker labs resulted; the eCDSS was opened for 73.9% (102/138), with orders or education signed for 81.4% (83/102).

CONCLUSIONS

Successful integration of an eCDSS into primary care workflows and high eCDSS utilization rates at eligible visits suggest this tailored electronic approach is feasible and has the potential to improve guideline-concordant CKD care.

TRIAL REGISTRATION

ClinicalTrials.gov NCT02925962; https://clinicaltrials.gov/ct2/show/NCT02925962 (Archived by WebCite at http://www.webcitation.org/78qpx1mjR).

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/14022.

Chat-based instant messaging support combined with brief smoking cessation interventions for Chinese community smokers in Hong Kong: Rationale and study protocol for a pragmatic, cluster-randomized controlled trial

BACKGROUND

Novel approaches to engage community smokers in smoking cessation are needed as smokers typically lack motivation to quit or use evidence-based tobacco dependence treatment. Mobile instant messaging apps (e.g., WhatsApp, Facebook Messenger) are widely used but under-studied as a mobile health modality for delivering smoking cessation support. This paper presents the rationale and study design of a trial which aims to evaluate the effectiveness of a chat-based intervention using mobile instant messaging combined with brief interventions for community smokers.

METHODS

This is a two-arm, parallel, accessor-blinded, pragmatic cluster-randomized controlled trial on an estimated 1172 daily cigarette smokers aged ≥18 years proactively recruited from 68 community sites (cluster) throughout Hong Kong. Subjects in intervention group received three months of chat-based, instant messaging support guided by acceptance and commitment therapy and other behavioural change techniques, integrated with brief advice and active referral to a smoking cessation service using the AWARD (Ask, Warn, Advise, Refer, Do-it-again) intervention model. Control group received brief advice to quit plus a self-help booklet at baseline. Outcomes were assessed at 1-, 2-, 3- and 6-month after baseline. The primary outcome is abstinence validated by exhaled carbon monoxide (<4 ppm) and salivary cotinine (<10 ng/mL) at 6-month after baseline. Primary analyses will be based on intention-to-treat.

COMMENTS

This is the first trial examining the effectiveness of a chat-based cessation support programme combined with brief interventions in promoting abstinence. The intervention model can be adapted for other behavioural change treatments and more advanced digital smoking cessation intervention.

Efficacy of motivating short interventions for smokers in primary care (COSMOS trial): study protocol for a cluster-RCT

Background

Tobacco abuse is a frequent issue in general practitioners' (GPs') offices, with doctors playing a key role in promoting smoking cessation to their patients. However, not all smokers are ready and willing to give up smoking. Thus, a GP focusing on smoking cessation alone might waste the opportunity to improve his patient’s health by supporting a change in another harmful behaviour pattern. The aim of this study is to determine whether multi-thematic coaching will lead to higher overall health benefits without resulting in a reduced rate of successful smoking cessations, compared with a monothematic smoking cessation approach.

Methods

The study is designed as a two-armed, double-blinded, cluster-randomised trial. GPs will be randomly assigned to the intervention or control group. In the intervention group, GPs will undergo training in patient-centred coaching, shared decision-making and motivational interviewing. The control group will be trained in a state-of-the-art smoking cessation algorithm.

GPs will approach adult cigarette-smoking patients and advise those included according to the GP’s group affiliation. The primary outcome is the between-group difference in the proportion of participants who achieve a beneficial change in at least one of seven different health-related behavioural dimensions, 12 months post baseline. Secondary outcomes include smoking cessation rates and the patients’ self-perceived smoking-related motivation, self-efficacy and planning behaviour. Additionally, covariates describing both GPs and patients will be collected before the start of the intervention, and process outcome measures in compliance with the RE-AIM (Reach Effectiveness Adoption Implementation Maintenance) framework will be recorded during the ongoing study.

Discussion

Tobacco consumption is still highly prevalent in the general population and often goes hand in hand with other behaviour patterns with adverse health effects. This study will add to the literature regarding effective strategies available to GPs to address unhealthy behaviour among their smoking patients beyond mere smoking cessation counselling. The study will also establish a basis for decisions about further promotion and dissemination of the coaching under study.

Trial registration

ISRCTN, ISRCTN38129107. Registered on 2 October 2017.

Electronic supplementary material

The online version of this article (10.1186/s13063-018-3071-z) contains supplementary material, which is available to authorized users.

Improving inappropriate medication and information transfer at hospital discharge: study protocol for a cluster RCT

BACKGROUND

Inappropriate medication and polypharmacy increase morbidity, hospitalisation rate, costs and mortality in multimorbid patients. At hospital discharge of elderly patients, polypharmacy is often even more pronounced than at admission. However, the optimal discharge strategy in view of sustained medication appropriateness remains unclear. In particular, unreflectingly switching back to the pre-hospitalisation medication must be avoided. Therefore, both the patients and the follow-up physicians should be involved in the discharge process. In this study, we aim to test whether a brief medication review which takes the patients' priorities into account, combined with a standardised communication strategy at hospital discharge, leads to sustained medication appropriateness and extends readmission times among elderly multimorbid patients.

METHODS

The study is designed as a two-armed, double-blinded, cluster-randomised trial, involving 42 senior hospital physicians (HPs) with their junior HPs and 2100 multimorbid patients aged 60 years or older. Using a randomised minimisation strategy, senior HPs will be assigned to either intervention or control group. Following instructions of the study team, the senior HPs in the intervention group will teach their junior HPs how to integrate a simple medication review tool combined with a defined communication strategy into their ward's discharge procedure. The untrained HPs in the control group will provide data on usual care, and their patients will be discharged following usual local routines. Primary outcome is the time until readmission within 6 months after discharge, and secondary outcomes cover readmission rates, number of emergency and GP visits, classes and numbers of drugs prescribed, proportions of potentially inappropriate medications, and the patients' quality of life after discharge. Additionally, the characteristics of both the HPs as well as the patients will be collected before the intervention. Process evaluation outcomes will be assessed parallel to the ongoing core study using qualitative research methods.

DISCUSSION

So far, interventions to reduce polypharmacy are still scarce at the crucial interface between HPs and GPs. To our knowledge, this trial is the first to analyse the combination of a brief deprescribing intervention with a standardised communication strategy at hospital discharge and in the early post-discharge period.

TRIAL REGISTRATION

ISRCTN, ISRCTN18427377 . Registered 11 January 2018.

Missing binary outcomes under covariate-dependent missingness in cluster randomised trials

Missing outcomes are a commonly occurring problem for cluster randomised trials, which can lead to biased and inefficient inference if ignored or handled inappropriately. Two approaches for analysing such trials are cluster‐level analysis and individual‐level analysis. In this study, we assessed the performance of unadjusted cluster‐level analysis, baseline covariate‐adjusted cluster‐level analysis, random effects logistic regression and generalised estimating equations when binary outcomes are missing under a baseline covariate‐dependent missingness mechanism. Missing outcomes were handled using complete records analysis and multilevel multiple imputation. We analytically show that cluster‐level analyses for estimating risk ratio using complete records are valid if the true data generating model has log link and the intervention groups have the same missingness mechanism and the same covariate effect in the outcome model. We performed a simulation study considering four different scenarios, depending on whether the missingness mechanisms are the same or different between the intervention groups and whether there is an interaction between intervention group and baseline covariate in the outcome model. On the basis of the simulation study and analytical results, we give guidance on the conditions under which each approach is valid. © 2017 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.

Effectiveness of a brief community outreach tobacco cessation intervention in India: a cluster-randomised controlled trial (the BABEX Trial)

BACKGROUND

Tobacco use kills half a million people every month, most in low-middle income countries (LMICs). There is an urgent need to identify potentially low-cost, scalable tobacco cessation interventions for these countries.

OBJECTIVE

To evaluate a brief community outreach intervention delivered by health workers to promote tobacco cessation in India.

DESIGN

Cluster-randomised controlled trial.

SETTING

32 low-income administrative blocks in Delhi, half government authorised ('resettlement colony') and half unauthorised ('J.J. cluster') communities.

PARTICIPANTS

1213 adult tobacco users.

INTERVENTIONS

Administrative blocks were computer randomised in a 1:1 ratio, to the intervention (16 clusters; n=611) or control treatment (16 clusters; n=602), delivered and assessed at individual level between 07/2012 and 11/2013. The intervention was single session quit advice (15 min) plus a single training session in yogic breathing exercises; the control condition comprised very brief quit advice (1 min) alone. Both were delivered via outreach, with contact made though household visits.

MEASUREMENTS

The primary outcome was 6-month sustained abstinence from all tobacco, assessed 7 months post intervention delivery, biochemically verified with salivary cotinine.

RESULTS

The smoking cessation rate was higher in the intervention group (2.6% (16/611)) than in the control group (0.5% (3/602)) (relative risk=5.32, 95% CI 1.43 to 19.74, p=0.013). There was no interaction with type of tobacco use (smoked vs smokeless). Results did not change materially in adjusted analyses, controlling for participant characteristics.

CONCLUSIONS

A single session community outreach intervention can increase tobacco cessation in LMIC. The effect size, while small, could impact public health if scaled up with high coverage.

TRIAL REGISTRATION NUMBER

ISRCTCN23362894.

A comparison of imputation strategies in cluster randomized trials with missing binary outcomes

In cluster randomized trials, clusters of subjects are randomized rather than subjects themselves, and missing outcomes are a concern as in individual randomized trials. We assessed strategies for handling missing data when analysing cluster randomized trials with a binary outcome; strategies included complete case, adjusted complete case, and simple and multiple imputation approaches. We performed a simulation study to assess bias and coverage rate of the population-averaged intervention-effect estimate. Both multiple imputation with a random-effects logistic regression model or classical logistic regression provided unbiased estimates of the intervention effect. Both strategies also showed good coverage properties, even slightly better for multiple imputation with a random-effects logistic regression approach. Finally, this latter approach led to a slightly negatively biased intracluster correlation coefficient estimate but less than that with a classical logistic regression model strategy. We applied these strategies to a real trial randomizing households and comparing ivermectin and malathion to treat head lice.

Statistical analysis and handling of missing data in cluster randomized trials: a systematic review

Background

Cluster randomized trials (CRTs) randomize participants in groups, rather than as individuals and are key tools used to assess interventions in health research where treatment contamination is likely or if individual randomization is not feasible. Two potential major pitfalls exist regarding CRTs, namely handling missing data and not accounting for clustering in the primary analysis. The aim of this review was to evaluate approaches for handling missing data and statistical analysis with respect to the primary outcome in CRTs.

Methods

We systematically searched for CRTs published between August 2013 and July 2014 using PubMed, Web of Science, and PsycINFO. For each trial, two independent reviewers assessed the extent of the missing data and method(s) used for handling missing data in the primary and sensitivity analyses. We evaluated the primary analysis and determined whether it was at the cluster or individual level.

Results

Of the 86 included CRTs, 80 (93 %) trials reported some missing outcome data. Of those reporting missing data, the median percent of individuals with a missing outcome was 19 % (range 0.5 to 90 %). The most common way to handle missing data in the primary analysis was complete case analysis (44, 55 %), whereas 18 (22 %) used mixed models, six (8 %) used single imputation, four (5 %) used unweighted generalized estimating equations, and two (2 %) used multiple imputation. Fourteen (16 %) trials reported a sensitivity analysis for missing data, but most assumed the same missing data mechanism as in the primary analysis. Overall, 67 (78 %) trials accounted for clustering in the primary analysis.

Conclusions

High rates of missing outcome data are present in the majority of CRTs, yet handling missing data in practice remains suboptimal. Researchers and applied statisticians should carry out appropriate missing data methods, which are valid under plausible assumptions in order to increase statistical power in trials and reduce the possibility of bias. Sensitivity analysis should be performed, with weakened assumptions regarding the missing data mechanism to explore the robustness of results reported in the primary analysis.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-016-1201-z) contains supplementary material, which is available to authorized users.

Design and analysis of trials with a partially nested design and a binary outcome measure

Where treatments are administered to groups of patients or delivered by therapists, outcomes for patients in the same group or treated by the same therapist may be more similar, leading to clustering. Trials of such treatments should take account of this effect. Where such a treatment is compared with an un-clustered treatment, the trial has a partially nested design. This paper compares statistical methods for this design where the outcome is binary. Investigation of consistency reveals that a random coefficient model with a random effect for group or therapist is not consistent with other methods for a null treatment effect, and so this model is not recommended for this design. Small sample performance of a cluster-adjusted test of proportions, a summary measures test and logistic generalised estimating equations and random intercept models are investigated through simulation. The expected treatment effect is biased for the logistic models. Empirical test size of two-sided tests is raised only slightly, but there are substantial biases for one-sided tests. Three formulae are proposed for calculating sample size and power based on (i) the difference of proportions, (ii) the log-odds ratio or (iii) the arc-sine transformation of proportions. Calculated power from these formulae is compared with empirical power from a simulations study. Logistic models appeared to perform better than those based on proportions with the likelihood ratio test performing best in the range of scenarios considered. For these analyses, the log-odds ratio method of calculation of power gave an approximate lower limit for empirical power.

Challenges of cluster randomized trials

Cluster randomized trials are trials that randomize clusters of people, rather than individuals. They are becoming increasingly common. A number of innovations have been developed recently, particularly in the calculation of the required size of a cluster trial, the handling of missing data, designs to minimize recruitment bias, the ethics of cluster randomized trials and the stepped wedge design. This article will highlight and illustrate these developments. It will also discuss issues with regards to the reporting of cluster randomized trials.

Detailed statistical analysis plan for the difficult airway management (DIFFICAIR) trial

Background

Preoperative airway assessment in Denmark is based on a non-specific clinical assessment left to the discretion of the responsible anesthesiologist. The DIFFICAIR trial compares the effect of using a systematic and consistent airway assessment versus a non-specific clinical assessment on the frequency of unanticipated difficult airway management.

To prevent outcome bias and selective reporting, we hereby present a detailed statistical analysis plan as an amendment (update) to the previously published protocol for the DIFFICAIR trial.

Method/Design

The DIFFICAIR trial is a stratified, parallel group, cluster (cluster = department) randomized multicenter trial involving 28 departments of anesthesia in Denmark randomized to airway assessment either by the Simplified Airway Risk Index (SARI) or by a usual non-specific assessment. Data from patients’ preoperative airway assessment are registered in the Danish Anesthesia Database. An objective score for intubation grading the severity, that is the severity of the intubations, as well as the frequency of unanticipated difficult intubation, is measured for each group.

Primary outcome measures are the fraction of unanticipated difficult and easy intubations.

The database is programmed so that the registration of the SARI is mandatory for the intervention group but invisible to controls.

Data recruitment was commenced in October 2012 and ended in ultimo December 2013.

Conclusion

We intend to increase the transparency of the data analyses regarding the DIFFICAIR trial by an a priori publication of a statistical analysis plan.

Trial registration

ClinicalTrials.gov: NCT01718561.