Are missing data adequately handled in cluster randomised trials? A systematic review and guidelines

Assessing treatment effect heterogeneity in the presence of missing effect modifier data in cluster-randomized trials

Understanding whether and how treatment effects vary across subgroups is crucial to inform clinical practice and recommendations. Accordingly, the assessment of heterogeneous treatment effects based on pre-specified potential effect modifiers has become a common goal in modern randomized trials. However, when one or more potential effect modifiers are missing, complete-case analysis may lead to bias and under-coverage. While statistical methods for handling missing data have been proposed and compared for individually randomized trials with missing effect modifier data, few guidelines exist for the cluster-randomized setting, where intracluster correlations in the effect modifiers, outcomes, or even missingness mechanisms may introduce further threats to accurate assessment of heterogeneous treatment effect. In this article, the performance of several missing data methods are compared through a simulation study of cluster-randomized trials with continuous outcome and missing binary effect modifier data, and further illustrated using real data from the Work, Family, and Health Study. Our results suggest that multilevel multiple imputation and Bayesian multilevel multiple imputation have better performance than other available methods, and that Bayesian multilevel multiple imputation has lower bias and closer to nominal coverage than standard multilevel multiple imputation when there are model specification or compatibility issues.

Statistical analysis plan for a cluster randomised trial in Madhya Pradesh, India: community health promotion and medical provision and impact on neonates (CHAMPION2)

BACKGROUND

Neonatal mortality in India has fallen steadily and was estimated to be 24 per 1000 live births in the year 2017. However, neonatal mortality remains high in rural parts of the country. The Community Health Promotion and Medical Provision and Impact On Neonates (CHAMPION2) trial investigates the effect of a complex health intervention on neonatal mortality in the Satna District of Madhya Pradesh.

METHODS/DESIGN

The CHAMPION2 trial forms one part of a cluster-randomised controlled trial with villages (clusters) randomised to receive either a health (CHAMPION2) or education (STRIPES2) intervention. Villages receiving the health intervention are controls for the education intervention and vice versa. The primary outcome is neonatal mortality. The effect of the active intervention on the primary outcome (compared to usual care) will be expressed as a risk ratio, estimated using a generalised estimating equation approach with robust standard errors that take account of clustering at village level. Secondary outcomes include maternal mortality, stillbirths, perinatal deaths, causes of death, health care and knowledge, hospital admissions of enrolled women during pregnancy or in the immediate post-natal care period or of their babies (during the neonatal period), maternal blood transfusions, and the cost effectiveness of the intervention. A total of 196 villages have been randomised and over 34,000 women have been recruited in CHAMPION2.

DISCUSSION

This update to the published trial protocol gives a detailed plan for the statistical analysis of the CHAMPION2 trial.

TRIAL REGISTRATION

Registry of India: CTRI/2019/05/019296. Registered on 23 May 2019. https://ctri.nic.in/Clinicaltrials/pmaindet2.php?EncHid=MzExOTg=&Enc=&userName=champion2.

Key considerations for designing, conducting and analysing a cluster randomized trial

Not only do cluster randomized trials require a larger sample size than individually randomized trials, they also face many additional complexities. The potential for contamination is the most commonly used justification for using cluster randomization, but the risk of contamination should be carefully weighed against the more serious problem of questionable scientific validity in settings with post-randomization identification or recruitment of participants unblinded to the treatment allocation. In this paper we provide some simple guidelines to help researchers conduct cluster trials in a way that minimizes potential biases and maximizes statistical efficiency. The overarching theme of this guidance is that methods that apply to individually randomized trials rarely apply to cluster randomized trials. We recommend that cluster randomization be only used when necessary-balancing the benefits of cluster randomization with its increased risks of bias and increased sample size. Researchers should also randomize at the lowest possible level-balancing the risks of contamination with ensuring an adequate number of randomization units-as well as exploring other options for statistically efficient designs. Clustering should always be allowed for in the sample size calculation; and the use of restricted randomization (and adjustment in the analysis for covariates used in the randomization) should be considered. Where possible, participants should be recruited before randomizing clusters and, when recruiting (or identifying) participants post-randomization, recruiters should be masked to the allocation. In the analysis, the target of inference should align with the research question, and adjustment for clustering and small sample corrections should be used when the trial includes less than about 40 clusters.

Defining and estimating effects in cluster randomized trials: A methods comparison

Across research disciplines, cluster randomized trials (CRTs) are commonly implemented to evaluate interventions delivered to groups of participants, such as communities and clinics. Despite advances in the design and analysis of CRTs, several challenges remain. First, there are many possible ways to specify the causal effect of interest (eg, at the individual-level or at the cluster-level). Second, the theoretical and practical performance of common methods for CRT analysis remain poorly understood. Here, we present a general framework to formally define an array of causal effects in terms of summary measures of counterfactual outcomes. Next, we provide a comprehensive overview of CRT estimators, including the t-test, generalized estimating equations (GEE), augmented-GEE, and targeted maximum likelihood estimation (TMLE). Using finite sample simulations, we illustrate the practical performance of these estimators for different causal effects and when, as commonly occurs, there are limited numbers of clusters of different sizes. Finally, our application to data from the Preterm Birth Initiative (PTBi) study demonstrates the real-world impact of varying cluster sizes and targeting effects at the cluster-level or at the individual-level. Specifically, the relative effect of the PTBi intervention was 0.81 at the cluster-level, corresponding to a 19% reduction in outcome incidence, and was 0.66 at the individual-level, corresponding to a 34% reduction in outcome risk. Given its flexibility to estimate a variety of user-specified effects and ability to adaptively adjust for covariates for precision gains while maintaining Type-I error control, we conclude TMLE is a promising tool for CRT analysis.

Statistical analysis plan for a cluster randomised trial in Madhya Pradesh, India: support to rural India's public education system and impact on numeracy and literacy scores (STRIPES2)

BACKGROUND

India has made steady progress in improving rates of primary school enrolment but levels of learning achievement remain low. The Support To Rural India's Public Education System (STRIPES) trial provided evidence that an after-school para-teacher intervention improved numeracy and literacy levels in Telangana, India. The STRIPES2 trial investigates whether such an intervention will have a similar effect on the literacy and numeracy of primary school age children in the Satna District of Madhya Pradesh, India.

METHODS/DESIGN

The STRIPES2 trial forms one part of a cluster-randomised controlled trial with villages (clusters) randomised to receive either a health (CHAMPION2) or education (STRIPES2) intervention. Building on the design of the earlier CHAMPION/STRIPES trial, villages receiving the health intervention are controls for the education intervention and vice versa. The primary outcome is a combined literacy and numeracy score. Secondary outcomes include separate scores for literacy and numeracy; caregivers' engagement with child's learning; expenditure on education; enrolment in school; caregiver's report of school attendance and the cost effectiveness of the intervention. Over 7000 primary school age children have been recruited and randomised in STRIPES2.

DISCUSSION

This update to the published trial protocol gives a detailed plan for the statistical analysis of the STRIPES 2 trial.

TRIAL REGISTRATION

Registry of India: CTRI/2019/05/019296. Registered on 23 May 2019. http://www.ctri.nic.in/Clinicaltrials/pdf_generate.php?trialid=31198&EncHid=&modid=&compid=%27,%2731198det%27.

A scoping review of statistical methods for trial-based economic evaluations: The current state of play

The statistical quality of trial-based economic evaluations is often suboptimal, while a comprehensive overview of available statistical methods is lacking. Therefore, this review summarized and critically appraised available statistical methods for trial-based economic evaluations. A literature search was performed to identify studies on statistical methods for dealing with baseline imbalances, skewed costs and/or effects, correlated costs and effects, clustered data, longitudinal data, missing data and censoring in trial-based economic evaluations. Data was extracted on the statistical methods described, their advantages, disadvantages, relative performance and recommendations of the study. Sixty-eight studies were included. Of them, 27 (40%) assessed methods for baseline imbalances, 39 (57%) assessed methods for skewed costs and/or effects, 27 (40%) assessed methods for correlated costs and effects, 18 (26%) assessed methods for clustered data, 7 (10%) assessed methods for longitudinal data, 26 (38%) assessed methods for missing data and 10 (15%) assessed methods for censoring. All identified methods were narratively described. This review provides a comprehensive overview of available statistical methods for dealing with the most common statistical complexities in trial-based economic evaluations. Herewith, it can provide valuable input for researchers when deciding which statistical methods to use in a trial-based economic evaluation.

The handling of missing data in trial-based economic evaluations: should data be multiply imputed prior to longitudinal linear mixed-model analyses?

INTRODUCTION

For the analysis of clinical effects, multiple imputation (MI) of missing data were shown to be unnecessary when using longitudinal linear mixed-models (LLM). It remains unclear whether this also applies to trial-based economic evaluations. Therefore, this study aimed to assess whether MI is required prior to LLM when analyzing longitudinal cost and effect data.

METHODS

Two-thousand complete datasets were simulated containing five time points. Incomplete datasets were generated with 10, 25, and 50% missing data in follow-up costs and effects, assuming a Missing At Random (MAR) mechanism. Six different strategies were compared using empirical bias (EB), root-mean-squared error (RMSE), and coverage rate (CR). These strategies were: LLM alone (LLM) and MI with LLM (MI-LLM), and, as reference strategies, mean imputation with LLM (M-LLM), seemingly unrelated regression alone (SUR-CCA), MI with SUR (MI-SUR), and mean imputation with SUR (M-SUR).

RESULTS

For costs and effects, LLM, MI-LLM, and MI-SUR performed better than M-LLM, SUR-CCA, and M-SUR, with smaller EBs and RMSEs as well as CRs closers to nominal levels. However, even though LLM, MI-LLM and MI-SUR performed equally well for effects, MI-LLM and MI-SUR were found to perform better than LLM for costs at 10 and 25% missing data. At 50% missing data, all strategies resulted in relatively high EBs and RMSEs for costs.

CONCLUSION

LLM should be combined with MI when analyzing trial-based economic evaluation data. MI-SUR is more efficient and can also be used, but then an average intervention effect over time cannot be estimated.

Sensitivity analyses for data missing at random versus missing not at random using latent growth modelling: a practical guide for randomised controlled trials

BACKGROUND

Missing data are ubiquitous in randomised controlled trials. Although sensitivity analyses for different missing data mechanisms (missing at random vs. missing not at random) are widely recommended, they are rarely conducted in practice. The aim of the present study was to demonstrate sensitivity analyses for different assumptions regarding the missing data mechanism for randomised controlled trials using latent growth modelling (LGM).

METHODS

Data from a randomised controlled brief alcohol intervention trial was used. The sample included 1646 adults (56% female; mean age = 31.0 years) from the general population who had received up to three individualized alcohol feedback letters or assessment-only. Follow-up interviews were conducted after 12 and 36 months via telephone. The main outcome for the analysis was change in alcohol use over time. A three-step LGM approach was used. First, evidence about the process that generated the missing data was accumulated by analysing the extent of missing values in both study conditions, missing data patterns, and baseline variables that predicted participation in the two follow-up assessments using logistic regression. Second, growth models were calculated to analyse intervention effects over time. These models assumed that data were missing at random and applied full-information maximum likelihood estimation. Third, the findings were safeguarded by incorporating model components to account for the possibility that data were missing not at random. For that purpose, Diggle-Kenward selection, Wu-Carroll shared parameter and pattern mixture models were implemented.

RESULTS

Although the true data generating process remained unknown, the evidence was unequivocal: both the intervention and control group reduced their alcohol use over time, but no significant group differences emerged. There was no clear evidence for intervention efficacy, neither in the growth models that assumed the missing data to be at random nor those that assumed the missing data to be not at random.

CONCLUSION

The illustrated approach allows the assessment of how sensitive conclusions about the efficacy of an intervention are to different assumptions regarding the missing data mechanism. For researchers familiar with LGM, it is a valuable statistical supplement to safeguard their findings against the possibility of nonignorable missingness.

TRIAL REGISTRATION

The PRINT trial was prospectively registered at the German Clinical Trials Register (DRKS00014274, date of registration: 12th March 2018).

Influential methods reports for group-randomized trials and related designs

BACKGROUND

This article identifies the most influential methods reports for group-randomized trials and related designs published through 2020. Many interventions are delivered to participants in real or virtual groups or in groups defined by a shared interventionist so that there is an expectation for positive correlation among observations taken on participants in the same group. These interventions are typically evaluated using a group- or cluster-randomized trial, an individually randomized group treatment trial, or a stepped wedge group- or cluster-randomized trial. These trials face methodological issues beyond those encountered in the more familiar individually randomized controlled trial.

METHODS

PubMed was searched to identify candidate methods reports; that search was supplemented by reports known to the author. Candidate reports were reviewed by the author to include only those focused on the designs of interest. Citation counts and the relative citation ratio, a new bibliometric tool developed at the National Institutes of Health, were used to identify influential reports. The relative citation ratio measures influence at the article level by comparing the citation rate of the reference article to the citation rates of the articles cited by other articles that also cite the reference article.

RESULTS

In total, 1043 reports were identified that were published through 2020. However, 55 were deemed to be the most influential based on their relative citation ratio or their citation count using criteria specific to each of the three designs, with 32 group-randomized trial reports, 7 individually randomized group treatment trial reports, and 16 stepped wedge group-randomized trial reports. Many of the influential reports were early publications that drew attention to the issues that distinguish these designs from the more familiar individually randomized controlled trial. Others were textbooks that covered a wide range of issues for these designs. Others were "first reports" on analytic methods appropriate for a specific type of data (e.g. binary data, ordinal data), for features commonly encountered in these studies (e.g. unequal cluster size, attrition), or for important variations in study design (e.g. repeated measures, cohort versus cross-section). Many presented methods for sample size calculations. Others described how these designs could be applied to a new area (e.g. dissemination and implementation research). Among the reports with the highest relative citation ratios were the CONSORT statements for each design.

CONCLUSIONS

Collectively, the influential reports address topics of great interest to investigators who might consider using one of these designs and need guidance on selecting the most appropriate design for their research question and on the best methods for design, analysis, and sample size.

How to Improve Healthcare for Patients with Multimorbidity and Polypharmacy in Primary Care: A Pragmatic Cluster-Randomized Clinical Trial of the MULTIPAP Intervention

(1) Purpose: To investigate a complex MULTIPAP intervention that implements the Ariadne principles in a primary care population of young-elderly patients with multimorbidity and polypharmacy and to evaluate its effectiveness for improving the appropriateness of prescriptions. (2) Methods: A pragmatic cluster-randomized clinical trial was conducted involving 38 family practices in Spain. Patients aged 65–74 years with multimorbidity and polypharmacy were recruited. Family physicians (FPs) were randomly allocated to continue usual care or to provide the MULTIPAP intervention based on the Ariadne principles with two components: FP training (eMULTIPAP) and FP patient interviews. The primary outcome was the appropriateness of prescribing, measured as the between-group difference in the mean Medication Appropriateness Index (MAI) score change from the baseline to the 6-month follow-up. The secondary outcomes were quality of life (EQ-5D-5 L), patient perceptions of shared decision making (collaboRATE), use of health services, treatment adherence, and incidence of drug adverse events (all at 1 year), using multi-level regression models, with FP as a random effect. (3) Results: We recruited 117 FPs and 593 of their patients. In the intention-to-treat analysis, the between-group difference for the mean MAI score change after a 6-month follow-up was −2.42 (95% CI from −4.27 to −0.59) and, between baseline and a 12-month follow-up was −3.40 (95% CI from −5.45 to −1.34). There were no significant differences in any other secondary outcomes. (4) Conclusions: The MULTIPAP intervention improved medication appropriateness sustainably over the follow-up time. The small magnitude of the effect, however, advises caution in the interpretation of the results given the paucity of evidence for the clinical benefit of the observed change in the MAI. Trial registration: Clinicaltrials.gov NCT02866799.

Interpretation of trial-based economic evaluations of musculoskeletal physical therapy interventions

BACKGROUND

As resources for healthcare are scarce, decision-makers increasingly rely on economic evaluations when making reimbursement decisions about new health technologies, such as drugs, procedures, devices, and equipment. Economic evaluations compare the costs and effects of two or more interventions. Musculoskeletal disorders have a high prevalence and result in high levels of disability and high costs worldwide. Because physical therapy interventions are usually the first line of treatment for musculoskeletal disorders, economic evaluations of such interventions are becoming increasingly important for stakeholders in the field of physical therapy, including physical therapists, decision-makers, and reseachers. However, economic evaluations are relatively difficult to interpret for the majority of stakeholders.

OBJECTIVE

To support physical therapists, decision-makers, and researchers in the field of physical therapy interpreting trial-based economic evaluations and translating the results of such studies to clinical practice.

METHODS

The design, analysis, and interpretation of economic evaluations performed alongside randomized controlled trials are discussed. To further illustrate and explain these concepts, we use a case study assessing the cost-effectiveness of exercise therapy compared to standard advice in patients with musculoskeletal disorders.

CONCLUSIONS

Economic evaluations are increasingly being used in healthcare decision-making. Therefore, it is of utmost importance that their design, conduct, and analysis are state-of-the-art and that their interpretation is adequate. This masterclass will help physical therapists, decision-makers, and researchers in the field of physical therapy to critically appraise the quality and results of trial-based economic evaluations and to apply the results of such studies to their own clinical practice and setting.

The statistical approach in trial-based economic evaluations matters: get your statistics together!

Background

Baseline imbalances, skewed costs, the correlation between costs and effects, and missing data are statistical challenges that are often not adequately accounted for in the analysis of cost-effectiveness data. This study aims to illustrate the impact of accounting for these statistical challenges in trial-based economic evaluations.

Methods

Data from two trial-based economic evaluations, the REALISE and HypoAware studies, were used. In total, 14 full cost-effectiveness analyses were performed per study, in which the four statistical challenges in trial-based economic evaluations were taken into account step-by-step. Statistical approaches were compared in terms of the resulting cost and effect differences, ICERs, and probabilities of cost-effectiveness.

Results

In the REALISE study and HypoAware study, the ICER ranged from 636,744€/QALY and 90,989€/QALY when ignoring all statistical challenges to − 7502€/QALY and 46,592€/QALY when accounting for all statistical challenges, respectively. The probabilities of the intervention being cost-effective at 0€/ QALY gained were 0.67 and 0.59 when ignoring all statistical challenges, and 0.54 and 0.27 when all of the statistical challenges were taken into account for the REALISE study and HypoAware study, respectively.

Conclusions

Not accounting for baseline imbalances, skewed costs, correlated costs and effects, and missing data in trial-based economic evaluations may notably impact results. Therefore, when conducting trial-based economic evaluations, it is important to align the statistical approach with the identified statistical challenges in cost-effectiveness data. To facilitate researchers in handling statistical challenges in trial-based economic evaluations, software code is provided.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06513-1.

What is the future of patient-reported outcomes in sickle-cell disease?

INTRODUCTION

Sickle cell disease (SCD) is a complex, chronic disease caused by abnormal polymerization of hemoglobin, which leads to severe pain episodes, fatigue, and end-organ damage. Patient reported outcomes (PROs) have emerged as a critical tool for measuring SCD disease severity and response to treatment.

AREAS COVERED

Authors review the key issues involved when deciding to use a PRO in a clinical trial. We describe the most highly recommended generic and disease-specific PRO tools in SCD and discuss the challenges of incorporating them in clinical practice.

EXPERT OPINION

PRO measures are essential to incorporate into SCD clinical trials either as primary or secondary outcomes. The use of PRO measures in SCD facilitates a patient-centered approach, which is likely to lead to improved outcomes. Significant challenges remain in adapting PRO tools to routine clinical use and in developing countries.

Comparison of glottic visualisation through supraglottic airway device (SAD) using bronchoscope in the ramped versus supine 'sniffing air' position: A pilot feasibility study

Background and Aims:

Airway management in obese patients is associated with increased risk of difficult airway and intubation. After failed intubation, supraglottic airway-guided flexible bronchoscopic intubation (SAGFBI) may be required. It is uncertain whether SAGFBI is best performed in the ramped versus conventional supine “sniffing air” position. We conducted a feasibility study to evaluate the logistics of positioning, compared glottic views, and evaluated SAGFBI success rates.

Methods:

We conducted a prospective, pilot study in patients with a body mass index (BMI) 30–40 kg/m² undergoing elective operations requiring tracheal intubation. All patients were placed in a ramped position. After induction, a supraglottic airway device (SAD) was inserted. A flexible bronchoscope was inserted into the SAD and a photograph of the glottic view taken. The patient was repositioned to the supine position. A second photograph was taken. SAGFBI was performed. Images were randomised and assessed by two independent anesthetists.

Results:

Of 17 patients recruited, 15 patients were repositioned successfully. There were no differences in glottic views observed in the two positions. SAGFBI was successful in 92.9% of patients (median time 91.5 s). Haemodynamic changes were noted in 42.7% of patients which resolved spontaneously.

Conclusion:

Our pilot study was completed within 5 months, achieved low dropout rate and protocol feasibility was established. SAGFBI was successfully and safely performed in obese patients, with a median time of 91.5 s. The time taken for SAGFBI was similar to awake intubation using FBI and videolaryngoscopy. Our study provided preliminary data supporting future, larger-scale studies to evaluate glottic views in the ramped versus supine positions.

Effectiveness of an Educational Intervention in Reducing New International Postgraduates' Acculturative Stress in Malaysian Public Universities: Protocol for a Cluster Randomized Controlled Trial

Background

Universities around the world, including Malaysia, have attracted many international students from different countries. Research has reported that acculturative stress resulting from international students’ attempts to adjust to the cultures of host countries is one of the most challenging issues that affects their lives in general and academic lives in particular.

Objective

This study aims to examine the effectiveness of an educational intervention on acculturative stress among new postgraduate international students joining Malaysian public universities.

Methods

A cluster randomized controlled trial design with Malaysian public universities as the unit of randomization will be used in this study. Public universities will be randomized in a 1:1 ratio to be either in the intervention (educational program) or control group (waiting list). Participants in the intervention group will receive 7 sessions in 9 hours delivered by an expert in psychology and the researcher. The control group will receive the intervention once the 3-month follow-up evaluation is completed.

Results

The data will be analyzed using the generalized estimation equation with a confidence interval value of 95%; significant differences between and within groups are determined as P<.05. The results of the study underlie the effectiveness of educational program in decreasing acculturative stress of new international students and enabling them to cope with a new environment. The results of this study will contribute to previous knowledge of acculturative stress, acculturation, and adjustment of international students. Furthermore, such results are expected to play a role in raising university policy makers’ awareness of their postgraduate international students’ acculturative stress issues and how they can help them avoid such stress and perform well in their academic life.

Conclusions

We expect that the intervention group will score significantly lower than the wait-list group on the immediate and 3-month postintervention evaluation of acculturative stress and achieve a higher level of adjustment. Results will have implications for international students, policy makers at universities, the Malaysian Ministry of Higher Education, and future research.

Trial Registration

Clinical Trials Registry India CTRI/2018/01/011223; http://ctri.nic.in/Clinicaltrials/showallp.php?mid1= 21978&amp;EncHid=&amp;userName=Muhamad%20Hanafiah%20Juni

International Registered Report Identifier (IRRID)

PRR1-10.2196/12950

Cost-effectiveness of offering an area-level financial incentive on breast feeding: a within-cluster randomised controlled trial analysis

OBJECTIVE

To provide the first estimate of the cost-effectiveness of financial incentive for breastfeeding intervention compared with usual care.

DESIGN

Within-cluster ('ward'-level) randomised controlled trial cost-effectiveness analysis (trial registration number ISRCTN44898617).

SETTING

Five local authority districts in the North of England.

PARTICIPANTS

5398 mother-infant dyads (intervention arm), 4612 mother-infant dyads (control arm).

INTERVENTIONS

Offering a financial incentive (over a 6-month period) on breast feeding to women living in areas with low breastfeeding prevalence (<40% at 6-8 weeks).

MAIN OUTCOME MEASURES

Babies breast fed (receiving breastmilk) at 6-8 weeks, and cost per additional baby breast fed.

METHODS

Costs were compared with differences in area-level data on babies' breast fed in order to estimate a cost per additional baby breast fed and the quality-adjusted life year (QALY) gains required over the lifetime of babies to justify intervention cost.

RESULTS

In the trial, the total cost of providing the intervention in 46 wards was £462 600, with an average cost per ward of £9989 and per baby of £91. At follow-up, area-level breastfeeding prevalence at 6-8 weeks was 31.7% (95% CI 29.4 to 34.0) in control areas and 37.9% (95% CI 35.0 to 40.8) in intervention areas. The adjusted difference between intervention and control was 5.7 percentage points (95% CI 2.7 to 8.6; p<0.001), resulting in 10 (95% CI 6 to 14) more additional babies breast fed in the intervention wards (39 vs 29). The cost per additional baby breast fed at 6-8 weeks was £974. At a cost per QALY threshold of £20 000 (recommended in England), an additional breastfed baby would need to show a QALY gain of 0.05 over their lifetime to justify the intervention cost. If decision makers are willing to pay £974 (or more) per additional baby breast fed at a QALY gain of 0.05, then this intervention could be cost-effective. Results were robust to sensitivity analyses.

CONCLUSION

This study provides information to help inform public health guidance on breast feeding. To make the economic case unequivocal, evidence on the varied and long-term health benefits of breast feeding to both the baby and mother and the effectiveness of financial incentives for breastfeeding beyond 6-8 weeks is required.

Essential Ingredients and Innovations in the Design and Analysis of Group-Randomized Trials

This article reviews the essential ingredients and innovations in the design and analysis of group-randomized trials. The methods literature for these trials has grown steadily since they were introduced to the biomedical research community in the late 1970s, and we summarize those developments. We review, in addition to the group-randomized trial, methods for two closely related designs, the individually randomized group treatment trial and the stepped-wedge group-randomized trial. After describing the essential ingredients for these designs, we review the most important developments in the evolution of their methods using a new bibliometric tool developed at the National Institutes of Health. We then discuss the questions to be considered when selecting from among these designs or selecting the traditional randomized controlled trial. We close with a review of current methods for the analysis of data from these designs, a case study to illustrate each design, and a brief summary.

Assessing longer-term effectiveness of a combined household-level piped water and sanitation intervention on child diarrhoea, acute respiratory infection, soil-transmitted helminth infection and nutritional status: a matched cohort study in rural Odisha, India

BACKGROUND

Open defecation is widespread in rural India, and few households have piped water connections. While government and other efforts have increased toilet coverage in India, and evaluations found limited immediate impacts on health, longer-term effects have not been rigorously assessed.

METHODS

We conducted a matched cohort study to assess the longer-term effectiveness of a combined household-level piped water and sanitation intervention implemented by Gram Vikas (an Indian NGO) in rural Odisha, India. Forty-five intervention villages were randomly selected from a list of those where implementation was previously completed at least 5 years before, and matched to 45 control villages. We conducted surveys and collected stool samples between June 2015 and October 2016 in households with a child <5 years of age (n = 2398). Health surveillance included diarrhoea (primary outcome), acute respiratory infection (ARI), soil-transmitted helminth infection, and anthropometry.

RESULTS

Intervention villages had higher improved toilet coverage (85% vs 18%), and increased toilet use by adults (74% vs 13%) and child faeces disposal (35% vs 6%) compared with control villages. There was no intervention association with diarrhoea [adjusted OR (aOR): 0.94, 95% confidence interval (CI): 0.74-1.20] or ARI. Compared with controls, children in intervention villages had lower helminth infection (aOR: 0.44, 95% CI: 0.18, 1.00) and improved height-for-age z scores (HAZ) (+0.17, 95% CI: 0.03-0.31).

CONCLUSIONS

This combined intervention, where household water connections were contingent on community-wide household toilet construction, was associated with improved HAZ, and reduced soil-transmitted helminth (STH) infection, though not reduced diarrhoea or ARI. Further research should explore the mechanism through which these heterogenous effects on health may occur.

What do national pharmacoeconomic guidelines recommend regarding the statistical analysis of trial-based economic evaluations?

Introduction: The statistical quality of many trial-based economic evaluations is poor. When conducting trial-based economic evaluations, researchers often turn to national pharmacoeconomic guidelines for guidance. Therefore, this study reviewed which recommendations are currently given by national pharmacoeconomic guidelines on the statistical analysis of trial-based economic evaluations.Areas covered: 40 national pharmacoeconomic guidelines were identified. Data were extracted on the guidelines' recommendations on how to deal with baseline imbalances, skewed costs, correlated costs and effects, clustering of data, longitudinal data, and missing data in trial-based economic evaluations. Four guidelines (10%) were found to include recommendations on how to deal with baseline imbalances, five (13%) on how to deal with skewed costs, and seven (18%) on how to deal with missing data. Recommendations were very general in nature and recommendations on dealing with correlated costs and effects, clustering of data, and longitudinal data were lacking.Expert opinion: Current national pharmacoeconomic guidelines provide little to no guidance on how to deal with the statistical challenges to trial-based economic evaluations. Since the use of suboptimal statistical methods may lead to biased results, and, therefore, possibly to a waste of scarce resources, national agencies are advised to include more statistical guidance in their pharmacoeconomic guidelines.

Properties and pitfalls of weighting as an alternative to multilevel multiple imputation in cluster randomized trials with missing binary outcomes under covariate-dependent missingness

The generalized estimating equation (GEE) approach can be used to analyze cluster randomized trial data to obtain population-averaged intervention effects. However, most cluster randomized trials have some missing outcome data and a GEE analysis of available data may be biased when outcome data are not missing completely at random. Although multilevel multiple imputation for GEE (MMI-GEE) has been widely used, alternative approaches such as weighted GEE are less common in practice. Using both simulations and a real data example, we evaluate the performance of inverse probability weighted GEE vs. MMI-GEE for binary outcomes. Simulated data are generated assuming a covariate-dependent missing data pattern across a range of missingness clustering (from none to high), where all covariates are measured at baseline and are fully observed (i.e. a type of missing-at-random mechanism). Two types of weights are estimated and used in the weighted GEE: (1) assuming no clustering of missingness (W-GEE) and (2) accounting for such clustering (CW-GEE). Results show that, even in settings with high missingness clustering, CW-GEE can lead to more bias and lower coverage than W-GEE, whereas W-GEE and MMI-GEE provide comparable results. W-GEE should be considered a viable strategy to account for missing outcomes in cluster randomized trials.

Bias through selective inclusion and attrition: Representativeness when comparing provider performance with routine outcome monitoring data

Background

Observational research based on routine outcome monitoring is prone to missing data, and outcomes can be biased due to selective inclusion at baseline or selective attrition at posttest. As patients with complete data may not be representative of all patients of a provider, missing data may bias results, especially when missingness is not random but systematic.

Methods

The present study establishes clinical and demographic patient variables relevant for representativeness of the outcome information. It applies strategies to estimate sample selection bias (weighting by inclusion propensity) and selective attrition bias (multiple imputation based on multilevel regression analysis) and estimates the extent of their impact on an index of provider performance. The association between estimated bias and response rate is also investigated.

Results

Provider‐based analyses showed that in current practice, the effect of selective inclusion was minimal, but attrition had a more substantial effect, biasing results in both directions: overstating and understating performance. For 22% of the providers, attrition bias was estimated to be in excess of 0.05 ES. Bias was associated with overall response rate (r = .50). When selective inclusion and attrition bring providers' response below 50%, it is more likely that selection bias increased beyond a critical level, and conclusions on the comparative performance of such providers may be misleading.

Conclusions

Estimates of provider performance were biased by selection, especially by missing data at posttest. Results on the extent and direction of bias and minimal requirements for response rates to arrive at unbiased performance indicators are discussed.

The West Midlands ActiVe lifestyle and healthy Eating in School children (WAVES) study: a cluster randomised controlled trial testing the clinical effectiveness and cost-effectiveness of a multifaceted obesity prevention intervention programme targeted at children aged 6-7 years

BACKGROUND

Systematic reviews suggest that school-based interventions can be effective in preventing childhood obesity, but better-designed trials are needed that consider costs, process, equity, potential harms and longer-term outcomes.

OBJECTIVE

To assess the clinical effectiveness and cost-effectiveness of the WAVES (West Midlands ActiVe lifestyle and healthy Eating in School children) study intervention, compared with usual practice, in preventing obesity among primary school children.

DESIGN

A cluster randomised controlled trial, split across two groups, which were randomised using a blocked balancing algorithm. Schools/participants could not be blinded to trial arm. Measurement staff were blind to allocation arm as far as possible.

SETTING

Primary schools, West Midlands, UK.

PARTICIPANTS

Schools within a 35-mile radius of the study centre and all year 1 pupils (aged 5-6 years) were eligible. Schools with a higher proportion of pupils from minority ethnic populations were oversampled to enable subgroup analyses.

INTERVENTIONS

The 12-month intervention encouraged healthy eating/physical activity (PA) by (1) helping teachers to provide 30 minutes of additional daily PA, (2) promoting 'Villa Vitality' (interactive healthy lifestyles learning, in an inspirational setting), (3) running school-based healthy cooking skills/education workshops for parents and children and (4) highlighting information to families with regard to local PA opportunities.

MAIN OUTCOME MEASURES

The primary outcomes were the difference in body mass index z-scores (BMI-zs) between arms (adjusted for baseline body mass index) at 3 and 18 months post intervention (clinical outcome), and cost per quality-adjusted life-year (QALY) (cost-effectiveness outcome). The secondary outcomes were further anthropometric, dietary, PA and psychological measurements, and the difference in BMI-z between arms at 27 months post intervention in a subset of schools.

RESULTS

Two groups of schools were randomised: 27 in 2011 (n = 650 pupils) [group 1 (G1)] and another 27 in 2012 (n = 817 pupils) [group 2 (G2)]. Primary outcome data were available at first follow-up (n = 1249 pupils) and second follow-up (n = 1145 pupils) from 53 schools. The mean difference (MD) in BMI-z between the control and intervention arms was -0.075 [95% confidence interval (CI) -0.183 to 0.033] and -0.027 (95% CI -0.137 to 0.083) at 3 and 18 months post intervention, respectively. The main analyses showed no evidence of between-arm differences for any secondary outcomes. Third follow-up included data on 467 pupils from 27 G1 schools, and showed a statistically significant difference in BMI-z (MD -0.20, 95% CI -0.40 to -0.01). The mean cost of the intervention was £266.35 per consented child (£155.53 per child receiving the intervention). The incremental cost-effectiveness ratio associated with the base case was £46,083 per QALY (best case £26,804 per QALY), suggesting that the intervention was not cost-effective.

LIMITATIONS

The presence of baseline primary outcome imbalance between the arms, and interschool variation in fidelity of intervention delivery.

CONCLUSIONS

The primary analyses show no evidence of clinical effectiveness or cost-effectiveness of the WAVES study intervention. A post hoc analysis, driven by findings at third follow-up, suggests a possible intervention effect, which could have been attenuated by baseline imbalances. There was no evidence of an intervention effect on measures of diet or PA and no evidence of harm.

FUTURE WORK

A realist evidence synthesis could provide insights into contextual factors and strategies for future interventions. School-based interventions need to be integrated within a wider societal framework and supported by upstream interventions.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN97000586.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 22, No. 8. See the NIHR Journals Library website for further project information.

Impact of oral health literacy on self-reported missing data in epidemiological research

OBJECTIVES

To investigate whether oral health literacy (OHL) impacts missing data obtained through self-reporting in oral health epidemiological research.

METHODS

A cross-sectional study was conducted with parents (n = 344) of 4- to 5-year-old children randomly selected from public schools within the city of Curitiba, Brazil. Parental OHL was measured using the Brazilian version of the Rapid Estimate of Adult Literacy in Dentistry (BREALD-30). Parents answered a set of questionnaires comprising 88 items concerning sociodemographic and economic data, children's access to dental services, oral hygiene behaviour, diet and mealtime behaviour. The total number of unanswered items (TUI) and the number of unanswered items in each type of question (open-ended, dichotomous, multiple choice with up to 4 options and with 5-9 options) for each participant was compared across different levels of OHL (chi-squared, Mann-Whitney, Kruskal-Wallis and Spearman's correlation test). Multiple Poisson regression was used to estimate rate ratios (RR) of TUI between OHL scores and their respective 95% confidence interval (95% CI).

RESULTS

Approximately one-third of studied parents (37%) exhibited low OHL (BREALD-30 ≤ 21). The prevalence of missing data in at least one item was 85.5%. Low OHL was associated with failing to respond open-ended items (P = 0.003) and multiple-choice items with up to 4 (P = 0.003) and between 5 and 9 options (P = 0.030). There was a negative correlation between OHL scores and TUI (r = -0.195; P < 0.001), as well as with the number of unanswered items in all types of questions (P < 0.01), except dichotomous questions. Parents with lower OHL were more likely to show higher values of TUI (RR 0.95: 0.93-0.98), when adjusted by income and education.

CONCLUSIONS

Participants with lower OHL were significantly more likely to fail to complete research questionnaires. The impact of OHL on missing data was greater with more complex types of items.

Design and analysis of group-randomized trials in cancer: A review of current practices

The purpose of this paper is to summarize current practices for the design and analysis of group-randomized trials involving cancer-related risk factors or outcomes and to offer recommendations to improve future trials. We searched for group-randomized trials involving cancer-related risk factors or outcomes that were published or online in peer-reviewed journals in 2011-15. During 2016-17, in Bethesda MD, we reviewed 123 articles from 76 journals to characterize their design and their methods for sample size estimation and data analysis. Only 66 (53.7%) of the articles reported appropriate methods for sample size estimation. Only 63 (51.2%) reported exclusively appropriate methods for analysis. These findings suggest that many investigators do not adequately attend to the methodological challenges inherent in group-randomized trials. These practices can lead to underpowered studies, to an inflated type 1 error rate, and to inferences that mislead readers. Investigators should work with biostatisticians or other methodologists familiar with these issues. Funders and editors should ensure careful methodological review of applications and manuscripts. Reviewers should ensure that studies are properly planned and analyzed. These steps are needed to improve the rigor and reproducibility of group-randomized trials. The Office of Disease Prevention (ODP) at the National Institutes of Health (NIH) has taken several steps to address these issues. ODP offers an online course on the design and analysis of group-randomized trials. ODP is working to increase the number of methodologists who serve on grant review panels. ODP has developed standard language for the Application Guide and the Review Criteria to draw investigators' attention to these issues. Finally, ODP has created a new Research Methods Resources website to help investigators, reviewers, and NIH staff better understand these issues.

Reporting non-adherence in cluster randomised trials: A systematic review

BACKGROUND

Treatment non-adherence in randomised trials refers to situations where some participants do not receive their allocated treatment as intended. For cluster randomised trials, where the unit of randomisation is a group of participants, non-adherence may occur at the cluster or individual level. When non-adherence occurs, randomisation no longer guarantees that the relationship between treatment receipt and outcome is unconfounded, and the power to detect the treatment effects in intention-to-treat analysis may be reduced. Thus, recording adherence and estimating the causal treatment effect adequately are of interest for clinical trials.

OBJECTIVES

To assess the extent of reporting of non-adherence issues in published cluster trials and to establish which methods are currently being used for addressing non-adherence, if any, and whether clustering is accounted for in these.

METHODS

We systematically reviewed 132 cluster trials published in English in 2011 previously identified through a search in PubMed.

RESULTS

One-hundred and twenty three cluster trials were included in this systematic review. Non-adherence was reported in 56 cluster trials. Among these, 19 reported a treatment efficacy estimate: per protocol in 15 and as treated in 4. No study discussed the assumptions made by these methods, their plausibility or the sensitivity of the results to deviations from these assumptions.

LIMITATIONS

The year of publication of the cluster trials included in this review (2011) could be considered a limitation of this study; however, no new guidelines regarding the reporting and the handling of non-adherence for cluster trials have been published since. In addition, a single reviewer undertook the data extraction. To mitigate this, a second reviewer conducted a validation of the extraction process on 15 randomly selected reports. Agreement was satisfactory (93%).

CONCLUSION

Despite the recommendations of the Consolidated Standards of Reporting Trials statement extension to cluster randomised trials, treatment adherence is under-reported. Among the trials providing adherence information, there was substantial variation in how adherence was defined, handled and reported. Researchers should discuss the assumptions required for the results to be interpreted causally and whether these are scientifically plausible in their studies. Sensitivity analyses to study the robustness of the results to departures from these assumptions should be performed.

Quality of reporting of pilot and feasibility cluster randomised trials: a systematic review

OBJECTIVES

To systematically review the quality of reporting of pilot and feasibility of cluster randomised trials (CRTs). In particular, to assess (1) the number of pilot CRTs conducted between 1 January 2011 and 31 December 2014, (2) whether objectives and methods are appropriate and (3) reporting quality.

METHODS

We searched PubMed (2011-2014) for CRTs with 'pilot' or 'feasibility' in the title or abstract; that were assessing some element of feasibility and showing evidence the study was in preparation for a main effectiveness/efficacy trial. Quality assessment criteria were based on the Consolidated Standards of Reporting Trials (CONSORT) extensions for pilot trials and CRTs.

RESULTS

Eighteen pilot CRTs were identified. Forty-four per cent did not have feasibility as their primary objective, and many (50%) performed formal hypothesis testing for effectiveness/efficacy despite being underpowered. Most (83%) included 'pilot' or 'feasibility' in the title, and discussed implications for progression from the pilot to the future definitive trial (89%), but fewer reported reasons for the randomised pilot trial (39%), sample size rationale (44%) or progression criteria (17%). Most defined the cluster (100%), and number of clusters randomised (94%), but few reported how the cluster design affected sample size (17%), whether consent was sought from clusters (11%), or who enrolled clusters (17%).

CONCLUSIONS

That only 18 pilot CRTs were identified necessitates increased awareness of the importance of conducting and publishing pilot CRTs and improved reporting. Pilot CRTs should primarily be assessing feasibility, avoiding formal hypothesis testing for effectiveness/efficacy and reporting reasons for the pilot, sample size rationale and progression criteria, as well as enrolment of clusters, and how the cluster design affects design aspects. We recommend adherence to the CONSORT extensions for pilot trials and CRTs.

Cluster-randomized controlled trials evaluating complex interventions in general practices are mostly ineffective: a systematic review

OBJECTIVES

The aim of this study was to evaluate how frequently complex interventions are shown to be superior to routine care in general practice-based cluster-randomized controlled studies (c-RCTs) and to explore whether potential differences explain results that come out in favor of a complex intervention.

STUDY DESIGN AND SETTING

We performed an unrestricted search in the Central Register of Controlled Trials, MEDLINE, and EMBASE. Included were all c-RCTs that included a patient-relevant primary outcome in a general practice setting with at least 1-year follow-up. We extracted effect sizes, P-values, intracluster correlation coefficients (ICCs), and 22 quality aspects.

RESULTS

We identified 29 trials with 99 patient-relevant primary outcomes. After adjustment for multiple testing on a trial level, four outcomes (4%) in four studies (14%) remained statistically significant. Of the 11 studies that reported ICCs, in 8, the ICC was equal to or smaller than the assumed ICC. In 16 of the 17 studies with available sample size calculation, effect sizes were smaller than anticipated.

CONCLUSION

More than 85% of the c-RCTs failed to demonstrate a beneficial effect on a predefined primary endpoint. All but one study were overly optimistic with regard to the expected treatment effect. This highlights the importance of weighing up the potential merit of new treatments and planning prospectively, when designing clinical studies in a general practice setting.

Missing binary outcomes under covariate-dependent missingness in cluster randomised trials

Missing outcomes are a commonly occurring problem for cluster randomised trials, which can lead to biased and inefficient inference if ignored or handled inappropriately. Two approaches for analysing such trials are cluster‐level analysis and individual‐level analysis. In this study, we assessed the performance of unadjusted cluster‐level analysis, baseline covariate‐adjusted cluster‐level analysis, random effects logistic regression and generalised estimating equations when binary outcomes are missing under a baseline covariate‐dependent missingness mechanism. Missing outcomes were handled using complete records analysis and multilevel multiple imputation. We analytically show that cluster‐level analyses for estimating risk ratio using complete records are valid if the true data generating model has log link and the intervention groups have the same missingness mechanism and the same covariate effect in the outcome model. We performed a simulation study considering four different scenarios, depending on whether the missingness mechanisms are the same or different between the intervention groups and whether there is an interaction between intervention group and baseline covariate in the outcome model. On the basis of the simulation study and analytical results, we give guidance on the conditions under which each approach is valid. © 2017 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.

The Spanish Version of the Unified Protocol for Transdiagnostic Treatment of Emotional Disorders in Adolescents (UP-A) Adapted as a School-Based Anxiety and Depression Prevention Program: Study Protocol for a Cluster Randomized Controlled Trial

Background

Anxiety and depression are common, impairing conditions that evidence high comorbidity rates in adolescence. The Unified Protocol for Transdiagnostic Treatment of Emotional Disorders in Adolescents (UP-A) is one of the few existing resources aimed at applying transdiagnostic treatment principles to target core dysfunctions associated with both anxiety and depression within a single protocol. To our knowledge, this is the first study examining the efficacy of the UP-A adapted as a universal preventive intervention program.

Objective

The primary aim of this study is to examine whether the Spanish version of the UP-A is more effective than a waitlist (WL) control group in reducing and preventing symptoms of anxiety and depression when employed as a universal, classroom-based preventive intervention. The secondary aim is to investigate changes in a broad range of secondary outcome measures, including negative and positive affect, anxiety sensitivity, emotional avoidance, top problems ratings, school grades, depression and anxiety-related interference, self-esteem, life satisfaction, quality of life, conduct problems, hyperactivity/inattention symptoms, peer problems, prosocial behavior, school adjustment, and discipline problems. Other aims are to assess a range of possible predictors of intervention effects and to examine the feasibility and the acceptability of implementing UP-A in a prevention group format and in a school setting.

Methods

A cluster, randomized, WL, controlled trial design with classroom as the unit of randomization was used in this study. Five classes including a total of 152 adolescents were randomized to the experimental or WL control groups. Participants in the experimental group received 9 55-minute sessions delivered by advanced doctoral and masters students in clinical psychology. The WL control group will receive the intervention once the 3-month follow-up assessment is completed.

Results

We have recruited participants to the cluster randomized controlled trial (RCT) and have conducted the intervention with the experimental group. We expect the WL control group to complete the intervention in July 2017. Data analysis will take place during the second semester of 2017.

Conclusions

We expect the experimental group to outperform the WL control group at post-intervention and 3-month follow-up. We also expect the WL control group to show improvements in primary and secondary outcome measures after receiving the intervention. Results will have implications for researchers, families, and education providers.

Trial Registration

Clinicaltrials.gov NCT03123991; https://clinicaltrials.gov/ct2/show/NCT03123991 (Archived by WebCite at http://www.webcitation.org/6qp7GIzcR)

Determining the efficacy of guppies and pyriproxyfen (Sumilarv® 2MR) combined with community engagement on dengue vectors in Cambodia: study protocol for a randomized controlled trial

Background

Evidence on the effectiveness of low-cost, sustainable, biological vector-control tools for the Aedes mosquitoes is limited. Therefore, the purpose of this trial is to estimate the impact of guppy fish (guppies), in combination with the use of the larvicide pyriproxyfen (Sumilarv® 2MR), and Communication for Behavioral Impact (COMBI) activities to reduce entomological indices in Cambodia.

Methods/design

In this cluster randomized controlled, superiority trial, 30 clusters comprising one or more villages each (with approximately 170 households) will be allocated, in a 1:1:1 ratio, to receive either (1) three interventions (guppies, Sumilarv® 2MR, and COMBI activities), (2) two interventions (guppies and COMBI activities), or (3) control (standard vector control). Households will be invited to participate, and entomology surveys among 40 randomly selected households per cluster will be carried out quarterly. The primary outcome will be the population density of adult female Aedes mosquitoes (i.e., number per house) trapped using adult resting collections. Secondary outcome measures will include the House Index, Container Index, Breteau Index, Pupae Per House, Pupae Per Person, mosquito infection rate, guppy fish coverage, Sumilarv® 2MR coverage, and percentage of respondents with knowledge about Aedes mosquitoes causing dengue. In the primary analysis, adult female Aedes density and mosquito infection rates will be aggregated over follow-up time points to give a single rate per cluster. This will be analyzed by negative binomial regression, yielding density ratios.

Discussion

This trial is expected to provide robust estimates of the intervention effect. A rigorous evaluation of these vector-control interventions is vital to developing an evidence-based dengue control strategy and to help direct government resources.

Trial registration

Current Controlled Trials, ID: ISRCTN85307778. Registered on 25 October 2015.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-017-2105-2) contains supplementary material, which is available to authorized users.

Review of Recent Methodological Developments in Group-Randomized Trials: Part 2-Analysis

In 2004, Murray et al. reviewed methodological developments in the design and analysis of group-randomized trials (GRTs). We have updated that review with developments in analysis of the past 13 years, with a companion article to focus on developments in design. We discuss developments in the topics of the earlier review (e.g., methods for parallel-arm GRTs, individually randomized group-treatment trials, and missing data) and in new topics, including methods to account for multiple-level clustering and alternative estimation methods (e.g., augmented generalized estimating equations, targeted maximum likelihood, and quadratic inference functions). In addition, we describe developments in analysis of alternative group designs (including stepped-wedge GRTs, network-randomized trials, and pseudocluster randomized trials), which require clustering to be accounted for in their design and analysis.

Handling Missing Data in Within-Trial Cost-Effectiveness Analysis: A Review with Future Recommendations

Cost-effectiveness analyses (CEAs) alongside randomised controlled trials (RCTs) are increasingly designed to collect resource use and preference-based health status data for the purpose of healthcare technology assessment. However, because of the way these measures are collected, they are prone to missing data, which can ultimately affect the decision of whether an intervention is good value for money. We examine how missing cost and effect outcome data are handled in RCT-based CEAs, complementing a previous review (covering 2003-2009, 88 articles) with a new systematic review (2009-2015, 81 articles) focussing on two different perspectives. First, we provide guidelines on how the information about missingness and related methods should be presented to improve the reporting and handling of missing data. We propose to address this issue by means of a quality evaluation scheme, providing a structured approach that can be used to guide the collection of information, elicitation of the assumptions, choice of methods and considerations of possible limitations of the given missingness problem. Second, we review the description of the missing data, the statistical methods used to deal with them and the quality of the judgement underpinning the choice of these methods. Our review shows that missing data in within-RCT CEAs are still often inadequately handled and the overall level of information provided to support the chosen methods is rarely satisfactory.

Treatment of missing data in follow-up studies of randomised controlled trials: A systematic review of the literature

BACKGROUND/AIMS

After completion of a randomised controlled trial, an extended follow-up period may be initiated to learn about longer term impacts of the intervention. Since extended follow-up studies often involve additional eligibility restrictions and consent processes for participation, and a longer duration of follow-up entails a greater risk of participant attrition, missing data can be a considerable threat in this setting. As a potential source of bias, it is critical that missing data are appropriately handled in the statistical analysis, yet little is known about the treatment of missing data in extended follow-up studies. The aims of this review were to summarise the extent of missing data in extended follow-up studies and the use of statistical approaches to address this potentially serious problem.

METHODS

We performed a systematic literature search in PubMed to identify extended follow-up studies published from January to June 2015. Studies were eligible for inclusion if the original randomised controlled trial results were also published and if the main objective of extended follow-up was to compare the original randomised groups. We recorded information on the extent of missing data and the approach used to treat missing data in the statistical analysis of the primary outcome of the extended follow-up study.

RESULTS

Of the 81 studies included in the review, 36 (44%) reported additional eligibility restrictions and 24 (30%) consent processes for entry into extended follow-up. Data were collected at a median of 7 years after randomisation. Excluding 28 studies with a time to event primary outcome, 51/53 studies (96%) reported missing data on the primary outcome. The median percentage of randomised participants with complete data on the primary outcome was just 66% in these studies. The most common statistical approach to address missing data was complete case analysis (51% of studies), while likelihood-based analyses were also well represented (25%). Sensitivity analyses around the missing data mechanism were rarely performed (25% of studies), and when they were, they often involved unrealistic assumptions about the mechanism.

CONCLUSION

Despite missing data being a serious problem in extended follow-up studies, statistical approaches to addressing missing data were often inadequate. We recommend researchers clearly specify all sources of missing data in follow-up studies and use statistical methods that are valid under a plausible assumption about the missing data mechanism. Sensitivity analyses should also be undertaken to assess the robustness of findings to assumptions about the missing data mechanism.

Effectiveness of a lumbopelvic monitor and feedback device to change postural behaviour: a protocol for the ELF cluster randomised controlled trial

INTRODUCTION

Low back pain (LBP) is the most common, costly and disabling musculoskeletal disorder worldwide, and is prevalent in healthcare workers. Posture is a modifiable risk factor for LBP shown to reduce the prevalence of LBP. Our feasibility research suggests that postural feedback might help healthcare workers avoid hazardous postures. The Effectiveness of Lumbopelvic Feedback (ELF) trial will investigate the extent to which postural monitor and feedback (PMF) can reduce exposure to hazardous posture associated with LBP.

METHODS

This is a participant-blinded, randomised controlled trial with blocked cluster random allocation. Participants will include volunteer healthcare workers recruited from aged care institutions and hospitals. A postural monitoring and feedback device will monitor and record lumbopelvic forward bending posture, and provide audio feedback whenever the user sustains a lumbopelvic forward bending posture that exceeds predefined thresholds. The primary outcome measure will be postural behaviour (exceeding thresholds). Secondary outcome measures will be incidence of LBP, participant-reported disability and adherence. Following baseline assessment, we will randomly assign participants to 1 of 2 intervention arms: a feedback group and a no-feedback control group. We will compare between-group differences of changes in postural behaviour by using a repeated measures mixed-effect model analysis of covariance (ANCOVA) at 6 weeks. Postural behaviour baseline scores, work-related psychosocial factors and disability scores will be input as covariates into the statistical models. We will use logistic mixed model analysis and Cox's proportional hazards for assessing the effect of a PMF on LBP incidence between groups.

DISCUSSION

Posture is a modifiable risk factor for low back disorders. Findings from the ELF trial will inform the design of future clinical trials assessing the effectiveness of wearable technology on minimising hazardous posture during daily living activities in patients with low back disorders.

TRIAL REGISTRATION NUMBER

ACTRN12616000449437.

Explanation and Elaboration Document for the STROBE-Vet Statement: Strengthening the Reporting of Observational Studies in Epidemiology-Veterinary Extension

The STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) statement was first published in 2007 and again in 2014. The purpose of the original STROBE was to provide guidance for authors, reviewers, and editors to improve the comprehensiveness of reporting; however, STROBE has a unique focus on observational studies. Although much of the guidance provided by the original STROBE document is directly applicable, it was deemed useful to map those statements to veterinary concepts, provide veterinary examples, and highlight unique aspects of reporting in veterinary observational studies. Here, we present the examples and explanations for the checklist items included in the STROBE-Vet statement. Thus, this is a companion document to the STROBE-Vet statement methods and process document (JVIM_14575 "Methods and Processes of Developing the Strengthening the Reporting of Observational Studies in Epidemiology-Veterinary (STROBE-Vet) Statement" undergoing proofing), which describes the checklist and how it was developed.

Increased risk of type I errors in cluster randomised trials with small or medium numbers of clusters: a review, reanalysis, and simulation study

BACKGROUND

Cluster randomised trials (CRTs) are commonly analysed using mixed-effects models or generalised estimating equations (GEEs). However, these analyses do not always perform well with the small number of clusters typical of most CRTs. They can lead to increased risk of a type I error (finding a statistically significant treatment effect when it does not exist) if appropriate corrections are not used.

METHODS

We conducted a small simulation study to evaluate the impact of using small-sample corrections for mixed-effects models or GEEs in CRTs with a small number of clusters. We then reanalysed data from TRIGGER, a CRT with six clusters, to determine the effect of using an inappropriate analysis method in practice. Finally, we reviewed 100 CRTs previously identified by a search on PubMed in order to assess whether trials were using appropriate methods of analysis. Trials were classified as at risk of an increased type I error rate if they did not report using an analysis method which accounted for clustering, or if they had fewer than 40 clusters and performed an individual-level analysis without reporting the use of an appropriate small-sample correction.

RESULTS

Our simulation study found that using mixed-effects models or GEEs without an appropriate correction led to inflated type I error rates, even for as many as 70 clusters. Conversely, using small-sample corrections provided correct type I error rates across all scenarios. Reanalysis of the TRIGGER trial found that inappropriate methods of analysis gave much smaller P values (P ≤ 0.01) than appropriate methods (P = 0.04-0.15). In our review, of the 99 trials that reported the number of clusters, 64 (65 %) were at risk of an increased type I error rate; 14 trials did not report using an analysis method which accounted for clustering, and 50 trials with fewer than 40 clusters performed an individual-level analysis without reporting the use of an appropriate correction.

CONCLUSIONS

CRTs with a small or medium number of clusters are at risk of an inflated type I error rate unless appropriate analysis methods are used. Investigators should consider using small-sample corrections with mixed-effects models or GEEs to ensure valid results.

Missing continuous outcomes under covariate dependent missingness in cluster randomised trials

Attrition is a common occurrence in cluster randomised trials which leads to missing outcome data. Two approaches for analysing such trials are cluster-level analysis and individual-level analysis. This paper compares the performance of unadjusted cluster-level analysis, baseline covariate adjusted cluster-level analysis and linear mixed model analysis, under baseline covariate dependent missingness in continuous outcomes, in terms of bias, average estimated standard error and coverage probability. The methods of complete records analysis and multiple imputation are used to handle the missing outcome data. We considered four scenarios, with the missingness mechanism and baseline covariate effect on outcome either the same or different between intervention groups. We show that both unadjusted cluster-level analysis and baseline covariate adjusted cluster-level analysis give unbiased estimates of the intervention effect only if both intervention groups have the same missingness mechanisms and there is no interaction between baseline covariate and intervention group. Linear mixed model and multiple imputation give unbiased estimates under all four considered scenarios, provided that an interaction of intervention and baseline covariate is included in the model when appropriate. Cluster mean imputation has been proposed as a valid approach for handling missing outcomes in cluster randomised trials. We show that cluster mean imputation only gives unbiased estimates when missingness mechanism is the same between the intervention groups and there is no interaction between baseline covariate and intervention group. Multiple imputation shows overcoverage for small number of clusters in each intervention group.

Quantity, topics, methods and findings of randomised controlled trials published by German university departments of general practice - systematic review

Background

Academic infrastructures and networks for clinical research in primary care receive little funding in Germany. We aimed to provide an overview of the quantity, topics, methods and findings of randomised controlled trials published by German university departments of general practice.

Methods

We searched Scopus (last search done in April 2015), publication lists of institutes and references of included articles. We included randomised trials published between January 2000 and December 2014 with a first or last author affiliated with a German university department of general practice or family medicine. Risk of bias was assessed with the Cochrane tool, and study findings were quantified using standardised mean differences (SMDs).

Results

Thirty-three trials met the inclusion criteria. Seventeen were cluster-randomised trials, with a majority investigating interventions aimed at improving processes compared with usual care. Sample sizes varied between 6 and 606 clusters and 168 and 7807 participants. The most frequent methodological problem was risk of selection bias due to recruitment of individuals after randomisation of clusters. Effects of interventions over usual care were mostly small (SMD <0.3). Sixteen trials randomising individual participants addressed a variety of treatment and educational interventions. Sample sizes varied between 20 and 1620 participants. The methodological quality of the trials was highly variable. Again, effects of experimental interventions over controls were mostly small.

Conclusions

Despite limited funding, German university institutes of general practice or family medicine are increasingly performing randomised trials. Cluster-randomised trials on practice improvement are a focus, but problems with allocation concealment are frequent.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-016-1328-y) contains supplementary material, which is available to authorized users.

Multiple imputation methods for bivariate outcomes in cluster randomised trials

Missing observations are common in cluster randomised trials. The problem is exacerbated when modelling bivariate outcomes jointly, as the proportion of complete cases is often considerably smaller than the proportion having either of the outcomes fully observed. Approaches taken to handling such missing data include the following: complete case analysis, single‐level multiple imputation that ignores the clustering, multiple imputation with a fixed effect for each cluster and multilevel multiple imputation.

We contrasted the alternative approaches to handling missing data in a cost‐effectiveness analysis that uses data from a cluster randomised trial to evaluate an exercise intervention for care home residents.

We then conducted a simulation study to assess the performance of these approaches on bivariate continuous outcomes, in terms of confidence interval coverage and empirical bias in the estimated treatment effects. Missing‐at‐random clustered data scenarios were simulated following a full‐factorial design.

Across all the missing data mechanisms considered, the multiple imputation methods provided estimators with negligible bias, while complete case analysis resulted in biased treatment effect estimates in scenarios where the randomised treatment arm was associated with missingness. Confidence interval coverage was generally in excess of nominal levels (up to 99.8%) following fixed‐effects multiple imputation and too low following single‐level multiple imputation. Multilevel multiple imputation led to coverage levels of approximately 95% throughout. © 2016 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.

How often are interventions in cluster-randomised controlled trials of complex interventions in general practices effective and reasons for potential shortcomings? Protocol and results of a feasibility project for a systematic review

INTRODUCTION

Most studies conducted at general practices investigate complex interventions and increasingly use cluster-randomised controlled trail (c-RCT) designs to do so. Our primary objective is to evaluate how frequently complex interventions are shown to be more, equally or less effective than routine care in c-RCTs with a superior design. The secondary aim is to discover whether the quality of a c-RCT determines the likelihood of the complex intervention being effective.

METHODS AND ANALYSIS

All c-RCTs of any design that have a patient-relevant primary outcome and with a duration of at least 1 year will be included. The search will be performed in three electronic databases (MEDLINE, EMBASE and the Cochrane Database of Systematic Reviews (CDSR)). The screening process, data collection, quality assessment and statistical data analyses (if suitably similar and of adequate quality) will be performed in accordance with requirements of the Cochrane Handbook for Systematic Reviews of Interventions. A feasibility project was carried out that was restricted to a search in MEDLINE and the CCTR for c-RCTs published in 1 of the 8 journals that are most relevant to general practice. The process from trial selection to data collection, assessment and results presentation was piloted. Of the 512 abstracts identified during the feasibility search, 21 studies examined complex interventions in a general practice setting. Extrapolating the preliminary search to include all relevant c-RCTs in three databases, about 5000 abstracts and 150 primary studies are expected to be identified in the main study. 14 studies included in the feasibility project (67%) did not show a positive effect on a primary patient-relevant end point.

ETHICS AND DISSEMINATION

Ethical approval is not being sought for this review. Findings will be disseminated via peer-reviewed journals that frequently publish articles on the results of c-RCTs and through presentations at international conferences.

TRIAL REGISTRATION NUMBER

PROSPERO CRD201400923.

Statistical analysis and handling of missing data in cluster randomized trials: a systematic review

Background

Cluster randomized trials (CRTs) randomize participants in groups, rather than as individuals and are key tools used to assess interventions in health research where treatment contamination is likely or if individual randomization is not feasible. Two potential major pitfalls exist regarding CRTs, namely handling missing data and not accounting for clustering in the primary analysis. The aim of this review was to evaluate approaches for handling missing data and statistical analysis with respect to the primary outcome in CRTs.

Methods

We systematically searched for CRTs published between August 2013 and July 2014 using PubMed, Web of Science, and PsycINFO. For each trial, two independent reviewers assessed the extent of the missing data and method(s) used for handling missing data in the primary and sensitivity analyses. We evaluated the primary analysis and determined whether it was at the cluster or individual level.

Results

Of the 86 included CRTs, 80 (93 %) trials reported some missing outcome data. Of those reporting missing data, the median percent of individuals with a missing outcome was 19 % (range 0.5 to 90 %). The most common way to handle missing data in the primary analysis was complete case analysis (44, 55 %), whereas 18 (22 %) used mixed models, six (8 %) used single imputation, four (5 %) used unweighted generalized estimating equations, and two (2 %) used multiple imputation. Fourteen (16 %) trials reported a sensitivity analysis for missing data, but most assumed the same missing data mechanism as in the primary analysis. Overall, 67 (78 %) trials accounted for clustering in the primary analysis.

Conclusions

High rates of missing outcome data are present in the majority of CRTs, yet handling missing data in practice remains suboptimal. Researchers and applied statisticians should carry out appropriate missing data methods, which are valid under plausible assumptions in order to increase statistical power in trials and reduce the possibility of bias. Sensitivity analysis should be performed, with weakened assumptions regarding the missing data mechanism to explore the robustness of results reported in the primary analysis.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-016-1201-z) contains supplementary material, which is available to authorized users.

Best (but oft-forgotten) practices: designing, analyzing, and reporting cluster randomized controlled trials

Cluster randomized controlled trials (cRCTs; also known as group randomized trials and community-randomized trials) are multilevel experiments in which units that are randomly assigned to experimental conditions are sets of grouped individuals, whereas outcomes are recorded at the individual level. In human cRCTs, clusters that are randomly assigned are typically families, classrooms, schools, worksites, or counties. With growing interest in community-based, public health, and policy interventions to reduce obesity or improve nutrition, the use of cRCTs has increased. Errors in the design, analysis, and interpretation of cRCTs are unfortunately all too common. This situation seems to stem in part from investigator confusion about how the unit of randomization affects causal inferences and the statistical procedures required for the valid estimation and testing of effects. In this article, we provide a brief introduction and overview of the importance of cRCTs and highlight and explain important considerations for the design, analysis, and reporting of cRCTs by using published examples.

Statistical analysis and handling of missing data in cluster randomised trials: protocol for a systematic review

INTRODUCTION

Cluster randomised trials (CRTs) randomise participants in groups, rather than as individuals, and are key tools used to assess interventions in health research where treatment contamination is likely or if individual randomisation is not feasible. Missing outcome data can reduce power in trials, including in CRTs, and is a potential source of bias. The current review focuses on evaluating methods used in statistical analysis and handling of missing data with respect to the primary outcome in CRTs.

METHODS AND ANALYSIS

We will search for CRTs published between August 2013 and July 2014 using PubMed, Web of Science and PsycINFO. We will identify relevant studies by screening titles and abstracts, and examining full-text articles based on our predefined study inclusion criteria. 86 studies will be randomly chosen to be included in our review. Two independent reviewers will collect data from each study using a standardised, prepiloted data extraction template. Our findings will be summarised and presented using descriptive statistics.

ETHICS AND DISSEMINATION

This methodological systematic review does not need ethical approval because there are no data used in our study that are linked to individual patient data. After completion of this systematic review, data will be immediately analysed, and findings will be disseminated through a peer-reviewed publication and conference presentation.