Internal and external validity of cluster randomised trials: systematic review of recent trials

Kids SIPsmartER reduces sugar-sweetened beverages among Appalachian middle-school students and their caregivers: a cluster randomized controlled trial

BACKGROUND

High consumption of sugar-sweetened beverages (SSB) is a global health concern. Additionally, sugar-sweetened beverage (SSB) consumption is disproportionately high among adolescents and adults in rural Appalachia. The primary study objective is to determine the intervention effects of Kids SIPsmartER on students' SSB consumption. Secondary objectives focus on caregivers' SSB consumption and secondary student and caregiver outcomes [e.g, body mass index (BMI), quality of life (QOL)].

METHODS

This Type 1 hybrid, cluster randomized controlled trial includes 12 Appalachian middle schools (6 randomized to Kids SIPsmartER and 6 to control). Kids SIPsmartER is a 6-month, 12 lesson, multi-level, school-based, behavior and health literacy program aimed at reducing SSB among 7th grade middle school students. The program also incorporates a two-way text message strategy for caregivers. In this primary prevention intervention, all 7th grade students and their caregivers from participating schools were eligible to participate, regardless of baseline SSB consumption. Validated instruments were used to assess SSB behaviors and QOL. Height and weight were objectively measured in students and self-reported by caregivers. Analyses included modified two-part models with time fixed effects that controlled for relevant demographics and included school cluster robust standard errors.

RESULTS

Of the 526 students and 220 caregivers, mean (SD) ages were 12.7 (0.5) and 40.6 (6.7) years, respectively. Students were 55% female. Caregivers were mostly female (95%) and White (93%); 25% had a high school education or less and 33% had an annual household income less than $50,000. Regardless of SSB intake at baseline and relative to control participants, SSB significantly decreased among students [-7.2 ounces/day (95% CI = -10.7, -3.7); p < 0.001, effect size (ES) = 0.35] and caregivers [-6.3 ounces/day (95% CI = -11.3, -1.3); p = 0.014, ES = 0.33]. Among students (42%) and caregivers (28%) who consumed > 24 SSB ounces/day at baseline (i.e., high consumers), the ES increased to 0.45 and 0.95, respectively. There were no significant effects for student or caregiver QOL indicators or objectively measured student BMI; however, caregiver self-reported BMI significantly decreased in the intervention versus control schools (p = 0.001).

CONCLUSIONS

Kids SIPsmartER was effective at reducing SSB consumption among students and their caregivers in the rural, medically underserved Appalachian region. Importantly, SSB effects were even stronger among students and caregivers who were high consumers at baseline.

TRIAL REGISTRATION

Clincialtrials.gov: NCT03740113. Registered 14 November 2018- Retrospectively registered, https://clinicaltrials.gov/ct2/show/NCT03740113 .

Estimating average treatment effects for clustered RCTs with recruitment bias

In clustered randomized controlled trials (RCTs), sample recruitment is often conducted after cluster randomization. This timing can lead to recruitment bias if access to the intervention affects the composition of study-eligible cluster entrants and study consenters. This article develops a potential outcomes framework in such settings that yields a causal estimand that pertains to the always-recruited in either research condition. A consistent inverse probability weighting (IPW) estimator is developed using data on recruits only, and a generalized estimating equations approach is used to obtain robust clustered SE estimators that adjust for estimation error in the IPW weights. A simple data collection strategy is discussed to improve the predictive accuracy of the logit propensity score models. Simulations show that the IPW estimator achieves nominal confidence interval coverage under the assumed identification conditions. An empirical application demonstrates the methods using data from an RCT testing the effects of a behavioral health intervention in schools. An R program for estimation is available for download.

Nutrition supplementation plus standard of care versus standard of care alone or standard of care plus unconditional cash transfer in the prevention of chronic malnutrition in Southern Angola: study protocol for the MuCCUA study, a cluster randomized controlled trial

BACKGROUND

Chronic malnutrition is a condition associated with negative impacts on physical and cognitive development. It is multi-causal and can start very early in life, already in utero, thus it is especially challenging to find appropriate interventions to tackle it. The government of Angola is implementing a standard of care program with potential to prevent it, and the provision of cash transfers and the supplementation with small quantity lipid-based nutrients (SQ-LNS) are also promising interventions. We aimed to evaluate the impact of the standard of care program alone and of the standard of care plus a cash transfer intervention in the lineal growth of children less than 2 years old and compare it to the effectiveness of a nutrition supplementation plus standard of care program in Southern Angola.

METHODS/DESIGN

The three-arm parallel cluster randomised controlled trial is set in four communes of Huila and Cunene provinces. Clusters are villages or neighbourhoods with a population around 1075 people. A total of twelve clusters were selected per arm and forty pregnant women are expected to be recruited in each cluster. Pregnant women receive the standard of care alone, or the standard of care plus unconditional cash transfer or plus nutritional supplementation during the first 1000 days, from pregnancy to the child reaching 24 months. The primary outcome is the prevalence of stunting measured as height-for-age Z-score (HAZ) < -2 in children below 2 years. Impact will be assessed at 3, 6, 12, 18 and 24 months of children's age. Secondary outcomes include mortality, morbidity, caring, hygiene and nutrition behaviours and practices, and women and children's dietary diversity. Quantitative data are also collected on women's empowerment, household food security, expenditure and relevant clinical and social events at baseline, endline and intermediate time points.

DISCUSSION

The results will provide valuable information on the impact of the standard of care intervention alone as well as combined with an unconditional cash transfer intervention compared to a nutrition supplementation plus standard of care intervention, carried out during the first 1000 days, in the children´s growth up to 2 years and related outcomes in Southern Angola.

TRIAL REGISTRATION

Clinical Trials NCT05571280. Registered 7 October 2022.

Effect of nutrition education integrating the health belief model and theory of planned behavior on dietary diversity of pregnant women in Southeast Ethiopia: a cluster randomized controlled trial

BACKGROUND

Maternal anemia, miscarriage, low birth weight (LBW), preterm birth (PTB), intrauterine growth restriction (IUGR), prenatal and infant mortality, morbidity, and the risk of chronic disease later in life are all increased by a lack of dietary diversity during pregnancy. However, evidence for the effect of nutrition education on the dietary diversity score (DDS) among pregnant women was sparse in Ethiopia, particularly in the study areas. This study aimed to assess the effect of nutrition education on dietary diversity among pregnant women in urban settings in Southeast Ethiopia.

METHODS

A community-based two-arm parallel cluster randomized controlled trial was conducted among 447 randomly selected pregnant women attending antenatal care (224 intervention group and 223 control group) at health facilities from February to December 2021. A multistage cluster sampling technique, followed by systematic sampling, was used to select the pregnant women. Pregnant women who participated in the interventions were given nutrition education starting at 16 weeks of gestation and continuing for 6 months. We used a pre-tested, interviewer-administered, structured questionnaire to collect the data. A 24-hour qualitative dietary recall was used to calculate the dietary diversity score (DDS). A multivariable generalized estimating equation (GEE) model was conducted to evaluate the intervention effect.

RESULTS

After the intervention, the proportion of adequate dietary diversity was 14.15% higher in the intervention arm compared to the control group (45.09% versus 30.94%, P = 0.002). The overall difference in adequate dietary diversity between the two groups was 8.5%. After adjusting for background characteristics, the multivariable GEE binary logistic model revealed that having received intervention [(AOR = 1.89, 95% CI: 1.27, 2.79)], being literate [(AOR = 3.41, 95% CI: 1.13, 10.23)], and having high wealth [(AOR = 1.60, 95% CI: 1.09, 2.35)] significantly improved adequate dietary diversity.

CONCLUSION

The findings indicated that having received the intervention, being literate, and having a high level of wealth significantly improved maternal dietary diversity. Efforts should be made to increase nutrition education using the health belief model (HBM) and the theory of planned behavior (TPB). Moreover, there is a need to improve literacy and economic empowerment through income-generating activities to enhance adequate dietary diversification during pregnancy.

TRIAL REGISTRATION

Clinicaltrials.gov (PACTR202201731802989, Retrospectively registered on 24 January 2022).

A Process Evaluation of an Antibiotic Stewardship Intervention for Urinary Tract Infections in Nursing Homes

OBJECTIVES

To assess the internal and external validity of a cluster randomized controlled trial (cRCT) evaluating a decision tool with supportive interventions for the empirical treatment of urinary tract infections (UTIs) in nursing homes (NHs), and to identify facilitators and barriers in implementing this antibiotic stewardship intervention.

DESIGN

Mixed-methods process evaluation study.

SETTING AND PARTICIPANTS

Physicians, nursing staff, client council members, and residents of Dutch NHs.

METHODS

We used cRCT data of the ANNA study (Antibiotic Prescribing and Non-prescribing in Nursing Home Residents With Signs and Symptoms Ascribed to Urinary Tract Infection). In addition, we sent out an online evaluation questionnaire, conducted semistructured interviews with physicians and nursing staff, and consulted client council members.

RESULTS

Internal validity was lowered: control group physicians participated in several non-study-related activities regarding UTI. External validity was good: almost all intervention components had a high fidelity (52%-74%) and were perceived as relevant (physicians: 7.2-8.6 of 10, nursing staff: 6.5-8.5 of 10) and feasible (physicians: 7.5 of 10, nursing staff 6.4 of 10), with feasibility for residents with dementia and urine incontinence needing attention. The most common reason for deviating from the advice generated by the decision tool was an unclear illness presentation. Identified facilitators to implementation were confidence in the intervention, repeated intervention encounter, and having "champions" in the NH. Barriers were limited involvement of nursing staff, unstable nursing teams, residents' and representatives' belief that antibiotics should be prescribed, and a low antibiotic prescribing threshold within the NH culture.

CONCLUSIONS AND IMPLICATIONS

Lowered internal validity may have reduced the study effect. Attention should be paid to the feasibility of the intervention in residents with dementia and urinary incontinence. Improvement opportunities for implementation were higher nursing staff involvement and repeated intervention offering.

Assessing the representativeness of cluster randomized trials: Evidence from two large pragmatic trials in United States nursing homes

BACKGROUND/AIMS

When the randomized clusters in a cluster randomized trial are selected based on characteristics that influence treatment effectiveness, results from the trial may not be directly applicable to the target population. We used data from two large nursing home-based pragmatic cluster randomized trials to compare nursing home and resident characteristics in randomized facilities to eligible non-randomized and ineligible facilities.

METHODS

We linked data from the high-dose influenza vaccine trial and the Music & Memory Pragmatic TRIal for Nursing Home Residents with ALzheimer's Disease (METRICaL) to nursing home assessments and Medicare fee-for-service claims. The target population for the high-dose trial comprised Medicare-certified nursing homes; the target population for the METRICaL trial comprised nursing homes in one of four US-based nursing home chains. We used standardized mean differences to compare facility and individual characteristics across the three groups and logistic regression to model the probability of nursing home trial participation.

RESULTS

In the high-dose trial, 4476 (29%) of the 15,502 nursing homes in the target population were eligible for the trial, of which 818 (18%) were randomized. Of the 1,361,122 residents, 91,179 (6.7%) were residents of randomized facilities, 463,703 (34.0%) of eligible non-randomized facilities, and 806,205 (59.3%) of ineligible facilities. In the METRICaL trial, 160 (59%) of the 270 nursing homes in the target population were eligible for the trial, of which 80 (50%) were randomized. Of the 20,262 residents, 973 (34.4%) were residents of randomized facilities, 7431 (36.7%) of eligible non-randomized facilities, and 5858 (28.9%) of ineligible facilities. In the high-dose trial, randomized facilities differed from eligible non-randomized and ineligible facilities by the number of beds (132.5 vs 145.9 and 91.9, respectively), for-profit status (91.8% vs 66.8% and 68.8%), belonging to a nursing home chain (85.8% vs 49.9% and 54.7%), and presence of a special care unit (19.8% vs 25.9% and 14.4%). In the METRICaL trial randomized facilities differed from eligible non-randomized and ineligible facilities by the number of beds (103.7 vs 110.5 and 67.0), resource-poor status (4.6% vs 10.0% and 18.8%), and presence of a special care unit (26.3% vs 33.8% and 10.9%). In both trials, the characteristics of residents in randomized facilities were similar across the three groups.

CONCLUSION

In both trials, facility-level characteristics of randomized nursing homes differed considerably from those of eligible non-randomized and ineligible facilities, while there was little difference in resident-level characteristics across the three groups. Investigators should assess the characteristics of clusters that participate in cluster randomized trials, not just the individuals within the clusters, when examining the applicability of trial results beyond participating clusters.

Key considerations for designing, conducting and analysing a cluster randomized trial

Not only do cluster randomized trials require a larger sample size than individually randomized trials, they also face many additional complexities. The potential for contamination is the most commonly used justification for using cluster randomization, but the risk of contamination should be carefully weighed against the more serious problem of questionable scientific validity in settings with post-randomization identification or recruitment of participants unblinded to the treatment allocation. In this paper we provide some simple guidelines to help researchers conduct cluster trials in a way that minimizes potential biases and maximizes statistical efficiency. The overarching theme of this guidance is that methods that apply to individually randomized trials rarely apply to cluster randomized trials. We recommend that cluster randomization be only used when necessary-balancing the benefits of cluster randomization with its increased risks of bias and increased sample size. Researchers should also randomize at the lowest possible level-balancing the risks of contamination with ensuring an adequate number of randomization units-as well as exploring other options for statistically efficient designs. Clustering should always be allowed for in the sample size calculation; and the use of restricted randomization (and adjustment in the analysis for covariates used in the randomization) should be considered. Where possible, participants should be recruited before randomizing clusters and, when recruiting (or identifying) participants post-randomization, recruiters should be masked to the allocation. In the analysis, the target of inference should align with the research question, and adjustment for clustering and small sample corrections should be used when the trial includes less than about 40 clusters.

A personalized intervention to prevent depression in primary care based on risk predictive algorithms and decision support systems: protocol of the e-predictD study

The predictD is an intervention implemented by general practitioners (GPs) to prevent depression, which reduced the incidence of depression-anxiety and was cost-effective. The e-predictD study aims to design, develop, and evaluate an evolved predictD intervention to prevent the onset of major depression in primary care based on Information and Communication Technologies, predictive risk algorithms, decision support systems (DSSs), and personalized prevention plans (PPPs). A multicenter cluster randomized trial with GPs randomly assigned to the e-predictD intervention + care-as-usual (CAU) group or the active-control + CAU group and 1-year follow-up is being conducted. The required sample size is 720 non-depressed patients (aged 18-55 years), with moderate-to-high depression risk, under the care of 72 GPs in six Spanish cities. The GPs assigned to the e-predictD-intervention group receive brief training, and those assigned to the control group do not. Recruited patients of the GPs allocated to the e-predictD group download the e-predictD app, which incorporates validated risk algorithms to predict depression, monitoring systems, and DSSs. Integrating all inputs, the DSS automatically proposes to the patients a PPP for depression based on eight intervention modules: physical exercise, social relationships, improving sleep, problem-solving, communication skills, decision-making, assertiveness, and working with thoughts. This PPP is discussed in a 15-min semi-structured GP-patient interview. Patients then choose one or more of the intervention modules proposed by the DSS to be self-implemented over the next 3 months. This process will be reformulated at 3, 6, and 9 months but without the GP-patient interview. Recruited patients of the GPs allocated to the control-group+CAU download another version of the e-predictD app, but the only intervention that they receive via the app is weekly brief psychoeducational messages (active-control group). The primary outcome is the cumulative incidence of major depression measured by the Composite International Diagnostic Interview at 6 and 12 months. Other outcomes include depressive symptoms (PHQ-9) and anxiety symptoms (GAD-7), depression risk (predictD risk algorithm), mental and physical quality of life (SF-12), and acceptability and satisfaction ('e-Health Impact' questionnaire) with the intervention. Patients are evaluated at baseline and 3, 6, 9, and 12 months. An economic evaluation will also be performed (cost-effectiveness and cost-utility analysis) from two perspectives, societal and health systems.

Trial registration

ClinicalTrials.gov, identifier: NCT03990792.

Strategies to minimise and monitor biases and imbalances by arm in surgical cluster randomised trials: evidence from ChEETAh, a trial in seven low- and middle-income countries

BACKGROUND

Cluster randomised controlled trials (cRCT) present challenges regarding risks of bias and chance imbalances by arm. This paper reports strategies to minimise and monitor biases and imbalances in the ChEETAh cRCT.

METHODS

ChEETAh was an international cRCT (hospitals as clusters) evaluating whether changing sterile gloves and instruments prior to abdominal wound closure reduces surgical site infection at 30 days postoperative. ChEETAh planned to recruit 12,800 consecutive patients from 64 hospitals in seven low-middle income countries. Eight strategies to minimise and monitor bias were pre-specified: (1) minimum of 4 hospitals per country; (2) pre-randomisation identification of units of exposure (operating theatres, lists, teams or sessions) within clusters; (3) minimisation of randomisation by country and hospital type; (4) site training delivered after randomisation; (5) dedicated 'warm-up week' to train teams; (6) trial specific sticker and patient register to monitor consecutive patient identification; (7) monitoring characteristics of patients and units of exposure; and (8) low-burden outcome-assessment.

RESULTS

This analysis includes 10,686 patients from 70 clusters. The results aligned to the eight strategies were (1) 6 out of 7 countries included ≥ 4 hospitals; (2) 87.1% (61/70) of hospitals maintained their planned operating theatres (82% [27/33] and 92% [34/37] in the intervention and control arms); (3) minimisation maintained balance of key factors in both arms; (4) post-randomisation training was conducted for all hospitals; (5) the 'warm-up week' was conducted at all sites, and feedback used to refine processes; (6) the sticker and trial register were maintained, with an overall inclusion of 98.1% (10,686/10,894) of eligible patients; (7) monitoring allowed swift identification of problems in patient inclusion and key patient characteristics were reported: malignancy (20.3% intervention vs 12.6% control), midline incisions (68.4% vs 58.9%) and elective surgery (52.4% vs 42.6%); and (8) 0.4% (41/9187) of patients refused consent for outcome assessment.

CONCLUSION

cRCTs in surgery have several potential sources of bias that include varying units of exposure and the need for consecutive inclusion of all eligible patients across complex settings. We report a system that monitored and minimised the risks of bias and imbalances by arm, with important lessons for future cRCTs within hospitals.

Bridging gaps across levels of care in rehabilitation of patients with rheumatic and musculoskeletal diseases: Results from a stepped-wedge cluster randomized controlled trial

OBJECTIVE

To compare the effectiveness of a structured goal-setting and tailored follow-up rehabilitation intervention with existing rehabilitation in patients with rheumatic and musculoskeletal diseases.

DESIGN

A pragmatic stepped-wedge cluster randomized trial.

SETTING

Eight rehabilitation centers in secondary healthcare, Norway.

PARTICIPANTS

A total of 374 adults with rheumatic and musculoskeletal diseases were included in either the experimental (168) or the control group (206).

INTERVENTIONS

A new rehabilitation intervention which comprised structured goal setting, action planning, motivational interviewing, digital self-monitoring of goal progress, and individual follow-up support after discharge according to patients' needs and available resources in primary healthcare (the BRIDGE-intervention), was compared to usual care.

MAIN MEASURES

Patient-reported outcomes were collected electronically on admission and discharge from rehabilitation, and after 2, 7, and 12 months. The primary outcome was patients' goal attainment measured by the Patient Specific Functional Scale (0-10, 10 best) at 7 months. Secondary outcome measures included physical function (30-s Sit-To-Stand test), health-related quality of life (EQ-5D-5L-index), and self-assessed health (EQ-VAS). The main statistical analyses were performed on an intention-to-treat basis using linear mixed models.

RESULTS

No significant treatment effects of the BRIDGE-intervention were found for either primary (Patient Specific Functional Scale mean difference 0.1 [95% CI: -0.5, 0.8], p = 0.70), or secondary outcomes 7 months after rehabilitation.

CONCLUSION

The BRIDGE-intervention was not shown to be more effective than existing rehabilitation for patients with rheumatic and musculoskeletal diseases. There is still a need for more knowledge about factors that can improve the quality, continuity, and long-term health effects of rehabilitation for this patient group.

Evaluating physical activity interventions for socioeconomically disadvantaged adults through the RE-AIM framework: A systematic review of experimental and non–/quasi-experimental trials

•

The review evaluated internal and external validity using the rE-AIM Framework.

•

Results indicate an overall low reporting of validity factors.

•

Information on adoption and maintenance was particularly low.

•

Future studies should improve the reporting across all rE-AIM dimensions.

•

Future studies should consider adaptations during the implementation phase.

The promotion of physical activity in people from lower social strata is a public health priority. Previous reviews of physical activity interventions among socioeconomically disadvantaged adults have focused on intervention effectiveness without considering their translation into practice. This review utilised the RE-AIM framework (Reach, Efficacy/Effectiveness, Adoption, Implementation, Maintenance) to (1) evaluate the extent to which experimental and non–/quasi-experimental trials of interventions to promote physical activity among socioeconomically disadvantaged adults report on issues of internal and external validity and (2) to provide recommendations for future intervention studies.

Four databases were searched through June 2021. We included studies published in English or German since 2000 that tested physical activity interventions for socioeconomically disadvantaged adults. Two researchers coded all studies using a validated RE-AIM data extraction tool with 61 indicators referring to internal and external validity. Binary coding (yes = 1/no = 0) was applied to calculate the number and percentage of studies reporting each of the indicators.

We included 39 studies of which 22 were non–/quasi-experimental trials. Indicators of reach were most frequently reported (59.2%), followed by implementation (38.9%) and efficacy/effectiveness (28.9%). Dimensions related to external validity were least frequently reported (adoption: 21.9%, maintenance: 17.8%). Few differences were found between experimental and non–/quasi-experimental trials.

Analysis showed overall poor reporting of components related to internal and external validity. We recommend that future research should increase attention on reporting indicators of internal and external validity to facilitate their translation and implementation into real world settings.

Trial registration: The review was registered with PROSPERO (CRD42021283688).

The challenges of assessing sample representativeness within community-based evaluations of parenting programmes delivered in England and Ireland

The findings from evaluations of parenting programmes can help inform policy and practice decisions, including how best to allocate scarce resources designed to support families. However, studies often fail to consider the extent to which the findings can be generalised to other settings or populations. One, yet unexplored solution, is to compare study findings and large-scale data sets including publicly available population data. The aims of this study were to assess the feasibility of assessing sample representativeness using publicly available data and to identify the challenges involved in considering the generalisability of study findings. Sociodemographic data from two community-based evaluations of parenting programmes conducted in England and Ireland between 2015 and 2018 were used in the study (N = 395 parent-infant dyads). The results indicated some differences between the trial samples and the wider population. However, it is difficult to reach definitive conclusions about these findings due to the limitations associated with using the comparative data sets. Our study revealed three key challenges, including: (1) how best to define and conceptualise representativeness; (2) the availability of comparative data sets; and (3) the quality of the available data. Our study suggests that there is a need for up-to-date, good-quality comparative data sets to allow for the assessment of representativeness. Further work is required to identify parameters for making claims about representativeness, specifically regarding the acceptable level of difference between the target population and the study sample. This is the first study to explore the feasibility of using publicly available population data in two jurisdictions, for the purpose of making judgements about the representativeness of the findings from parent programmes. It is hoped that our results will encourage further investigation around the reporting of trial external validity to enable effective decision-making at policy and practice level.

Influential methods reports for group-randomized trials and related designs

BACKGROUND

This article identifies the most influential methods reports for group-randomized trials and related designs published through 2020. Many interventions are delivered to participants in real or virtual groups or in groups defined by a shared interventionist so that there is an expectation for positive correlation among observations taken on participants in the same group. These interventions are typically evaluated using a group- or cluster-randomized trial, an individually randomized group treatment trial, or a stepped wedge group- or cluster-randomized trial. These trials face methodological issues beyond those encountered in the more familiar individually randomized controlled trial.

METHODS

PubMed was searched to identify candidate methods reports; that search was supplemented by reports known to the author. Candidate reports were reviewed by the author to include only those focused on the designs of interest. Citation counts and the relative citation ratio, a new bibliometric tool developed at the National Institutes of Health, were used to identify influential reports. The relative citation ratio measures influence at the article level by comparing the citation rate of the reference article to the citation rates of the articles cited by other articles that also cite the reference article.

RESULTS

In total, 1043 reports were identified that were published through 2020. However, 55 were deemed to be the most influential based on their relative citation ratio or their citation count using criteria specific to each of the three designs, with 32 group-randomized trial reports, 7 individually randomized group treatment trial reports, and 16 stepped wedge group-randomized trial reports. Many of the influential reports were early publications that drew attention to the issues that distinguish these designs from the more familiar individually randomized controlled trial. Others were textbooks that covered a wide range of issues for these designs. Others were "first reports" on analytic methods appropriate for a specific type of data (e.g. binary data, ordinal data), for features commonly encountered in these studies (e.g. unequal cluster size, attrition), or for important variations in study design (e.g. repeated measures, cohort versus cross-section). Many presented methods for sample size calculations. Others described how these designs could be applied to a new area (e.g. dissemination and implementation research). Among the reports with the highest relative citation ratios were the CONSORT statements for each design.

CONCLUSIONS

Collectively, the influential reports address topics of great interest to investigators who might consider using one of these designs and need guidance on selecting the most appropriate design for their research question and on the best methods for design, analysis, and sample size.

Interim data monitoring in cluster randomised trials: Practical issues and a case study

BACKGROUND

There is an abundance of guidance for the interim monitoring of individually randomised trials. While methodological literature exists on how to extend these methods to cluster randomised trials, there is little guidance on practical implementation. Cluster trials have many features which make their monitoring needs different. We outline the methodological and practical challenges of interim monitoring of cluster trials; and apply these considerations to a case study.

CASE STUDY

The E-MOTIVE study is an 80-cluster randomised trial of a bundle of interventions to treat postpartum haemorrhage. The proposed data monitoring plan includes (1) monitor sample size assumptions, (2) monitor for evidence of selection bias, and (3) an interim assessment of the primary outcome, as well as monitoring data completeness. The timing of the sample size monitoring is chosen with both consideration of statistical precision and to allow time to recruit more clusters. Monitoring for selection bias involves comparing individual-level characteristics and numbers recruited between study arms to identify any post-randomisation participant identification bias. An interim analysis of outcomes presented with 99.9% confidence intervals using the Haybittle-Peto approach should mitigate any concern regarding the inflation of type-I error. The pragmatic nature of the trial means monitoring for adherence is not relevant, as it is built into a process evaluation.

CONCLUSIONS

The interim analyses of cluster trials have a number of important differences to monitoring individually randomised trials. In cluster trials, there will often be a greater need to monitor nuisance parameters, yet there will often be considerable uncertainty in their estimation. This means the utility of sample size re-estimation can be questionable particularly when there are practical or funding difficulties associated with making any changes to planned sample sizes. Perhaps most importantly interim monitoring has the potential to identify selection bias, particularly in trials with post-randomisation identification or recruitment. Finally, the pragmatic nature of cluster trials might mean that the utility of methods to allow for interim monitoring of outcomes based on statistical testing, or monitoring for adherence to study interventions, are less relevant. Our intention is to facilitate the planning of future cluster randomised trials and to promote discussion and debate to improve monitoring of these studies.

Cluster randomized trials of individual-level interventions were at high risk of bias

OBJECTIVES

To describe the prevalence of risks of bias in cluster-randomized trials of individual-level interventions, according to the Cochrane Risk of Bias tool.

STUDY DESIGN AND SETTING

Review undertaken in duplicate of a random sample of 40 primary reports of cluster-randomized trials of individual-level interventions.

RESULTS

The most common reported reasons for adopting cluster randomization were the need to avoid contamination (17, 42.5%) and practical considerations (14, 35%). Of the 40 trials all but one was assessed as being at risk of bias. A majority (27, 67.5%) were assessed as at risk due to the timing of identification and recruitment of participants; many (21, 52.5%) due to an apparent lack of adequate allocation concealment; and many due to selectively reported results (22, 55%), arising from a mixture of reasons including lack of documentation of primary outcome. Other risks mostly occurred infrequently.

CONCLUSION

Many cluster-randomized trials evaluating individual-level interventions appear to be at risk of bias, mostly due to identification and recruitment biases. We recommend that investigators carefully consider the need for cluster randomization; follow recommended procedures to mitigate risks of identification and recruitment bias; and adhere to good reporting practices including clear documentation of primary outcome and allocation concealment methods.

Impact of a health literacy intervention combining general practitioner training and a consumer facing intervention to improve colorectal cancer screening in underserved areas: protocol for a multicentric cluster randomized controlled trial

BACKGROUND

Colorectal cancer (CRC) is a leading cause of cancer burden worldwide. In France, it is the second most common cause of cancer death after lung cancer. Systematic uptake of CRC screening can improve survival rates. However, people with limited health literacy (HL) and lower socioeconomic position rarely participate. Our aim is to assess the impact of an intervention combining HL and CRC screening training for general practitioners (GPs) with a pictorial brochure and video targeting eligible patients, to increase CRC screening and other secondary outcomes, after 1 year, in several underserved geographic areas in France.

METHODS

We will use a two-arm multicentric randomized controlled cluster trial with 32 GPs primarily serving underserved populations across four regions in France with 1024 patients recruited. GPs practicing in underserved areas (identified using the European Deprivation Index) will be block-randomized to: 1) a combined intervention (HL and CRC training + brochure and video for eligible patients), or 2) usual care. Patients will be included if they are between 50 and 74 years old, eligible for CRC screening, and present to recruited GPs. The primary outcome is CRC screening uptake after 1 year. Secondary outcomes include increasing knowledge and patient activation. After trial recruitment, we will conduct semi-structured interviews with up to 24 GPs (up to 8 in each region) and up to 48 patients (6 to 12 per region) based on data saturation. We will explore strategies that promote the intervention's sustained use and rapid implementation using Normalization Process Theory. We will follow a community-based participatory research approach throughout the trial. For the analyses, we will adopt a regression framework for all quantitative data. We will also use exploratory mediation analyses. We will analyze all qualitative data using a framework analysis guided by Normalization Process Theory.

DISCUSSION

Limited HL and its impact on the general population is a growing public health and policy challenge worldwide. It has received limited attention in France. A combined HL intervention could reduce disparities in CRC screening, increase screening rates among the most vulnerable populations, and increase knowledge and activation (beneficial in the context of repeated screening).

TRIAL REGISTRATION

Registry: ClinicalTrials.gov.

TRIAL REGISTRATION NUMBER

2020-A01687-32 . Date of registration: 17th November 2020.

Implementing a multidisciplinary psychotropic medication review among nursing home residents with dementia: a process evaluation

OBJECTIVES

Before drawing conclusions on the contribution of an effective intervention to daily practice and initiating dissemination, its quality and implementation in daily practice should be optimal. The aim of this process evaluation was to study these aspects alongside a randomized controlled trial investigating the effects of a multidisciplinary biannual medication review in long-term care organizations (NTR3569).

DESIGN

Process evaluation with multiple measurements.

SETTING

Thirteen units for people with dementia in six long-term care organizations in the Netherlands.

PARTICIPANTS

Physicians, pharmacists, and nursing staff of participating units.

INTERVENTION

The PROPER intervention is a structured and biannually repeated multidisciplinary medication review supported by organizational preparation and education, evaluation, and guidance.

MEASUREMENTS

Web-based questionnaires, interviews, attendance lists of education sessions, medication reviews and evaluation meetings, minutes, evaluation, and registration forms.

RESULTS

Participation rates in education sessions (95%), medication reviews (95%), and evaluation meetings (82%) were high. The intervention's relevance and feasibility and applied implementation strategies were highly rated. However, the education sessions and conversations during medication reviews were too pharmacologically oriented for several nursing staff members. Identified barriers to implementation were required time, investment, planning issues, and high staff turnover; facilitators were the positive attitude of professionals toward the intervention, the support of higher management, and the appointment of a local implementation coordinator.

CONCLUSION

Implementation was successful. The commitment of both higher management and professionals was an important factor. This may partly have been due to the subject being topical; Dutch long-term-care organizations are pressed to lower inappropriate psychotropic drug use.

Effectiveness of a tailored intervention to reduce antibiotics for urinary tract infections in nursing home residents: a cluster, randomised controlled trial

BACKGROUND

When suspecting a urinary tract infection (UTI), the nursing home staff contacts a physician with clinical information on behalf of the resident; hence, poor understanding of UTI or a lack of clinical communicative skills can cause overtreatment with antibiotics. We investigated whether a tailored intervention that improves knowledge about UTI and communication skills in nursing home staff influences antibiotic prescriptions for UTI.

METHODS

This open-label, parallel-group, cluster randomised controlled trial was done at 22 participating nursing homes in Denmark. Patients were eligible if they were nursing home residents aged 65 years or older, had the nursing home listed as their permanent address, and resided in a living space designated for those with dementia or somatic health-care needs. We included nursing homes that were not participating in other UTI projects and those in which staff were present at all hours. Using computer-generated random numbers and stratification by municipality, a statistician randomised the nursing homes (1:1) to receive either interactive educational sessions and use of a dialogue tool or to continue standard practice. The statistical analysis was blinded. Staff attended 75 min sessions over 8 weeks to learn how to distinguish between UTIs and asymptomatic bacteriuria, evaluate non-specific symptoms, and use the dialogue tool. The primary outcome was the number of antibiotic prescriptions for acute UTI per resident per days at risk, defined as the number of days the resident had been present at the nursing home during the trial period. The trial is registered at ClinicalTrials.gov, NCT03715062.

FINDINGS

Between June 1, 2017, and June 1, 2018, 22 of 68 invited nursing homes were recruited. Of 22 randomised nursing homes (n=1625 residents), 11 received the intervention (770 [92·2%] of 835 allocated residents) and 11 were in the control group (705 [89·2%] of 790 allocated residents). The standardised number of nursing home staff was 572 in the intervention group and 535 in the control group. All nursing homes completed the trial. 65 residents were excluded from data collection in the intervention group and 85 were excluded in the control group. 1470 residents (intervention n=765; control n=705) were analysed for the primary endpoint. The number of antibiotic prescriptions for UTI per resident was 134 per 84 035 days at risk in the intervention group and 228 per 77817 days at risk in the control group. The rate ratio (RR) of receiving an antibiotic for UTI was 0·51 (95% CI 0·37-0·71) in the unadjusted model and 0·42 (0·31-0·57) in the adjusted model. Of 140 diary entries of suspected UTIs, no deaths were reported. 421 (28·5%) of 1475 residents were admitted to hospital. The risk of all-cause hospitalisation increased in the intervention group (adjusted model RR 1·28, 95% CI 0·95-1·74), whereas all-cause mortality was lower in the intervention group (0·91, 0·62-1·33).

INTERPRETATION

The intervention effectively reduced antibiotic prescriptions and inappropriate treatments for UTI without substantially influencing all-cause hospitalisations and mortality.

FUNDING

Danish Ministry of Health and the Velux Foundation.

Differences in treatment choices between prostate cancer patients using a decision aid and patients receiving care as usual: results from a randomized controlled trial

Objective

To determine whether or not decision aid (DA) use influences treatment decisions in patients with low and intermediate risk prostate cancer (PC).

Patients and methods

In a cluster randomized controlled trial, patients were randomized to either DA use (DA group) or no DA use (control group). Between 2014 and 2016, newly diagnosed patients with low or intermediate risk PC were recruited in 18 hospitals in the Netherlands. DA users had access to a web-based DA that provided general PC information, PC-treatment information, and values clarification exercises to elicit personal preferences towards the treatment options. Control group patients received care as usual. Differences in treatment choice were analysed using multilevel logistic regressions. Differences in eligible treatment options between groups were compared using Pearson Chi-square tests.

Results

Informed consent was given by 382 patients (DA group N = 273, control group N = 109). Questionnaire response rate was 88% (N = 336). Active surveillance (AS) was an option for 38%, radical prostatectomy (RP) for 98%, external beam radiotherapy (EBRT) for 88%, and brachytherapy (BT) for 79% of patients. DA users received AS significantly more often than control group. Patients (29 vs 16%, p = 0.01), whereas the latter more often chose BT (29 vs 18%, p < 0.01). No differences were found between groups regarding RP and EBRT. DA users who were not eligible for AS, received surgery more often compared to the control group (53 vs 35%, p = 0.01). Patient and disease characteristics were evenly distributed between groups.

Conclusion

DA-using PC patients chose the AS treatment option more often than non-DA-using patients did.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00345-021-03782-7.

Quality Assessment of Studies Included in Cochrane Oral Health Systematic Reviews: A Meta-Research

Objectives: To assess the Risk of Bias (RoB) and other characteristics of published randomised clinical trials within Cochrane oral health systematic reviews. Materials and methods: All the published clinical trials within Cochrane oral health systematic reviews until 1 June 2020 were identified and examined. RoB was assessed for all the included clinical trials according to the Cochrane review standards. The Overall Risk of Bias (ORoB) was defined in this study using Cochrane’s RoB tool-v2. Descriptive analyses were carried out to determine the frequency of each variable in the study sample. Results: Out of a total of 2565 included studies, the majority (n = 1600) had sample sizes of 50 or higher. Regarding blinding, 907 studies were labelled as double-blind. Among the various domains of bias, the performance bias showed the highest rate of high risk (31.4%). Almost half of the studies had a high ORoB, compared to 11.1% with a low ORoB. The studies that used placebos had a higher percentage of low ORoB (14.8% vs. 10.7%). Additionally, the double- and triple-blind studies had higher percentages of low ORoB (23.6% and 23.3%, respectively), while the studies with a crossover design had the highest percentage of low ORoB (28.8%). Conclusion: The RoB of oral health studies published as Cochrane reviews was deemed high.

The role and challenges of cluster randomised trials for global health

Evaluating whether an intervention works when trialled in groups of individuals can pose complex challenges for clinical research. Cluster randomised controlled trials involve the random allocation of groups or clusters of individuals to receive an intervention, and they are commonly used in global health research. In this paper, we describe the potential reasons for the increasing popularity of cluster trials in low-income and middle-income countries. We also draw on key areas of global health research for an assessment of common trial planning practices, and we address their methodological shortcomings and pitfalls. Lastly, we discuss alternative approaches for population-level intervention trials that could be useful for research undertaken in low-income and middle-income countries for situations in which the use of cluster randomisation might not be appropriate.

Contamination: How much can an individually randomized trial tolerate?

Cluster randomization results in an increase in sample size compared to individual randomization, referred to as an efficiency loss. This efficiency loss is typically presented under an assumption of no contamination in the individually randomized trial. An alternative comparator is the sample size needed under individual randomization to detect the attenuated treatment effect due to contamination. A general framework is provided for determining the extent of contamination that can be tolerated in an individually randomized trial before a cluster randomized design yields a larger sample size. Results are presented for a variety of cluster trial designs including parallel arm, stepped-wedge and cluster crossover trials. Results reinforce what is expected: individually randomized trials can tolerate a surprisingly large amount of contamination before they become less efficient than cluster designs. We determine the point at which the contamination means an individual randomized design to detect an attenuated effect requires a larger sample size than cluster randomization under a nonattenuated effect. This critical rate is a simple function of the design effect for clustering and the design effect for multiple periods as well as design effects for stratification or repeated measures under individual randomization. These findings are important for pragmatic comparisons between a novel treatment and usual care as any bias due to contamination will only attenuate the true treatment effect. This is a bias that operates in a predictable direction. Yet, cluster randomized designs with post-randomization recruitment without blinding, are at high risk of bias due to the differential recruitment across treatment arms. This sort of bias operates in an unpredictable direction. Thus, with knowledge that cluster randomized trials are generally at a greater risk of biases that can operate in a nonpredictable direction, results presented here suggest that even in situations where there is a risk of contamination, individual randomization might still be the design of choice.

Rationale, Methodological Quality, and Reporting of Cluster-Randomized Controlled Trials in Critical Care Medicine: A Systematic Review

OBJECTIVE

Compared with individual-patient randomized controlled trials, cluster randomized controlled trials have unique methodological and ethical considerations. We evaluated the rationale, methodological quality, and reporting of cluster randomized controlled trials in critical care studies.

DATA SOURCES

Systematic searches of Medline, Embase, and Cochrane Central Register were performed.

STUDY SELECTION

We included all cluster randomized controlled trials conducted in adult, pediatric, or neonatal critical care units from January 2005 to September 2019.

DATA EXTRACTION

Two reviewers independently screened citations, reviewed full texts, protocols, and supplements of potentially eligible studies, abstracted data, and assessed methodology of included studies.

DATA SYNTHESIS

From 1,902 citations, 59 cluster randomized controlled trials met criteria. Most focused on quality improvement (24, 41%), antimicrobial therapy (9, 15%), or infection control (9, 15%) interventions. Designs included parallel-group (25, 42%), crossover (21, 36%), and stepped-wedge (13, 22%). Concealment of allocation was reported in 21 studies (36%). Thirteen studies (22%) reported at least one method of blinding. The median total sample size was 1,660 patients (interquartile range, 813-4,295); the median number of clusters was 12 (interquartile range, 5-24); and the median patients per cluster was 141 (interquartile range, 54-452). Sample size calculations were reported in 90% of trials, but only 54% met Consolidated Standards of Reporting Trials guidance for sample size reporting. Twenty-seven of the studies (46%) identified a fixed number of available clusters prior to trial commencement, and only nine (15%) prespecified both the number of clusters and patients required to detect the expected effect size. Overall, 36 trials (68%) achieved the total prespecified sample size. When analyzing data, 44 studies (75%) appropriately adjusted for clustering when analyzing the primary outcome. Only 12 (20%) reported an intracluster coefficient (median 0.047 [interquartile range, 0.01-0.13]).

CONCLUSIONS

Cluster randomized controlled trials in critical care typically involve a small and fixed number of relatively large clusters. The reporting of key methodological aspects of these trials is often inadequate.

Designing and undertaking randomised implementation trials: guide for researchers

Implementation science is the study of methods to promote the systematic uptake of evidence based interventions into practice and policy to improve health. Despite the need for high quality evidence from implementation research, randomised trials of implementation strategies often have serious limitations. These limitations include high risks of bias, limited use of theory, a lack of standard terminology to describe implementation strategies, narrowly focused implementation outcomes, and poor reporting. This paper aims to improve the evidence base in implementation science by providing guidance on the development, conduct, and reporting of randomised trials of implementation strategies. Established randomised trial methods from seminal texts and recent developments in implementation science were consolidated by an international group of researchers, health policy makers, and practitioners. This article provides guidance on the key components of randomised trials of implementation strategies, including articulation of trial aims, trial recruitment and retention strategies, randomised design selection, use of implementation science theory and frameworks, measures, sample size calculations, ethical review, and trial reporting. It also focuses on topics requiring special consideration or adaptation for implementation trials. We propose this guide as a resource for researchers, healthcare and public health policy makers or practitioners, research funders, and journal editors with the goal of advancing rigorous conduct and reporting of randomised trials of implementation strategies.

Analysis and reporting of stratified cluster randomized trials-a systematic survey

Background

In order to correctly assess the effect of intervention from stratified cluster randomized trials (CRTs), it is necessary to adjust for both clustering and stratification, as failure to do so leads to misleading conclusions about the intervention effect. We have conducted a systematic survey to examine the current practices about analysis and reporting of stratified CRTs.

Method

We used the search terms to identify the stratified CRTs from MEDLINE since the inception to July 2019. In phase 1, we screened the title and abstract for English-only studies and selected, including the main results paper of the identified protocols, for the next phase. In phase 2, we screened the full text and selected studies for data abstraction. The data abstraction form was piloted and developed using the REDCap. We abstracted data on multiple design and methodological aspects of the study including whether the primary method adjusted for both clustering and stratification, reporting of sample size, randomization, and results.

Results

We screened 2686 studies in the phase 1 and selected 286 studies for phase 2—among them 185 studies were selected for data abstraction. Most of the selected studies were two-arm 140/185 (76%) and parallel-group 165/185 (89%) trials. Among these 185 studies, 27 (15%) of them did not provide any sample size or power calculation, while 105 (57%) studies did not mention any method used for randomization within each stratum. Further, 43 (23%) and 150 (81%) of 185 studies did not provide the definition of all the strata, while more than 60% of the studies did not include all the stratification variable(s) in the flow chart or baseline characteristics table. More than half 114/185 (62%) of the studies did not adjust the primary method for both clustering and stratification.

Conclusion

Stratification helps to achieve the balance among intervention groups. However, to correctly assess the intervention effect from stratified CRTs, it is important to adjust the primary analysis for both stratification and clustering. There are significant deficiencies in the reporting of methodological aspects of stratified CRTs, which require substantial improvements in several areas including definition of strata, inclusion of stratification variable(s) in the flow chart or baseline characteristics table, and reporting the stratum-specific number of clusters and individuals in the intervention groups.

Supplementary information

Supplementary information accompanies this paper at 10.1186/s13063-020-04850-w.

The effect of access to water, sanitation and handwashing facilities on child growth indicators: Evidence from the Ethiopia Demographic and Health Survey 2016

INTRODUCTION

Poor access to water, sanitation, and handwashing (WASH) facilities frequently contribute to child growth failure. The role of access to WASH facilities on child growth outcomes in Ethiopia is largely unknown. The aim of this study was to determine individual and combined effects of access to WASH facilities on child growth outcomes.

METHODS

Data for this analysis was sourced from the recent Ethiopia Demographic and Health Survey (EDHS) 2016. A multivariable logistic regression model was applied to identify the separate and combined association of access to WASH facilities with child growth outcomes. Odds ratio (OR) and 95% confidence interval (CI) were estimated. Statistical significance was declared at p < 0.05.

RESULTS

Included in the analyses were data for children 0-59 months of age, which amounted to valid data for 9588 children with a height-for-age z-score (HAZ), 9752 children with a weight-for-age z-score (WAZ) and 9607 children with a weight-for-height z-score (WHZ). Children with access to improved combined sanitation with handwashing facilities had 29% lower odds of linear growth failure (stunting) (adjusted odds ratio (AOR) = 0.71; 95% CI: 0.51-0.99) compared with those with unimproved. Children with access to combined improved WASH facilities were 33% less likely to have linear growth failure (AOR = 0.67; 95% CI: 0.45-0.98). Access to improved handwashing alone reduced the odds of being underweight by 17% (AOR = 0.83; 95% CI: 0.71-0.98) compared with unimproved. Improved water and sanitation separately as well as combined WASH were not associated with decreased odds of underweight and wasting.

CONCLUSIONS

Combined access to improved water, sanitation and handwashing was associated with reduced child linear growth failure. Further research with robust methods is needed to examine whether combined WASH practices have synergistic effect on child growth outcomes.

Challenges in Protocol Development and Interpretation of the Schistosomiasis Consortium for Operational Research and Evaluation Intervention Studies

In 2010, the Schistosomiasis Consortium for Operational Research and Evaluation (SCORE) began the design of randomized controlled trials to compare different strategies for praziquantel mass drug administration, whether for gaining or sustaining control of schistosomiasis or for approaching local elimination of Schistosoma transmission. The goal of this operational research was to expand the evidence base for policy-making for regional and national control of schistosomiasis in sub-Saharan Africa. Over the 10-year period of its research programs, as SCORE operational research projects were implemented, their scope and scale posed important challenges in terms of research performance and the final interpretation of their results. The SCORE projects yielded valuable data on program-level effectiveness and strengths and weaknesses in performance, but in most of the trials, a greater-than-expected variation in community-level responses to assigned schedules of mass drug administration meant that identification of a dominant control strategy was not possible. This article critically reviews the impact of SCORE’s cluster randomized study design on performance and interpretation of large-scale operational research such as ours.

The Effects of Workplace Nature-Based Interventions on the Mental Health and Well-Being of Employees: A Systematic Review

Mental health in the workplace is a societal challenge with serious economical and human costs. Most prevalent mental disorders in the workforce (e.g., depression), however, are preventable. There is widespread agreement about the favorable effects of nature exposure and consequently, nature-based interventions (NBI) in the workplace have been proposed as a cost-effective approach to promote good health among employees. The objective of the present study was to systematically review scientific evidence on the effectiveness of NBI to promote mental health and well-being among actual employees in actual workplace settings. The review was conducted and presented in accordance with the PRISMA guidelines. The literature search was performed on five databases (PubMed, Embase, CENTRAL, CINHAL, and PsycINFO), hand-searching of field-specific journals, and the reference lists of retrieved papers over the past 5 years up to November (13th, 2018). Studies were eligible for inclusion if they (i) were randomized or nonrandomized controlled trials; (ii) comprised samples of actual employees; (iii) implemented a workplace-based intervention with exposure to nature; (iv) included comparison conditions that displayed a clear contrast to NBIs; and (v) investigated the quantitative effects on mental health or well-being. No restrictions on type of employees or workplace, publication period, or language of the publication were set. Risk of bias was assessed using the Cochrane's RoB2 tool. Narrative synthesis was performed due to large heterogeneity in outcome variables. Of the 510 articles identified, 10 NBIs (nine papers) met the eligibility criteria. The outcomes were grouped in five categories: (i) mental health indices, (ii) cognitive ability, (iii) recovery and restoration, (iv) work and life satisfaction, and (v) psychophysiological indicators. Narrative synthesis indicates consistently positive effects on mental health indices and cognitive ability, while mixed results were found for the other outcome categories. Caution must be given when interpreting the current evidence in this emerging research field because of the diversity of NBIs and the overall high risk of bias in the individual studies. Although in this field often researchers have to balance scientific rigor and ecological validity, there is a need for large, well-designed and rigorously conducted trials grounded in contemporary theories.

Essential Ingredients and Innovations in the Design and Analysis of Group-Randomized Trials

This article reviews the essential ingredients and innovations in the design and analysis of group-randomized trials. The methods literature for these trials has grown steadily since they were introduced to the biomedical research community in the late 1970s, and we summarize those developments. We review, in addition to the group-randomized trial, methods for two closely related designs, the individually randomized group treatment trial and the stepped-wedge group-randomized trial. After describing the essential ingredients for these designs, we review the most important developments in the evolution of their methods using a new bibliometric tool developed at the National Institutes of Health. We then discuss the questions to be considered when selecting from among these designs or selecting the traditional randomized controlled trial. We close with a review of current methods for the analysis of data from these designs, a case study to illustrate each design, and a brief summary.

Applying Intervention Mapping to Improve the Applicability of Precious Memories, an Intervention for Depressive Symptoms in Nursing Home Residents

Precious memories (PM) is a life review intervention for depression in older adults with no to mild cognitive decline that has been implemented in multiple nursing homes (NHs) in the Netherlands. Previous research suggested its relevance but questioned its applicability. Therefore, this research aimed to (1) investigate the applicability of PM, and (2) increase its applicability, if necessary. Intervention mapping (IM) was used to achieve these goals: process evaluation through semi-structured interviews with psychologists (n = 11) and clients (n = 2) to identify potential improvements for PM and to set an improvement goal (IM-step 1); three focus groups with stakeholders (n = 20) to specify behaviors necessary to reach the improvement goal (IM-step 2); and selection of behavior change techniques and applications to facilitate attainment of these behaviors (IM-step 3). Results showed that psychologists perceived a high drop-out rate, which was partly due to PM being provided to clients that did not belong to the target group. Although PM was generally considered relevant, psychologists articulated its longer-term effects should be improved. To improve PM’s applicability, concrete maintenance strategies were developed aiming to maintain clients’ well-being by stimulating positive contact with others. Future research must pilot, implement and evaluate these strategies.

The impact of three types of writing intervention on students' writing quality

Students' writing constitutes a topic of major concern due to its importance in school and in daily life. To mitigate students' writing problems, school-based interventions have been implemented in the past, but there is still a need to examine the effectiveness of different types of writing interventions that use robust design methodologies. Hence, the present study followed a longitudinal cluster-randomized controlled design using a multilevel modeling analysis with 370 fourth-grade students (nested in 20 classes). The classes were randomly assigned to four conditions: one comparison group and three writing types of writing interventions (i.e., week-journals, Self-Regulation Strategy Development (SRSD) instruction and SRSD plus Self-Regulated Learning (SRL) program using a story-tool), with five classes participating in each condition. Data supports our hypothesis by showing differences between the treatment groups in students' writing quality over time. Globally, the improvement of students' writing quality throughout time is related to the level of specialization of the writing interventions implemented. This is an important finding with strong implications for educational practice. Week-journals and writing activities can be easily implemented in classrooms and provides an opportunity to promote students' writing quality. Still, students who participated in the instructional programs (i.e., SRSD and SRSD plus story-tool) exhibited higher writing quality than the students who wrote week-journals. Current data did not find statistical significant differences between results from the two instructional writing tools.

Structured Goal Planning and Supportive Telephone Follow-up in Rheumatology Care: Results From a Pragmatic, Stepped-Wedge, Cluster-Randomized Trial

OBJECTIVE

To evaluate patient-reported health effects of an add-on structured goal-planning and supportive telephone follow-up rehabilitation program compared with traditional rehabilitation programs in patients with rheumatic diseases.

METHODS

In this pragmatic stepped-wedge, cluster-randomized, controlled trial, 389 patients with rheumatic diseases recruited from 6 rehabilitation centers received either traditional rehabilitation or traditional rehabilitation extended with an add-on program tailored to individual needs. The add-on program comprised a self-management booklet, motivational interviewing in structured individualized goal planning, and 4 supportive follow-up phone calls after discharge. Data were collected by questionnaires on admission and discharge from rehabilitation stay, and at 6 months and 12 months after discharge. The primary outcome was health-related quality of life (HRQoL) measured by the Patient Generated Index (range 0-100, where 0 = low). Secondary outcomes included patient-reported health status, self-efficacy, pain, fatigue, global disease activity, and motivation for change. The main statistical analysis was a linear repeated measures mixed model performed on the intent-to-treat population using all available data.

RESULTS

A significant treatment effect of the add-on intervention on HRQoL was found on discharge (mean difference 3.32 [95% confidence interval 0.27, 6.37]; P = 0.03). No significant between-group differences were found after 6 or 12 months. Both groups showed positive changes in HRQoL following rehabilitation, which gradually declined, although the values remained at higher levels after 6 and 12 months compared with baseline values.

CONCLUSION

The add-on program enhanced the short-term effect of rehabilitation with respect to patient-specific HRQoL, but it did not prolong the effect as intended.

Intervention effect estimates in cluster randomized versus individually randomized trials: a meta-epidemiological study

BACKGROUND

Cluster randomized trials (CRTs) and individually randomized trials (IRTs) are often pooled together in meta-analyses (MAs) of randomized trials. However, the potential systematic differences in intervention effect estimates between these two trial types has never been investigated. Therefore, we conducted a meta-epidemiological study comparing intervention effect estimates between CRTs and IRTs.

METHODS

All Cochrane MAs including at least one CRT and one IRT, published between 1 January 2010 and 31 December 2014, were included. For each MA, we estimated a ratio of odds ratios (ROR) for binary outcomes or a difference of standardized differences (DSMD) for continuous outcomes, where less than 1 (or 0, respectively) indicated a greater intervention effect estimate with CRTs.

RESULTS

Among 1301 screened reviews, we selected 121 MAs, of which 76 had a binary outcome and 45 had a continuous outcome. For binary outcomes, intervention effect estimates did not differ between CRTs and IRTs [ROR 1.00, 95% confidence interval (0.93 to 1.08)]. Subgroup and adjusted analyses led to consistent results. For continuous outcomes, the DSMD was 0.13 (0.06 to 0.19). It was lower for MAs with a pharmacological intervention [-0.03, (-0.12 to 0.07)], an objective outcome [0.05, (-0.08 to 0.17)] or after adjusting for trial size [0.06, (-0.01 to 0.15)].

CONCLUSION

For binary outcomes, CRTs and IRTs can safely be pooled in MAs because of an absence of systematic differences between effect estimates. For continuous outcomes, the results were less clear although accounting for trial sample sizes led to a non-significant difference. More research is needed for continuous outcomes and, meanwhile, MAs should be completed with subgroup analyses (CRTs vs IRTs).

The frequency of intrapartum caesarean section use with the WHO partograph versus Zhang's guideline in the Labour Progression Study (LaPS): a multicentre, cluster-randomised controlled trial

BACKGROUND

There is an ongoing debate concerning which guidelines and monitoring tools are most beneficial for assessing labour progression, to help prevent use of intrapartum caesarean section (ICS). The WHO partograph has been used for decades with the assumption of a linear labour progression; however, in 2010, Zhang introduced a new guideline suggesting a more dynamic labour progression. We aimed to investigate whether the frequency of ICS use differed when adhering to the WHO partograph versus Zhang's guideline for labour progression.

METHODS

We did a multicentre, cluster-randomised controlled trial at obstetric units in Norway, and each site was required to deliver more than 500 fetuses per year to be eligible for inclusion. The participants were nulliparous women who had a singleton, full-term fetus with cephalic presentation, and who entered spontaneous active labour. The obstetric units were treated as clusters, and women treated within these clusters were all given the same treatment. We stratified these clusters by size and number of previous caesarean sections. The clusters containing the obstetric units were then randomly assigned (1:1) to the control group, which adhered to the WHO partograph, or to the intervention group, which adhered to Zhang's guideline. The randomisation was computer-generated and was done in the Unit of Biostatistics and Epidemiology, Oslo University Hospital, Oslo, Norway, and investigators in this unit had no further involvement in the trial. Our study design did not enable masking of participants or health-care providers, but the investigators who were analysing the data were masked to group allocation. The primary outcome was use of ICS during active labour (cervical dilatation of 4-10 cm) in all participating women. The Labour Progression Study (LaPS) is registered with ClinicalTrials.gov, number NCT02221427.

FINDINGS

Between Aug 1, 2014, and Sept 1, 2014, 14 clusters were enrolled in the LaPS trial, and on Sept 11, 2014, seven obstetric units were randomly assigned to the control group (adhering to the WHO partograph) and seven obstetric units were randomly assigned to the intervention group (adhering to Zhang's guideline). Between Dec 1, 2014, and Jan 31, 2017, 11 615 women were judged to be eligible for recruitment in the trial, which comprised 5421 (46·7%) women in the control group units and 6194 (53·3%) women in the intervention group units. In the control group, 2100 (38·7%) of 5421 women did not give signed consent to participate and 16 (0·3%) women abstained from participation. In the intervention group, 2181 (35·2%) of 6194 women did not give signed consent to participate and 41 (0·7%) women abstained from participation. 7277 (62·7%) of 11 615 eligible women were therefore included in the analysis of the primary endpoint. Of these women, 3305 (45·4%) participants were in an obstetric unit that was randomly assigned to the control group (adhering to the WHO partograph) and 3972 (54·6%) participants were in an obstetric unit that was randomly assigned to the intervention group (adhering to Zhang's guideline). No women dropped out during the trial. Before the start of the trial, ICS was used in 9·5% of deliveries in the control group obstetric units and in 9·3% of intervention group obstetric units. During our trial, there were 196 (5·9%) ICS deliveries in women in the control group (WHO partograph) and 271 (6·8%) ICS deliveries in women in the intervention group (Zhang's guideline), and the frequency of ICS use did not differ between the groups (adjusted relative risk 1·17, 95% CI 0·98-1·40; p=0·08; adjusted risk difference 1·00%, 95% CI -0·1 to 2·1). We identified no maternal or neonatal deaths during our study.

INTERPRETATION

We did not find any significant difference in the frequency of ICS use between the obstetric units assigned to adhere to the WHO partograph and those assigned to adhere to Zhang's guideline. The overall decrease in ICS use that we observed relative to the previous frequency of ICS use noted in these obstetric units might be explained by the close focus on assessing labour progression more than use of the guidelines. Our results represent an important contribution to the discussion on implementation of the new guideline.

FUNDING

Østfold Hospital Trust.

Promoting the use of self-management in novice chiropractors treating individuals with spine pain: the design of a theory-based knowledge translation intervention

Background

Clinical practice guidelines generally recommend clinicians use self-management support (SMS) when managing patients with spine pain. However, even within the educational setting, the implementation of SMS remains suboptimal. The objectives of this study were to 1) estimate the organizational readiness for change toward using SMS at the Canadian Memorial Chiropractic College (CMCC), Toronto, Ontario from the perspective of directors and deans, 2) estimate the attitudes and self-reported behaviours towards using evidence-based practice (EBP), and beliefs about pain management among supervisory clinicians and chiropractic interns, 3) identify potential barriers and enablers to using SMS, and 4) design a theory-based tailored Knowledge Translation (KT) intervention to increase the use of SMS.

Methods

Mixed method design. We administered three self-administered questionnaires to assess clinicians’ and interns’ attitudes and behaviours toward EBP, beliefs about pain management, and practice style. In addition, we conducted 3 focus groups with clinicians and interns based on the Theoretical Domain Framework (TDF) to explore their beliefs about using SMS for patients with spine pain. Data were analysed using deductive thematic analysis by 2 independent assessors. A panel of 7 experts mapped behaviour change techniques to key barriers identified informing the design of a KT intervention.

Results

Participants showed high level of EBP knowledge, positive attitude of EBP, and moderate frequency of EBP use. A number of barrier factors were identified from clinicians (N = 6) and interns (N = 16) corresponding to 7 TDF domains: Knowledge; Skills; Environmental context and resources; Emotion; Beliefs about Capabilities; Memory, attention & decision making; and Social Influence. To address these barriers, the expert panel proposed a multifaceted KT intervention composed of a webinar and online educational module on a SMS guided by the Brief Action Planning, clinical vignettes, training workshop, and opinion leader support.

Conclusion

SMS strategies can help maximizing the health care services for patients with spine pain. This may in turn optimize patients’ health. The proposed theory-based KT intervention may facilitate the implementation of SMS among clinicians and interns.

Electronic supplementary material

The online version of this article (10.1186/s12891-018-2241-1) contains supplementary material, which is available to authorized users.

A Liftless Intervention to Prevent Preterm Birth and Low Birthweight Among Pregnant Ghanaian Women: Protocol of a Stepped-Wedge Cluster Randomized Controlled Trial

Background

Preterm birth (PTB) is a leading cause of infant morbidity and mortality worldwide. Every year, 20 million babies are born with low birthweight (LBW), about 96% of which occur in low-income countries. Despite the associated dangers, in about 40%-50% of PTB and LBW cases, the causes remain unexplained. Existing evidence is inconclusive as to whether occupational physical activities such as heavy lifting are implicated. African women bear the transport burden of accessing basic needs for their families. Ghana’s PTB rate is 14.5%, whereas the global average is 9.6%. The proposed liftless intervention aims to decrease lifting exposure during pregnancy among Ghanaian women. We hypothesize that a reduction in heavy lifting among pregnant women in Ghana will increase gestational age and birthweight.

Objective

To investigate the effects of the liftless intervention on the incidence of PTB and LBW among pregnant Ghanaian women.

Methods

A cohort stepped-wedge cluster randomized controlled trial in 10 antenatal clinics will be carried out in Ghana. A total of 1000 pregnant participants will be recruited for a 60-week period. To be eligible, the participant should have a singleton pregnancy between 12 and 16 weeks gestation, be attending any of the 10 antenatal clinics, and be exposed to heavy lifting. All participants will receive standard antenatal care within the control phase; by random allocation, two clusters will transit into the intervention phase. The midwife-led 3-component liftless intervention consists of health education, a take-home reminder card mimicking the colors of a traffic light, and a shopping voucher. The primary outcome are gestational ages of <28, 28-32, and 33-37 weeks. The secondary outcomes are LBW (preterm LBW, term but LBW, and postterm), compliance, prevalence of low back and pelvic pain, and premature uterine contractions. Study midwives and participants will not be blinded to the treatment allocation.

Results

Permission to conduct the study at all 10 antenatal clinics has been granted by the Ghana Health Service. Application for funding to begin the trial is ongoing. Findings from the main trial are expected to be published by the end of 2019.

Conclusions

To the best of our knowledge, there has been no randomized trial of this nature in Ghana. Minimizing heavy lifting among pregnant African women can reduce the soaring rates of PTB and LBW. The findings will increase the knowledge of the prevention of PTB and LBW worldwide.

Trial Registration

Pan African Clinical Trial Register (PACTR201602001301205); http://apps.who.int/trialsearch/ Trial2.aspx?TrialID=PACTR201602001301205 (Archived by WebCite at http://www.webcitation.org/71TCYkHzu)

Registered Report Identifier

RR1-10.2196/10095

Engagement of Husbands in a Maternal Nutrition Program Substantially Contributed to Greater Intake of Micronutrient Supplements and Dietary Diversity during Pregnancy: Results of a Cluster-Randomized Program Evaluation in Bangladesh

Background

Although husbands may provide support during pregnancy, limited evidence exists on how to promote husbands' engagement and what impact it has. Alive & Thrive integrated nutrition-focused interventions, targeting both wives and husbands, through an existing Maternal, Neonatal, and Child Health (MNCH) platform in Bangladesh.

Objectives

We evaluated 1) the impact of a nutrition-focused MNCH program, compared with the standard MNCH program, on husbands' behavioral determinants (i.e., awareness, knowledge, self-efficacy) and support to wives to adopt optimal nutrition practices and 2) how much of the previously documented impact on women's supplement intake and dietary diversity was explained by husbands' behavioral determinants and support.

Methods

We used a cluster-randomized design with cross-sectional surveys at baseline (2015) and endline (2016) (n = ∼1000 women and ∼700 husbands/survey). We used mixed linear regression accounting for clustering to estimate difference-in-differences (DIDs) for impact on husbands' behavioral determinants and path analysis to examine how much these determinants explained the impact on women's nutrition behaviors.

Results

Of husbands in the nutrition-focused MNCH group, 62% were counseled by health workers, 66% attended a husbands' forum, and 34% saw video shows. The nutrition-focused MNCH, compared with the standard MNCH group, resulted in greater husbands' awareness (DID: 2.74 of 10 points), knowledge (DID: 1.31), self-efficacy and social norms with regard to optimal nutrition practices (difference: 1.08), and support to their wives (DID: 1.86). Husbands' behavioral determinants and support explained nearly half of the program impact for maternal supplement intake and one-quarter for dietary diversity.

Conclusions

A nutrition-focused MNCH program that promoted and facilitated husbands' engagement during their wives' pregnancies significantly improved husbands' awareness, knowledge, self-efficacy, and support. These improvements substantially explained the program's impact on women's intake of micronutrient supplements and dietary diversity. Targeting wives and husbands and designing activities to engage men in maternal nutrition programs are important to maximize impact. This trial was registered at www.clinicaltrials.gov as NCT02745249.

Comparison of registered and published intervention fidelity assessment in cluster randomised trials of public health interventions in low- and middle-income countries: systematic review

BACKGROUND

Cluster randomised trials (CRTs) are a key instrument to evaluate public health interventions. Fidelity assessment examines study processes to gauge whether an intervention was delivered as initially planned. Evaluation of implementation fidelity (IF) is required to establish whether the measured effects of a trial are due to the intervention itself and may be particularly important for CRTs of complex interventions conducted in low- and middle-income countries (LMICs). However, current CRT reporting guidelines offer no guidance on IF assessment. The objective of this review was to study current practices concerning the assessment of IF in CRTs of public health interventions in LMICs.

METHODS

CRTs of public health interventions in LMICs that planned or reported IF assessment in either the trial protocol or the main trial report were included. The MEDLINE/PubMed, CINAHL and EMBASE databases were queried from January 2012 to May 2016. To ensure availability of a study protocol, CRTs reporting a registration number in the abstract were included. Relevant data were extracted from each study protocol and trial report by two researchers using a predefined screening sheet. Risk of bias for individual studies was assessed.

RESULTS

We identified 90 CRTs of public health interventions in LMICs with a study protocol in a publicly available trial registry published from January 2012 to May 2016. Among these 90 studies, 25 (28%) did not plan or report assessing IF; the remaining 65 studies (72%) addressed at least one IF dimension. IF assessment was planned in 40% (36/90) of trial protocols and reported in 71.1% (64/90) of trial reports. The proportion of overall agreement between the trial protocol and trial report concerning occurrence of IF assessment was 66.7% (60/90). Most studies had low to moderate risk of bias.

CONCLUSIONS

IF assessment is not currently a systematic practice in CRTs of public health interventions carried out in LMICs. In the absence of IF assessment, it may be difficult to determine if CRT results are due to the intervention design, to its implementation, or to unknown or external factors that may influence results. CRT reporting guidelines should promote IF assessment.

TRIAL REGISTRATION

Protocol published and available at: https://doi.org/10.1186/s13643-016-0351-0.

Design and analysis of group-randomized trials in cancer: A review of current practices

The purpose of this paper is to summarize current practices for the design and analysis of group-randomized trials involving cancer-related risk factors or outcomes and to offer recommendations to improve future trials. We searched for group-randomized trials involving cancer-related risk factors or outcomes that were published or online in peer-reviewed journals in 2011-15. During 2016-17, in Bethesda MD, we reviewed 123 articles from 76 journals to characterize their design and their methods for sample size estimation and data analysis. Only 66 (53.7%) of the articles reported appropriate methods for sample size estimation. Only 63 (51.2%) reported exclusively appropriate methods for analysis. These findings suggest that many investigators do not adequately attend to the methodological challenges inherent in group-randomized trials. These practices can lead to underpowered studies, to an inflated type 1 error rate, and to inferences that mislead readers. Investigators should work with biostatisticians or other methodologists familiar with these issues. Funders and editors should ensure careful methodological review of applications and manuscripts. Reviewers should ensure that studies are properly planned and analyzed. These steps are needed to improve the rigor and reproducibility of group-randomized trials. The Office of Disease Prevention (ODP) at the National Institutes of Health (NIH) has taken several steps to address these issues. ODP offers an online course on the design and analysis of group-randomized trials. ODP is working to increase the number of methodologists who serve on grant review panels. ODP has developed standard language for the Application Guide and the Review Criteria to draw investigators' attention to these issues. Finally, ODP has created a new Research Methods Resources website to help investigators, reviewers, and NIH staff better understand these issues.

The clinical reasoning process in randomized clinical trials with patients with non-specific neck pain is incomplete: A systematic review

OBJECTIVE

Primarily to evaluate the completeness of the description of the clinical reasoning process in RCTs with patients with non-specific neck pain with an argued or diagnosed cause i.e. an impairment or activity limitation. Secondly, to determine the association between the completeness of the clinical reasoning process and the degree of risk of bias.

DATA SOURCES

Pubmed, Cinahl and PEDro were systematically searched from inception to July 2016.

STUDY SELECTION

RCTs (n = 122) with patients with non-specific neck pain receiving physiotherapy treatment published in English were included.

DATA EXTRACTION

Data extraction included study characteristics and important features of the clinical reasoning process based on the Hypothesis-Oriented Algorithm for Clinicians II (HOAC II)].

DATA SYNTHESIS

Thirty-seven studies (30%) had a complete clinical reasoning process of which 8 (6%) had a 'diagnosed cause' and 29 (24%) had an 'argued cause'. The Spearmans rho association between the extent of the clinical reasoning process and the risk of bias was -0.2.

CONCLUSIONS

In the majority of studies (70%) the described clinical reasoning process was incomplete. A very small proportion (6%) had a 'diagnosed cause'. Therefore, a better methodological quality does not necessarily imply a better described clinical reasoning process.

A school-based intervention improved dietary intake outcomes and reduced waist circumference in adolescents: a cluster randomized controlled trial

BACKGROUND

In Ecuador, adolescents' food intake does not comply with guidelines for a healthy diet. Together with abdominal obesity adolescent's inadequate diets are risk factors for non-communicable diseases. We report the effectiveness of a school-based intervention on the dietary intake and waist circumference among Ecuadorian adolescents.

METHODS

A pair-matched cluster randomized controlled trial including 1430 adolescents (12-14 years old) was conducted. The program aimed at improving the nutritional value of dietary intake, physical activity (primary outcomes), body mass index, waist circumference and blood pressure (secondary outcomes). This paper reports: (i) the effect on fruit and vegetable intake, added sugar intake, unhealthy snacking (consumption of unhealthy food items that are not in line with the dietary guidelines eaten during snack time; i.e. table sugar, sweets, salty snacks, fast food, soft drinks and packaged food), breakfast intake and waist circumference; and, (ii) dose and reach of the intervention. Dietary outcomes were estimated by means of two 24-h recall at baseline, after the first 17-months (stage one) and after the last 11-months (stage two) of implementation. Dose and reach were evaluated using field notes and attendance forms. Educational toolkits and healthy eating workshops with parents and food kiosks staff in the schools were implemented in two different stages. The overall effect was assessed using linear mixed models and regression spline mixed effect models were applied to evaluate the effect after each stage.

RESULTS

Data from 1046 adolescents in 20 schools were analyzed. Participants from the intervention group consumed lower quantities of unhealthy snacks (-23.32 g; 95% CI: -45.25,-1.37) and less added sugar (-5.66 g; 95% CI: -9.63,-1.65) at the end of the trial. Daily fruit and vegetable intake decreased in both the intervention and control groups compared to baseline, albeit this decrease was 23.88 g (95% CI: 7.36, 40.40) lower in the intervention group. Waist circumference (-0.84 cm; 95% CI: -1.68, 0.28) was lower in the intervention group at the end of the program; the effect was mainly observed at stage one. Dose and reach were also higher at stage one.

CONCLUSIONS

The trial had positive effects on risk factors for non-communicable diseases, i.e. decreased consumption of unhealthy snacks. The program strategies must be implemented at the national level through collaboration between the academia and policy makers to assure impact at larger scale.

TRIAL REGISTRATION

ClinicalTrial.gov-NCT01004367 .

Real-world Data for Clinical Evidence Generation in Oncology

Conventional cancer clinical trials can be slow and costly, often produce results with limited external validity, and are difficult for patients to participate in. Recent technological advances and a dynamic policy landscape in the United States have created a fertile ground for the use of real-world data (RWD) to improve current methods of clinical evidence generation. Sources of RWD include electronic health records, insurance claims, patient registries, and digital health solutions outside of conventional clinical trials. A definition focused on the original intent of data collected at the point of care can distinguish RWD from conventional clinical trial data. When the intent of data collection at the point of care is research, RWD can be generated using experimental designs similar to those employed in conventional clinical trials, but with several advantages that include gains in efficient execution of studies with an appropriate balance between internal and external validity. RWD can support active pharmacovigilance, insights into the natural history of disease, and the development of external control arms. Prospective collection of RWD can enable evidence generation based on pragmatic clinical trials (PCTs) that support randomized study designs and expand clinical research to the point of care. PCTs may help address the growing demands for access to experimental therapies while increasing patient participation in cancer clinical trials. Conducting valid real-world studies requires data quality assurance through auditable data abstraction methods and new incentives to drive electronic capture of clinically relevant data at the point of care.

Cluster-randomized controlled trials evaluating complex interventions in general practices are mostly ineffective: a systematic review

OBJECTIVES

The aim of this study was to evaluate how frequently complex interventions are shown to be superior to routine care in general practice-based cluster-randomized controlled studies (c-RCTs) and to explore whether potential differences explain results that come out in favor of a complex intervention.

STUDY DESIGN AND SETTING

We performed an unrestricted search in the Central Register of Controlled Trials, MEDLINE, and EMBASE. Included were all c-RCTs that included a patient-relevant primary outcome in a general practice setting with at least 1-year follow-up. We extracted effect sizes, P-values, intracluster correlation coefficients (ICCs), and 22 quality aspects.

RESULTS

We identified 29 trials with 99 patient-relevant primary outcomes. After adjustment for multiple testing on a trial level, four outcomes (4%) in four studies (14%) remained statistically significant. Of the 11 studies that reported ICCs, in 8, the ICC was equal to or smaller than the assumed ICC. In 16 of the 17 studies with available sample size calculation, effect sizes were smaller than anticipated.

CONCLUSION

More than 85% of the c-RCTs failed to demonstrate a beneficial effect on a predefined primary endpoint. All but one study were overly optimistic with regard to the expected treatment effect. This highlights the importance of weighing up the potential merit of new treatments and planning prospectively, when designing clinical studies in a general practice setting.