Design and analysis of group-randomized trials: a review of recent practices

Influential methods reports for group-randomized trials and related designs

BACKGROUND

This article identifies the most influential methods reports for group-randomized trials and related designs published through 2020. Many interventions are delivered to participants in real or virtual groups or in groups defined by a shared interventionist so that there is an expectation for positive correlation among observations taken on participants in the same group. These interventions are typically evaluated using a group- or cluster-randomized trial, an individually randomized group treatment trial, or a stepped wedge group- or cluster-randomized trial. These trials face methodological issues beyond those encountered in the more familiar individually randomized controlled trial.

METHODS

PubMed was searched to identify candidate methods reports; that search was supplemented by reports known to the author. Candidate reports were reviewed by the author to include only those focused on the designs of interest. Citation counts and the relative citation ratio, a new bibliometric tool developed at the National Institutes of Health, were used to identify influential reports. The relative citation ratio measures influence at the article level by comparing the citation rate of the reference article to the citation rates of the articles cited by other articles that also cite the reference article.

RESULTS

In total, 1043 reports were identified that were published through 2020. However, 55 were deemed to be the most influential based on their relative citation ratio or their citation count using criteria specific to each of the three designs, with 32 group-randomized trial reports, 7 individually randomized group treatment trial reports, and 16 stepped wedge group-randomized trial reports. Many of the influential reports were early publications that drew attention to the issues that distinguish these designs from the more familiar individually randomized controlled trial. Others were textbooks that covered a wide range of issues for these designs. Others were "first reports" on analytic methods appropriate for a specific type of data (e.g. binary data, ordinal data), for features commonly encountered in these studies (e.g. unequal cluster size, attrition), or for important variations in study design (e.g. repeated measures, cohort versus cross-section). Many presented methods for sample size calculations. Others described how these designs could be applied to a new area (e.g. dissemination and implementation research). Among the reports with the highest relative citation ratios were the CONSORT statements for each design.

CONCLUSIONS

Collectively, the influential reports address topics of great interest to investigators who might consider using one of these designs and need guidance on selecting the most appropriate design for their research question and on the best methods for design, analysis, and sample size.

A systematic review of effective strategies for chronic disease management in humanitarian settings; opportunities and challenges

Large number of people with non-communicable diseases (NCDs) face barriers to adequate healthcare in humanitarian settings. We conducted a systematic literature review in MEDLINE/PubMed, Web of Science, EMBASE/DARE, Cochrane, and grey literature from 1990 to 2021 to evaluate effective strategies in addressing NCDs (diabetes, cardiovascular diseases, COPD, cancer) in humanitarian settings. From 2793 articles, 2652 were eliminated through title/abstract screening; 141 articles were reviewed in full; 93 were eliminated for not meeting full criteria. Remaining 48 articles were reviewed qualitatively to assess populations, settings, interventions, outcome, and efficacy and effectiveness; 38 studies addressed treatments, 9 prevention, and 7 epidemiology. Prevention studies broadly addressed capacity-building. Treatment and epidemiology studies largely addressed hypertension and diabetes. Interventions included web-based/mobile health strategies, pharmacy-level interventions, portable imaging, and capacity building including physical clinics, staff training, forging collaborations, guideline development, point-of-care labs, health promotion activities, EMR, and monitoring interventions. Collaboration between academia and implementing agencies was limited. Models of care were largely not well-described and varied between studies due to contextual constraints. Barriers to interventions included financial, logistical, organizational, sociocultural, and security. Cancer care is significantly understudied. Simplified care models adapted to contexts and program evaluations of implemented strategies could address gaps in applied research. Inherent challenges in humanitarian settings pose unavoidable perils to evidence generation which requires a shift in research mindset to match aspirations with practicality, research collaborations at the inception of projects, reworking of desired conventional level of research evidence considering resource-intense constraints (HR, time, cost), and adapted research tools, methods, and procedures.

Constrained randomization and statistical inference for multi‐arm parallel cluster randomized controlled trials

A practical limitation of cluster randomized controlled trials (cRCTs) is that the number of available clusters may be small, resulting in an increased risk of baseline imbalance under simple randomization. Constrained randomization overcomes this issue by restricting the allocation to a subset of randomization schemes where sufficient overall covariate balance across comparison arms is achieved. However, for multi-arm cRCTs, several design and analysis issues pertaining to constrained randomization have not been fully investigated. Motivated by an ongoing multi-arm cRCT, we elaborate the method of constrained randomization and provide a comprehensive evaluation of the statistical properties of model-based and randomization-based tests under both simple and constrained randomization designs in multi-arm cRCTs, with varying combinations of design and analysis-based covariate adjustment strategies. In particular, as randomization-based tests have not been extensively studied in multi-arm cRCTs, we additionally develop most-powerful randomization tests under the linear mixed model framework for our comparisons. Our results indicate that under constrained randomization, both model-based and randomization-based analyses could gain power while preserving nominal type I error rate, given proper analysis-based adjustment for the baseline covariates. Randomization-based analyses, however, are more robust against violations of distributional assumptions. The choice of balance metrics and candidate set sizes and their implications on the testing of the pairwise and global hypotheses are also discussed. Finally, we caution against the design and analysis of multi-arm cRCTs with an extremely small number of clusters, due to insufficient degrees of freedom and the tendency to obtain an overly restricted randomization space.

When should matching be used in the design of cluster randomized trials?

For cluster randomized trials (CRTs) with a small number of clusters, the matched-pair (MP) design, where clusters are paired before randomizing one to each trial arm, is often recommended to minimize imbalance on known prognostic factors, add face-validity to the study, and increase efficiency, provided the analysis recognizes the matching. Little evidence exists to guide decisions on when to use matching. We used simulation to compare the efficiency of the MP design with the stratified and simple designs, based on the mean confidence interval width of the estimated intervention effect. Matched and unmatched analyses were used for the MP design; a stratified analysis was used for the stratified design; and analyses without and with post-stratification adjustment for factors that would otherwise have been used for restricted allocation were used for the simple design. Results showed the MP design was generally the most efficient for CRTs with 10 or more pairs when the correlation between cluster-level outcomes within pairs (matching correlation) was moderate to strong (0.3-0.5). There was little gain in efficiency for the MP or stratified designs compared to simple randomization when the matching correlation was weak (0.05-0.1). For trials with four pairs of clusters, the simple and stratified designs were more efficient than the MP design because greater degrees of freedom were available for the analysis, although an unmatched analysis of the MP design recovered precision for weak matching correlations. Practical guidance on choosing between the MP, stratified, and simple designs is provided.

Characteristics and practices of school-based cluster randomised controlled trials for improving health outcomes in pupils in the UK: a systematic review protocol

INTRODUCTION

Cluster randomised trials (CRTs) are studies in which groups (clusters) of participants rather than the individuals themselves are randomised to trial arms. CRTs are becoming increasingly relevant for evaluating interventions delivered in school settings for improving the health of children. Schools are a convenient setting for health interventions targeted at children and the CRT design respects the clustered structure in schools (ie, pupils within classrooms/teachers within schools). Some of the methodological challenges of CRTs, such as ethical considerations for enrolment of children into trials and how best to handle the analysis of data from participants (pupils) that change clusters (schools), may be more salient for the school setting. A better understanding of the characteristics and methodological considerations of school-based CRTs of health interventions would inform the design of future similar studies. To our knowledge, this is the only systematic review to focus specifically on the characteristics and methodological practices of CRTs delivered in schools to evaluate interventions for improving health outcomes in pupils in the UK.

METHODS AND ANALYSIS

We will search for CRTs published from inception to 30 June 2020 inclusively indexed in MEDLINE (Ovid). We will identify relevant articles through title and abstract screening, and subsequent full-text screening for eligibility against predefined inclusion criteria. Disagreements will be resolved through discussion. Two independent reviewers will extract data for each study using a prepiloted data extraction form. Findings will be summarised using descriptive statistics and graphs.

ETHICS AND DISSEMINATION

This methodological systematic review does not require ethical approval as only secondary data extracted from papers will be analysed and the data are not linked to individual participants. After completion of the systematic review, the data will be analysed, and the findings disseminated through peer-reviewed publications and scientific meetings.

PROSPERO REGISTRATION NUMBER

CRD42020201792.

Maternal and Child Health Handbook use for maternal and child care: a cluster randomized controlled study in rural Java, Indonesia

Background

Effectiveness of the Maternal and Child Health Handbook (MCHHB), a home-based booklet for pregnancy, delivery and postnatal/child health, was evaluated on care acquisition and home care in rural Java, a low service-coverage area.

Methods

We conducted a health centre-based randomized trial, with a 2-year follow-up. Intervention included (i) MCHHB provision at antenatal care visits; (ii) records and guides by health personnel on and with the MCHHB; and (iii) sensitization of care by volunteers using the MCHHB.

Results

The follow-up rate was 70.2% (183, intervention area; 271, control area). Respondents in the intervention area received consecutive MCH services including two doses of tetanus toxoid injections and antenatal care four times or more during pregnancy, professional assistance during child delivery and vitamin A supplements administration to their children, after adjustment for confounding variables and cluster effects (OR = 2.03, 95% CI: 1.19–3.47). In the intervention area, home care (continued breastfeeding; introducing complementary feeding; proper feeding order; varied foods feeding; self-feeding training; and care for cough), perceived support by husbands, and lower underweight rates and stunting rates among children were observed.

Conclusion

MCHHB use promoted continuous care acquisition and care at home from pregnancy to early child-rearing stages in rural Java.

The mixed model for repeated measures for cluster randomized trials: a simulation study investigating bias and type I error with missing continuous data

BACKGROUND

Cluster randomized trials (CRTs) are a design used to test interventions where individual randomization is not appropriate. The mixed model for repeated measures (MMRM) is a popular choice for individually randomized trials with longitudinal continuous outcomes. This model's appeal is due to avoidance of model misspecification and its unbiasedness for data missing completely at random or at random.

METHODS

We extended the MMRM to cluster randomized trials by adding a random intercept for the cluster and undertook a simulation experiment to investigate statistical properties when data are missing at random. We simulated cluster randomized trial data where the outcome was continuous and measured at baseline and three post-intervention time points. We varied the number of clusters, the cluster size, the intra-cluster correlation, missingness and the data-generation models. We demonstrate the MMRM-CRT with an example of a cluster randomized trial on cardiovascular disease prevention among diabetics.

RESULTS

When simulating a treatment effect at the final time point we found that estimates were unbiased when data were complete and when data were missing at random. Variance components were also largely unbiased. When simulating under the null, we found that type I error was largely nominal, although for a few specific cases it was as high as 0.081.

CONCLUSIONS

Although there have been assertions that this model is inappropriate when there are more than two repeated measures on subjects, we found evidence to the contrary. We conclude that the MMRM for CRTs is a good analytic choice for cluster randomized trials with a continuous outcome measured longitudinally.

TRIAL REGISTRATION

ClinicalTrials.gov, ID: NCT02804698.

Impact of baseline covariate imbalance on bias in treatment effect estimation in cluster randomized trials: Race as an example

Individual-level baseline covariate imbalance could happen more frequently in cluster randomized trials, and may influence the observed treatment effect. Using computer and real-data simulations, this paper quantifies the extent and impact of covariate imbalance on the estimated treatment effect for both continuous and binary outcomes, and relates it to the degree of imbalance for different numbers of clusters, cluster sizes, and covariate intraclass correlation coefficients. We focused on the impact of race as a covariate, given the emphasis of regulatory and funding bodies on understanding the influence of demographic characteristics on treatment effectiveness. We found that bias in the treatment effect is proportional to both the degree of baseline covariate imbalance and the covariate effect size. Larger numbers of clusters result in lower covariate imbalance, and increasing cluster size is less effective in reducing imbalance compared to increasing the number of clusters. Models adjusted for important baseline confounders are superior to unadjusted models for minimizing bias in both model-based simulations and an innovative simulation based on real clinical trial data. Higher outcome intraclass correlation coefficients did not affect bias but resulted in greater variance in treatment estimates.

Essential Ingredients and Innovations in the Design and Analysis of Group-Randomized Trials

This article reviews the essential ingredients and innovations in the design and analysis of group-randomized trials. The methods literature for these trials has grown steadily since they were introduced to the biomedical research community in the late 1970s, and we summarize those developments. We review, in addition to the group-randomized trial, methods for two closely related designs, the individually randomized group treatment trial and the stepped-wedge group-randomized trial. After describing the essential ingredients for these designs, we review the most important developments in the evolution of their methods using a new bibliometric tool developed at the National Institutes of Health. We then discuss the questions to be considered when selecting from among these designs or selecting the traditional randomized controlled trial. We close with a review of current methods for the analysis of data from these designs, a case study to illustrate each design, and a brief summary.

Program Impact Pathways and Contexts: A Commentary on Theoretical Issues and Research Applications to Support the EsIAN Component of Mexico's Conditional Cash Transfer Program

This commentary on the Integrated Strategy for Attention to Nutrition (EsIAN) journal supplement begins with a discussion about the challenges that implementation researchers confront with respect to analyzing complex impact pathways. We note that the research on the implementation of the EsIAN component of Mexico's conditional cash transfer program was based implicitly or explicitly on a program impact pathway approach, which used both quantitative and qualitative methods to examine bottlenecks in program implementation. We then identify 5 categories of contexts that affect the impact, implementation, and survival of intervention programs: 1) biological, 2) social-cultural, 3) delivery modalities and platforms, 4) bureaucratic, and 5) political. Each of these contexts presents theoretical and methodological challenges for investigators. In this commentary, we focus primarily on biological and social-cultural contexts, discussing the theoretical and methodological challenges the investigators faced and the research strategies they used to address them, which have produced a unique compilation of "learning by doing" studies. We also touch briefly on the political context in which the Prospera program research was conducted. We conclude with statements that highlight the exceptional value of the journal supplement, not only with respect to the analysis of the interventions the studies cover and the sustained examination of a long-term program but also as a major contribution to the literature in implementation science in nutrition.

Disseminating and implementing a lifestyle-based healthy weight program for mothers in a national organization: a study protocol for a cluster randomized trial

Background

Excessive weight gain among young adult women age 18–45 years is an alarming and overlooked trend that must be addressed to reverse the epidemics of obesity and chronic disease. During this vulnerable period, women tend to gain disproportionally large amounts of weight compared to men and to other life periods. Healthy Eating and Active Living Taught at Home (HEALTH) is a lifestyle modification intervention developed in partnership with Parents as Teachers (PAT), a national home visiting, community-based organization with significant reach in this population. HEALTH prevented weight gain, promoted sustained weight loss, and reduced waist circumference. PAT provides parent–child education and services free of charge to nearly 170,000 families through up to 25 free home visits per year until the child enters kindergarten.

Methods

This study extends effectiveness findings with a pragmatic cluster randomized controlled trial to evaluate dissemination and implementation (D&I) of HEALTH across three levels (mother, parent educator, PAT site). The trial will evaluate the effect of HEALTH and the HEALTH training curriculum (implementation strategy) on weight among mothers with overweight and obesity across the USA (N = 252 HEALTH; N = 252 usual care). Parent educators from 28 existing PAT sites (14 HEALTH, 14 usual care) will receive the HEALTH training curriculum through PAT National Center, using PAT’s existing training infrastructure, as a continuing education opportunity. An extensive evaluation, guided by RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance), will determine implementation outcomes (acceptability, adoption, appropriateness, feasibility, fidelity, and adaptation) at the parent educator level. The Conceptual Framework for Implementation Research will characterize determinants that influence HEALTH D&I at three levels: mother, parent educator, and PAT site to enhance external validity (reach and maintenance).

Discussion

Embedding intervention content within existing delivery channels can help expand the reach of evidence-based interventions. Interventions, which have been adapted, can still be effective even if the effect is reduced and can still achieve population impact by reaching a broader set of the population. The current study will build on this to test not only the effectiveness of HEALTH in real-world PAT implementation nationwide, but also elements critical to D&I, implementation outcomes, and the context for implementation.

Trial registration

https://ClinicalTrials.gov, NCT03758638. Registered 29 November 2018

Electronic supplementary material

The online version of this article (10.1186/s13012-019-0916-0) contains supplementary material, which is available to authorized users.

Sustaining sleep: Results from the randomized controlled work, family, and health study

Although calls for intervention designs are numerous within the organizational literature and increasing efforts are being made to conduct rigorous randomized controlled trials, existing studies have rarely evaluated the long-term sustainability of workplace health intervention outcomes, or mechanisms of this process. This is especially the case with regard to objective and subjective sleep outcomes. We hypothesized that a work-family intervention would increase both self-reported and objective actigraphic measures of sleep quantity and sleep quality at 6 and 18 months post-baseline in a sample of information technology workers from a U.S. Fortune 500 company. Significant intervention effects were found on objective actigraphic total sleep time and self-reported sleep insufficiency at the 6- and 18-month follow-up, with no significant decay occurring over time. However, no significant intervention effects were found for objective actigraphic wake after sleep onset or self-reported insomnia symptoms. A significant indirect effect was found for the effect of the intervention on objective actigraphic total sleep time through the proximal intervention target of 6-month control over work schedule and subsequent more distal 12-month family time adequacy. These results highlight the value of long-term occupational health intervention research, while also highlighting the utility of this work-family intervention with respect to some aspects of sleep. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Harsh Parenting and Violence Against Children: A Trial with Ultrapoor Families in Francophone West Africa

Few culturally congruent interventions are available to reduce abusive practices in families living in abject poverty in francophone West Africa. This study tests the effects of economic intervention-alone and in combination with a family-focused component-on parenting outcomes and children's reports of violence in rural Burkina Faso. Female caregivers and their 10- to 15-year-old children from 360 ultrapoor families were recruited to participate in a parallel cluster randomized control trial with 3 study arms: the waitlist (control) group, the economic intervention group (Trickle Up [TU]), and the economic intervention plus family coaching group (TU+). Effects were tested using repeated-measures mixed-effects regressions. At 12 months from baseline, caregivers from the TU+ group reported a reduced use of harsh discipline compared to the control group (Cohen's d = -0.57, p = .001) and the TU group (d = -0.48, p = .001). Changes were maintained at 24 months. TU+ caregivers also expressed more supportive parenting attitudes at 12 months compared to the control group (d = 0.39, p = .022) and the TU group (d = 0.55, p = .001). Compared to TU caregivers, caregivers in the TU+ group also reported a better quality of child-parent relationship (d = 0.40, p = .041). At 24 months, children in the TU+ group had lower odds of experiencing physical (odds ratio = 0.35, p = .050), 95% confidence interval [0.12, 1.00], and emotional (odds ratio = 0.52, p = .033), 95% confidence interval [0.28, 0.95], violence at home, compared to the control group children. The evidence suggests that involving all family members in sessions on child protection in addition to economic strengthening strategies can foster supportive parenting environments and reduce family violence among children living in ultralevel poverty in West Africa.

An Integrated Approach to Increasing Women's Empowerment and Reducing Domestic Violence: Results of a Cluster-Randomized Controlled Trial in a West African Country

OBJECTIVE

This is the first experimental study testing the effects of an economic intervention alone and in combination with a family-focused component, on women's empowerment status and family violence in Burkina Faso.

METHOD

The three-arm cluster randomized controlled trial with baseline and one-year follow-up included 360 ultra-poor married women from 12 villages. Villages were randomized to the three study arms: economic intervention for women (Trickle Up/TU), a combination of economic intervention and family coaching (Trickle Up Plus), and waitlist (Control arm). Analysis utilized repeated-measures mixed effects regressions.

RESULTS

Compared to the control group, there was a significant improvement in both the TU arm and the TU Plus arm in women's financial autonomy and in quality of marital relationships. In addition, women in both intervention arms reported a significant reduction in emotional spousal violence in the past year, with the effect size greater for the combined intervention (TU group OR=0.28, 95% CI 0.10,0.82 p=0.02 and TU+ group OR=0.19, 95% CI 0.06,0.64, p=0.007).

CONCLUSIONS

Economic intervention shows benefits that go beyond changes in financial wellbeing and may increase women's status and improve family relationships. Integrating psychosocial components with economic strategies may be more effective for improving women's empowerment status in West Africa.

Design and analysis of group-randomized trials in cancer: A review of current practices

The purpose of this paper is to summarize current practices for the design and analysis of group-randomized trials involving cancer-related risk factors or outcomes and to offer recommendations to improve future trials. We searched for group-randomized trials involving cancer-related risk factors or outcomes that were published or online in peer-reviewed journals in 2011-15. During 2016-17, in Bethesda MD, we reviewed 123 articles from 76 journals to characterize their design and their methods for sample size estimation and data analysis. Only 66 (53.7%) of the articles reported appropriate methods for sample size estimation. Only 63 (51.2%) reported exclusively appropriate methods for analysis. These findings suggest that many investigators do not adequately attend to the methodological challenges inherent in group-randomized trials. These practices can lead to underpowered studies, to an inflated type 1 error rate, and to inferences that mislead readers. Investigators should work with biostatisticians or other methodologists familiar with these issues. Funders and editors should ensure careful methodological review of applications and manuscripts. Reviewers should ensure that studies are properly planned and analyzed. These steps are needed to improve the rigor and reproducibility of group-randomized trials. The Office of Disease Prevention (ODP) at the National Institutes of Health (NIH) has taken several steps to address these issues. ODP offers an online course on the design and analysis of group-randomized trials. ODP is working to increase the number of methodologists who serve on grant review panels. ODP has developed standard language for the Application Guide and the Review Criteria to draw investigators' attention to these issues. Finally, ODP has created a new Research Methods Resources website to help investigators, reviewers, and NIH staff better understand these issues.

Effect of a Health Care Professional Communication Training Intervention on Adolescent Human Papillomavirus Vaccination: A Cluster Randomized Clinical Trial

IMPORTANCE

The incidence of human papillomavirus (HPV)-related cancers is more than 35 000 cases in the United States each year. Effective HPV vaccines have been available in the United States for several years but are underused among adolescents, the target population for vaccination. Interventions to increase uptake are needed.

OBJECTIVE

To evaluate the effect of a 5-component health care professional HPV vaccine communication intervention on adolescent HPV vaccination.

DESIGN, SETTING, AND PARTICIPANTS

A cluster randomized clinical trial using covariate-constrained randomization to assign study arms and an intent-to-treat protocol was conducted in 16 primary care practices in the Denver, Colorado, metropolitan area. Participants included 188 medical professionals and 43 132 adolescents.

INTERVENTIONS

The 5 components of the intervention were an HPV fact sheet library to create customized information sheets relevant to each practice's patient population, a tailored parent education website, a set of HPV-related disease images, an HPV vaccine decision aid, and 2½ hours of communication training on using a presumptive vaccine recommendation, followed by motivational interviewing if parents were resistant to vaccination. Each practice participated in a series of 2 intervention development meetings over a 6-month period (August 1, 2014, to January 31, 2015) before the intervention.

MAIN OUTCOMES AND MEASURES

Differences between control and intervention changes over time (ie, difference in differences between the baseline and intervention period cohorts of patients) in HPV vaccine series initiation (≥1 dose) and completion (≥3 doses) among patients aged 11 to 17 years seen at the practices between February 1, 2015, and January 31, 2016. Vaccination data were obtained from the practices' records and augmented with state immunization information system data.

RESULTS

Sixteen practices and 43 132 patients (50.3% female; median age, 12.6 years [interquartile range, 10.8-14.7 years] at the beginning of the study period) participated in this trial. Adolescents in the intervention practices had significantly higher odds of HPV vaccine series initiation (adjusted odds ratio [aOR], 1.46; 95% CI, 1.31-1.62) and completion (aOR, 1.56; 95% CI, 1.27-1.92) than those in the control practices (a 9.5-absolute percentage point increase in HPV vaccine series initiation and a 4.4-absolute percentage point increase in HPV vaccine series completion in intervention practices). The intervention had a greater effect in pediatric practices compared with family medicine practices and in private practices compared with public ones. Health care professionals reported that communication training and the fact sheets were the most used and useful intervention components.

CONCLUSIONS AND RELEVANCE

A health care professional communication intervention significantly improved HPV vaccine series initiation and completion among adolescent patients.

TRIAL REGISTRATION

clinicaltrials.gov Identifier: NCT02456077.

Preventing Adolescent Substance Use Through an Evidence-Based Program: Effects of the Italian Adaptation of Life Skills Training

Evidence-based preventive interventions for adolescent substance use, violence, and mental health issues are increasingly being adapted and disseminated internationally. In the present paper, we report the results of an effectiveness study that was part of a comprehensive initiative by a coalition of health promotion organizations in the Lombardy region of Italy to select, culturally adapt, implement, evaluate, and sustain an evidence-based drug abuse prevention program developed in the USA. Findings are presented from a large-scale effectiveness study of the Life Skills Training prevention program among over 3000 students attending 55 middle schools in Italy. The prevention program taught drug refusal skills, antidrug norms, personal self-management skills, and general social skills. Relative to comparison group students, students who received the prevention program were less likely to initiate smoking at the post-test and 2-year follow-up, and less likely to initiate weekly drunkenness at the 1-year follow-up. The program had direct positive effects on several cognitive, attitudinal, and skill variables believed to play a protective role in adolescent substance use. The findings from this study show that a drug abuse prevention program originally designed for adolescents in the USA is effective in a sample of Italian youth when a rigorous and systematic approach to cultural adaptation is followed that incorporates the input of multiple stakeholders.

Caring for the elderly at work and home: Can a randomized organizational intervention improve psychological health?

Although job stress models suggest that changing the work social environment to increase job resources improves psychological health, many intervention studies have weak designs and overlook influences of family caregiving demands. We tested the effects of an organizational intervention designed to increase supervisor social support for work and nonwork roles, and job control in a results-oriented work environment on the stress and psychological distress of health care employees who care for the elderly, while simultaneously considering their own family caregiving responsibilities. Using a group-randomized organizational field trial with an intent-to-treat design, 420 caregivers in 15 intervention extended-care nursing facilities were compared with 511 caregivers in 15 control facilities at 4 measurement times: preintervention and 6, 12, and 18 months. There were no main intervention effects showing improvements in stress and psychological distress when comparing intervention with control sites. Moderation analyses indicate that the intervention was more effective in reducing stress and psychological distress for caregivers who were also caring for other family members off the job (those with elders and those "sandwiched" with both child and elder caregiving responsibilities) compared with employees without caregiving demands. These findings extend previous studies by showing that the effect of organizational interventions designed to increase job resources to improve psychological health varies according to differences in nonwork caregiving demands. This research suggests that caregivers, especially those with "double-duty" elder caregiving at home and work and "triple-duty" responsibilities, including child care, may benefit from interventions designed to increase work-nonwork social support and job control. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Review of Recent Methodological Developments in Group-Randomized Trials: Part 2-Analysis

In 2004, Murray et al. reviewed methodological developments in the design and analysis of group-randomized trials (GRTs). We have updated that review with developments in analysis of the past 13 years, with a companion article to focus on developments in design. We discuss developments in the topics of the earlier review (e.g., methods for parallel-arm GRTs, individually randomized group-treatment trials, and missing data) and in new topics, including methods to account for multiple-level clustering and alternative estimation methods (e.g., augmented generalized estimating equations, targeted maximum likelihood, and quadratic inference functions). In addition, we describe developments in analysis of alternative group designs (including stepped-wedge GRTs, network-randomized trials, and pseudocluster randomized trials), which require clustering to be accounted for in their design and analysis.

Increased risk of type I errors in cluster randomised trials with small or medium numbers of clusters: a review, reanalysis, and simulation study

BACKGROUND

Cluster randomised trials (CRTs) are commonly analysed using mixed-effects models or generalised estimating equations (GEEs). However, these analyses do not always perform well with the small number of clusters typical of most CRTs. They can lead to increased risk of a type I error (finding a statistically significant treatment effect when it does not exist) if appropriate corrections are not used.

METHODS

We conducted a small simulation study to evaluate the impact of using small-sample corrections for mixed-effects models or GEEs in CRTs with a small number of clusters. We then reanalysed data from TRIGGER, a CRT with six clusters, to determine the effect of using an inappropriate analysis method in practice. Finally, we reviewed 100 CRTs previously identified by a search on PubMed in order to assess whether trials were using appropriate methods of analysis. Trials were classified as at risk of an increased type I error rate if they did not report using an analysis method which accounted for clustering, or if they had fewer than 40 clusters and performed an individual-level analysis without reporting the use of an appropriate small-sample correction.

RESULTS

Our simulation study found that using mixed-effects models or GEEs without an appropriate correction led to inflated type I error rates, even for as many as 70 clusters. Conversely, using small-sample corrections provided correct type I error rates across all scenarios. Reanalysis of the TRIGGER trial found that inappropriate methods of analysis gave much smaller P values (P ≤ 0.01) than appropriate methods (P = 0.04-0.15). In our review, of the 99 trials that reported the number of clusters, 64 (65 %) were at risk of an increased type I error rate; 14 trials did not report using an analysis method which accounted for clustering, and 50 trials with fewer than 40 clusters performed an individual-level analysis without reporting the use of an appropriate correction.

CONCLUSIONS

CRTs with a small or medium number of clusters are at risk of an inflated type I error rate unless appropriate analysis methods are used. Investigators should consider using small-sample corrections with mixed-effects models or GEEs to ensure valid results.

Estimates of Intraclass Correlation Coefficients and Other Design Parameters for Studies of School-Based Nutritional Interventions

INTRODUCTION

When cluster randomized trials are used to evaluate school-based nutritional interventions such as school lunch programs, design-stage estimates of the required sample size must take into account the correlation in outcomes among individuals within each cluster (e.g., classrooms, schools, or districts). Estimates of the necessary parameters have been carefully developed for educational interventions, but for nutritional interventions the literature is thin.

METHODS

Using data from two large multi-school, multi-district impact evaluations conducted in the United States, this article calculates estimates of the design parameters required for sizing school-based nutritional studies. The large size of the trials (252 and 1,327 schools) yields precise estimates of the parameters of interest. Variance components are estimated by fitting random-intercept multilevel models in Stata.

RESULTS

School-level intraclass correlations are similar to those typically found for educational outcomes. In particular, school-level estimates range from less than .01 to .26 across the two studies, and district-level estimates ranged from less than .01 to .19. This suggests that cluster randomized trials of nutritional interventions may require samples with numbers of schools similar to the education studies to detect similar effect sizes.

Generalized estimating equations in cluster randomized trials with a small number of clusters: Review of practice and simulation study

Background/aims:

Generalized estimating equations are a common modeling approach used in cluster randomized trials to account for within-cluster correlation. It is well known that the sandwich variance estimator is biased when the number of clusters is small (≤40), resulting in an inflated type I error rate. Various bias correction methods have been proposed in the statistical literature, but how adequately they are utilized in current practice for cluster randomized trials is not clear. The aim of this study is to evaluate the use of generalized estimating equation bias correction methods in recently published cluster randomized trials and demonstrate the necessity of such methods when the number of clusters is small.

Methods:

Review of cluster randomized trials published between August 2013 and July 2014 and using generalized estimating equations for their primary analyses. Two independent reviewers collected data from each study using a standardized, pre-piloted data extraction template. A two-arm cluster randomized trial was simulated under various scenarios to show the potential effect of a small number of clusters on type I error rate when estimating the treatment effect. The nominal level was set at 0.05 for the simulation study.

Results:

Of the 51 included trials, 28 (54.9%) analyzed 40 or fewer clusters with a minimum of four total clusters. Of these 28 trials, only one trial used a bias correction method for generalized estimating equations. The simulation study showed that with four clusters, the type I error rate ranged between 0.43 and 0.47. Even though type I error rate moved closer to the nominal level as the number of clusters increases, it still ranged between 0.06 and 0.07 with 40 clusters.

Conclusions:

Our results showed that statistical issues arising from small number of clusters in generalized estimating equations is currently inadequately handled in cluster randomized trials. Potential for type I error inflation could be very high when the sandwich estimator is used without bias correction.

Intervention effects on safety compliance and citizenship behaviors: Evidence from the Work, Family, and Health Study

We tested the effects of a work-family intervention on employee reports of safety compliance and organizational citizenship behaviors in 30 health care facilities using a group-randomized trial. Based on conservation of resources theory and the work-home resources model, we hypothesized that implementing a work-family intervention aimed at increasing contextual resources via supervisor support for work and family, and employee control over work time, would lead to improved personal resources and increased employee performance on the job in the form of self-reported safety compliance and organizational citizenship behaviors. Multilevel analyses used survey data from 1,524 employees at baseline and at 6-month and 12-month postintervention follow-ups. Significant intervention effects were observed for safety compliance at the 6-month, and organizational citizenship behaviors at the 12-month, follow-ups. More specifically, results demonstrate that the intervention protected against declines in employee self-reported safety compliance and organizational citizenship behaviors compared with employees in the control facilities. The hypothesized mediators of perceptions of family-supportive supervisor behaviors, control over work time, and work-family conflict (work-to-family conflict, family-to-work conflict) were not significantly improved by the intervention. However, baseline perceptions of family-supportive supervisor behaviors, control over work time, and work-family climate were significant moderators of the intervention effect on the self-reported safety compliance and organizational citizenship behavior outcomes.

Recommendations for choosing an analysis method that controls Type I error for unbalanced cluster sample designs with Gaussian outcomes

We used theoretical and simulation-based approaches to study Type I error rates for one-stage and two-stage analytic methods for cluster-randomized designs. The one-stage approach uses the observed data as outcomes and accounts for within-cluster correlation using a general linear mixed model. The two-stage model uses the cluster specific means as the outcomes in a general linear univariate model. We demonstrate analytically that both one-stage and two-stage models achieve exact Type I error rates when cluster sizes are equal. With unbalanced data, an exact size α test does not exist, and Type I error inflation may occur. Via simulation, we compare the Type I error rates for four one-stage and six two-stage hypothesis testing approaches for unbalanced data. With unbalanced data, the two-stage model, weighted by the inverse of the estimated theoretical variance of the cluster means, and with variance constrained to be positive, provided the best Type I error control for studies having at least six clusters per arm. The one-stage model with Kenward-Roger degrees of freedom and unconstrained variance performed well for studies having at least 14 clusters per arm. The popular analytic method of using a one-stage model with denominator degrees of freedom appropriate for balanced data performed poorly for small sample sizes and low intracluster correlation. Because small sample sizes and low intracluster correlation are common features of cluster-randomized trials, the Kenward-Roger method is the preferred one-stage approach.

Designing Intervention Studies: Selected Populations, Range Restrictions, and Statistical Power

An appropriate estimate of statistical power is critical for the design of intervention studies. Although the inclusion of a pretest covariate in the test of the primary outcome can increase statistical power, samples selected on the basis of pretest performance may demonstrate range restriction on the selection measure and other correlated measures. This can result in attenuated pretest-posttest correlations, reducing the variance explained by the pretest covariate. We investigated the implications of two potential range restriction scenarios: direct truncation on a selection measure and indirect range restriction on correlated measures. Empirical and simulated data indicated direct range restriction on the pretest covariate greatly reduced statistical power and necessitated sample size increases of 82%-155% (dependent on selection criteria) to achieve equivalent statistical power to parameters with unrestricted samples. However, measures demonstrating indirect range restriction required much smaller sample size increases (32%-71%) under equivalent scenarios. Additional analyses manipulated the correlations between measures and pretest-posttest correlations to guide planning experiments. Results highlight the need to differentiate between selection measures and potential covariates and to investigate range restriction as a factor impacting statistical power.

Estimates of Intraclass Correlation Coefficients From Longitudinal Group-Randomized Trials of Adolescent HIV/STI/Pregnancy Prevention Programs

INTRODUCTION

Group-randomized trials (GRTs) are one of the most rigorous methods for evaluating the effectiveness of group-based health risk prevention programs. Efficiently designing GRTs with a sample size that is sufficient for meeting the trial's power and precision goals while not wasting resources exceeding them requires estimates of the intraclass correlation coefficient (ICC)-the degree to which outcomes of individuals clustered within groups (e.g., schools) are correlated. ICC estimates vary widely depending on outcome, population, and setting, and small changes in ICCs can have large effects on the sample size needed to estimate intervention effects. This study addresses a gap in the literature by providing estimates of ICCs for adolescent sexual risk-taking outcomes under a range of study conditions.

METHOD

Multilevel regression analyses were applied to existing data from four federally funded GRTs of school-based HIV/STI/pregnancy prevention programs to obtain a variety of ICC estimates.

RESULTS

ICCs ranged from 0 to 0.15, with adjustment for covariates and repeated measurements reducing the ICC in the majority of cases. Minimum detectable effect sizes with 80% power and 0.05 significance levels ranged from small to medium Cohen's d (0.13 to 0.53) assuming 20 schools of 100 students each.

CONCLUSIONS

This study provides the first known set of ICC estimates for investigators to use when planning studies of school-based programs to prevent sexual risk behaviors in youth. The results provide further evidence of the importance of using the appropriate adjusted ICC estimate at the design stage to maximize resources in costly GRTs.

Statistical analysis and handling of missing data in cluster randomised trials: protocol for a systematic review

INTRODUCTION

Cluster randomised trials (CRTs) randomise participants in groups, rather than as individuals, and are key tools used to assess interventions in health research where treatment contamination is likely or if individual randomisation is not feasible. Missing outcome data can reduce power in trials, including in CRTs, and is a potential source of bias. The current review focuses on evaluating methods used in statistical analysis and handling of missing data with respect to the primary outcome in CRTs.

METHODS AND ANALYSIS

We will search for CRTs published between August 2013 and July 2014 using PubMed, Web of Science and PsycINFO. We will identify relevant studies by screening titles and abstracts, and examining full-text articles based on our predefined study inclusion criteria. 86 studies will be randomly chosen to be included in our review. Two independent reviewers will collect data from each study using a standardised, prepiloted data extraction template. Our findings will be summarised and presented using descriptive statistics.

ETHICS AND DISSEMINATION

This methodological systematic review does not need ethical approval because there are no data used in our study that are linked to individual patient data. After completion of this systematic review, data will be immediately analysed, and findings will be disseminated through a peer-reviewed publication and conference presentation.

The Maternal and Child Health (MCH) handbook in Mongolia: a cluster-randomized, controlled trial

OBJECTIVE

To assess the effectiveness of the Maternal and Child Health (MCH) handbook in Mongolia to increase antenatal clinic attendance, and to enhance health-seeking behaviors and other health outcomes.

METHODS

A cluster randomized trial was conducted using the translated MCH handbook in Bulgan, Mongolia to assess its effectiveness in promoting antenatal care attendance. Pregnant women were recruited from 18 randomly allocated districts using shuffled, sealed envelopes. The handbook was implemented immediately for women at their first antenatal visit in the intervention group, and nine months later in the control group. The primary outcome was the number of antenatal care visits of all women residing in the selected districts. Cluster effects were adjusted for using generalized estimation equation. Masking was not possible among care providers, pregnant women and assessors.

FINDINGS

Nine districts were allocated to the intervention group and the remainder to the control group. The intervention group (253 women) attended antenatal clinics on average 6•9 times, while the control group (248 women) attended 6•2 times. Socioeconomic status affected the frequency of clinic attendance: women of higher socioeconomic status visited antenatal clinics more often. Pregnancy complications were more likely to be detected among women using the handbook.

CONCLUSION

The MCH handbook promotes continuous care and showed an increase in antenatal visits among the intervention group. The intervention will help to identify maternal morbidities during pregnancy and promote health-seeking behaviors.

TRIAL REGISTRATION

UMIN Clinical Trial Registry UMIN000001748.

A workplace intervention improves sleep: results from the randomized controlled Work, Family, and Health Study

STUDY OBJECTIVES

The Work, Family, and Health Network Study tested the hypothesis that a workplace intervention designed to increase family-supportive supervision and employee control over work time improves actigraphic measures of sleep quantity and quality.

DESIGN

Cluster-randomized trial.

SETTING

A global information technology firm.

PARTICIPANTS

US employees at an information technology firm.

INTERVENTIONS

Randomly selected clusters of managers and employees participated in a 3-month, social, and organizational change process intended to reduce work-family conflict. The intervention included interactive sessions with facilitated discussions, role playing, and games. Managers completed training in family-supportive supervision.

MEASUREMENTS AND RESULTS

Primary outcomes of total sleep time (sleep duration) and wake after sleep onset (sleep quality) were collected from week-long actigraphy recordings at baseline and 12 months. Secondary outcomes included self-reported sleep insufficiency and insomnia symptoms. Twelve-month interviews were completed by 701 (93% retention), of whom 595 (85%) completed actigraphy. Restricting analyses to participants with e3 valid days of actigraphy yielded a sample of 473-474 for intervention effectiveness analyses. Actigraphy-measured sleep duration was 8 min/d greater among intervention employees relative to controls (P < .05). Sleep insufficiency was reduced among intervention employees (P = .002). Wake after sleep onset and insomnia symptoms were not different between groups. Path models indicated that increased control over work hours and subsequent reductions in work-family conflict mediated the improvement in sleep sufficiency.

CONCLUSIONS

The workplace intervention did not overtly address sleep, yet intervention employees slept 8 min/d more and reported greater sleep sufficiency. Interventions should address environmental and psychosocial causes of sleep deficiency, including workplace factors.

School-located vaccination of adolescents with insurance billing: cost, reimbursement, and vaccination outcomes

PURPOSE

To assess, in a school-located adolescent vaccination program that billed health insurance, the program costs, the proportion of costs reimbursed, and the likelihood of vaccination.

METHODS

During the 2010-2011 school year, vaccination clinics were held for sixth- to eighth-grade students at seven Denver public schools. Vaccine administration and purchase costs were compared with reimbursement by insurers. Multivariate analyses were used to compare the likelihood of vaccination among students in intervention schools with students in control schools who did not participate in the program, with analyses stratified by grade (sixth grade vs. seventh-eighth grades).

RESULTS

Fifteen percent (466 of 3,144) of students attending intervention schools were vaccinated at school-located vaccination clinics. Among students vaccinated at school, 41% were uninsured, 37% publicly insured, and 22% privately insured. Estimated vaccine administration costs were $23.98 per vaccine dose. Seventy-eight percent of vaccine purchase costs and 14% of vaccine administration costs were reimbursed by insurers; 41% of total program costs were reimbursed. Sixth-grade students in intervention schools were more likely than those in control schools to receive tetanus-diphtheria-acellular pertussis (risk ratio [RR], 1.30; 95% confidence interval [CI], 1.08, 1.57), meningococcal conjugate (RR, 1.42; CI, 1.18, 1.70), and human papillomavirus (for females only, RR, 1.69; CI, 1.21, 2.36) vaccines during the 2010-2011 school year, with similar results for seventh- to eighth-grade students.

CONCLUSIONS

Although school-located adolescent vaccination with billing appears feasible and likely to improve vaccination rates, improvements in insurance coverage and reimbursement rates may be needed for the long-term financial sustainability of such programs.

An Analysis of Methodologies That Can Be Used to Validate if a Perioperative Surgical Home Improves the Patient-centeredness, Evidence-based Practice, Quality, Safety, and Value of Patient Care

Approximately 80 million inpatient and outpatient surgeries are performed annually in the United States. Widely variable and fragmented perioperative care exposes these surgical patients to lapses in expected standard of care, increases the chance for operational mistakes and accidents, results in unnecessary and potentially detrimental care, needlessly drives up costs, and adversely affects the patient healthcare experience. The American Society of Anesthesiologists and other stakeholders have proposed a more comprehensive model of perioperative care, the Perioperative Surgical Home (PSH), to improve current care of surgical patients and to meet the future demands of increased volume, quality standards, and patient-centered care. To justify implementation of this new healthcare delivery model to surgical colleagues, administrators, and patients and maintain the integrity of evidenced-based practice, the nascent PSH model must be rigorously evaluated. This special article proposes comparative effectiveness research aims or objectives and an optimal study design for the novel PSH model.

A systematic review of cluster randomised trials in residential facilities for older people suggests how to improve quality

BACKGROUND

Previous reviews of cluster randomised trials have been critical of the quality of the trials reviewed, but none has explored determinants of the quality of these trials in a specific field over an extended period of time. Recent work suggests that correct conduct and reporting of these trials may require more than published guidelines. In this review, our aim was to assess the quality of cluster randomised trials conducted in residential facilities for older people, and to determine whether (1) statistician involvement in the trial and (2) strength of journal endorsement of the Consolidated Standards of Reporting Trials (CONSORT) statement influence quality.

METHODS

We systematically identified trials randomising residential facilities for older people, or parts thereof, without language restrictions, up to the end of 2010, using National Library of Medicine (Medline) via PubMed and hand-searching. We based quality assessment criteria largely on the extended CONSORT statement for cluster randomised trials. We assessed statistician involvement based on statistician co-authorship, and strength of journal endorsement of the CONSORT statement from journal websites.

RESULTS

73 trials met our inclusion criteria. Of these, 20 (27%) reported accounting for clustering in sample size calculations and 54 (74%) in the analyses. In 29 trials (40%), methods used to identify/recruit participants were judged by us to have potentially caused bias or reporting was unclear to reach a conclusion. Some elements of quality improved over time but this appeared not to be related to the publication of the extended CONSORT statement for these trials. Trials with statistician/epidemiologist co-authors were more likely to account for clustering in sample size calculations (unadjusted odds ratio 5.4, 95% confidence interval 1.1 to 26.0) and analyses (unadjusted OR 3.2, 1.2 to 8.5). Journal endorsement of the CONSORT statement was not associated with trial quality.

CONCLUSIONS

Despite international attempts to improve methods in cluster randomised trials, important quality limitations remain amongst these trials in residential facilities. Statistician involvement on trial teams may be more effective in promoting quality than further journal endorsement of the CONSORT statement. Funding bodies and journals should promote statistician involvement and co-authorship in addition to adherence to CONSORT guidelines.

Concordance between patient and family reports of family functioning in bipolar I disorder and major depressive disorder

Despite the extensive literature on family functioning and mood disorders, less is known about concordance between patient- and family-reported family functioning. To address this question, adults with bipolar I disorder (BD; n = 92) or major depressive disorder (MDD; n = 121) and their family members (n = 135 and 201, respectively) were recruited from hospital sources. All patients and their family members completed the Family Assessment Device (Epstein, Baldwin, Bishop. J Marital Fam Ther. 9:171-180, 1983). Intraclass correlation coefficients revealed that, in contrast to the moderate degree of concordance in the MDD sample, degree of concordance between patient- and family-reported family functioning was significantly weaker in BD. Subsequent analysis revealed that this discordance was driven by the reports of the child and young adolescent family members of the patients with BD. Results highlight the importance of collateral reports in the assessment of family functioning, especially among families of patients with BD, in research and treatment.

Estimating the Effect of a Community-Based Intervention with Two Communities

Due to the need to evaluate the effectiveness of community-based programs in practice, there is substantial interest in methods to estimate the causal effects of community-level treatments or exposures on individual level outcomes. The challenge one is confronted with is that different communities have different environmental factors affecting the individual outcomes, and all individuals in a community share the same environment and intervention. In practice, data are often available from only a small number of communities, making it difficult if not impossible to adjust for these environmental confounders. In this paper we consider an extreme version of this dilemma, in which two communities each receives a different level of the intervention, and covariates and outcomes are measured on a random sample of independent individuals from each of the two populations; the results presented can be straightforwardly generalized to settings in which more than two communities are sampled. We address the question of what conditions are needed to estimate the causal effect of the intervention, defined in terms of an ideal experiment in which the exposed level of the intervention is assigned to both communities and individual outcomes are measured in the combined population, and then the clock is turned back and a control level of the intervention is assigned to both communities and individual outcomes are measured in the combined population. We refer to the difference in the expectation of these outcomes as the marginal (overall) treatment effect. We also discuss conditions needed for estimation of the treatment effect on the treated community. We apply a nonparametric structural equation model to define these causal effects and to establish conditions under which they are identified. These identifiability conditions provide guidance for the design of studies to investigate community level causal effects and for assessing the validity of causal interpretations when data are only available from a few communities. When the identifiability conditions fail to hold, the proposed statistical parameters still provide nonparametric treatment effect measures (albeit non-causal) whose statistical interpretations do not depend on model specifications. In addition, we study the use of a matched cohort sampling design in which the units of different communities are matched on individual factors. Finally, we provide semiparametric efficient and doubly robust targeted MLE estimators of the community level causal effect based on i.i.d. sampling and matched cohort sampling.

An Integrative, Multilevel, and Transdisciplinary Research Approach to Challenges of Work, Family, and Health

Recognizing a need for rigorous, experimental research to support the efforts of workplaces and policymakers in improving the health and wellbeing of employees and their families, the National Institutes of Health and the Centers for Disease Control and Prevention formed the Work, Family & Health Network (WFHN). The WFHN is implementing an innovative multisite study with a rigorous experimental design (adaptive randomization, control groups), comprehensive multilevel measures, a novel and theoretically based intervention targeting the psychosocial work environment, and translational activities. This paper describes challenges and benefits of designing a multilevel and transdisciplinary research network that includes an effectiveness study to assess intervention effects on employees, families, and managers; a daily diary study to examine effects on family functioning and daily stress; a process study to understand intervention implementation; and translational research to understand and inform diffusion of innovation. Challenges were both conceptual and logistical, spanning all aspects of study design and implementation. In dealing with these challenges, however, the WFHN developed innovative, transdisciplinary, multi-method approaches to conducting workplace research that will benefit both the research and business communities.

Disseminating Organizational Screening and Brief Intervention Services (DO-SBIS) for alcohol at trauma centers study design

OBJECTIVE

In 2005, the American College of Surgeons passed a mandate requiring that Level I trauma centers have a mechanism to identify patients who are problem drinkers and have the capacity to provide an intervention for patients who screen positive. The aim of the Disseminating Organizational Screening and Brief Intervention Services (DO-SBIS) cluster randomized trial is to test a multilevel intervention targeting the implementation of high-quality alcohol screening and brief intervention (SBI) services at trauma centers.

METHOD

Twenty sites selected from all United States Level I trauma centers were randomized to participate in the trial. Intervention site providers receive a combination of workshop training in evidence-based motivational interviewing (MI) interventions and organizational development activities prior to conducting trauma-center-based alcohol SBI with blood-alcohol-positive injured patients. Control sites implement care as usual. Provider MI skills, patient alcohol consumption, and organizational acceptance of SBI implementation outcomes are assessed.

RESULTS

The investigation has successfully recruited provider, patient and trauma center staff samples into the study, and outcomes are being followed longitudinally.

CONCLUSION

When completed, the DO-SBIS trial will inform future American College of Surgeons' policy targeting the sustained integration of high-quality alcohol SBI at trauma centers nationwide.

Cluster randomized clinical trials in orthodontics: design, analysis and reporting issues

Cluster randomized trials (CRTs) use as the unit of randomization clusters, which are usually defined as a collection of individuals sharing some common characteristics. Common examples of clusters include entire dental practices, hospitals, schools, school classes, villages, and towns. Additionally, several measurements (repeated measurements) taken on the same individual at different time points are also considered to be clusters. In dentistry, CRTs are applicable as patients may be treated as clusters containing several individual teeth. CRTs require certain methodological procedures during sample calculation, randomization, data analysis, and reporting, which are often ignored in dental research publications. In general, due to similarity of the observations within clusters, each individual within a cluster provides less information compared with an individual in a non-clustered trial. Therefore, clustered designs require larger sample sizes compared with non-clustered randomized designs, and special statistical analyses that account for the fact that observations within clusters are correlated. It is the purpose of this article to highlight with relevant examples the important methodological characteristics of cluster randomized designs as they may be applied in orthodontics and to explain the problems that may arise if clustered observations are erroneously treated and analysed as independent (non-clustered).

Person mobility in the design and analysis of cluster-randomized cohort prevention trials

Person mobility is an inescapable fact of life for most cluster-randomized (e.g., schools, hospitals, clinic, cities, state) cohort prevention trials. Mobility rates are an important substantive consideration in estimating the effects of an intervention. In cluster-randomized trials, mobility rates are often correlated with ethnicity, poverty and other variables associated with disparity. This raises the possibility that estimated intervention effects may generalize to only the least mobile segments of a population and, thus, create a threat to external validity. Such mobility can also create threats to the internal validity of conclusions from randomized trials. Researchers must decide how to deal with persons who leave study clusters during a trial (dropouts), persons and clusters that do not comply with an assigned intervention, and persons who enter clusters during a trial (late entrants), in addition to the persons who remain for the duration of a trial (stayers). Statistical techniques alone cannot solve the key issues of internal and external validity raised by the phenomenon of person mobility. This commentary presents a systematic, Campbellian-type analysis of person mobility in cluster-randomized cohort prevention trials. It describes four approaches for dealing with dropouts, late entrants and stayers with respect to data collection, analysis and generalizability. The questions at issue are: 1) From whom should data be collected at each wave of data collection? 2) Which cases should be included in the analyses of an intervention effect? and 3) To what populations can trial results be generalized? The conclusions lead to recommendations for the design and analysis of future cluster-randomized cohort prevention trials.

Allocation techniques for balance at baseline in cluster randomized trials: a methodological review

Reviews have repeatedly noted important methodological issues in the conduct and reporting of cluster randomized controlled trials (C-RCTs). These reviews usually focus on whether the intracluster correlation was explicitly considered in the design and analysis of the C-RCT. However, another important aspect requiring special attention in C-RCTs is the risk for imbalance of covariates at baseline. Imbalance of important covariates at baseline decreases statistical power and precision of the results. Imbalance also reduces face validity and credibility of the trial results. The risk of imbalance is elevated in C-RCTs compared to trials randomizing individuals because of the difficulties in recruiting clusters and the nested nature of correlated patient-level data. A variety of restricted randomization methods have been proposed as way to minimize risk of imbalance. However, there is little guidance regarding how to best restrict randomization for any given C-RCT. The advantages and limitations of different allocation techniques, including stratification, matching, minimization, and covariate-constrained randomization are reviewed as they pertain to C-RCTs to provide investigators with guidance for choosing the best allocation technique for their trial.

Building the evidence base for decision making in cancer genomic medicine using comparative effectiveness research

The clinical utility is uncertain for many cancer genomic applications. Comparative effectiveness research (CER) can provide evidence to clarify this uncertainty. The aim of this study was to identify approaches to help stakeholders make evidence-based decisions and to describe potential challenges and opportunities in using CER to produce evidence-based guidance. We identified general CER approaches for genomic applications through literature review, the authors' experiences, and lessons learned from a recent, seven-site CER initiative in cancer genomic medicine. Case studies illustrate the use of CER approaches. Evidence generation and synthesis approaches used in CER include comparative observational and randomized trials, patient-reported outcomes, decision modeling, and economic analysis. Significant challenges to conducting CER in cancer genomics include the rapid pace of innovation, lack of regulation, and variable definitions and evidence thresholds for clinical and personal utility. Opportunities to capitalize on CER methods in cancer genomics include improvements in the conduct of evidence synthesis, stakeholder engagement, increasing the number of comparative studies, and developing approaches to inform clinical guidelines and research prioritization. CER offers a variety of methodological approaches that can address stakeholders' needs and help ensure an effective translation of genomic discoveries.

After-school based obesity prevention interventions: a comprehensive review of the literature

The purpose of this article was to review primary prevention interventions targeting childhood obesity implemented in the after school environment from 2006 and 2011. A total of 20 interventions were found from 25 studies. Children in the interventions ranged from kindergarten to middle schoolers, however a majority was in the 4th and 5th grades. Most of the interventions targeted both physical activity and dietary behaviors. Among those that focused on only one dimension, physical activity was targeted more than diet. The duration of the interventions greatly varied, but many were short-term or brief. Many interventions were also based on some behavioral theory, with social cognitive theory as the most widely used. Most of the interventions focused on short-term changes, and rarely did any perform a follow-up evaluation. A major limitation among after school interventions was an inadequate use of process evaluations. Overall, interventions resulted in modest changes in behaviors and behavioral antecedents, and results were mixed and generally unfavorable with regards to indicators of obesity. Recommendations for enhancing the effectiveness of after school based childhood obesity interventions are presented.

Scientific rigour in psycho-oncology trials: why and how to avoid common statistical errors

OBJECTIVE

It is well documented that statistical and methodological flaws are common in much of the health research literature, including psycho-oncology. These can have far-reaching effects, including the publishing of misleading results; the wasting of time, effort, and financial resources; exposure of patients to the potential harms of research and decreased confidence in science and researchers by the public.

METHODS

Several of the most common statistical errors and methodological pitfalls that occur in the field of psycho-oncology are discussed, including those that occur at the design, analysis, reporting and conclusion stages.

RESULTS

Fourteen topics are briefly discussed, explaining why there is a problem and how to avoid it. These include proper approaches to power, clustering, missing data, categorization of continuous variables, subgroup analyses, multiple comparisons, statistical interactions, confidence intervals and correct interpretation of p-values. Extensive referencing points the reader to more in-depth explanations.

CONCLUSIONS

To increase the scientific rigour in psycho-oncology, researchers should involve a biostatistician from the beginning of the study and should commit to continuing education on best practices in the fields of statistics and reporting.

Quality of cluster randomized controlled trials in oral health: a systematic review of reports published between 2005 and 2009

OBJECTIVES

To assess the quality of methods and reporting of recently published cluster randomized trials (CRTs) in oral health.

METHODS

We searched PubMed for CRTs that included at least one oral health-related outcome and were published from 2005 to 2009 inclusive. We developed a list of criteria for assessing trial quality and reporting. This was influenced largely by the extended CONSORT statement for CRTs but also included criteria suggested by other authors. We examined the extent to which trials were consistent with these criteria.

RESULTS

Twenty-three trials were included in the review. In 15 (65%) trials, clustering had been accounted for in sample size calculations, and in 18 (78%) authors had accounted for clustering in analysis. Intraclass correlation coefficients (ICCs) were reported for eight (35%) trials; the outcome assessor was reported as having been blinded to allocation in 12 (52%) trials; 17 (74%) described eligibility criteria at individual level, but only nine (39%) described such criteria at cluster level. Sixteen of 20 trials (80%), in which individuals were recruited, reported that individual informed consent was obtained.

CONCLUSIONS

These results suggest that the quality of recent CRTs in oral health is relatively high and appears to compare favourably with other fields. However, there remains room for improvement. Authors of future trials should endeavour to ensure sample size calculations and analyses properly account for clustering (and are reported as such), consider the potential for recruitment/identification bias at the design stage, describe the steps taken to avoid this in the final report and report observed ICCs and cluster-level eligibility criteria.

Study design and baseline description of the BMI2 trial: reducing paediatric obesity in primary care practices

OBJECTIVES

This study will test the efficacy of motivational interviewing (MI) conducted by primary care providers and dieticians among children ages 2-8 years old with a body mass index (BMI) ≥ 85th and ≤ 97th percentile.

METHODS

Forty-two practices from the American Academy of Pediatrics, Pediatric Research in Office Settings Network were assigned to one of three groups. Group 1 (usual care) measures BMI percentile at baseline, and at 1- and 2-year follow-ups and receives standard health education materials. Group 2 providers deliver three proactive MI counselling sessions with a parent of the index child in Year 1 and one additional 'booster' visit in Year 2. Group 3 adds six MI counselling sessions from a trained dietician. The primary outcome is the child's BMI percentile at 2-year follow-up. Secondary outcomes include parent report of the child's screen time, physical activity, intake of fruits and vegetables, and sugar-sweetened beverages.

RESULTS

We enrolled 633 eligible children whose mean BMI percentile was 92.0 and mean age of 5.1. The cohort was 57% female. Almost 70% of parents reported a household income of ≥ $40,000 per year, and 39% had at least a college education. The cohort was 63% white, 23% Hispanic, 7% black and 7% Asian. Parent self-reported confidence that their child will achieve a healthy weight was on average an 8 (out of 10).

CONCLUSION

To date, several aspects of the study can inform similar efforts including our ability to use volunteer clinicians to recruit participants and their willingness to dedicate their time, without pay, to receive training in MI.