Evaluating Group-Based Interventions When Control Participants Are Ungrouped

The impact of a small-group mammography video discussion on promoting screening uptake among nonadherent Chinese American immigrant women: A randomized controlled trial

BACKGROUND

The objective of this study was to evaluate the efficacy of an in-person, small-group mammography video discussion (SMVD) intervention on mammography uptake among nonadherent Chinese American immigrant women.

METHODS

Women (N = 956) were randomized into either an SMVD group, where Chinese-speaking community health workers (CHWs) used an effective, culturally appropriate video to discuss mammography, or a video-only group, which viewed the cultural video sent by mail. Outcomes were mammography uptake at 6 months and 21 months postintervention.

RESULTS

Women in both groups increased mammography uptake, and an outcome analysis revealed no group differences (adjusted odds ratio [AOR], 1.18; 95% confidence interval [CI], .68-2.06). Overall, 61.2% of the SMVD group and 55.3% of the video-only group had at least one mammogram during the 21-month follow-up period. When considering attendance to the SMVD, SMVD attendees had higher mammography uptake than the video-only group (AOR, 1.51; 95% CI, 1.19-1.92), and SMVD nonattendees had lower mammography uptake than the video-only group (AOR, .33; 95% CI, .22-.50).

CONCLUSIONS

Both intervention strategies were associated with increased mammography uptake. The authors observed that the increase in use was greater among women who participated in the SMVD session compared with those who viewed the cultural video only. Future research may explore a virtual SMVD intervention for higher session attendance and increased mammography uptake (ClinicalTrials.gov identifier NCT01292200).

Hierarchical Bayesian modeling of heterogeneous outcome variance in cluster randomized trials

BACKGROUND

Heterogeneous outcome correlations across treatment arms and clusters have been increasingly acknowledged in cluster randomized trials with binary endpoints, where analytical methods have been developed to study such heterogeneity. However, cluster-specific outcome variances and correlations have yet to be studied for cluster randomized trials with continuous outcomes.

METHODS

This article proposes models fitted in the Bayesian setting with hierarchical variance structure to quantify heterogeneous variances across clusters and explain it with cluster-level covariates when the outcome is continuous. The models can also be extended to analyzing heterogeneous variances in individually randomized group treatment trials, with arm-specific cluster-level covariates, or in partially nested designs. Simulation studies are carried out to validate the performance of the newly introduced models across different settings.

RESULTS

Simulations showed that overall the newly introduced models have good performance, reporting low bias and approximately 95% coverage for the intraclass correlation coefficients and regression parameters in the variance model. When variances are heterogeneous, our proposed models had improved model fit over models with homogeneous variances. When used to analyze data from the Kerala Diabetes Prevention Program study, our models identified heterogeneous variances and intraclass correlation coefficients across clusters and examined cluster-level characteristics associated with such heterogeneity.

CONCLUSION

We proposed new hierarchical Bayesian variance models to accommodate cluster-specific variances in cluster randomized trials. The newly developed methods inform the understanding of how an intervention strategy is implemented and disseminated differently across clusters and can help improve future trial design.

Proficiency With Number Concepts and Operations: Replicating the Efficacy of a First-Grade Mathematics Intervention

Conceptual replications are part and parcel of education science. Methodologically rigorous conceptual replication studies permit researchers to test and strengthen the generalizability of a study's initial findings. The current conceptual replication sought to replicate the efficacy of a small-group, first-grade mathematics intervention with 240 first-grade students with mathematics difficulties in a new geographical region. Participating students were randomized into one of three conditions: (a) 2:1 mathematics intervention group, (b) 5:1 mathematics intervention group, or (c) business-as-usual instruction. Relative to the original study, findings from the replication varied. When comparing the treatment groups to the control, results suggested positive effects on all outcome measures, including a follow-up assessment administered one year later. However, differences between the two treatment groups based on group size were not found in the mathematics outcome measures. Both groups also received commensurate levels of observed instructional interactions. Implications for unpacking contextual differences between original research and their replications as well as using future research to explore the quantity and quality of instructional interactions as ways to explain variation in findings of group size are discussed.

Designing individually randomized group treatment trials with repeated outcome measurements using generalized estimating equations

Individually randomized group treatment (IRGT) trials, in which the clustering of outcome is induced by group-based treatment delivery, are increasingly popular in public health research. IRGT trials frequently incorporate longitudinal measurements, of which the proper sample size calculations should account for correlation structures reflecting both the treatment-induced clustering and repeated outcome measurements. Given the relatively sparse literature on designing longitudinal IRGT trials, we propose sample size procedures for continuous and binary outcomes based on the generalized estimating equations approach, employing the block exchangeable correlation structures with different correlation parameters for the treatment arm and for the control arm, and surveying five marginal mean models with different assumptions of time effect: no-time constant treatment effect, linear-time constant treatment effect, categorical-time constant treatment effect, linear time by treatment interaction, and categorical time by treatment interaction. Closed-form sample size formulas are derived for continuous outcomes, which depends on the eigenvalues of the correlation matrices; detailed numerical sample size procedures are proposed for binary outcomes. Through simulations, we demonstrate that the empirical power agrees well with the predicted power, for as few as eight groups formed in the treatment arm, when data are analyzed using the matrix-adjusted estimating equations for the correlation parameters with a bias-corrected sandwich variance estimator.

Designing multicenter individually randomized group treatment trials

In an individually randomized group treatment (IRGT) trial, participant outcomes can be positively correlated due to, for example, shared therapists in treatment delivery. Oftentimes, because of limited treatment resources or participants at one location, an IRGT trial can be carried out across multiple centers. This design can be subject to potential correlations in the participant outcomes between arms within the same center. While the design of a single-center IRGT trial has been studied, little is known about the planning of a multicenter IRGT trial. To address this gap, this paper provides analytical sample size formulas for designing multicenter IRGT trials with a continuous endpoint under the linear mixed model framework. We found that accounting for the additional center-level correlation at the design stage can lead to sample size reduction, and the magnitude of reduction depends on the amount of between-therapist correlation. However, if the variance components of therapist-level random effects are considered as input parameters in the design stage, accounting for the additional center-level variance component has no impact on the sample size estimation. We presented our findings through numeric illustrations and performed simulation studies to validate our sample size procedures under different scenarios. Optimal design configurations under the multicenter IRGT trials have also been discussed, and two real-world trial examples are drawn to illustrate the use of our method.

A blended module (STAIRS) to promote functional and personal recovery in patients with a major depressive disorder in remission: study protocol of a concurrent mixed methods randomized controlled trial

BACKGROUND

Despite the availability of a wide variety of evidence-based treatments for major depressive disorder (MDD), many patients still experience impairments in their lives after remission. Programs are needed that effectively support patients in coping with these impairments. The program Storytelling and Training to Advance Individual Recovery Skills (STAIRS) was developed to address this need and combines the use of peer contact, expert-by-experience guidance, family support and professional blended care. The aim of the planned study is (1) to assess the efficacy of the STAIRS program in patients with remitted MDD, (2) to investigate patients' subjective experiences with STAIRS, and (3) to evaluate the program's cost-effectiveness.

METHODS

A concurrent mixed-methods randomized controlled trial design will be used. Patients aged between 18 and 65 years with remitted MDD (N = 140) will be randomized to either a group receiving care as usual (CAU) + the STAIRS-program or a control group receiving CAU + some basic psychoeducation. Quantitative efficacy data on functional and personal recovery and associated aspects will be collected using self-report questionnaires at the start of the intervention, immediately following the intervention, and at the six-month follow-up. Insights into patients' experiences on perceived effects and the way in which different program elements contribute to this effect, as well as the usability and acceptability of the program, will be gained by conducting qualitative interviews with patients from the experimental group, who are selected using maximum variation sampling. Finally, data on healthcare resource use, productivity loss and quality of life will be collected and analysed to assess the cost-effectiveness and cost-utility of the STAIRS-program.

DISCUSSION

Well-designed recovery-oriented programs for patients suffering from MDD are scarce. If efficacy and cost-effectiveness are demonstrated with this study and patients experience the STAIRS program as usable and acceptable, this program can be a valuable addition to CAU. The qualitative interviews may give insights into what works for whom, which can be used to promote implementation.

TRIAL REGISTRATION

This trial was registered at ClinicalTrials.gov on 1 July 2021, registration number NCT05440812.

A randomized controlled trial of a brief motivational interviewing-based group intervention for emerging adults experiencing homelessness: 12-Month effects on substance use and sexual risk behavior

INTRODUCTION

Few brief evidence-based risk reduction programs for emerging adults experiencing homelessness focus on the interrelated problems of substance use and sexual risk behavior. This study examines outcomes from a 12-month evaluation of AWARE, a brief Motivational Interviewing (MI)-based group risk reduction intervention for this population.

METHODS

In a cluster randomized crossover trial, N = 276 18-25-year-olds received AWARE or usual care at drop-in centers serving homeless youth in Los Angeles County. We evaluated intervention effects on substance use and condomless sex (primary outcomes), as well as drinking consequences and protective strategies, number of casual partners, self-efficacy, and motivation for change (secondary outcomes).

RESULTS

AWARE participants self-reported reductions in their alcohol use and negative consequences from drinking, and an increase in use of drinking protective strategies. AWARE participants also reported an initial decrease in drug use other than marijuana, followed by a slight uptake later on, as well as an initial increase in importance of cutting down on other drug use followed by a decrease. Control group participants did not show change in these outcomes. Among those who reported casual sex partners at both baseline and 12-month surveys, exploratory analyses indicated that AWARE participants had a 29 % decline in condomless sex with casual partners compared to a 6 % decline for control group participants.

CONCLUSIONS

Findings build on our pilot work by demonstrating that AWARE has long-term benefits on drinking among emerging adults experiencing homelessness. Further work should seek to strengthen its long-term effectiveness in reducing drug use in this population.

Using Peer Support to Prevent Diabetes: Results of a Pragmatic RCT

INTRODUCTION

High-contact structured diabetes prevention programs are effective in lowering weight and HbA1cs, yet their intensity level can create barriers to participation. Peer support programs improve clinical outcomes among adults with Type 2 diabetes, but their effectiveness in diabetes prevention is unknown. This study examined whether a low-intensity peer support program improved outcomes more than enhanced usual care in a diverse population with prediabetes.

STUDY DESIGN

The intervention was tested in a pragmatic 2-arm RCT.

SETTING/PARTICIPANTS

Participants were adults with prediabetes at three healthcare centers.

INTERVENTION

Participants randomized to the enhanced usual care arm received educational materials. Participants in the Using Peer Support to Aid in Prevention and Treatment in Prediabetes arm were matched with a peer supporter: another patient who had made healthy lifestyle changes and was trained in autonomy-supportive action planning. Peer supporters were instructed to provide weekly telephone support to their peers on specific action steps toward behavioral goals for 6 months, then monthly support for 6 months.

MAIN OUTCOME MEASURES

Changes in primary outcomes of weight and HbA1c and secondary outcomes of enrollment in formal diabetes prevention programs, self-reported diet, physical activity, health-specific social support, self-efficacy, motivation, and activation at 6 and 12 months were examined.

RESULTS

Data collection occurred from October 2018 to March 2022, with analyses completed in September 2022. Among 355 randomized patients, in intention-to-treat analyses, there were no between-group differences in HbA1c or weight changes at 6 and 12 months. Using Peer Support to Aid in Prevention and Treatment in Prediabetes participants were more likely to enroll in structured programs at 6 (AOR=2.45, p=0.009) and 12 (AOR=2.21, p=0.016) months and to report eating whole grains at 6 (4.49, p=0.026) and 12 (4.22, p=0.034) months. They reported greater improvements in perceived social support for diabetes prevention behaviors at 6 (6.39, p<0.001) and 12 (5.48, p<0.001) months, with no differences in other measures.

CONCLUSIONS

A stand-alone, low-intensity peer support program improved social support and participation in formal diabetes prevention programs but not weight or HbA1c. It will be important to examine whether peer support could effectively complement higher-intensity, structured diabetes prevention programs.

TRIAL REGISTRATION

This trial is registered at ClinicalTrials.gov, NCT03689530. Full protocol available at https://clinicaltrials.gov/ct2/show/NCT03689530.

Comparison of Prior Setting Methods for Multilevel Model Effect Estimation Based on Small Sample Imbalanced Nested Data in Bayesian Framework

In the fields of education and psychology, nested data with small samples and imbalances are very common. Bauer et al. (2008) first proposed adjusting the traditional multilevel model to analyze the small sample imbalanced nested data (SSIND). In terms of parameter estimation, the Bayesian method shows the possibility of providing unbiased estimation when the sample size is small. This study proposes that the Bayesian method should be used to analyze the SSIND. This study explores the performance of different treatment effects and nesting effects estimation methods in the multilevel model based on the Bayesian method that performs well in the case of small samples, to provide an appropriate and scientific method reference for the subsequent analysis of the model. Two prior setting methods are compared for multilevel model effect estimation based on a small sample of imbalanced nested data in the Bayesian framework. Two prior setting methods are gamma prior setting method and uniform prior setting method. The research results show that when the treatment condition ICC is small (0.05), the bias and RMSE values of the parameter estimation by the gamma prior setting method are larger and the performance is unstable, while the bias and RMSE values of the parameter estimation by the uniform prior setting method are smaller and the performance is relatively stable, so the uniform prior setting method is recommended; when the treatment condition ICC is large (0.15), the bias and RMSE values of the parameter estimation by the uniform prior setting method are larger and the performance is unstable, while the bias and RMSE values of the parameter estimation by the gamma prior setting method are smaller and the performance is relatively stable, so the gamma prior setting method is recommended; when the treatment condition ICC is between 0.05 and 0.15, both prior setting methods have similar effects. Furthermore, when the number of treatment groups is small (8), the gamma prior setting method is recommended; when the number of treatment groups is large (16), the uniform prior setting method is recommended; when the number of treatment groups is between 8 and 16, both prior setting methods have similar effects. Summarily, when we choose which prior setting method to use for the SSIND, we must consider the interaction between the ICC and the number of treatment groups.

Rumination and Sleep Quality Among Older Adults: Examining the Role of Social Support

OBJECTIVES

Although the adverse link between rumination and sleep quality is well established, much of the literature neglects the role of social factors. This study examined the role of older adults' perceived social support from spouses and from family/friends in modifying the association between trait rumination and sleep quality. Existing hypotheses suggest that social support may play 3 unique roles, each tested within the current study: (H1) support may act as a protective factor that buffers negative effects of rumination on sleep quality, (H2) support may curtail rumination and, in turn, promote sleep quality, and (H3) rumination may erode support and, in turn, undermine sleep quality.

METHOD

Data came from 86 partnered older adults in independent-living or retirement communities (Mage = 75.70 years). We utilized 3 waves of interview data collected annually between 2017 and 2019. The first hypothesis was tested using moderation in multilevel models; the second 2 hypotheses were evaluated with prospective associations using multilevel mediation.

RESULTS

Negative effects of high-trait rumination on time-varying sleep quality were attenuated among those who reported high, stable levels of support from their spouses. Perceived family/friend support did not yield the same protective effect. There was no evidence that support preempted, or was eroded by, rumination.

DISCUSSION

Perceived spousal support may act as a psychosocial resource that mitigates negative effects of trait rumination on older adults' sleep quality. Interventions aimed at mitigating maladaptive outcomes of rumination on sleep quality for older adults should consider spousal support as a key target.

Impact of complex, partially nested clustering in a three-arm individually randomized group treatment trial: A case study with the wHOPE trial

BACKGROUND/AIMS

When participants in individually randomized group treatment trials are treated by multiple clinicians or in multiple group treatment sessions throughout the trial, this induces partially nested clusters which can affect the power of a trial. We investigate this issue in the Whole Health Options and Pain Education trial, a three-arm pragmatic, individually randomized clinical trial. We evaluate whether partial clusters due to multiple visits delivered by different clinicians in the Whole Health Team arm and dynamic participant groups due to changing group leaders and/or participants across treatment sessions during treatment delivery in the Primary Care Group Education arm may impact the power of the trial. We also present a Bayesian approach to estimate the intraclass correlation coefficients.

METHODS

We present statistical models for each treatment arm of Whole Health Options and Pain Education trial in which power is estimated under different intraclass correlation coefficients and mapping matrices between participants and clinicians or treatment sessions. Power calculations are based on pairwise comparisons. In practice, sample size calculations depend on estimates of the intraclass correlation coefficients at the treatment sessions and clinician levels. To accommodate such complexities, we present a Bayesian framework for the estimation of intraclass correlation coefficients under different participant-to-session and participant-to-clinician mapping scenarios. We simulated continuous outcome data based on various clinical scenarios in Whole Health Options and Pain Education trial using a range of intraclass correlation coefficients and mapping matrices and used Gibbs samplers with conjugate priors to obtain posteriors of the intraclass correlation coefficients under those different scenarios. Posterior means and medians and their biases are calculated for the intraclass correlation coefficients to evaluate the operating characteristics of the Bayesian intraclass correlation coefficient estimators.

RESULTS

Power for Whole Health Team versus Primary Care Group Education is sensitive to the intraclass correlation coefficient in the Whole Health Team arm. In these two arms, an increased number of clinicians, more evenly distributed workload of clinicians, or more homogeneous treatment group sizes leads to increased power. Our simulation study for the intraclass correlation coefficient estimation indicates that the posterior mean intraclass correlation coefficient estimator has less bias when the true intraclass correlation coefficients are large (i.e. 0.10), but when the intraclass correlation coefficient is small (i.e. 0.01), the posterior median intraclass correlation coefficient estimator is less biased.

CONCLUSION

Knowledge of intraclass correlation coefficients and the structure of clustering are critical to the design of individually randomized group treatment trials with partially nested clusters. We demonstrate that the intraclass correlation coefficient of the Whole Health Team arm can affect power in the Whole Health Options and Pain Education trial. A Bayesian approach provides a flexible procedure for estimating the intraclass correlation coefficients under complex scenarios. More work is needed to educate the research community about the individually randomized group treatment design and encourage publication of intraclass correlation coefficients to help inform future trial designs.

Causal inference under interference with prognostic scores for dynamic group therapy studies

Group therapy is a common treatment modality for behavioral health conditions. Patients often enter and exit groups on an ongoing basis, leading to dynamic therapy groups. Examining the effect of high versus low session attendance on patient outcomes is a research question of interest. However, there are several challenges to identifying causal effects in this setting, including the lack of randomization, interference among patients, and the interrelatedness of patient participation. Dynamic therapy groups motivate a unique causal inference scenario, as the treatment statuses are completely defined by the patient attendance record for the therapy session, which is also the structure inducing interference. We adopt the Rubin causal model framework to define the causal effect of high versus low session attendance of group therapy at both the individual patient and peer levels. We propose a strategy to identify individual, peer, and total effects of high attendance versus low attendance on patient outcomes by the prognostic score stratification. We examine performance of our approach via simulation and apply it to data from a group cognitive behavioral therapy trial for treating depression among patients in a substance use disorders treatment setting.

Effects of North Carolina's Pre-Kindergarten Program at the End of Kindergarten: Contributions of School-Wide Quality

The positive effects of pre-kindergarten (pre-K) programming may be enhanced in later grades for children who subsequently experience high-quality educational environments in elementary school. The current study tested this hypothesis in relation to the effects of North Carolina's NC Pre-K program on child outcomes at the end of kindergarten, including language, literacy, mathematics, and working memory. Measures of elementary school quality were examined as moderators of the NC Pre-K effects, including school-wide academic proficiency and school-wide growth in academic achievement. We found no reliable effects of NC Pre-K participation for children attending elementary schools with average levels of quality. However, the positive effects of NC Pre-K participation on language and working memory skills were evident for children attending elementary schools with higher levels of academic proficiency and academic growth, respectively. No evidence of moderation was found in relation to literacy and mathematics skills.

Evaluating treatments for posttraumatic stress disorder, alcohol and other drug use disorders using meta-analysis of individual patient data: Design and methodology of a virtual clinical trial

This paper describes Project Harmony, a Virtual Clinical Trial (VCT) funded by the National Institute on Alcohol Abuse and Alcoholism (NIAAA) to harmonize and analyze data from over 40 independent psychological, pharmacologic and/or combined pharmacological treatment studies for posttraumatic stress disorder and comorbid alcohol and other drug use disorders (PTSD/AOD). The study attends to three distinct analysis challenges: (1) variation in measurement of PTSD/AOD across studies, time, populations and reporters, (2) cross-study variation in treatment effect sizes and (3) non-randomized, cross-study variation in the classification of treatments (despite within-study randomization of treatment arms). To address these challenges, the study combines meta-analysis of individual patient data (MIPD), integrative data analysis (IDA) and propensity score weighting (PSW) to integrate raw data from these clinical trials. This protocol shows how this VCT analytic framework was used to (1) develop commensurate scale scores of PTSD and AOD severity when measures vary across studies, (2) compare the efficacy of evidence-based treatment models for PTSD/AOD, (3) test for potential mediators of treatment effects on AOD and PTSD across treatment models, and (4) explore individual- and study-level moderators to inform for whom each of the treatment models works best. The advantages of the general VCT approach are juxtaposed against the limitations of single randomized controlled trials and conventional meta-analysis.

Pilot Randomized Clinical Trial of a Text Messaging-Based Intervention for Smoking Cessation Among Young People Experiencing Homelessness

INTRODUCTION

Smoking rates are alarmingly high among young people experiencing homelessness (YEH), yet there are no evidence-based cessation programs for this population. This paper presents results from a pilot evaluation of a text messaging-based smoking cessation treatment, as an adjunct to brief group cessation counseling, to improve abstinence rates among 18-25 year old smokers experiencing homelessness. The goal of this study was to estimate effect sizes for a larger trial and it was not powered to detect group differences.

METHODS

YEH smokers who had a working cell phone with them at recruitment were randomized to receive a group counseling session, nicotine patches, and written material on quitting (n=37) or a similar program that also included a 6-week automated text messaging intervention (TMI) to provide ongoing support for quitting (n=40). Smoking outcomes were evaluated through a 90-day follow-up.

RESULTS

7-day point prevalence abstinence at 90-day follow-up was higher in the TMI condition than standard condition (17.50% vs. 8.11%, respectively; Cohen's h=.37); however, the 90-day continuous abstinence rate was not statistically different from zero in either condition. Reductions in the number of days smoked in the past 30 days from baseline to follow-up were greater in the TMI condition than the standard condition (-14.24 vs. -8.62, respectively; Cohen's d=.49).

CONCLUSIONS

Adding a 6-week TMI support to a brief group counseling and pharmacotherapy protocol holds promise for smoking reduction and abstinence among YEH smokers. Results indicate that further development and evaluation of the TMI in this population is warranted.

IMPLICATIONS

This is the first study to evaluate the feasibility of using a text messaging-based intervention (TMI) for behavior change with 18-25 year olds experiencing homelessness, and more specifically, the first to test a TMI to provide ongoing support for smoking cessation. Small to medium effect sizes for the TMI are promising in terms of implementing a TMI using participants' own cell phones, as well as the efficacy of this approach as an adjunct to standard care (brief group counseling and pharmacotherapy) for smoking cessation among young people experiencing homelessness.

School-Based Secondary Obesity Prevention for Eight- to Twelve-Year-Olds: Results from the Students, Nurses, and Parents Seeking Healthy Options Together Randomized Trial

Objective: The primary aim of this randomized controlled trial, conducted in Minneapolis/St. Paul, Minnesota (2014-2019), was to evaluate the effects of a school-based, school nurse-delivered, secondary obesity prevention intervention to reduce excess weight gain among preadolescent children with obesity or at risk of developing obesity. Methods: Parent/child dyads (n = 132) were randomized to the 9-month Students, Nurses, and Parents Seeking Healthy Options Together (SNAPSHOT) intervention (32.5 contact hours) or newsletter-only control group. Eligible children were 8 to 12 years old, proficient in English, and with a BMI ≥75th percentile, calculated using height/weight reported by a parent, school nurse, or clinician. The primary outcome was child BMI for sex/age z-score (BMIz) at postintervention (12 months) and follow-up (24 months). Results: Among children, 63% were non-White, 51% were male, and 51% with obesity, including 21% with severe obesity. Among families, 59% received economic assistance and 30% reported food insecurity. The mean number of intervention contact hours received was 20 (range: 0-32.5). Among dyads (n = 54) receiving the intervention, parents were very satisfied/satisfied with SNAPSHOT and SNAPSHOT staff, 96% and 100%, respectively, and very likely/likely (97%) to recommend SNAPSHOT to others. Most (70%) children liked the kid group sessions "a lot." In an intent-to-treat analysis, there were no significant between-group differences in child BMIz at 12 [0.04; 95% confidence interval (CI) -0.07 to 0.16] or 24 months (0.06; 95% CI -0.08 to 0.20), with participant retention of 92% and 93%, respectively. Conclusions: The SNAPSHOT intervention was well received, but did not improve BMI in a mostly diverse, low-income preadolescent population. Approaches to intervention delivery that are feasible, maximize accessibility, and optimize clinician and school nurse collaboration warrant consideration. Clinical trial registration identifier NCT02029976.

Modular cognitive behavioral therapy for autism-related symptoms in children: A randomized controlled trial

OBJECTIVE

To date, no one-on-one psychotherapy protocol for elementary and middle school-aged children with autism spectrum disorder (ASD) has been found to be efficacious for treating autism-related symptoms such as failure to initiate social interactions. This study compared modular cognitive behavioral therapy (CBT) with enhanced standard community treatment (ESCT) in terms of impact on the severity of autism-related symptoms.

METHOD

Children with ASD (N = 107; aged 6-13 years) were randomly assigned to a treatment condition (CBT or ESCT). Both treatments provided 32 therapy sessions. The CBT condition utilized a modular design, matching specific evidence-based treatment elements to each child's clinical needs (e.g., social-communication symptoms). The ESCT condition provided social skills training and cognitive behavioral training in a structured and linear group therapy format. The primary outcome measure was independent evaluator ratings of peer engagement during school recess using a structured and validated observation system. Parents also made session-by-session ratings on personalized autism-related symptom profiles throughout treatment.

RESULTS

CBT outperformed ESCT on the primary outcome measure (p < .001; d = .50; 95% CI [.06, .93]) and the secondary outcome measure (p = .003; d = .87; 95% CI [.45, 1.27]).

CONCLUSIONS

The modular one-on-one CBT program evaluated in this study may be beneficial for reducing the severity of autism-related symptoms in some children with ASD. Further research is needed to clarify the extent of the treatment effect and the feasibility of implementation for therapists in the community. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Improving Social Knowledge and Skills among Adolescents with Autism: Systematic Review and Meta-Analysis of UCLA PEERS® for Adolescents

UCLA PEERS® for Adolescents is a widely applied program among a number of social skills training programs developed over the years. We synthesized current research evidence on the PEERS program to evaluate the treatment effect on four commonly used outcome measures. 12 studies met inclusion criteria for the review and nine met the criteria for meta-analysis. Results showed moderate to large pooled effects across measures and informants in favor of the PEERS program, with the largest effect seen in social knowledge improvement and the smallest effect in the frequency of get-togethers. The heterogeneity of effects across studies were examined and the limitations of the current evidence were discussed.

Management of Return to School Following Brain Injury: An Evaluation Model

Traumatic brain injury (TBI) affects children's ability to succeed at school. Few educators have the necessary training and knowledge needed to adequately monitor and treat students with a TBI, despite schools regularly serving as the long-term service provider. In this article, we describe a return to school model used in Oregon that implements best practices indicated by the extant literature, as well as our research protocol for evaluating this model. We discuss project aims and our planned procedures, including the measures used, our quasi-experimental design using matched controls, statistical power, and impact analyses. This project will provide the evidential base for implementation of a return to school model at scale.

Peer Support for Self-Management of Chronic Pain: the Evaluation of a Peer Coach-Led Intervention to Improve Pain Symptoms (ECLIPSE) Trial

BACKGROUND

Pain self-management is an effective, evidence-based treatment for chronic pain. Peer support, in which patients serve as coaches for other patients, has been effective in other chronic conditions and is a potentially promising approach to implementing pain self-management programs using fewer clinical resources.

OBJECTIVE

To test a peer coach-delivered pain self-management program for chronic pain.

DESIGN

Randomized controlled trial.

PARTICIPANTS

Veterans with chronic musculoskeletal pain.

INTERVENTION

Intervention patients were assigned a trained peer coach for 6 months. Coaches, who were volunteers, were asked to contact their assigned patients, either by phone or in person, twice per month. Coaches and patients were given an intervention manual to guide sessions. The control group was offered a 2-hour pain self-management class.

MAIN MEASURES

The primary outcome was total pain, assessed by the Brief Pain Inventory (BPI). Secondary outcomes were anxiety, depression, pain catastrophizing, self-efficacy, social support, patient activation, health-related quality of life, and healthcare utilization. Outcomes were measured at baseline, 6 months, and 9 months.

KEY RESULTS

Two hundred fifteen patients enrolled (120 intervention, 95 control). Adherence to intervention protocol was low, with only 13% of patients reporting having at least the recommended 12 peer coach meetings over the 6-month intervention. BPI total decreased from baseline to 6 months and baseline to 9 months in both groups. At 9 months, this change was statistically significant (intervention, - 0.40, p = 0.018; control, - 0.47, p = 0.006). There was not a statistically significant difference between groups on BPI at either time point. No secondary outcomes improved significantly in either group after adjusting for multiple comparisons.

CONCLUSIONS

Patients randomized to peer support did not differ from control patients on primary and secondary outcomes. Other peer support models that do not rely on volunteers might be more effective.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT02380690.

Study protocol for a group-based motivational interviewing brief intervention to reduce substance use and sexual risk behavior among young adults experiencing homelessness

BACKGROUND

Young people experiencing homelessness have alarmingly high rates of alcohol and other drug (AOD) use, which is associated with sexual risk behaviors such as unprotected sex, trading sex, and sex with multiple casual partners. Few risk reduction programs for this population have been developed and rigorously evaluated, particularly those that address both of these interrelated behaviors, use a collaborative and non-judgmental approach, and are feasible to deliver in settings where homeless young people seek services. This paper describes the protocol of a study evaluating a four-session Motivational Interviewing (MI)-based group risk reduction intervention for this population. The protocol has been shown to be efficacious in pilot work over 3 months with 200 homeless young adults [1]. The current study seeks to refine the intervention protocol and evaluate the program on a larger scale.

METHODS/DESIGN

In a cluster-cross-over randomized controlled trial, 18-25 year olds will receive the AWARE risk reduction program (n = 200) or standard care (n = 200) at one of three drop-in centers serving homeless youth in the Los Angeles area. We will evaluate intervention effects on primary outcomes of AOD use and sexual risk behavior, as well as secondary outcomes of health-related quality of life and social stability, over a 12-month period.

DISCUSSION

This project has the potential to fill a significant gap in prevention services by demonstrating that a brief intervention, feasible to deliver within settings where young people experiencing homelessness typically seeks services, can significantly reduce the interrelated problems of AOD use and sexual risk behavior. Trial registration ClinicalTrials.gov Identifier: NCT03735784. Registered November 18, 2018, https://clinicaltrials.gov/ct2/show/record/NCT03735784 (retrospectively registered).

A Wearable Technology Delivering a Web-Based Diabetes Prevention Program to People at High Risk of Type 2 Diabetes: Randomized Controlled Trial

BACKGROUND

Intensive lifestyle interventions are effective in reducing the risk of type 2 diabetes, but the implementation of learnings from landmark studies is expensive and time consuming. The availability of digital lifestyle interventions is increasing, but evidence of their effectiveness is limited.

OBJECTIVE

This randomized controlled trial (RCT) aimed to test the feasibility of a web-based diabetes prevention program (DPP) with step-dependent feedback messages versus a standard web-based DPP in people with prediabetes.

METHODS

We employed a two-arm, parallel, single-blind RCT for people at high risk of developing diabetes. Patients with a hemoglobin A_1c (HbA_1c) level of 39-47 mmol/mol were recruited from 21 general practices in London. The intervention integrated a smartphone app delivering a web-based DPP course with SMS texts incorporating motivational interviewing techniques and step-dependent feedback messages delivered via a wearable device over 12 months. The control group received the wearable technology and access to the web-based DDP but not the SMS texts. As this was a feasibility study, the primary aim was to estimate potential sample size at different stages of the study, including the size of the target study population and the proportion of participants who consented, were randomized, and completed follow-up. We also measured the main outcomes for a full-scale RCT, namely, change in weight and physical activity at 6- and 12-month follow-ups, and secondary outcomes, including changes in the HbA_1c level, blood pressure, waist circumference, waist-to-hip ratio, and lipid levels.

RESULTS

We enrolled 200 participants: 98 were randomized to the intervention and 102 were randomized to the control group. The follow-up rate was higher in the control group (87/102, 85.3%) than in the intervention group (69/98, 70%) at 12 months. There was no treatment effect on weight at 6 months (mean difference 0.15; 95% CI -0.93 to 1.23) or 12 months (mean difference 0.07 kg; 95% CI -1.29 to 1.44) or for physical activity levels at 6 months (mean difference -382.90 steps; 95% CI -860.65 to 94.85) or 12 months (mean difference 92.64 steps; 95% CI -380.92 to 566.20). We did not observe a treatment effect on the secondary outcomes measured at the 6-month or 12-month follow-up. For the intervention group, the mean weight was 92.33 (SD 15.67) kg at baseline, 91.34 (SD 16.04) kg at 6 months, and 89.41 (SD 14.93) kg at 12 months. For the control group, the mean weight was 92.59 (SD 17.43) kg at baseline, 91.71 (SD 16.48) kg at 6 months, and 91.10 (SD 15.82) kg at 12 months. In the intervention group, the mean physical activity was 7308.40 (SD 4911.93) steps at baseline, 5008.76 (SD 2733.22) steps at 6 months, and 4814.66 (SD 3419.65) steps at 12 months. In the control group, the mean physical activity was 7599.28 (SD 3881.04) steps at baseline, 6148.83 (SD 3433.77) steps at 6 months, and 5006.30 (SD 3681.1) steps at 12 months.

CONCLUSIONS

This study demonstrates that it is feasible to successfully recruit and retain patients in an RCT of a web-based DPP.

TRIAL REGISTRATION

ClinicalTrials.gov NCT02919397; http://clinicaltrials.gov/ct2/show/NCT02919397.

Effectiveness of Task-Shifted Trauma-Focused Cognitive Behavioral Therapy for Children Who Experienced Parental Death and Posttraumatic Stress in Kenya and Tanzania: A Randomized Clinical Trial

IMPORTANCE

Approximately 140 million children worldwide have experienced the death of one or both parents. These children, mostly in low- and middle-income countries, have higher rates of mental health problems than those who have not experienced parental death. Cognitive behavioral therapy (CBT) may improve the well-being of these children, but to our knowledge there have been no randomized clinical trials specifically focused on this population.

OBJECTIVES

To test the effectiveness of trauma-focused CBT (TF-CBT) for improving posttraumatic stress (PTS) in children in Kenya and Tanzania who have experienced parental death, to test the effects of TF-CBT on other mental health symptoms, and to examine the feasibility of task-shifting with greater reliance on experienced, local lay counselors as trainers and supervisors.

DESIGN, SETTING, AND PARTICIPANTS

A randomized clinical trial conducted in urban and rural areas of Tanzania and Kenya compared TF-CBT and usual care (UC) for 640 children aged 7 to 13 years who were recruited from February 13, 2013, to July 24, 2015. All children had experienced the death of one or both parents and had elevated PTS and/or prolonged grief. Interviewers were masked to study condition. Participants were followed up for 12 months after the randomized clinical trial. Statistical analysis was performed from February 3, 2017, to August 26, 2019. All analyses were on an intent-to-treat basis.

INTERVENTIONS

In the intervention condition, 320 children received 12 weeks of group TF-CBT delivered by lay counselors who were supervised weekly. In the UC condition, 320 children received community services typically offered to this population.

MAIN OUTCOMES AND MEASURES

The primary outcome was PTS, evaluated using a continuous, standardized measure. Other mental health symptoms and child-guardian relationship were also measured.

RESULTS

A total of 640 children (320 girls and 320 boys; mean [SD] age, 10.6 [1.6] years) were included in the study. Trauma-focused CBT was more effective than UC for PTS in 3 of 4 sites after treatment (end of 3-month randomized clinical trial): rural Kenya (Cohen d = 1.04 [95% CI, 0.72-1.36]), urban Kenya (Cohen d = 0.56 [95% CI, 0.29-0.83]), and urban Tanzania (Cohen d = 0.45 [95% CI, 0.10-0.80]). At 12-month follow-up, TF-CBT remained more effective than UC in both rural (Cohen d = 0.86 [95% CI, 0.64-1.07]) and urban (Cohen d = 0.99 [95% CI, 0.75-1.23]) Kenya. At 12-month follow-up in Tanzania, children who received TF-CBT and UC had comparable rates of improvement (rural Tanzania, Cohen d = 0.09 [95% CI, -0.08 to 0.26]; urban Tanzania, Cohen d = 0.11 [95% CI, -0.09 to 0.31]). A similar pattern was seen for secondary outcomes, with stronger effects observed in Kenya, where children experienced greater stress and adversity (eg, more food scarcity, poorer guardian health, and greater exposure to traumatic events).

CONCLUSIONS AND RELEVANCE

This study found that TF-CBT was more effective than UC in reducing PTS among children who experienced parental death in 3 of 4 sites in Kenya and Tanzania. At 12-month follow-up, TF-CBT was more effective in reducing PTS only among children in rural and urban Kenya.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT01822366.

A text message intervention for quitting cigarette smoking among young adults experiencing homelessness: study protocol for a pilot randomized controlled trial

BACKGROUND

Cigarette smoking is much more prevalent among young people experiencing homelessness than in the general population of adolescents and young adults. Although many young homeless smokers are motivated to quit, there are no empirically-evaluated smoking cessation programs for this population. It is important that any such program address the factors known to be associated with quitting-related outcomes among homeless young people, to provide ongoing support in a way that accommodates the mobility of this population, and does not rely on scarce service provider resources for its delivery. The objective of this project is to develop and pilot test a text messaging-based intervention (TMI), as an adjunct to brief cessation counseling and provision of nicotine patches, to help homeless young people who want to quit smoking.

METHODS/DESIGN

This pilot study will utilize a cluster cross-over randomized controlled design with up to 80 current smokers who desire to quit and are recruited from three drop-in centers serving young people experiencing homelessness in the Los Angeles area. All participants will be provided with a minimum standard of care: a 30-min group-based smoking cessation counseling session and free nicotine replacement. Half of these smokers will then also receive the TMI, as an adjunct to this standard care, which will provide 6 weeks of ongoing support for quitting. This support includes continued and more intensive education regarding nicotine dependence, quitting smoking, and relapse; does not require additional agency resources; can be available "on demand" to users; and includes features to personalize the quitting experience. This study will investigate whether receiving the TMI adjunct to standard smoking cessation care results in greater reductions in cigarette smoking compared to standard care alone over a 3-month period.

DISCUSSION

This study has the potential to address an important gap in the clinical research literature on cigarette smoking cessation and provide empirical support for using a TMI to provide ongoing assistance and support for quitting among young smokers experiencing homelessness. Trial registration ClinicalTrials.gov Identifier NCT03874585. Registered March 14, 2019, https://clinicaltrials.gov/ct2/show/record/NCT03874585.

Statistical power in partially nested designs probing multilevel mediation

Objective: Analysis of the intermediate behaviors and mechanisms through which innovative therapies come to shape outcomes is a critical objective in many areas of psychotherapy research because it supports the iterative exploration, development and refinement of theories and therapies. Despite widespread interest in the intermediate behaviors and mechanisms that convey treatment effects, there is limited guidance on how to effectively and efficiently design studies to detect such mediated effects in the types of partially nested designs that commonly arise in psychotherapy research. In this study, we develop statistical power formulas to identify requisite sample sizes and guide the planning of studies probing mediation under two- and three-level partially nested designs. Method: We investigate multilevel mediation in partially nested structures and models for two- and three-level designs. Results: Well-powered studies probing mediation using partially nested designs will typically require moderate to large sample sizes or moderate to large effects. Discussion: We implement these formulas in the R package PowerUpR and a simple Shiny web application (https://poweruprshiny.shinyapps.io/PartiallyNestedMediationPower/) and demonstrate their use to plan studies using partially nested designs.

Essential Ingredients and Innovations in the Design and Analysis of Group-Randomized Trials

This article reviews the essential ingredients and innovations in the design and analysis of group-randomized trials. The methods literature for these trials has grown steadily since they were introduced to the biomedical research community in the late 1970s, and we summarize those developments. We review, in addition to the group-randomized trial, methods for two closely related designs, the individually randomized group treatment trial and the stepped-wedge group-randomized trial. After describing the essential ingredients for these designs, we review the most important developments in the evolution of their methods using a new bibliometric tool developed at the National Institutes of Health. We then discuss the questions to be considered when selecting from among these designs or selecting the traditional randomized controlled trial. We close with a review of current methods for the analysis of data from these designs, a case study to illustrate each design, and a brief summary.

Using n-Level Structural Equation Models for Causal Modeling in Fully Nested, Partially Nested, and Cross-Classified Randomized Controlled Trials

Complex data structures are ubiquitous in psychological research, especially in educational settings. In the context of randomized controlled trials, students are nested in classrooms but may be cross-classified by other units, such as small groups. Furthermore, in many cases only some students may be nested within a unit while other students may not. Such instances of partial nesting requires a more flexible framework for estimating treatment effects so that the model coefficients are correctly estimated. Although several recommendations have been offered to the field on handling partially nested data, few are comprehensive in their treatment of manifest and latent variables in the context of partial nesting, full nesting, and cross-classification. The present study introduces n-level structural equation modeling (SEM) as a flexible measurement and analytic framework for the estimation of treatment effects for complex data structures that frequently present in randomized controlled trials. In this tutorial, we explore how the notation of n-level SEM allows for parsimonious model specification whether data are observed or latent and in the presence of partial nested or cross-classified designs. By using the xxm package in R, the advantage of using n-level SEM framework is demonstrated through five examples for single outcome manifest variables, as in the traditional multilevel model, as well as latent applications as in multilevel SEM.

Enhanced motivational interviewing for reducing weight and increasing physical activity in adults with high cardiovascular risk: the MOVE IT three-arm RCT

BACKGROUND

Motivational interviewing (MI) enhanced with behaviour change techniques (BCTs) and deployed by health trainers targeting multiple risk factors for cardiovascular disease (CVD) may be more effective than interventions targeting a single risk factor.

OBJECTIVES

The clinical effectiveness and cost-effectiveness of an enhanced lifestyle motivational interviewing intervention for patients at high risk of CVD in group settings versus individual settings and usual care (UC) in reducing weight and increasing physical activity (PA) were tested.

DESIGN

This was a three-arm, single-blind, parallel randomised controlled trial.

SETTING

A total of 135 general practices across all 12 South London Clinical Commissioning Groups were recruited.

PARTICIPANTS

A total of 1742 participants aged 40-74 years with a ≥ 20.0% risk of a CVD event in the following 10 years were randomised.

INTERVENTIONS

The intervention was designed to integrate MI and cognitive-behavioural therapy (CBT), delivered by trained healthy lifestyle facilitators in 10 sessions over 1 year, in group or individual format. The control group received UC.

RANDOMISATION

Simple randomisation was used with computer-generated randomisation blocks. In each block, 10 participants were randomised to the group, individual or UC arm in a 4 : 3 : 3 ratio. Researchers were blind to the allocation.

MAIN OUTCOME MEASURES

The primary outcomes are change in weight (kg) from baseline and change in PA (average number of steps per day over 1 week) from baseline at the 24-month follow-up, with an interim follow-up at 12 months. An economic evaluation estimates the relative cost-effectiveness of each intervention. Secondary outcomes include changes in low-density lipoprotein cholesterol and CVD risk score.

RESULTS

The mean age of participants was 69.75 years (standard deviation 4.11 years), 85.5% were male and 89.4% were white. At the 24-month follow-up, the group and individual intervention arms were not more effective than UC in increasing PA [mean 70.05 steps, 95% confidence interval (CI) -288 to 147.9 steps, and mean 7.24 steps, 95% CI -224.01 to 238.5 steps, respectively] or in reducing weight (mean -0.03 kg, 95% CI -0.49 to 0.44 kg, and mean -0.42 kg, 95% CI -0.93 to 0.09 kg, respectively). At the 12-month follow-up, the group and individual intervention arms were not more effective than UC in increasing PA (mean 131.1 steps, 95% CI -85.28 to 347.48 steps, and mean 210.22 steps, 95% CI -19.46 to 439.91 steps, respectively), but there were reductions in weight for the group and individual intervention arms compared with UC (mean -0.52 kg, 95% CI -0.90 to -0.13 kg, and mean -0.55 kg, 95% CI -0.95 to -0.14 kg, respectively). The group intervention arm was not more effective than the individual intervention arm in improving outcomes at either follow-up point. The group and individual interventions were not cost-effective.

CONCLUSIONS

Enhanced MI, in group or individual formats, targeted at members of the general population with high CVD risk is not effective in reducing weight or increasing PA compared with UC. Future work should focus on ensuring objective evidence of high competency in BCTs, identifying those with modifiable factors for CVD risk and improving engagement of patients and primary care.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN84864870.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 23, No. 69. See the NIHR Journals Library website for further project information. This research was part-funded by the NIHR Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King's College London.

Scleroderma Patient-centered Intervention Network-Scleroderma Support group Leader EDucation (SPIN-SSLED) program: non-randomised feasibility trial

OBJECTIVES

The Scleroderma Patient-centered Intervention Network-Scleroderma Support group Leader EDucation (SPIN-SSLED) Programme was designed to improve confidence and self-efficacy and to reduce burden for support group leaders. Objectives were to (1) evaluate feasibility of programme delivery, including required resources, management issues and scientific aspects (eg, performance of outcome measures) and (2) assess user satisfaction and identify any modifications needed to improve programme content or delivery based on participant feedback.

DESIGN

Non-randomised feasibility trial.

SETTING

North American patient organisations.

PARTICIPANTS

Current support group leaders or potential new leaders referred by patient organisations.

INTERVENTION

The programme included 13 modules delivered live via videoconference over 3 months (April to July 2018) in 60 to 90 min sessions.

OUTCOME MEASURES

(1) Elements of feasibility, including enrolment and consent procedures, percentage of referred group leaders who consented to participate, session attendance and technical support requirements; (2) programme usability, understandability, organisation and clarity; (3) leader satisfaction with the programme and (4) planned trial outcome measures, including support group leader self-efficacy, burnout, emotional distress and physical function.

RESULTS

All 12 referred potential participants consented to enrol, and 10 were included in two training groups of five participants each. Participants attended 95% of sessions. Required technical support was minimal, and videoconferencing technology functioned well. Overall programme satisfaction rating was 9.4/10. Mean item rating on the eight items of the Client Satisfaction Questionnaire-8 was 3.83 (1=low satisfaction; 4=high satisfaction). Pre-post scores on the Scleroderma Support Group Leader Self-efficacy Scale increased by 1.7 SDs (large effect); scores on burnout, emotional distress and physical function improved by 0.44, 0.38 and 0.45 SDs (moderate effects).

CONCLUSION

The SPIN-SSLED Programme was feasibly delivered, including management, resource and scientific aspects. Participant satisfaction was high. The programme is ready to be tested in a full-scale randomised controlled trial.

TRIAL REGISTRATION NUMBER

NCT03508661.

Evaluation of a peer coach-led intervention to improve pain symptoms (ECLIPSE): Rationale, study design, methods, and sample characteristics

Chronic pain is prevalent, costly, and a leading cause of disability. Pain self-management (i.e., employing self-management strategies including behavioral modifications) is an effective, evidence-based treatment. However, implementation and delivery of a pain self-management model is challenging because of time and resources. Peer supported pain self-management offers a promising approach to implementing pain self-management programs using fewer clinical resources. Evaluation of a Peer Coach-Led Intervention for the Improvement of Pain Symptoms (ECLIPSE) is a randomized controlled trial testing effectiveness of peer coach-delivered pain self-management intervention versus controls receiving a class on pain and pain self-management. ECLIPSE is a Hybrid Type 1 study testing effectiveness while examining implementation factors. ECLIPSE enrolled 215 veterans randomly assigned to the peer coaching (N = 120) or control (N = 95) arm. The peer coaching intervention lasts 6 months, with patient-peer coach pairs instructed to talk twice per month. Coaches attend initial training, are provided a detailed training manual, and attend monthly booster sessions. Outcomes are assessed at baseline, 6 months, and 9 months. The primary outcome is overall pain (intensity and interference), measured by the Brief Pain Inventory (BPI). Secondary outcomes are self-efficacy, social support, pain catastrophizing, patient activation, health-related quality of life, and health care utilization. To maximize implementation potential of pain self-management, innovative delivery methods are needed that do not require additional resources from healthcare teams. A novel and promising approach is a peer-coaching model, in which patients who are successfully managing their pain offer information, ongoing support, and advice to other patients with pain. Clinical Trials Registration: NCT02380690.

Appropriate statistical methods for analysing partially nested randomised controlled trials with continuous outcomes: a simulation study

Background

In individually randomised trials we might expect interventions delivered in groups or by care providers to result in clustering of outcomes for participants treated in the same group or by the same care provider. In partially nested randomised controlled trials (pnRCTs) this clustering only occurs in one trial arm, commonly the intervention arm. It is important to measure and account for between-cluster variability in trial design and analysis. We compare analysis approaches for pnRCTs with continuous outcomes, investigating the impact on statistical inference of cluster sizes, coding of the non-clustered arm, intracluster correlation coefficient (ICCs), and differential variance between intervention and control arm, and provide recommendations for analysis.

Methods

We performed a simulation study assessing the performance of six analysis approaches for a two-arm pnRCT with a continuous outcome. These include: linear regression model; fully clustered mixed-effects model with singleton clusters in control arm; fully clustered mixed-effects model with one large cluster in control arm; fully clustered mixed-effects model with pseudo clusters in control arm; partially nested homoscedastic mixed effects model, and partially nested heteroscedastic mixed effects model. We varied the cluster size, number of clusters, ICC, and individual variance between the two trial arms.

Results

All models provided unbiased intervention effect estimates. In the partially nested mixed-effects models, methods for classifying the non-clustered control arm had negligible impact. Failure to account for even small ICCs resulted in inflated Type I error rates and over-coverage of confidence intervals. Fully clustered mixed effects models provided poor control of the Type I error rates and biased ICC estimates. The heteroscedastic partially nested mixed-effects model maintained relatively good control of Type I error rates, unbiased ICC estimation, and did not noticeably reduce power even with homoscedastic individual variances across arms.

Conclusions

In general, we recommend the use of a heteroscedastic partially nested mixed-effects model, which models the clustering in only one arm, for continuous outcomes similar to those generated under the scenarios of our simulations study. However, with few clusters (3–6), small cluster sizes (5–10), and small ICC (≤0.05) this model underestimates Type I error rates and there is no optimal model.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0559-x) contains supplementary material, which is available to authorized users.

Predictors of long-term outcome of CBT for youth with anxiety disorders treated in community clinics

Cognitive behavioral therapy (CBT) has proven long-term effects in youth with anxiety disorders. However, only a few studies have examined predictors of long-term outcomes of CBT treatment. The present study investigated possible predictors of long-term treatment outcomes in youth with mixed anxiety disorders treated in community mental health clinics. A total of 139 youth (mean age at assessment 15.5 years, range 11-21 years) with a principal diagnosis of separation anxiety disorder, social anxiety disorder, and/or generalized anxiety disorder were evaluated a mean of 3.9 years post-treatment (range 2.2-5.9 years). Outcomes were loss of all inclusion anxiety diagnoses, loss of the principal inclusion anxiety diagnosis, and changes in youth- and parent-rated youth anxiety symptoms. Predictors encompassed youth, parent and demographic factors, and post-treatment recovery. The most consistent finding was that low family social class predicted poorer outcomes. Higher treatment motivation was associated with better outcome whereas a diagnosis of social anxiety was associated with poorer outcome. Identified predictors extend on previous findings from efficacy trials, and the results indicate a need for more specific treatment protocols.

Multilevel factorial designs with experiment-induced clustering

Factorial experimental designs have many applications in the behavioral sciences. In the context of intervention development, factorial experiments play a critical role in building and optimizing high-quality, multicomponent behavioral interventions. One challenge in implementing factorial experiments in the behavioral sciences is that individuals are often clustered in social or administrative units and may be more similar to each other than to individuals in other clusters. This means that data are dependent within clusters. Power planning resources are available for factorial experiments in which the multilevel structure of the data is due to individuals' membership in groups that existed before experimentation. However, in many cases clusters are generated in the course of the study itself. Such experiment-induced clustering (EIC) requires different data analysis models and power planning resources from those available for multilevel experimental designs in which clusters exist prior to experimentation. Despite the common occurrence of both experimental designs with EIC and factorial designs, a bridge has yet to be built between EIC and factorial designs. Therefore, resources are limited or nonexistent for planning factorial experiments that involve EIC. This article seeks to bridge this gap by extending prior models for EIC, developed for single-factor experiments, to factorial experiments involving various types of EIC. We also offer power formulas to help investigators decide whether a particular experimental design involving EIC is feasible. We demonstrate that factorial experiments can be powerful and feasible even with EIC. We discuss design considerations and directions for future research. (PsycINFO Database Record

Do clinical experience, formal cognitive behavioural therapy training, adherence, and competence predict outcome in cognitive behavioural therapy for anxiety disorders in youth?

OBJECTIVE

The aim was to investigate whether clinical experience, formal cognitive behavioural therapy (CBT) training, adherence, and competence predict outcome in CBT for anxiety disorders in youth.

METHOD

Videotapes (N = 181) from the sessions in a randomized controlled effectiveness trial (Wergeland et al., 2014, Behaviour Research and Therapy, 57, 1-12) comprising youth (N = 182, M age = 11.5 years) with mixed anxiety disorders were assessed for treatment adherence and competence using the Competence and Adherence Scale for CBT for anxiety disorders in youth (Bjaastad et al., 2016, Psychological Assessment, 28, 908-916). Therapists' (N = 17) clinical experience and educational background were assessed. Participants completed a diagnostic interview (Anxiety Disorders Interview Schedule, child and parent versions) and an anxiety symptom measure (Spence Children's Anxiety Scale, child and parent versions) at pretreatment, posttreatment, and 1-year follow-up.

RESULTS

Higher therapist adherence was related to better treatment outcomes, whereas number of years of clinical experience and competence was related to worse outcomes. However, these findings were not consistent across informants and the time points for the assessments. Interaction effects suggested that competence among therapists with formal CBT training was related to better patient outcomes.

CONCLUSIONS

Therapist adherence, competence, and clinical experience are associated with outcomes of manualized CBT for youth anxiety disorders, but mixed findings indicate the need for more research in this area.

Youth Cognitive-Behavioral Depression Prevention: Testing Theory in a Randomized Controlled Trial

This study tested the plausibility of a theoretical model of change for the Penn Resiliency Program (PRP), a cognitive-behavioral (CB) depression prevention program for adolescents. Middle school students (N = 697) were randomized to PRP, an assessment-only control condition (CON), or a placebo-control condition (PLA). Explanatory style and depressive symptoms were evaluated over 24 months of follow-up. Relative to both CON and PLA, there were significant indirect effects of PRP on 12-month levels of depressive symptoms through improvements in explanatory style in two of three participating schools. Within a third school, where PRP was not effective in targeting depressive symptoms (Gillham et al., 2007), there was no evidence of group differences in growth in explanatory style or indirect effects. When effective, PRP's CB training provides incremental value over non-specific components and there are indirect effects on depressive symptoms through improvements in explanatory style.

The development and evaluation of a web-based programme to support problem-solving skills following brain injury

BACKGROUND AND AIM

Cognitive impairments following brain injury, including difficulty with problem solving, can pose significant barriers to successful community reintegration. Problem-solving strategy training is well-supported in the cognitive rehabilitation literature. However, limitations in insurance reimbursement have resulted in fewer services to train such skills to mastery and to support generalization of those skills into everyday environments. The purpose of this project was to develop and evaluate an integrated, web-based programme, ProSolv, which uses a small number of coaching sessions to support problem solving in everyday life following brain injury.

METHOD

We used participatory action research to guide the iterative development, usability testing, and within-subject pilot testing of the ProSolv programme. The finalized programme was then evaluated in a between-subjects group study and a non-experimental single case study.

RESULTS

Results were mixed across studies. Participants demonstrated that it was feasible to learn and use the ProSolv programme for support in problem solving. They highly recommended the programme to others and singled out the importance of the coach. Limitations in app design were cited as a major reason for infrequent use of the app outside of coaching sessions.

CONCLUSIONS

Results provide mixed evidence regarding the utility of web-based mobile apps, such as ProSolv to support problem solving following brain injury. Implications for Rehabilitation People with cognitive impairments following brain injury often struggle with problem solving in everyday contexts. Research supports problem solving skills training following brain injury. Assistive technology for cognition (smartphones, selected apps) offers a means of supporting problem solving for this population. This project demonstrated the feasibility of a web-based programme to address this need.

Early intervention for children at risk for reading disabilities: The impact of grade at intervention and individual differences on intervention outcomes

Across multiple schools in three sites, the impact of grade-at-intervention was evaluated for children at risk or meeting criteria for reading disabilities. A multiple-component reading intervention with demonstrated efficacy was offered to small groups of children in 1st, 2nd, or 3rd grade. In a quasi-experimental design, 172 children received the Triple-Focus Program (PHAST + RAVE-O), and 47 were control participants. Change during intervention and 1-3 years later (6-8 testing points), and the influence of individual differences in predicting outcomes, were assessed using reading and reading-related repeated measures. Intervention children out-performed control children at posttest on all 14 outcomes, with average effect sizes (Cohen's d) on standardized measures of .80 and on experimental measures of 1.69. On foundational word reading skills (standardized measures), children who received intervention earlier, in 1st and 2nd grade, made gains relative to controls almost twice that of children receiving intervention in 3rd grade. At follow-up, the advantage of 1st grade intervention was even clearer: First graders continued to grow at faster rates over the follow-up years than 2nd graders on six of eight key reading outcomes. For some outcomes with metalinguistic demands beyond the phonological, however, a posttest advantage was revealed for 2nd grade Triple participants and for 3rd grade Triple participants relative to controls. Estimated IQ predicted growth during intervention on seven of eight outcomes. Growth during follow-up was predicted by vocabulary and visual sequential memory. These findings provide evidence on the importance of early intensive evidence-based intervention for reading problems in the primary grades.

Circle of Security-Parenting: A randomized controlled trial in Head Start

Although evidence shows that attachment insecurity and disorganization increase risk for the development of psychopathology (Fearon, Bakermans-Kranenburg, van IJzendoorn, Lapsley, & Roisman, 2010; Groh, Roisman, van IJzendoorn, Bakermans-Kranenburg, & Fearon, 2012), implementation challenges have precluded dissemination of attachment interventions on the broad scale at which they are needed. The Circle of Security-Parenting Intervention (COS-P; Cooper, Hoffman, & Powell, 2009), designed with broad implementation in mind, addresses this gap by training community service providers to use a manualized, video-based program to help caregivers provide a secure base and a safe haven for their children. The present study is a randomized controlled trial of COS-P in a low-income sample of Head Start enrolled children and their mothers. Mothers (N = 141; 75 intervention, 66 waitlist control) completed a baseline assessment and returned with their children after the 10-week intervention for the outcome assessment, which included the Strange Situation. Intent to treat analyses revealed a main effect for maternal response to child distress, with mothers assigned to COS-P reporting fewer unsupportive (but not more supportive) responses to distress than control group mothers, and a main effect for one dimension of child executive functioning (inhibitory control but not cognitive flexibility when maternal age and marital status were controlled), with intervention group children showing greater control. There were, however, no main effects of intervention for child attachment or behavior problems. Exploratory follow-up analyses suggested intervention effects were moderated by maternal attachment style or depressive symptoms, with moderated intervention effects emerging for child attachment security and disorganization, but not avoidance; for inhibitory control but not cognitive flexibility; and for child internalizing but not externalizing behavior problems. This initial randomized controlled trial of the efficacy of COS-P sets the stage for further exploration of "what works for whom" in attachment intervention.

A group-based motivational interviewing brief intervention to reduce substance use and sexual risk behavior among homeless young adults

Homeless young adults ages 18-25 exhibit high rates of alcohol and other drug (AOD) use, and sexual risk behaviors such as unprotected sex. Yet few programs exist for this population that are both effective and can be easily incorporated into settings serving this population. This pilot cluster cross-over randomized controlled trial evaluates AWARE, a voluntary four session group-based motivational interviewing (MI) intervention to reduce AOD use and sexual risk behavior. We evaluated AWARE with 200 homeless young adults using drop-in center services in Los Angeles County (mean age=21.8years; 73% male; 79% heterosexual; 31% non-Hispanic White, 25% African American, 24% Hispanic, 21% multiracial/other). Surveys were completed at baseline and three months after program completion. Retention in the AWARE program was excellent (79% attended multiple sessions) and participants reported high levels of satisfaction with the program. AWARE participants self-reported positive change in their past 3month and past 30day alcohol use (ps≤0.05), motivation to change drug use (ps<0.05), and condom use self-efficacy (p=0.05) compared to the control group. Among those with multiple sex partners, AWARE participants showed a decrease in unprotected sexual events (p<0.05), whereas the control group did not. Results from this pilot evaluation are promising, suggesting that a brief group-MI risk reduction intervention can be effective in helping homeless young adults make positive changes in their alcohol and condom use. Further work is needed to more fully evaluate the efficacy of AWARE on AOD behavior and sexual risk behavior outcomes.

Recommendations for the analysis of individually randomised controlled trials with clustering in one arm - a case of continuous outcomes

Background

In an individually randomised controlled trial where the treatment is delivered by a health professional it seems likely that the effectiveness of the treatment, independent of any treatment effect, could depend on the skill, training or even enthusiasm of the health professional delivering it. This may then lead to a potential clustering of the outcomes for patients treated by the same health professional, but similar clustering may not occur in the control arm. Using four case studies, we aim to provide practical guidance and recommendations for the analysis of trials with some element of clustering in one arm.

Methods

Five approaches to the analysis of outcomes from an individually randomised controlled trial with clustering in one arm are identified in the literature. Some of these methods are applied to four case studies of completed randomised controlled trials with clustering in one arm with sample sizes ranging from 56 to 539. Results are obtained using the statistical packages R and Stata and summarised using a forest plot.

Results

The intra-cluster correlation coefficient (ICC) for each of the case studies was small (<0.05) indicating little dependence on the outcomes related to cluster allocations. All models fitted produced similar results, including the simplest approach of ignoring clustering for the case studies considered.

Conclusions

A partially clustered approach, modelling the clustering in just one arm, most accurately represents the trial design and provides valid results. Modelling homogeneous variances between the clustered and unclustered arm is adequate in scenarios similar to the case studies considered. We recommend treating each participant in the unclustered arm as a single cluster. This approach is simple to implement in R and Stata and is recommended for the analysis of trials with clustering in one arm only. However, the case studies considered had small ICC values, limiting the generalisability of these results.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-016-0249-5) contains supplementary material, which is available to authorized users.

Estimating Standardized Effect Sizes for Two- and Three-Level Partially Nested Data

Although previous research has discussed an effect size estimator for partially nested cluster randomized designs, the existing estimator (a) is not efficient when used with primary data, (b) can be biased when the homogeneity of variance assumption is violated, and (c) has not yet been empirically evaluated for its finite sample properties. The present paper addresses these limitations by proposing an alternative maximum likelihood estimator for obtaining standardized mean difference effect size and the corresponding sampling variance for partially nested data, as well as the variants that do not make an assumption of homogeneity of variance. The typical estimator, denoted as d (d_W with pooled SD and d_C with control arm SD), requires input of summary statistics such as observed means, variances, and the intraclass correlation, and is useful for meta-analyses and secondary data analyses; the newly proposed estimator [Formula: see text] ([Formula: see text] and [Formula: see text]) takes parameter estimates from a correctly specified multilevel model as input and is mainly of interest to researchers doing primary research. The simulation results showed that the two methods (d and [Formula: see text]) produced unbiased point and variance estimates for effect size. As expected, in general, [Formula: see text] was more efficient than d with unequal cluster sizes, especially with large average cluster size and large intraclass correlation. Furthermore, under heterogeneous variances, [Formula: see text] demonstrated a greater relative efficiency with small sample size for the unclustered control arm. Real data examples, one from a youth preventive program and one from an eating disorder intervention, were used to demonstrate the methods presented. In addition, we extend the discussion to a scenario with a three-level treatment arm and an unclustered control arm, and illustrate the procedures for effect size estimation using a hypothetical example of multiple therapy groups of clients clustered within therapists.

Anxiety and Depression in Children of Depressed Parents: Dynamics of Change in a Preventive Intervention

The current study examined effects of a preventive intervention on patterns of change in symptoms of anxiety and depression in a sample of children of depressed parents. Parents with a history of depression (N = 180) and their children (N = 242; 50% female; M_age = 11.38; 74% Euro-American) enrolled in an intervention to prevent psychopathology in youth. Families were randomized to a family group cognitive behavioral intervention (FGCB) or a written information (WI) control condition. Parents and youth completed the Child Behavior Checklist and Youth Self Report at baseline, 6-, 12-, 18-, and 24-month follow up. Youth in the FGCB intervention reported significantly greater declines in symptoms of both anxiety and depression at 6, 12, and 18 months compared to youth in the WI condition. Youth with higher baseline levels of each symptom (e.g., anxiety) reported greater declines in the other symptom (e.g., depression) from 0 to 6 months in the FGCB intervention only. Changes in anxiety symptoms from 0 to 6 months predicted different patterns of subsequent changes in depressive symptoms from 6 to 12 months for the two conditions, such that declines in anxiety preceded and predicted greater declines in depression for FGCB youth but lesser increases in depression for WI youth. Findings inform transdiagnostic approaches to preventive interventions for at-risk youth, suggesting that both initial symptom levels and initial magnitude of change in symptoms are important to understand subsequent patterns of change in response to intervention.

Sample size determinations for group-based randomized clinical trials with different levels of data hierarchy between experimental and control arms

We derived sample size formulae for detecting main effects in group-based randomized clinical trials with different levels of data hierarchy between experimental and control arms. Such designs are necessary when experimental interventions need to be administered to groups of subjects whereas control conditions need to be administered to individual subjects. This type of trial, often referred to as a partially nested or partially clustered design, has been implemented for management of chronic diseases such as diabetes and is beginning to emerge more commonly in wider clinical settings. Depending on the research setting, the level of hierarchy of data structure for the experimental arm can be three or two, whereas that for the control arm is two or one. Such different levels of data hierarchy assume correlation structures of outcomes that are different between arms, regardless of whether research settings require two or three level data structure for the experimental arm. Therefore, the different correlations should be taken into account for statistical modeling and for sample size determinations. To this end, we considered mixed-effects linear models with different correlation structures between experimental and control arms to theoretically derive and empirically validate the sample size formulae with simulation studies.

Evaluating Long-term Outcomes of NHS Stop Smoking Services (ELONS): a prospective cohort study

BACKGROUND

NHS Stop Smoking Services (SSSs) provide free at the point of use treatment for smokers who would like to stop. Since their inception in 1999 they have evolved to offer a variety of support options. Given the changes that have happened in the provision of services and the ongoing need for evidence on effectiveness, the Evaluating Long-term Outcomes for NHS Stop Smoking Services (ELONS) study was commissioned.

OBJECTIVES

The main aim of the study was to explore the factors that determine longer-term abstinence from smoking following intervention by SSSs. There were also a number of additional objectives.

DESIGN

The ELONS study was an observational study with two main stages: secondary analysis of routine data collected by SSSs and a prospective cohort study of service clients. The prospective study had additional elements on client satisfaction, well-being and longer-term nicotine replacement therapy (NRT) use.

SETTING

The setting for the study was SSSs in England. For the secondary analysis, routine data from 49 services were obtained. For the prospective study and its added elements, nine services were involved. The target population was clients of these services.

PARTICIPANTS

There were 202,804 cases included in secondary analysis and 3075 in the prospective study.

INTERVENTIONS

A combination of behavioural support and stop smoking medication delivered by SSS practitioners.

MAIN OUTCOME MEASURES

Abstinence from smoking at 4 and 52 weeks after setting a quit date, validated by a carbon monoxide (CO) breath test.

RESULTS

Just over 4 in 10 smokers (41%) recruited to the prospective study were biochemically validated as abstinent from smoking at 4 weeks (which was broadly comparable with findings from the secondary analysis of routine service data, where self-reported 4-week quit rates were 48%, falling to 34% when biochemical validation had occurred). At the 1-year follow-up, 8% of prospective study clients were CO validated as abstinent from smoking. Clients who received specialist one-to-one behavioural support were twice as likely to have remained abstinent than those who were seen by a general practitioner (GP) practice and pharmacy providers [odds ratio (OR) 2.3, 95% confidence interval (CI) 1.2 to 4.6]. Clients who received group behavioural support (either closed or rolling groups) were three times more likely to stop smoking than those who were seen by a GP practice or pharmacy providers (OR 3.4, 95% CI 1.7 to 6.7). Satisfaction with services was high and well-being at baseline was found to be a predictor of abstinence from smoking at longer-term follow-up. Continued use of NRT at 1 year was rare, but no evidence of harm from longer-term use was identified from the data collected.

CONCLUSIONS

Stop Smoking Services in England are effective in helping smokers to move away from tobacco use. Using the 52-week CO-validated quit rate of 8% found in this study, we estimate that in the year 2012-13 the services supported 36,249 clients to become non-smokers for the remainder of their lives. This is a substantial figure and provides one indicator of the ongoing value of the treatment that the services provide. The study raises a number of issues for future research including (1) examining the role of electronic cigarettes (e-cigarettes) in smoking cessation for service clients [this study did not look at e-cigarette use (except briefly in the longer-term NRT study) but this is a priority for future studies]; (2) more detailed comparisons of rolling groups with other forms of behavioural support; (3) further exploration of the role of practitioner knowledge, skills and use of effective behaviour change techniques in supporting service clients to stop smoking; (4) surveillance of the impact of structural and funding changes on the future development and sustainability of SSSs; and (5) more detailed analysis of well-being over time between those who successfully stop smoking and those who relapse. Further research on longer-term use of non-combustible nicotine products that measures a wider array of biomarkers of smoking-related harm such as lung function tests or carcinogen metabolites.

FUNDING

The National Institute for Health Research Health Technology Assessment programme. The UK Centre for Tobacco and Alcohol Studies provided funding for the longer-term NRT study.