Efficacy and effectiveness trials: examples from smoking cessation and bullying prevention

Comparative Effectiveness of Community-Based vs Clinic-Based Healthy Choices Motivational Intervention to Improve Health Behaviors Among Youth Living With HIV: A Randomized Clinical Trial

IMPORTANCE

Youth living with HIV make up one-quarter of new infections and have high rates of risk behaviors but are significantly understudied. Effectiveness trials in real-world settings are needed to inform program delivery.

OBJECTIVE

To compare the effectiveness of the Healthy Choices intervention delivered in a home or community setting vs a medical clinic.

DESIGN, SETTING, AND PARTICIPANTS

This randomized clinical trial was conducted from November 1, 2014, to January 31, 2018, with 52 weeks of follow-up. Participants, recruited from 5 adolescent HIV clinics in the United States, were youths and young adults living with HIV aged 16 to 24 years who were fluent in English, were currently prescribed HIV medication, had a detectable viral load, and had used alcohol in the past 12 weeks. Individuals with an active psychosis that resulted in an inability to complete questionnaires were excluded. Data were analyzed from May to December, 2019.

INTERVENTIONS

Participants were randomized to receive the Healthy Choices intervention in either a home or clinic setting. Four 30-minute individual sessions based on motivational interviewing to improve (1) medication adherence and (2) drinking behavior were delivered during 10 weeks by trained community health workers. In session 1, participants chose which behavior to discuss first. Using motivational interviewing strategies, the community health worker elicited motivational language, guided the development of an individualized change plan while supporting autonomy, delivered feedback, and addressed knowledge gaps. Session 2 focused on the second target behavior. In subsequent sessions, community health workers reviewed the individualized change plan, monitored progress, guided problem solving, and helped maintain changes made.

MAIN OUTCOMES AND MEASURES

Primary outcomes were viral load and alcohol use change trajectories during 52 weeks of follow-up. Alcohol use severity and frequency were measured using the Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) (scores range from 0 to 33, with higher scores indicating greater severity of alcohol-related problems) and number of drinks consumed each day over a 30-day period, with timeline followback.

RESULTS

A total of 183 young people living with HIV (145 male [79.2%]; mean [SD] age, 21.4 [1.9] y) were randomized to the home setting (n = 90) or clinical setting (n = 93). Using growth-curve analysis, both groups showed declines in viral load after the intervention: among participants with available viral load information, in the home group, 12 participants (21%) had an undetectable viral load at 16 weeks, 12 (22%) at 28 weeks, and 10 (20%) at 52 weeks; in the clinic group, 16 participants (24%) had an undetectable viral load at 16 weeks, 20 (39%) at 28 weeks, and 18 (35%) at 52 weeks. However, the clinic group maintained gains, whereas those counseled at home had a significantly different and increasing trajectory during follow-up (unstandardized β = -0.07; 95% CI,-0.14 to -0.01; P = .02). A similar pattern was observed in ASSIST scores during follow-up, with reduced ASSIST scores in the clinic group (unstandardized β = -0.44; 95% CI,-0.81 to -0.07; P = .02).

CONCLUSIONS AND RELEVANCE

In this trial, the Healthy Choices intervention resulted in improvements in viral load and alcohol use over 12 months. Unexpectedly, the clinic setting outperformed home-based delivery for viral suppression. Although cross-sectional differences in ASSIST scores were nonsignificant, clinic delivery did improve the trajectory of ASSIST scores during follow-up. Thus, clinics may be the more effective site for interventions aimed at viral load reduction for young people living with HIV.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT01969461.

A Population-Level, Randomized Effectiveness Trial of Recruitment Strategies for Parenting Programs in Elementary Schools

Objective: A population-level, randomized controlled trial was conducted to test the effectiveness of a parent recruitment package for increasing initial engagement into a school-based parenting program and to identify strategies responsible for effects.Method: Participants were caregivers of kindergarten- to third-grade students (N = 1,276) attending one of five schools serving ethnically diverse families living in mostly low-income, urban conditions. First, families were randomly assigned to be recruited for research surveys or not, and then to a parenting program recruitment condition: 1) Engagement-as-usual (EAU) informational flyer; 2) EAU + testimonial booklet; 3) EAU + teacher endorsement; 4) EAU + recruitment call; or 5) all strategies (full package). Caregivers were offered a free parenting program at their child's school. Primary dependent variables were parenting program enrollment and attending at least one session (initiation). Exploratory analyses were conducted on program completion, attendance across sessions, homework completion, and in-session participation.Results: In the population-level sample, enrollment and initiation were higher for the full package compared to all other conditions except the recruitment call condition. Enrollment, initiation, and program completion were higher for the recruitment call and full package conditions compared to the EAU condition. In the subsample of initiators, parents in the full package condition attended fewer parenting sessions than in the EAU condition. Controlling for attendance across sessions, there were no condition effects on homework completion or in-session participation.Conclusions: The recruitment call can increase the public health impact of evidence-based parenting programs by improving enrollment, initiation, and program completion.

Effectiveness of motivational interviewing, health education and brief advice in a population of smokers who are not ready to quit

Background

Motivational Interviewing (MI), Brief Advice (BA) and Health Education (HE) are established smoking cessation induction methods for smokers with low desire to quit. Although randomized controlled trials (RCT’s) have been frequently used to assess these interventions the temporal efficacy and effectiveness of these interventions have been poorly elaborated. The present work endeavors to fill the gap by considering the full range of possible motivational outcomes for all of the participants.

Methods

As a two-step process, Markov Chain (MC) and Ordinary Differential Equation (ODE) models were successively employed to examine the temporal efficacy and effectiveness of these interventions by computing the gradual movements of participants from an initial stage of unmotivated smoker to stages of increased motivation to quit and cessation. Specifically, in our re-analysis of data from the RCT we examined the proportion of participants in 4 stages of readiness to quit (unmotivated, undecided, motivated, former smokers) over 6 months, across treatment groups [MI (n = 87), BA (n = 43) and HE (n = 91)].

Results

Although HE had greater efficacy compared to MI and BA (i.e., the highest smoking cessation rates), it had lower effectiveness at certain time points. This was due to the fact that HE had the greatest proportion of motivated smokers who quit smoking but simultaneously a large proportion of the motivated smokers became unmotivated to quit. The effectiveness of HE dropped substantially in weeks 3–12 and remained below the effectiveness of BA from week 12 onward. The 2-year ODE model projections show that the prevalence of motivated smokers in HE group may fall below 5%. The prevalence of HE former smokers can reach an equilibrium of 26%, where the prevalence of both BA and MI former smokers exceeds this equilibrium.

Conclusions

The methodology proposed in this paper strongly benefits from the capabilities of both MC and ODE modeling approaches, in the event of low observations over the time. Particularly, the temporal population sizes are first estimated by the MC model. Then they are used to parametrize the ODE model and predict future values. The methodology enabes us to determine and compare the temporal efficacy and effectiveness of smoking cessation interventions, yielding predictive and analytic insights related to temporal characteristics and capabilities of these interventions during the study period and beyond.

Trial registration

Testing Counseling Styles to Motivate Smokers to Quit, NCT01188018, (July 4, 2012). This study is registered at www.clinicaltrials.gov NCT01188018.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0511-0 contains supplementary material, which is available to authorized users.

Community-Based Effectiveness Trials as a Means to Disseminate Evidence-Based and Culturally Responsive Behavioral Health Interventions

This article adds to the existing knowledge about innovative community-based approaches used to reduce and eliminate behavioral health disparities. Special attention is given to culturally appropriate interventions that increase access to care and that improve the quality of behavioral health services across different geographic regions and communities. Effectiveness trials are presented as appropriate tools to translate existing efficacious interventions into real-world settings. This article describes how to transition from efficacy to effectiveness, including factors essential to consider from the perspectives of university-based research teams and community-based organizations (CBOs). A case study illustrates how a university-CBO partnership mastered the complexities of transitioning an efficacious family intervention to a community setting while nurturing a new partnership. Included are implications of the study related to social work education and social work practice. Lessons learned about critical components of the community-based research collaboration and recommendations for future research are provided.

Effectiveness of Program Modification Strategies of the Diabetes Prevention Program: A Systematic Review

PURPOSE

The purpose of this study is to review the effectiveness of commonly used program modifications classified under cultural adaptation and program translational strategies for the Diabetes Prevention Program (DPP) in terms of risk reduction for type 2 diabetes.

METHODS

Authors extracted data about weight, body mass index (BMI), and 5 areas of program modification strategies from 28 interventions and analyzed them in SPSS software. Bivariate analyses examined the odds of achieving a significant reduction in outcomes by each modification of the DPP and by presence of a maintenance component, as well as the mean reduction of weight and BMI by more versus fewer modifications and by the presence of a maintenance component.

RESULTS

There were no statistically significant differences in achieving a significant reduction in weight or BMI by any type of modification or by the presence of a maintenance component. Programs with fewer modifications reported significantly greater reduction in mean weight at 12 months postintervention and the furthest time point extracted. Programs with a maintenance component achieved significantly greater reduction in mean weight measured at the furthest time point extracted.

CONCLUSIONS

The DPP appears to be programmatically robust to a variety of cultural adaptation and translational strategies. Potentially cost-saving modifications do not seem to reduce effectiveness, which should encourage implementation on a broader scale. Program planners should, however, make efforts to include maintenance components because they appear to significantly reduce risk for acquiring type 2 diabetes.

'Better Health Choices' by telephone: a feasibility trial of improving diet and physical activity in people diagnosed with psychotic disorders

The study objective was to evaluate the feasibility of a telephone delivered intervention consisting of motivational interviewing and cognitive behavioural strategies aimed at improving diet and physical activity in people diagnosed with psychotic disorders. Twenty participants diagnosed with a non-acute psychotic disorder were recruited. The intervention consisted of eight telephone delivered sessions targeting fruit and vegetable (F&V) consumption and leisure screen time, as well as smoking and alcohol use (as appropriate). F&V frequency and variety, and overall diet quality (measured by the Australian Recommended Food Score, ARFS), leisure screen time, overall sitting and walking time, smoking, alcohol consumption, mood, quality of life, and global functioning were examined before and 4-weeks post-treatment. Nineteen participants (95%) completed all intervention sessions, and 17 (85%) completed follow-up assessments. Significant increases from baseline to post-treatment were seen in ARFS fruit, vegetable and overall diet quality scores, quality of life and global functioning. Significant reductions in leisure screen time and overall sitting time were also seen. Results indicated that a telephone delivered intervention targeting key cardiovascular disease risk behaviours appears to be feasible and relatively effective in the short-term for people diagnosed with psychosis. A randomized controlled trial is warranted to replicate and extend these findings.

The Gold Standard Programme: smoking cessation interventions for disadvantaged smokers are effective in a real-life setting

OBJECTIVES

To evaluate the real-life effect of an evidence-based Gold Standard Programme (GSP) for smoking cessation interventions in disadvantaged patients and to identify modifiable factors that consistently produce the highest abstinence rates.

DESIGN

Observational prospective cohort study.

SETTING

GSPs in pharmacies, hospitals and communities in Denmark, reporting to the national Smoking Cessation Database.

PARTICIPANTS

Disadvantaged patients, defined as patients with a lower level of education and those receiving unemployment benefits.

INTERVENTIONS

6-week manualised GSP smoking cessation interventions performed by certified staff.

MAIN OUTCOME MEASURES

6 months of continuous abstinence, response rate: 80%.

RESULTS

Continuous abstinence of the 16 377 responders was 34% (of all 20 588 smokers: 27%). Continuous abstinence was lower in 5738 smokers with a lower educational level (30% of responders and 23% of all) and in 840 unemployed (27% of responders and 19% of all). In respect to modifiable factors, continuous abstinence was found more often after programmes in one-on-one formats (vs group formats) among patients with a lower educational level, 34% (vs 25%, p=0.037), or among unemployed, 35% (vs 24%, p=0.099). The variable 'format' stayed in the final model of multivariable analyses in patients with a lower educational level, OR=1.31 (95% CI 1.05 to 1.63).

CONCLUSIONS

Although continuous abstinence was lower among disadvantaged smokers, the absolute difference was small. If the programme had been as effective in disadvantaged as in non-disadvantaged groups, there would have been an extra 46 or 8 quitters annually, respectively. Promoting individual interventions among those with a low education may increase the effectiveness of GSP.

The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change

BACKGROUND

Despite growth in implementation research, limited scientific attention has focused on understanding and improving sustainability of health interventions. Models of sustainability have been evolving to reflect challenges in the fit between intervention and context.

DISCUSSION

We examine the development of concepts of sustainability, and respond to two frequent assumptions -'voltage drop,' whereby interventions are expected to yield lower benefits as they move from efficacy to effectiveness to implementation and sustainability, and 'program drift,' whereby deviation from manualized protocols is assumed to decrease benefit. We posit that these assumptions limit opportunities to improve care, and instead argue for understanding the changing context of healthcare to continuously refine and improve interventions as they are sustained. Sustainability has evolved from being considered as the endgame of a translational research process to a suggested 'adaptation phase' that integrates and institutionalizes interventions within local organizational and cultural contexts. These recent approaches locate sustainability in the implementation phase of knowledge transfer, but still do not address intervention improvement as a central theme. We propose a Dynamic Sustainability Framework that involves: continued learning and problem solving, ongoing adaptation of interventions with a primary focus on fit between interventions and multi-level contexts, and expectations for ongoing improvement as opposed to diminishing outcomes over time.

SUMMARY

A Dynamic Sustainability Framework provides a foundation for research, policy and practice that supports development and testing of falsifiable hypotheses and continued learning to advance the implementation, transportability and impact of health services research.

Effectiveness of over-the-counter nicotine replacement therapy: a qualitative review of nonrandomized trials

AIM

Randomized trials conducted in over-the-counter (OTC) settings have shown that nicotine replacement therapy (NRT) is effective. This paper reviews nonrandomized tests of the effectiveness of OTC NRT.

METHODS

Literature search via computer and other methods located (a) retrospective cohort studies of users versus nonusers of OTC NRT and (b) studies of quit rates before versus after NRT went OTC or before versus after NRT was given free to quitline callers. The methods were too heterogeneous to allow meta-analysis.

RESULTS

The results were similar for cohort and pre- versus post-studies. Most of the studies found numerically greater quitting among NRT users than nonusers. Often when NRT was not found effective, other assumed effective treatments (e.g., phone counseling) were also not found effective, suggesting biased or insensitive study methods. Only about half of the studies found statistically greater quitting among NRT users, and the most rigorous studies did not find greater quitting among users. Many studies found selection bias, for example, NRT users are more dependent smokers.

CONCLUSIONS

Some lines of evidence appear to confirm the effectiveness of OTC NRT, but others do not. We believe further secondary analyses using nonrandomized comparisons are unlikely to resolve this issue due to sensitivity, specificity, and selection bias problems.

Design, Operation, and Interpretation of Clinical Trials

Randomized controlled clinical trials offer the best evidence for changing clinical practice and informing public health policy. Using examples from the literature, this paper reviews clinical trials for those who may be unfamiliar with their design, operation, and interpretation. In the design of a clinical trial, the question to be answered and a clinically meaningful outcome must be clearly defined. Ethics must be considered, sample size carefully estimated, and use of biomarkers and surrogate outcomes understood. Prominent issues in trial implementation include developing a manual of operations, trial registration, subject recruitment and retention, use of a data coordinating center, and data and safety monitoring. Interpretation of clinical trials requires understanding differences between efficacy and effectiveness; superiority, equivalence, and non-inferiority; intent-to-treat; primary and secondary analyses; and limitations of unregistered small clinical trials compared with large multi-center Phase III trials that are more likely to be representative of a population and change clinical practice or public health policy.

Effectiveness of medication combined with intensive behavioral intervention for reducing aggression in youth with autism spectrum disorder

BACKGROUND

The use of antipsychotic medications to treat aggression in youths with autism spectrum disorders (ASD) is based on open-label trials and efficacy studies. There are no studies examining the combined effectiveness of antipsychotic medications and intensive behavioral intervention (IBI) to treat aggression in ASD.

METHODS

Youths with ASD and aggressive behavior received IBI. Medication use remained stable during the study period and was coded into antipsychotic, mood-stabilizing, and nonstimulant attention-deficit/hyperactivity disorder (ADHD)/sleep medication classes. Analysis of covariance (ANCOVA) and survival analyses examined the effects of medication classes on the average number of aggressive behaviors and time to behavior plan success.

RESULTS

Thirty-two youths (mean age = 11.16, standard deviation [SD] = 3.31, range = 4-16 years, 75% male) with ASD received aggression reduction plans. Of these, 25 youths were taking at least one psychiatric medication (antipsychotic n = 18, mood stabilizing n = 10, and nonstimulant ADHD/sleep n = 12). Aggression dropped substantially following implementation of IBI (p < 0.001; d = 1.70). Antipsychotic medication use predicted significantly fewer sessions to achieve behavior plan success (chi(2)(1) = 5.67, p = 0.017; d = 0.93). No other medication classes influenced aggressive behavior (largest chi(2)(1) = 0.16, p = 0.694).

CONCLUSIONS

Behavioral treatment combined with antipsychotic medication was the most effective approach to reducing aggressive behaviors in youths with ASD. Mood-stabilizing and nonstimulant ADHD/sleep medications did not contribute to aggression reduction.

Sexual violence prevention: the role of stages of change

Increasing numbers of empirical studies and theoretical frameworks for preventing sexual violence are appearing in the research- and practice-based literatures. The consensus of this work is that although important lessons have been learned, the field is still in the early stages of developing and fully researching effective models, particularly for the primary prevention of this problem in communities. The purpose of this article is to discuss the utility of applying the transtheoretical model of readiness for change to sexual violence prevention and evaluation. A review of this model and its application in one promising new primary prevention program is provided, along with exploratory data about what is learned about program design and effectiveness when the model is used. The study also represents one of the first attempts to operationalize and create specific measures to quantify readiness for change in the context of sexual violence prevention and evaluation. Implications for program development and evaluation research are discussed.

Development and implementation of an integrated, multi-modality, user-centered interactive dietary change program

Computer-tailored behavior change programs offer the potential for reaching large populations at a much lower cost than individual or group-based programs. However, few of these programs to date appear to integrate behavioral theory with user choice, or combine different electronic modalities. We describe the development of an integrated CD-ROM and interactive voice response dietary change intervention that combines behavioral problem-solving theory with a high degree of user choice. The program, WISE CHOICES, is being evaluated as part of an ongoing trial. This paper describes the program development, emphasizing how user preferences are accommodated, and presents implementation and user satisfaction data. The program was successfully implemented; the linkages among the central database, the CD-ROM and the automated telephone components were robust, and participants liked the program almost as well as a counselor-delivered dietary change condition. Multi-modality programs that emphasize the strengths of each approach appear to be feasible. Future research is needed to determine the program impact and cost-effectiveness compared with counselor-delivered intervention.

Percutaneous coronary interventions for non-acute coronary artery disease: a quantitative 20-year synopsis and a network meta-analysis

BACKGROUND

Over the past 20 years, percutaneous transluminal balloon coronary angioplasty (PTCA), bare-metal stents (BMS), and drug-eluting stents (DES) succeeded each other as catheter-based treatments for coronary artery disease. We undertook a systematic overview of randomised trials comparing these interventions with each other and with medical therapy in patients with non-acute coronary artery disease.

METHODS

We searched Medline for trials contrasting at least two of the four interventions (PTCA, BMS, DES, and medical therapy). Eligible outcomes were death, myocardial infarction, coronary artery bypass grafting, target lesion or vessel revascularisation, and any revascularisation. Random effects meta-analyses summarised head-to-head (direct) comparisons, and network meta-analyses integrated direct and indirect evidence.

FINDINGS

61 eligible trials (25 388 patients) investigated four of six possible comparisons between the four interventions; no trials directly compared DES with medical therapy or PTCA. In all direct or indirect comparisons, succeeding advancements in percutaneous coronary intervention did not produce detectable improvements in deaths or myocardial infarction. The risk ratio (RR) for indirect comparisons between DES and medical therapy was 0.96 (95% CI 0.60-1.52) for death and 1.15 (0.73-1.82) for myocardial infarction. By contrast, we recorded sequential significant reductions in target lesion or vessel revascularisation with BMS compared with PTCA (RR 0.68 [0-60.0.77]) and with DES compared with BMS (0.44 [0.35-0.56]). The RR for the indirect comparison between DES and PTCA for target lesion or vessel revascularisation was 0.30 (0.17-0.51).

INTERPRETATION

Sequential innovations in the catheter-based treatment of non-acute coronary artery disease showed no evidence of an effect on death or myocardial infarction when compared with medical therapy. These results lend support to present recommendations to optimise medical therapy as an initial management strategy in patients with this disease.

Translating the Olweus Bullying Prevention Program into real-world practice

This article describes the application of a community collaboration model implementing an evidence-based bullying-prevention program in elementary and middle schools. Between 2001 and 2006, the Office of School Climate and Safety, Office of Research and Evaluation, the local chapter of Physicians for Social Responsibility, and a local researcher partnered to implement the Olweus Bullying Prevention Program (BPP). The BPP is a blueprints program for youth violence prevention, following the public health model of needs assessment, problem definition, planning, implementation, and evaluation. This article describes the practical application of the model in a large urban district with multiple obstacles to implementation. The description provides a model for other community collaborations in community public health practice. A strength of collaborative practice is the ability to draw on the different capabilities of partners to provide a seamless program. A suggestion for policy is to promote community collaborations of evidence-based programs through preferential funding.

Two key strategies for teaching prevention: specialized course and infusion

Prevention is taught only rarely in counseling and counseling psychology curricula. Failure to teach it suggests that graduates may be less likely to conduct prevention. In this article, we describe two key strategies for addressing this problem, where prevention is being taught through (a) required courses, and (b) infusion within existing courses. Four training programs, two examples of each mode, are presented from the University of Cincinnati, Pennsylvania State University, Boston College, and the University of Minnesota. We describe the processes involved in developing and implementing these key pedagogical strategies that lead, we intend, to broadened application.

Donation intentions among African American college students: decisional balance and self-efficacy measures

Although the need for transplantation among African Americans is high, their donation rates are disproportionately low. This study describes the development and validation of culturally adapted psychosocial measures, including Transtheoretical Model constructs, Stages of Change, Decisional Balance, and Self-efficacy, related to deceased organ and tissue donation for an African American college population. Exploratory and confirmatory analyses for Decisional Balance and Self-efficacy measures demonstrated factor structures similar to previous studies of other behavioral applications, indicated excellent model fit and showed good internal and external validity. This study developed brief measures with good psychometric properties for an emerging behavior change domain in a new population.