An evaluation of the fidelity of implementation of a school-based drug abuse prevention program: project toward no drug abuse (TND)

Fidelity of implementation of Reptes, an alcohol and cannabis selective prevention program for youngsters

OBJECTIVE

Implementation fidelity is a key dimension in process evaluation but has been scarcely studied. Reptes is a selective alcohol and cannabis use prevention program performed in vulnerable young people (16-21 years) from different educational and leisure settings. The study aimed to describe the components of this program, its implementation fidelity, and satisfaction among participants and facilitators.

METHOD

A descriptive cross-sectional study was conducted. Implementation fidelity was assessed in terms of adherence, dose, participant responsiveness, and quality of the intervention through self-reports completed by 15 facilitators and self-administered questionnaires by 99 youth from the 15 groups completing the program.

RESULTS

70.7% of the groups performed 30 of the 43 planned activities with high use of various teaching methods, except for Information and Communication Technology resources. Fidelity implementation was above 70%. 2/3 groups completed an acceptable intervention and 1/3 completed a qualified intervention. Satisfaction was higher than 7.

CONCLUSIONS

Our results show high implementation fidelity and satisfaction compared with those from similar studies.

Community Health Workers in Schools: A Systematic Review

BACKGROUND

Community health workers (CHWs) are trusted community members who provide health education and care. However, no consensus exists regarding whether community health worker-based interventions are effective within the school setting.

OBJECTIVE

To determine outcomes and best practices of school-based community health worker interventions.

DATA SOURCES

PubMed, CINAHL, and SCOPUS databases.

STUDY ELIGIBILITY CRITERIA

This systematic literature review examined articles that described an intervention led by community health workers, targeted children and/or parents, and took place primarily within a Kindergarten-12th grade school setting. Articles were excluded if they described an intervention outside the United States.

PARTICIPANTS

Community health workers, children, and/or their parents INTERVENTIONS: School-based community health worker programs RESULTS: Of 1875 articles identified, 13 met inclusion criteria and were included in the final analysis. Of these, 5 described a statistically significant primary outcome. Seven articles provided details regarding community health worker recruitment, training, and roles that would enable reproduction of the intervention.

LIMITATIONS

This review focused on interventions in the United States. Bias of individual studies had a wide range of scores (9-21). Heterogeneity of studies also precluded a meta-analysis of primary outcomes.

CONCLUSIONS AND IMPLICATIONS OF KEY FINDINGS

The utilization of Community health workers in school-based interventions for children and/or parents is promising. This review identified a lack of detail and uniformity in program presentation, specifically with Community health worker recruitment, training, and roles. A standardized reporting mechanism for Community health worker interventions in schools would better allow for reproducibility and scalability of existing studies.

Reducing the Prevalence of Alcohol-Exposed Pregnancies in the United States: A Simulation Modeling Study

BACKGROUND

Public health efforts to prevent alcohol-exposed pregnancies (AEPs) primarily focus on promoting abstinence from alcohol among women if pregnant or seeking pregnancy and using effective contraception to prevent unintended pregnancies if consuming alcohol. Little is known about how programs to improve adherence to these recommendations would affect the prevalence of AEPs.

METHODS

We developed an individual-based simulation model of US women of reproductive age to project the prevalence of AEPs under different public health strategies. The model varies each woman's risk of an AEP over time depending on fertility, contraceptive use, awareness of pregnancy, sexual activity, and drinking patterns. We used the 2013-2015 National Survey on Family Growth data set to parameterize the model.

RESULTS

We estimate that 54% (95% uncertainty interval: 48%-59%) of pregnancies that result in a live birth in the United States are exposed to alcohol, 12% (10%-15%) are ever exposed to ≥5 alcoholic drinks in a week, and 3.0% (1.3%-4.2%) to ≥9 drinks. Unintended pregnancies (either due to contraceptive failure or sex without contraceptives) account for 80% (75%-87%) of pregnancies unknowingly exposed to alcohol. We project that public health efforts that focus only on promoting alcohol abstinence among women who are aware of their pregnancy or seeking pregnancy could reduce the prevalence of AEPs by at most 42% (36%-48%). Augmenting this strategy with efforts to avert unintended pregnancies could yield an 80% (73%-86%) reduction in the prevalence of AEPs.

CONCLUSIONS

Promoting alcohol abstinence among women who are aware of their pregnancy or seeking pregnancy offers limited potential to reduce the prevalence of AEPs. Programs to avert unintended pregnancies are essential to achieve more substantial reductions in AEPs in the United States.

Exploring the factors affecting the implementation of tobacco and substance use interventions within a secondary school setting: a systematic review

BACKGROUND

The aim of this mixed-methods, systematic literature review was to develop an understanding of the factors affecting the implementation of tobacco and substance use intervention programmes in the secondary school setting using NPT as an analytical framework.

METHODS

A search strategy was developed that combined implementation, school and intervention search terms. Literature searches were conducted in MEDLINE, Embase, PsycHINFO, Scopus, ERIC, CINAHL, Web of Science and the Cochrane Library. PROSPERO was also searched for similar systematic reviews and a grey literature search of policy documents and relevant material was also conducted. Papers were eligible for inclusion if they were based in a secondary school and focused on the implementation of a tobacco or substance use programme. Both quantitative and qualitative methodologies were considered for inclusion. Normalisation Process Theory (NPT) was used as a conceptual framework to identify facilitators and barriers of implementation and to structure the synthesis.

RESULTS

Inclusion criteria were met by 15 papers. The included papers were both quantitative and qualitative and focused on a range of tobacco and substance use interventions, delivered by differing providers. Key facilitating factors for implementation were positive organisational climate, adequate training and teacher's and pupil's motivation. Barriers to implementation included heavy workloads, budget cuts and lack of resources or support. Quality appraisal identified papers to be of moderate to weak quality, as papers generally lacked detail.

CONCLUSION

NPT highlighted the need for studies to extend their focus to include reflexive monitoring around appraisal and the evaluation processes of implementing new tobacco or substance use programs. Future research should also focus on employing implementation theory as a tool to facilitate bridging the gap between school health research and practice.

Comparación de tres programas de prevención escolar para reducir el consumo de cannabis, en función del tipo de aplicador

El propósito general de este trabajo se encuadra en la línea de investigaciones que pretenden elevar los niveles de calidad de las estrategias preventivas desarrolladas, analizando la eficacia diferencial de los programas escolares al variar el perfil de las personas encargadas de implementarlos. Para ello, compara los resultados de la aplicación de tres programas de prevención escolar sobre el consumo de cannabis y las actitudes hacia su consumo, en función del tipo de aplicador empleado. La muestra se compuso de 280 alumnos de 1º de Educación Secundaria Obligatoria (ESO), distribuidos en siete condiciones experimentales: seis de tratamiento (grupos experimentales) y otra en lista de espera (grupo control). Los grupos experimentales estuvieron determinados por la existencia de dos factores: Tipo de programa (programa Construyendo Salud, programa Saluda, programa Barbacana y Grupo Control) y tipo de Agente de aplicación (profesor habitual o psicólogo externo). Los profesores obtienen mejores resultados cuando aplican los programas Barbacana y Saluda, y los psicólogos cuando aplican el programa Construyendo Salud, tanto en el consumo de cannabis como en las actitudes relacionadas. Los resultados hallados apuntan hacia un efecto de interacción entre el tipo de programa y el tipo de aplicador.

Methods used to address fidelity of receipt in health intervention research: a citation analysis and systematic review

BACKGROUND

The American Behaviour Change Consortium (BCC) framework acknowledges patients as active participants and supports the need to investigate the fidelity with which they receive interventions, i.e. receipt. According to this framework, addressing receipt consists in using strategies to assess or enhance participants' understanding and/or performance of intervention skills. This systematic review aims to establish the frequency with which receipt is addressed as defined in the BCC framework in health research, and to describe the methods used in papers informed by the BCC framework and in the wider literature.

METHODS

A forward citation search on papers presenting the BCC framework was performed to determine the frequency with which receipt as defined in this framework was addressed. A second electronic database search, including search terms pertaining to fidelity, receipt, health and process evaluations was performed to identify papers reporting on receipt in the wider literature and irrespective of the framework used. These results were combined with forward citation search results to review methods to assess receipt. Eligibility criteria and data extraction forms were developed and applied to papers. Results are described in a narrative synthesis.

RESULTS

19.6% of 33 studies identified from the forward citation search to report on fidelity were found to address receipt. In 60.6% of these, receipt was assessed in relation to understanding and in 42.4% in relation to performance of skill. Strategies to enhance these were present in 12.1% and 21.1% of studies, respectively. Fifty-five studies were included in the review of the wider literature. Several frameworks and operationalisations of receipt were reported, but the latter were not always consistent with the guiding framework. Receipt was most frequently operationalised in relation to intervention content (16.4%), satisfaction (14.5%), engagement (14.5%), and attendance (14.5%). The majority of studies (90.0%) included subjective assessments of receipt. These relied on quantitative (76.0%) rather than qualitative (42.0%) methods and studies collected data on intervention recipients (50.0%), intervention deliverers (28.0%), or both (22.0%). Few studies (26.0%) reported on the reliability or validity of methods used.

CONCLUSIONS

Receipt is infrequently addressed in health research and improvements to methods of assessment and reporting are required.

The Effects of Implementation Fidelity in the Towards No Drug Abuse Dissemination Trial

PURPOSE

The current study examines the influence of contextual and provider-level factors on the implementation fidelity of a research-based substance abuse prevention program. Also, it investigates whether two provider-level factors, self-efficacy and beliefs about the value of the program, statistically moderate and mediate the effects of a provider training intervention on implementation fidelity.

DESIGN/METHODOLOGY/APPROACH

Using generalized mixed-linear modeling, we examine relationships between program provider-, organizational, and community-level factors and implementation fidelity in a sample of 50 high school teachers from 43 high schools in 8 states across the U.S. Fidelity of implementation was assessed utilizing an observation procedure.

FINDINGS

Implementation fidelity was negatively associated with the urbanicity of the community and the level of teachers' beliefs about the value of the program, and positively predicted by the organizational capacity of the school. Comprehensive training significantly increased teachers' self-efficacy, which resulted in an increase in implementation fidelity.

RESEARCH IMPLICATIONS

School-based prevention program implementation is influenced by a variety of contextual factors occurring at multiple ecological levels. Future effectiveness and dissemination studies need to account for the complex nature of schools in analyses of implementation fidelity and outcomes.

PRACTICAL IMPLICATIONS

Our findings suggest that both provider- and organizational-level factors are influential in promoting implementation fidelity. Before implementation begins, as well as throughout the implementation process, training and ongoing technical assistance should be conducted to increase teachers' skills, self-efficacy, and comfort with prevention curricula.

ORIGINALITY/VALUE

The present study is one of the few to examine contextual and provider-level correlates of implementation fidelity and use mediation analyses to explore whether provider-level factors mediate the effects of a provider training intervention on implementation fidelity.

Component analysis of a school-based substance use prevention program in Spain: contributions of problem solving and social skills training content

The objective of the present research was to examine the contribution of two intervention components, social skills training and problem solving training, to alcohol- and drug-related outcomes in a school-based substance use prevention program. Participants included 341 Spanish students from age 12 to 15 who received the prevention program Saluda in one of four experimental conditions: full program, social skills condition, problem solving condition, and a wait-list control group. Students completed self-report surveys at the pretest, posttest and 12-month follow-up assessments. Compared to the wait-list control group, the three intervention conditions produced reductions in alcohol use and intentions to use other substances. The intervention effect size for alcohol use was greatest in magnitude for the full program with all components. Problem-solving skills measured at the follow-up were strongest in the condition that received the full program with all components. We discuss the implications of these findings, including the advantages and disadvantages of implementing tailored interventions to students by selecting intervention components after a skills-based needs assessment.

An evaluation of immediate outcomes and fidelity of a drug abuse prevention program in continuation high schools: project towards no drug abuse (TND)

The present study provides an implementation fidelity, process, and immediate outcomes evaluation of Project Towards No Drug Abuse (TND), a drug prevention program targeting continuation high school youth (n=1426) at risk for drug abuse. A total of 24 schools participated in three randomized conditions: TND Only, TND and motivational interviewing follow-up, and no treatment control. Fidelity was high: across program schools the curriculum was implemented as intended and was received favorably by students. Relative to controls, intervention conditions produced effects on hypothesized mediators, including greater gains in program related knowledge, greater reductions in drug use intentions, and positive changes in motivation. However, few generalizations to attitudes and intentions regarding risky sexual behavior were found. The pattern of results suggests that the experimental manipulations worked as intended.

Mentoring adolescents to prevent drug and alcohol use

BACKGROUND

Many adolescents receive mentoring. There is no systematic review if mentoring prevents alcohol and drug use.

OBJECTIVES

Assess effectiveness of mentoring to prevent adolescent alcohol/drug use.

SEARCH METHODS

Cochrane CENTRAL (issue 4), MEDLINE (1950-to July 2011), EMBASE (1980-to July 2011), 5 other electronic and 11 Grey literature electronic databases, 10 websites, reference lists, experts in addictions and mentoring.

SELECTION CRITERIA

Randomised controlled trials (RCTs) of mentoring in adolescents to prevent alcohol/drug use.

DATA COLLECTION AND ANALYSIS

We identified 2,113 abstracts, independently assessed 233 full-text articles, 4 RCTs met inclusion criteria. Two reviewers independently extracted data and assessed risks of bias. We contacted investigators for missing information.

MAIN RESULTS

We identified 4 RCTs (1,194 adolescents). No RCT reported enough detail to assess whether a strong randomisation method was used or allocation was concealed. Blinding was not possible as the intervention was mentoring. Three RCTs provided complete data. No selective reporting.Three RCTs provided evidence about mentoring and preventing alcohol use. We pooled two RCTs (RR for mentoring compared to no intervention = 0.71 (95% CI = 0.57 to 0.90, P value = 0.005). A third RCT found no significant differences.Three RCTs provided evidence about mentoring and preventing drug use, but could not be pooled. One found significantly less use of "illegal" drugs," one did not, and one assessed only marijuana use and found no significant differences.One RCT measured "substance use" without separating alcohol and drugs, and found no difference for mentoring.

AUTHORS' CONCLUSIONS

All four RCTs were in the US, and included "deprived" and mostly minority adolescents. Participants were young (in two studies age 12, and in two others 9-16). All students at baseline were non-users of alcohol and drugs. Two RCTs found mentoring reduced the rate of initiation of alcohol, and one of drug usage. The ability of the interventions to be effective was limited by the low rates of commencing alcohol and drug use during the intervention period in two studies (the use of marijuana in one study increased to 1% in the experimental and to 1.6% in the control group, and in another study drug usage rose to 6% in the experimental and 11% in the control group). However, in a third study there was scope for the intervention to have an effect as alcohol use rose to 19% in the experimental and 27% in the control group. The studies assessed structured programmes and not informal mentors.

Evidence-based practice in school substance use prevention: fidelity of implementation under real-world conditions

Fidelity of program implementation under real-world conditions is a critical issue in the dissemination of evidence-based school substance use prevention curricula. Program effects are diminished when programs are implemented with poor fidelity. We assessed five domains of fidelity--adherence, exposure (dosage), quality of delivery, participant responsiveness and program differentiation (lack of contamination from other programs)--in a subset of respondents (N = 342) from a national random sample of public schools with middle school grades (N = 1721). Respondents taught 1 of 10 evidence-based universal substance use prevention programs as their primary program during the 2004-05 school year. Their responses to survey questions about their recent implementation practices indicated that fidelity was high for quality of delivery and participant responsiveness, low for program differentiation and modest for adherence and exposure--the two core domains of fidelity. Results suggest the need for continued emphasis on fidelity in program materials, trainings and on-going technical support. Particular attention should be paid to supporting use of interactive delivery strategies.

One-year follow-up evaluation of the Project Towards No Drug Abuse (TND) dissemination trial

OBJECTIVE

The aims of this trial, conducted 2004-2008, were to examine (1) the effectiveness of Project Towards No Drug Abuse (TND) at the one-year follow-up when implemented on a large scale; and (2) the relative effectiveness of two training approaches for program implementers.

METHOD

A total of 65 high schools from 14 school districts across the United States were randomized to one of three conditions: regular workshop training, comprehensive implementation support, or standard care control. Physical education and health teachers delivered the program to students (n=2538). Program effectiveness was assessed with dichotomous measures of 30-day substance use at baseline and one-year follow-up.

RESULTS

When the program conditions were considered in aggregate and compared to controls, the program showed a marginally significant effect in lowering marijuana use from baseline to the one-year follow-up. Significant program effects on hard drug use were achieved for baseline non-users only. There were no differences in the effects of the two program conditions.

CONCLUSION

Positive outcomes may be achieved by trained teachers when they implement Project TND in real-world high school environments; however, program effects are likely to be weaker than those achieved in efficacy trials. Training workshops may be adequate to build capacity for successful program implementation.

The Project Towards No Drug Abuse (TND) dissemination trial: implementation fidelity and immediate outcomes

One of the important research issues in the emerging area of research on dissemination of prevention programs relates to the type and extent of training needed by program providers to prepare them to implement effective programs with fidelity. The present paper describes the immediate outcomes of a dissemination and implementation trial of Project Toward No Drug Abuse, an evidence-based prevention program for high school students. A total of 65 high schools in 14 school districts across the USA were recruited and randomly assigned to one of three experimental conditions: comprehensive implementation support for teachers, regular workshop training only, or standard care control. The comprehensive intervention was comprised of on-site coaching, web-based support, and technical assistance, in addition to the regular workshop. Students (n = 2,983) completed self-report surveys before and immediately after program implementation. Fidelity of implementation was assessed with a classroom observation procedure that focused on program process. Results indicated that relative to the controls, both intervention conditions produced effects on hypothesized program mediators, including greater gains in program-related knowledge; greater reductions in cigarette, marijuana and hard drug use intentions; and more positive changes in drug-related beliefs. There were stronger effects on implementation fidelity in the comprehensive, relative to the regular, training condition. However, seven of the ten immediate student outcome measures showed no significant differences between the two training conditions. The implications of these findings for dissemination research and practice are discussed.

Implementation fidelity of packaged teen smoking cessation treatments delivered in community-based settings

Efficacious 'packaged' teen smoking cessation treatment programs, those developed by national organizations, are widely disseminated to local communities to help teens quit smoking. The implementation fidelity of these programs in community settings has not been documented. The efficacy of these programs could be lessened if they are not implemented as intended. Data from Helping Young Smokers Quit describe the frequency and types of modifications made to packaged teen cessation treatment programs for community delivery. A national sample of 591 community-based teen tobacco cessation treatment programs was profiled and 59% used a single packaged treatment program. Bivariate analyses found that 63% of program administrators reported implementing their program as planned; 37% modified their selected program. The most frequently reported modifications were made to the length and format of the program. Of those who modified their programs, >90% reported multiple program modifications (e.g. length and content). Administrators modified their programs to accommodate implementation barriers, such as time constraints and low participant enrollment, and to address the needs of participants with multiple risk behaviors that are co-morbid with tobacco use.

Preventing youths' use of inhalants and other harmful legal products in frontier Alaskan communities: a randomized trial

This study tests for the efficacy of a school-based drug prevention curriculum (Think Smart) that was designed to reduce use of Harmful Legal Products (HLPs, such as inhalants and over-the-counter drugs), alcohol, tobacco, and other drugs among fifth- and sixth-grade students in frontier Alaska. The curriculum consisted of 12 core sessions and 3 booster sessions administered 2 to 3 months later, and was an adaptation of the Schinke life skills training curriculum for Native Americans. Fourteen communities, which represented a mixture of Caucasian and Alaska Native populations in various regions of the state, were randomly assigned to intervention or control conditions. Single items measuring 30-day substance use and multi-item scales measuring the mediators under study were taken from prior studies. Scales for the mediators demonstrated satisfactory construct validity and internal reliability. A pre-intervention survey was administered in classrooms in each school in the fall semester of the fifth and sixth grades prior to implementing the Think Smart curriculum, and again in the spring semester immediately following the booster session. A follow-up survey was administered 6 months later in the fall semester of the sixth and seventh grades. A multi-level analysis found that the Think Smart curriculum produced a decrease (medium size effect) in the proportion of students who used HLPs over a 30-day period at the 6 month follow-up assessment. There were no effects on other drug use. Further, the direct effect of HLPs use was not mediated by the measured risk and protective factors that have been promoted in the prevention field. Alternative explanations and implications of these results are discussed.

Acculturation and substance use: social influence as a mediator among Hispanic alternative high school youth

Research suggests that acculturation increases the risk of substance use among Hispanic youth. However, this process is not well understood. This study examined associations between acculturation and several substance use indicators among a sample of 714 Hispanic youth attending alternative high schools in southern California. Peer social influence was assessed as a potential mediator. Acculturation, measured by language use, was associated with increased risk of lifetime alcohol, marijuana, and current alcohol, cigarettes, marijuana, and hard drug use, controlling for age, socioeconomic status, and gender. Results of mediation analyses indicate that peer social influence mediated the relationship between acculturation and lifetime alcohol and current alcohol, cigarettes, and hard drug use. Evidence for partial mediation was observed with lifetime and current marijuana use. These results provide evidence that peer social influence is an important mediating variable that should be considered when examining the relationship between acculturation and substance use.

Physical and relational aggression as predictors of drug use: gender differences among high school students

The present study investigated the longitudinal relationships between physical and relational aggression and later drug use, as moderated by gender. Self-reported data were gathered from 2064 high school students at pretest and 1-year post-test to test the hypotheses that (1) males would engage in more physical aggression than females, whereas females would engage in more relational aggression than males; and (2) physical aggression would be a stronger drug use predictor for males and relational aggression a stronger predictor for females. Results indicated that males reported engaging in more physical aggression than females at baseline; however, females and males reported engaging in similar rates of relational aggression. After controlling for relational aggression, baseline drug use, and demographic variables, physical aggression at baseline was found to predict alcohol use 1-year later for males but not for females. After controlling for physical aggression, baseline drug use, and demographic variables, relational aggression was found to predict cigarette use and marijuana use for females but not for males. However, relational aggression was found to predict later alcohol and hard drug equally across gender. These findings suggest that both physical and relational aggression are predictive of subsequent drug use and have important implications for violence and drug use prevention intervention efforts.

One-year follow-up evaluation of Project Towards No Drug Abuse (TND-4)

OBJECTIVES

This paper describes the one-year outcomes of the fourth experimental trial of Project Towards No Drug Abuse. Two theoretical content components of the program were examined to increase our understanding of the relative contribution of each to the effectiveness of the program.

METHODS

High schools in Southern California (n=18) were randomly assigned to one of three conditions: cognitive perception information curriculum, cognitive perception information+behavioral skills curriculum, or standard care (control). The curricula were delivered to high school students (n=2734) by project health educators and regular classroom teachers. Program effectiveness was assessed with both dichotomous and continuous measures of 30-day substance use at baseline and one-year follow-up.

RESULTS

Across all program schools, the two different curricula failed to significantly reduce dichotomous measures of substance use (cigarette, alcohol, marijuana, and hard drugs) at one-year follow-up. Both curricula exerted an effect only on the continuous measure of hard drug use, indicating a 42% (p=0.02) reduction in the number of times hard drugs were used in the last 30 days in the program groups relative to the control.

CONCLUSIONS

The lack of main effects of the program on dichotomous outcomes was contrary to previous studies. An effect on an ordinal count measure of hard drug use among both intervention conditions replicates previous work and suggests that this program effect may have been due to changes in cognitive misperception of drug use rather than behavioral skill.

Fidelity of implementation in Project Towards No Drug Abuse (TND): a comparison of classroom teachers and program specialists

This paper presents the results of an effectiveness trial of Project Towards No Drug Abuse [TND], in which we compared program delivery by regular classroom teachers and program specialists within the same high schools. Within 18 schools that were randomly assigned to the program or control conditions, health classrooms were assigned to program delivery by teachers or (outside) specialists. Classroom sessions were observed by pairs of observers to assess three domains of implementation fidelity: adherence, classroom process, and perceived student acceptance of the program. Pre- and immediate posttest survey data were collected from 2331 students. Of the four composite indexes of implementation fidelity that were examined, only one (quality of delivery) showed a difference between specialists and teachers, with marginally higher ratings of specialists (p < .10). Both teachers and program specialists achieved effects on three of the five immediate outcome measures, including program-specific knowledge, addiction concern, and social self-control. Students' posttest ratings of the program overall and the quality of program delivery failed to reveal differences between the teacher- and specialist-led classrooms. These results suggest that motivated, trained classroom teachers can implement evidence-based prevention programs with fidelity and achieve immediate effects.